DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction  Via Gaussian Consensus
Authors: Yu Chen
Gim Hee Lee
Venue: NeurIPS 2024 poster (2024)
Keywords: Gaussian Splatting
distributed training
3D Reconstruction
Novel View Synthesis
Primary area: machine_vision

Abstract: The recent advances in 3D Gaussian Splatting (3DGS) show promising results on the novel view synthesis (NVS) task. With its superior rendering performance and high-fidelity rendering quality, 3DGS is excelling at its previous NeRF counterparts. The most recent 3DGS method focuses either on improving the instability of rendering efficiency or reducing the model size. On the other hand, the training efficiency of 3DGS on large-scale scenes has not gained much attention. In this work, we propose DoGaussian, a method that trains 3DGS distributedly. Our method first decomposes a scene into $K$ blocks and then introduces the Alternating Direction Method of Multipliers (ADMM) into the training procedure of 3DGS. During training, our DoGaussian maintains one global 3DGS model on the master node and $K$ local 3DGS models on the slave nodes. The $K$ local 3DGS models are dropped after training and we only query the global 3DGS model during inference. The training time is reduced by scene decomposition, and the training convergence and stability are guaranteed through the consensus on the shared 3D Gaussians. Our method accelerates the training of 3DGS by $6+$ times when evaluated on large-scale scenes while concurrently achieving state-of-the-art rendering quality. Our code is publicly available at [https://github.com/AIBluefisher/DOGS](https://github.com/AIBluefisher/DOGS).

DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus
Yu Chen
Gim Hee Lee National University of Singapore gimhee lee@nus edu ag
National University of Singapore yuchen01 nus .
Figure 1: DOGS accelerates 3D GS training on large-scale UrbanScene3D dataset (29 by 6+ times with better rendering quality. Top: 3D Gaussian primitives (8.27M); Bottom: rendered images.
Abstract
The recent advances in 3D Gaussian Splatting (3DGS) show promising results on the novel view synthesis (NVS) task. With its superior rendering performance and high-fidelity rendering quality, 3DGS is excelling at its previous NeRF counterparts. The most recent 3DGS method focuses either on improving rendering efficiency or reducing the model size. On the other hand, the training efficiency of 3DGS on large-scale scenes has not gained much attention. In this work, propose DOGS, method that trains 3DGS distributedly. Our method first decomposes scene into K blocks and then introduces the Alternating Direction Method of Multipliers (ADMM) into the training procedure of 3DGS. During training, our DOGS maintains one global 3DGS model on the master node and local 3DGS models the slave nodes. The K local 3DGS models dropped after training and we only query the global 3DGS model during inference. The training time is reduced by scene decomposition, and the training convergence and stability are guaranteed through the consensus on the shared 3D Gaussians. Our method accelerates the training of 3DGS by 6+ times when evaluated large-scale scenes while concurrently achieving state-of-the-art rendering quality. Our code is publicly available at htttps://giteee wwt./oooo/Aiieae /her/DGs
instability
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Introduction
Neural 3D scene reconstruction has taken a giant step beyond the limitations of traditional photogram- metry tools. Neural radiance fields (NeRFs) [36 17 47 which encode scenes implicitly MLP or explicitly in voxels, exhibit superior resilience to non-Lambertian effects, appearance changes, dynamic scenes, etc. However, most NeRF-based methods are inefficient in rendering scenes due to the need to query massive points for volume rendering to infer scene geometry and color. Recently, 3D Gaussian Splatting (3DGS) [2T] has shown promising results on real-time applications and inspires many follow-up works. 3DGS encodes scenes into set of 3D anisotropic Gaussians, where each 3D Gaussian is represented by a covariance matrix, a center position, opacity, and the latent features to encode color information. Pixel colors by projecting 3D Gaussians into 2D image space can be efficiently computed via rasterization, which is highly optimized on modern graphics processors. However, 3DGS often requires larger memory during training compared to NeRF methods. This is scene
because 3DGS needs millions of 3D Gaussians to represent a scene to recover details. Consequently, the memory footprint increases drastically for training 3DGS on larger scenes, e.g. city-scale scenes. Moreover, the training of a huge number of 3D Gaussians on larger scenes leads to longer training time. Unfortunately, in comparison to NeRF where the rays can be conveniently distributed into different compute nodes, dispatching 3D Gaussians into different compute nodes is much more difficult due to the highly customized rasterization procedure of 3DGS. In summary, the two challenges for reconstructing large-scale scenes with 3DGS are: 1) High GPU memory to hold the large 3D model during training: 2) Long training time due to the large areas of scenes.
Previous large-scale NeRF methods 49 51 351 solve the menteoee two issues by embracing a divide-and-conquee approach, where scenes split into individual blocks with smaller models fitted into each block. However, these methods require querying multiple sub-models during inference, which slows down the rendering efficiency. This leads to the following question:
"Can we apply a similar methodology to 3DGS during training while querying only a global consistent model during inference?"
In this work, we propose DOGS to answer the aforementioned question. Following previous large- scale NeRF methods, our DOGS splits scene structures into blocks for distributed training. Inspired by previously distributed bundle adjustment methods [15] 62), we apply the Alternating Direction Method of Multipliers (ADMM) [5] to ensure the consistency of the shared 3D Gaussians between different blocks. Specifically, we first estimate a tight bounding box for each scene. Subsequently, we split training views and point clouds into blocks. To guarantee each block has similar size, we split scenes into two blocks each time along the longer axis of the bounding box. Scenes are split recursively in the same way until we obtain the desired number of blocks. Finally, we re-estimate bounding box for each block and expand the bounding box to construct shared 3D Gaussians between blocks. During training, we maintain a global 3D Gaussian model on the master node and dispatch local 3D Gaussians into other slave nodes. We further guarantee the consistency of the shared 3D Gaussians and the convergence of the training through 3D Gaussian consensus. Specifically, the local 3D Gaussians are collected onto the master node and averaged to update the current global 3D Gaussian model at the end of each training iteration, and then the updated global 3D Gaussian model is shared to all slave nodes to regularize the training of local 3D) Gaussians. In this way, our method guarantees that the local 3D Gaussians converge to the global 3D Gaussian model during training. By training 3DGS in a distributed way, our DOGS can reduce the training time by 6+ times compared to the original 3DGS. Furthermore, our DOGS guarantees training convergence and therefore achieves better rendering quality than its counterparts with the 3D Gaussians consensus in the distributed training. After training, we can drop all local 3D Gaussians and maintain only the global 3D Gaussians. During inference, we only need to query the global model while maintaining the rendering performance of 3DGS.
Our contributions are summarized as follows:
We propose a recursive approach to split scenes into blocks with balanced sizes.
We introduce distributed approach to train 3D Gaussians for large-scale scenes. The training time is reduced by 6+ times compared to the original 3DGS.
We conduct exhaustive experiments on standard large-scale datasets to validate the effective- ness of our method.

---

2 Related Work
Neural Radiance Fields. Neural radiance fields [36] enable rendering from novel viewpoints with encoded frequency features [50]. To improve training and rendering efficiency, follow-up works [27 [59 17 either encodes scenes into sparse voxels, multi-resolution hash table [77 where the hash collision is implicitly handled during optimization. TensoRF [6] uses CP-decomposition or VM-decomposition to encode scenes into three orthogonal axes and planes. While the previously mentioned work focuses on per-scené reconstruction, other methods also focus on the generalizability
of NeRF [54 19 46 28 , bundle-adjusting camera poses and NeRF 25
34 8), and leveraging
sparse or dense depth to supervise the training of NeRF [56] 14 42).
etc. To address the aliasing issue in vanilla NeRF, Mip-NeRF [T] proposed to use Gaussian to approximate the cone sampling, the integrated positional encodings are therefore scale-aware and can be used to address the aliasing issue of NeRF. Mip-NeRF360 2 further uses space contraction model unbounded scenes. Zip- NeRF [3] adopted hexagonal sampling strategy to handle the aliasing issue for Instant-NGii [37]. assumes images are only available during training in each block, and they propose methods to register NeRF blocks together.
NeRF2NeRF [8]] and DReg-Nekt NeRF [|]
Gaussian Splatting. Gaussian Splatting 21 initializes 3D Gaussians from a sparse point cloud. The 3D Gaussians are used as explicit scene representation and dynamically merged and split during training. Real-time novel view synthesis is enabled by the fast splatting operation through rasterization. Scaffold-GS [3]] initializes a sparse voxel grid from the initial point cloud and encodes the features of 3D Gaussians into corresponding feature vectors. The introduction of the sparse voxel reduced the Gaussian densities by avoiding unnecessary densification on the scene surface. Octree-GS [4] introduces the level-of-details to dynamically select the appropriate level from the set of multi- renlliiossolution anchor points, which ensures consistent rendering performance with adaptive LOD adjustment and maintains high-fidelity rendering results. To reduce the model size, methods [16] 38 also remove redundant 3D Gaussians through exhaustive quantization. Other methods also focus on or leveraging the efficient rasterizer of point
alleviating the aliasing issue of 3D Gaussians
60 58
rendering for real-time indoor reconstruction
20 33
Large-Scale 3D Reconstruction. The classical photogrammetry methods utilized Structure- from- Motion (SfM) 433 and keypoint extraction with SIFT 30 to reconstruct sparse scene structures. One of such foundational software is Phototourism
To handle city-scale scenes, the 'divide- and-conquer' strategy is widely adopted for the extensibility and scalability of the 3D reconstruction system. SfM methods [4] 63 10 11 splitting scenes based on the view graph, where images with strong connections are divided into the same block. By estimating similarity transformations and merging tracks, all local SfM points and camera poses are fused into a global reference system. Existing NeRF methods also follow similar strategy for reconstructing city-scale scenes. When camera poses are known, the scene can be split into grid cells. Block-NeRF [49] focus on the day-night even cross-season street views. It utilizes the appearance encoding in NeRF-W [32 to fix the appearance inconsistency issue between different blocks, while Mega-NeRF [51] aims at encouraging the sparsity of the network under aerial scenes. Urban-NeRF (40) leverages lidar points to supervise the depth of NeRF in outdoor scenes. SUDS [52] further extended Mega-NeRF into dynamic scenes. Different from previous large-scale NeRF methods, Switch-NeRF [35] uses a switch transformer that learns to assign rays to different blocks during training. Grid-NeRF [57 designed a two-branch network architecture, where the NeRF branch can encourage the feature plane (6] branch recover more scene details under large-scale scenes. However, the two-branch training scheme is trivial and needs long time to train. Concurrent works our method are VastGaussian (26) and Hierarchy-Gr (22). which also utilize 3D Gaussians for large-scale scene reconstruction. VastGaussian and Hierarchy-GS split scenes into independent chunks and train independent chunks simultaneously. However, VastGaussian relies on exhaustive searching of the training views and initial points to guarantee the convergence of training, and each block trained without data sharing. Hierarchy-GS consolidates independent chunks into intermediate nodes for further rendering. However, the hierarchical approach needs to preserve redundant models and is specially designed for street view scenes. Our method, on the other hand, focuses on the distributed training of 3DGS and built upon the consensus of shared 3D Gaussians between different blocks has guaranteed convergence rate that achieves better performance.
45
3

---

Our Method
3.1 Preliminary
Gaussian Splatting, 3D Gaussian Splatting represents a scene with a set of anisotropic 3D Gaus- sians G {G, e N). Each 3D Gaussian primitive G, has center R and covariance E g3x and can be described by:
(p) = (p- u.) T (p-u,)}
During training, the covariance is decomposed into rotation matrix R E R3x3 3 and diagonal scaling matrix R3x3, i.e. E; RSS R. to ensure the covariance matrix is positive semi-definite. To render the color for a pixel p. the 3D Gaussians are projected into the image space for alpha blending:
i-1
c- c II a1)))
j=1
where is the rendering opacity and is computed by When training 3D Gaussian Splatting, we minimize the loss function below:
(p))
[(x) Ligh + XCssim,
(3)
where {u, qi, 8,, f,, 0;), q are quaternions corresponds to the rotation matrix R, vectors corresponds to the three diagonal elements of S, and are coefficients of the spherical harmonics.
3.2 Distributed 3D Gaussian Consensus
The -orid--and-conquer method is a common paradigm for large-scale 3D reconstruction, which we also adopt in our framework. Different from previous methods such as Block-NeRF [49 and VastGaussian [26] which are pipeline parallelized, our method is optimization parallelized with guaranteed convergence. The pipeline of our algorithm is shown Fig. 2 Firstly, we split scene (training views and point clouds) into K intersected blocks. Secondly, we assign training views and points different blocks. By introducing the ADMM into the training process, we also maintain globally consistent 3DGS model on a master node. Thirdly, during training, we collect and average the local 3D Gaussians to update the global 3DGS model each consensus step. The global 3D Gaussians are also shared with each block before we distributedly train the local 3D Gaussians in each block. Finally, we drop all local 3D Gaussians while only using the global 3D Gaussians to render novel views.
Scene Splitting
Block 3D Gaussians
Global 3D Gaussians
Synthesized Views
Assign poincs & views
GS Consensus
Splatting
Figure 2: The pipeline of our distributed 3D Gaussian Splatting method. 1) We first split the scene into blocks with similar sizes. Each block is extended to a larger size to construct overlapping parts. 2) Subsequently, we assign views and points into different blocks. The shared local 3D Gaussians (connected by solid lines in figure) are a copy the global 3D Gaussians. 3) The local 3D Gaussians are then collected and averaged to the global 3D Gaussians in each consensus step, and the global 3D Gaussians are shared with each block before training all blocks. 4) Finally, we use the final global 3D Gaussians to synthesize novel views.
In this section, we first introduce the ADMM algorithm. Subsequently, we derive the distributed 3DGS training algorithm. We also present a scene splitting algorithm, which recursively and evenly splits the scene into two blocks each time.

---

ADMM.
A general form for consensus ADMM is given by:
minimize Σ fi(x), S.L.;, == E (1.
(4)
By definition, the constraints applied such that all the local variables x, agree with the global variable z. By applying the augmented Lagrangian method, we have:
L,(x, z, Σ (f((x)) + (x: - z) + Llx, - -212),
(5)
where y; is the dual variable, is the penalty parameter. During optimization, ADMM alternatively updates the local variables x, global variable and the dual variables y; at the + iteration by:
++1 arg min (xi(x:) + vi (x: 22) P -1x:: 22' 211))
(6a)
N
1
z 1+
(x: ++1 + y:),
(6b)
N
y +1 = y + p(x?+ - 2 11
(6c)
Distributed Training. We apply the ADMM methad to distributedly train a large-scale 3D Gaussian
Splatting model. In our problem, fi(-) in Eq. 4 corresponds to the loss function L(-) in Eq. 3
To
simplify the implementation for Eq. (6a). we adopt a scaled form of ADMM by defining u, y
We can then rewrite Eq. (5) into (see supplementary for the derivation):
N C,( Σ (f((x.) + - u
u 12).
Compared to Eq. (5) Eq. 7 can be made easier to implement by expressing all terms in the squared difference errors. Suppose the variables are decomposed into K blocks, we then denote the ith 3D Gaussians in the kith block as x² Accordingly, we revise the ADMM updating rule by:
(xx)) = arg min (f(x)) + L (((xt) - z' + (ut))11)
(8a)
z' K
(x))+t1
(8b)
(u/) = (ut))* + (x/))+1 - +11
(8c)
Eq. (a)) the original loss function in Gaussian Splatting with an additional regularizer term
(x z' + (u.) Note that Eq. (8b) should be +1+1 == R Σx=1 ((x+)++I + (uf)*). However, the dual variables have an average value of zero after the first iteration. Consequently, Eq. (8b) can be simplified as the global 3D Gaussians are formed by the average of the local 3D Gaussians from all blocks. Moreover, Eq. (Bb) is called a "consensus' step and it requires collecting the local 3D Gaussians from all blocks. After updating the global model z, we update the dual variables u, in Eq. (8c) and share the global 3D Gaussians z to each block for optimizing the local 3D Gaussians in Eq. (8a). Note that each 3D Gaussian x; has different properties {u;, q; S; f. o.). As a result, the penalty terms and dual variables should be represented separately according to these properties. The detailed form of Eq. K is given in the supplementary.
Scene Splitting, We decompose the scene into K blocks before applying the updating rule in Eq. (B). Unlike VastGaussian (26). which focuses mostly on splitting large-scale scenes and needs exhaustive search on the training views and point clouds to ensure the consistency of 3D Gaussians in different blocks, our method relies on the consensus step ensure the consistency of the 3DGS. However, scene splitting is still important to the convergence of our method. We propose two constraints for the scene-splittng method to best balance the training efficiency and rendering quality:
1. Individual blocks should have similar size.
5

---

2. Adjacent blocks should have enough overlaps to boost the convergence of training.
The first constraint is proposed to ensure that: 1) Each block can be fed into GPUs with the same capacity. This important since a larger block can cause an out-of-memory of the GPU during training due to the imbalanced splitting results. 2) Each block has similar training time at each iteration. After every iteration, we collect all local 3D Gaussians from each block. Intuitively, larger blocks require a longer time to train. Consequently, the master node and all other slave nodes have to wait for the nodes with larger blocks to finish, which increases the training time unnecessarily.
The second constraint is used to boost the convergence of ADMM. From Eq. (b). the local 3D Gaussians would converge to the global 3D Gaussians by averaging the shared local 3D Gaussians during training. Sufficient shared 3D Gaussians encourage reconstruction consistency between different blocks. Too many shared local 3D Gaussians can bring more communication overhead, which inevitably slows down the training while a lack of shared 3D Gaussians leads to divergence of the algorithm. Although there is no theoretical analysis show the optimal value of overlapping parts, we empirically use constant value which we will introduce later in our experiments.
(a) Splitting results from VastGaus- (b) Splitting results from our method sian (camera trajectories)
(c) Splitting results from our method (camera trajectories and 3D points)
(camera trajectories)
Figure 3:
Scene splitting results of our method v.s. VastGaussian (26] (a) VastGaussian can result in imbalanced blocks. (b) Our recursive bipartite strategy solves the imbalanced splitting issue. (c) Points and views with the same grid coordinate are assigned to the same block.
We assume one of the axes of the scene is aligned to physical ground, which can usually be done under the Manhattan world assumption. VastGaussian (26) adopts a grid-splitting method that first splits the scene into cells along the x-axis, and then splits each of the cells into sub-cells along the y-axis. As we show in Fig, 3a this strategy can result in imbalanced blocks. Our splitting method is inherited from VastGaussian while adopting recursive spitting method to resolve the imbalanced issue. Specifically, we first estimate a tight bounding box for the initial scene. We then split the scene into two parts along the longer axis of the scene. Splitting the scene along the longer axis can prevent the blocks from becoming too shallow on one axis. We re-estimate tighter bounding box for each of the two cells and split them into smaller blocks. This step repeated until the number of blocks reaches our requirement. We present the result of this recursive method in Fig. 35 Compared to Fig. Bal we produce more balanced blocks. To construct overlapping areas, we expand the bounding box of each block by scale factor s, any points/views that are covered by the same bounding box are grouped into the same block. The training views and point clouds are split in the same way as is shown in Fig. 3c
3.3 Improving Convergence Rate
ADMM is known to be sensitive to the initialization of penalty parameters. Since improper initial penalty parameters can slow down the training, we introduce the adaptive penalty parameters and over-relaxation to improve the convergence rate.
Primal Residual and Dual Residual.
We define the primal residual r' and the dual residual s' as
x - z', s' p<zz z -²²
(9)

---

In ADMM, the primal residual and dual residual are used as the stopping criteria which terminate the optimization. In our method, we use hard threshold of training iteration to terminate the algorithm. The primal residual and dual residual are used to adaptively adjust the penalty parameters.
Adaptive Penalty Parameters. parameters:
We adopt a similar scheme from to adaptively adjust the penalty
inc
||' 2| /
0++1_
|| 212 > ~|| 12.
(10)
7
p²
||s 2|2 = /|||"* 12:
where > 1, Tim 7 are hyper-parannn peee-paaameters...
The existing convergence proof of the ADMM
algorithm based on the fixed penalty parameters [3] To guarantee the convergence of our algorithm, we stop adjusting the penalty parameters after 2000 iterations in all of our experiments.
Over Relaxation. Similar to [5) we replace x with a'x (1 - (-2")) in Eq. (8h) and Eq. (8c). where a² E (0,2) is the relaxation parameter and experiments show that the over-relaxation with a' E [1.5, 1.8] can improve convergence.
Experiments
Datasets. We evaluate our method on the two large-scale urban datasets, the Mill19 [51] dataset, the UrbanScene3D 2 dataset, and the MatrixCity [24] dataset. Both datasets are captured by drones and each scene contains thousands of high-resolution images. During training and evaluation, we adopt the original image splitting in Mega-NeRF [T], and downsample the images by 6+ times from the original resolution.
Implementation Details. For Mill19 and UrbanScene3D, the official site of Mega-NeRF [51]. The y-axis of the scene is aligned to the horizontal plane by COLMAP [3) under the Manhattan world assumption. We use the CPU version of SIFT (SIFT- CPU) in COLMAP [43] to extract keypoints and match each image to its nearest 100 images using vocabulary trees. With known camera poses and keypoint matches, we further triangulate 3D points and bundle adjust them. The SIFT-CPU can extract more points than the SIFT-GPU, which can benefit the initialization of 3D Gaussians. For the original 3D Gaussian Splatting (denoted as *3DGS'), we train it in 500,000 iterations and densify it for every 200 iterations until it reaches 30,000 steps. We train both VastGaussian and our method in 80,000 iterations. The densification intervals and termination steps are the same as the 3DGS. We reimplement VastGaussian [26) since its code was not released during this work. Note that we did not implement the decoupled appearance embedding in VastGaussian, which can be used to remove floaters caused by inconsistent image appearance. We argue that we still provided fair comparison since this module can be applied all 3DGS- based methods. For our method, consensus and sharing are enabled every 100 iterations. We leverage the remote procedure call (RPC) framework of PyTorch 39 to implement our distributed algorithm and transmit data across different compute nodes.
the camera poses provided by
Results. We employ PSNR, SSIM (55) and L.PIS [6] as metrics for novel view synthesis. We
compare our method against the state-of-the-e large-scale NeRF-based methods: Mega-NeRFF (5I),
Switch-NeRF [35 and 3DGS-basel based methods: 3DGS [21 Fed3DGS [48). VastGaussian [26). Hierarchy-GS 22 For Mega-NeRF and Switch-NeRF, we use the officially provided checkpoints on 8 blocks for evaluation. The results of Hierarchy-GS are cited from the original paper since its code has not been released during this work.
We present the quantitative visual quality results in Table I The training time and rendering efficiency also provided in Table 2 As shown in the tables, methods based on 3D Gaussians achieved better results than F-Rsed methods. Although NeRF-based methods are comparable to 3DGS methods in PSNR, the rendered images lack details in such large-scale scenes. This is also validated from
httpes//ggtaabbbccmm//mmaaaaaaap
https://ee..eehh//aaaassnss]
7

---

Fig. and Fig. 5 Moreover, NeRF-basee methods are much slower than 3DGS based methods and take longer time to train even if they are trained distributedly. Notably, our method achieves the best results in almost all the scenes. The original 3DGS has results comparable to ours. However, it takes 8x more training time than our method. We also emphasize that we build a strong baseline of 3DGS for fair comparison: the densification interval is 200 iterations, which is 8 times more frequent than the 3DGS baseline in VastGaussian; the training iteration 500K (in comparison, the training iteration of the 3DGS baseline is 450K in VastGaussian) our method. This is because our method requires additional time for data transmission. However, method achieves better rendering quality than VastGaussian. Moreover, the data transmission time does not become the bottleneck of our method due to our balanced scene splitting method. Particularly, each block has similar training time and the master node does not need to wait for long time for the fat nodes to finish its job.
VastGaussian trains 3DGS faster than
To further show the applicability of our method to larger-scale scenes, we evaluated our method on the 2.7km² Small City scene in the MatrixCity [24] dataset, which contains 5, 620 training views and 741 validation views. We early terminated the training of the original 3DGS since did not finish the training within two days. VastGaussian failed on this dataset since two blocks produce no 3D Gaussian primitives due to the imbalanced splitting. From Table [3] our method achieved the best results in rendering quality. The visual qualitative results are shown in Fig. 6
Table 1: Quantitative results of novel view synthesis on Mill19 [51 dataset and Urban- Scene3D 29 dataset. t: higher is better, J: lower is better. The red orange and yellow
colors respectively denote the best, the second best, and the third best results. denotes without applying the decoupled appearance encoding.
Boibling
Campusc
Residumes
PENR T
SSIM
LPIPS
PSNR
SSDE1
1PIPS
FSNR
SSIM
LPIES
PSNR
SSIM
LPIPS 1
PSNR.
SSDX 1
LPMPS
Mega: NeRF F31
20.52
0.547
0.454
24106
0:555
0.508
23.42
0.531
10.335
22.06
0.628
0.491
25.60
0.770
0.312
Sweich-NeREE
0.579
24.31
0.478
0.541
0.65
23.57
0.654
26.31
0.271
VDGS 10
0.745
Fe030GS
0.602
2062
0.477
20.00
0.344
21.03
0.335
0.728
25:20
0.264
21.01
22.64
Hiaaadyy 05 23
21.52
0.723
0.207
24.44
0.355
0.254
DOGS
22.78
0.739
0.384
25.78
0.365
0.277
24.01
0.81
0.57
21.94
0.740
0.244
24.42
0.804
0219
Table 2: Quantitative results of novel view synthesis on Mill19 dataset and UrbanScene3D
dataset. We present the training time (hh:mm), the number of final points (10%), the allocated memory (GB), and the framerate (FPS) during evaluation. f denotes without applying the decoupled appearance encoding.
Builing
Campun
Resllace
Train
Prines
Men
1T55
Trals
Prine
Meri
FPS
Thill
Priess
Mer
375
Twin
Porrs
Men
IT5
Trils
Poinec
Mins
FPS
Swiud--M-
133
24:46
3.54
0,099
38.30
5.87
0.003
36:15
5.85
0.007
35:11
5.6
0.007
34.36
5.00
0,038
SOGS
21.57
7.98
4.02
38.00
15:40
3.85
2.18
16657
23:03
13,6
7.69
99.53
23113
5.35
325
143.86
21.38
1.61
240.95
01:28
5.80
121.38
01:9)
471
2254
16399
00231
MA3
47.84
00:12
6.26
387
138.4%
02235
3.54
120133
Ground Truth
Mega-NeRF
Switch-NeRF
VastGaussian
Ours
Figure 4: Qualitative comparisons of our method and others on the Mill19 dataset. The first row and second row are respectively the results of scene 'building' and 'rubble'.
Ablation Study. We ablate the effectiveness of our method and present the results in Table 4 Our method without applying the 3D Gaussian consensus is denoted as w.o. CS, our method without adopting the self-adaptation of penalty parameters is denoted as w.o. SD, our method without adopting
8

---

Ground Truth
Mega-NeRF
Switch-NeRF
3D GS
VastGaussian
Ours
Figure 5:
Qualitative comparisons of our method and others on the UrbanScene3D dataset. From top to bottom are respectively the results of scenes 'campus', 'residence', and 'sci-art'.
Table 3: Quantitative results of novel view synthesis on the MatrixCity (24] dataset. t: higher better, 1: lower is better. The red, orange and yellow colors respectively denote the best, the second best, and the third best results.
1
Mom
SSIM
LPIPS
Time
f
27.36
0.818
0.237
47:60
11.9
6.31
15.57
20.03
0.643
0.650
14:24
1.85
2.33
193.32
YauGGusssaan'
28.33
0.835
0.230
05:53
12.5
6.99
40.04
DOGS
28.58
0.847
0.199
06:34
10.3
5.82
18.34
21.61
0.632
0.619
02:33
2.37
2.89
180.51
the over-relaxation is denoted as w.o. OR. As shown in the table, the performance drastically drops without the ADMM consensus step. Furthermore. the results without applying the self-adaptation of penalty parameters about 1.5 dB lower than the full model in PSNR. The model without applying over-relaxation is comparable to the full model in SSIM and LPIPS but has lower PSNR. We thus employ over-relaxation in our method. We also present the qualitative differences Fig. 7 and clearly shows the full model has better quality in the rendered images and geometries. We include more qualitative results Fig. g to show the importance of the consensus step. We can observe that the distributed training presents noisy results without the consensus step. From the two bothom-right figures, we can observe obvious artifacts along the block boundary without the consensus step.
Moreover, we ablate the scale factor in constructing the overlapping areas in Table 5 We can find that the performance of our method is improved with larger scale factor. Our method has similar performance when the scale factor is 1 1.4 and 1.5. However, we select 1.4 in our experiments since larger scale factor brings longer time and more memory requirement.
Conclusion
In this paper, proposed DOGS, scalable method for training 3DGS distributedly under large-scale scenes. Our method first splits scenes into multiple intersected blocks recursively and then trains each block distributedly model on the master node. The global 3DGS model is shared with each block to encourage each block converge to the global model. The local 3DGS of all blocks is collected update the global 3DGS model. When evaluated on large-scale datasets, our method accelerates the 3DGS training time by 6x 8x while achieving the best rendering quality in novel view synthesis.
different workers. Our method additionally maintains global 3DGS
Table 4: Ablation study of our method.
Table Ablation study of the scale factor in our method.
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
Times 1
Points
FPS
w.o. CS
22.80
0.677
0.326
1.2
24.25
0.739
0.276
02:27
4.95
129.87
w.o.
0.729
0.285
1.3
24.86
0.270
02:27
5.02
128.73
OR
24.45
0.766
0.259
1.4
25.78
0.765
0.257
02:25
4.74
147.06
fall model
25.78
0.765
0.257
1.5
25.97
0.767
0.257
02:39
4,84
130.28

---

Full Ground
VastGaussian
Figure 6: Qualitative results on the MatrixCity 24 dataset.
CS
full model
Figure 7: Ablation study of our method. Top: rendered images; Bottom: rendered depths.
full model
model
Figure 8: Importance of the consensus step.
10

---

This research project is supported by the National Research Foundation (NRF) Singapore, under its VRF-Ivvestigttorshipp Programme (Award ID. NRF-NRF109-00083, Yu Chen is also partially supported by Google PhD) Fellowship.
Acknowledgement.
References
[1] Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin Martin-Brualla, and Pratul P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In IEEE/CVF International Conference on Computer Vision, pages 5835-5844, 2021. [2] Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5460-5469, 2022.
[3] Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. CoRR, abs/2304.06706, 2023.
[4] Brojeshwar Bhowmick, Suvam Patra, Avishek Chatterjee, Venu Madhav Govindu, and Sub- hashis Banerjee. Divide and conquer: Efficient large-scale structure from motion using graph partitioning. In Asian Conference on Computer Vision, pages 273-287, 2014.
[5] Stephen P. Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed opti- mization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1): 2011.
[6] Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In Computer Vision ECCV 2022 17th European Conference, volume 13692, pages 333-350, 2022.
[7] Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao
Su. Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo. In 202/ IEEF/CVF International Conference on Computer Vision, pages 14104-14113, IEEF, 2021.
[8] Yu Chen and Gim Hee Lee. Dbarf: Deep bundle-adjusting generalizable neural radiance fields.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24 34, June 2023.
[9] Yu Chen and Gim Hee Lee. Dreg-nerf: Deep registration for neural radiance fields. In
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 22703-22713, October 2023.
[10] Yu Chen, Shuhan Shen, Yisong Chen, and Guoping Wang. Graph-based based parallel large scale structure from motion. Pattern Recognition, 107:107537, 2020.
[11] Yu Chen, Zihao Yu, Shu Song, Tianning Yu, Jianming Li, and Gim Hee Lee. Adasfm: From coarse global to fine incremental adaptive structure from motion. In IEEE International Conference Robotics and Automation, pages 2054-2061. IEEE, 2023.
[12] Yu Chen, Ji Zhao, and Laurent Kneip. Hybrid rotation averaging: A fast and robust rotation averaging approach. In IEEE Conference on Computer Vision and Pattern Recognition, pages 10358-10367, 2021.
[13] Kai Cheng, Xiaoxiao Long, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, and Xuejin Chen. Gaussianpro: 3d gaussian splatting with progressive propagation. arXiv preprint arXiv:2402.14650, 2024.
[14] Kangle Deng, Andrew Liu, Jun- Yan Zhu, and Deva Ramanan. Depth-supervised NeRF: Fewer views and faster training for free. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), June 2022.
[15] Anders P. Eriksson, John Bastian, Tat-Jun Chin, and Mats Isaksson. A consensus-based framework for distributed bundle adjustment. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, pages 1754-1 1722, 2016.
[16] Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, and Zhangyang Wang. Light- gaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps, 2023.
[17] Sara Fridovich-Ken Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5491-5500, 2022.
[18] Lily Goli, Daniel Rebain, Sara Sabour, Animesh Garg, and Andrea Tagliasacchi. nerf2nerf: Pairwise registration of neural radiance fields. CoRR, abs/22 1.01600, 2022.
11

---

[19] Mohammad Mahdi Johari. Yann Lepoittevin, and François Fleuret. Geonerf: Generalizing nerf with geometry priors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 18344-18347. IEFF, 2022.
[20] Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, and Jonathon Luiten. Splatam: Splat, track and map 3d gaussians for dense rgb-d slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024.
[21] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(44::39:11139 14, 2023,
[22] Bernhard Kerbl, Andreas Meuleman, Georgios Kopanas, Michael Wimmer, Alexandre Lanvin, and George Drettakis. A hierarchical 3d gaussian representation for real-time rendering of very large datasets. ACM Trunsuctions Graphics, 43(4), July 2024.
[23] Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning, Jun Zhou, and Lin Gu. Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization. arXiv preprint arXiv:2403.000 06912, 2024.
[24] Yixuan Li. Lihan Jiang, Linning Xu, Yuanbo Xiangli, Zhenzhi Wang. Dahua Lin, and Bo Dai. Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3205-3215, 2023.
[25] Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey. BARF: bundle-adjusting neural radiance fields. In 2021 IEEE/CVF International Conference on Computer Vision, pages
5721-5731. IEEE, 2021.
[26] Jiaqi Lin, Zhihao Li, Xiao Tang, Jianzhuang Liu, Shiyong Liu, Jiayue Liu, Yangdi Lu, Xiaofei Wu, Songeen Xu, Youliang Yan, and Wenming Yang. Vastgaussian: Vast 3d gaussians for large scene reconstruction. In CVPR, 2024.
[27] Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. In Advances in Neural Information Processing Systems 33, 2020.
[28] Yuan Liu, Sida Peng, Lingjie Liu, Qianqian Wang, Peng Wang, Christian Theobalt, Xiaowei Zhou, and Wenping Wang. Neural rays for occlusion-aware image-based rendering. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 7814-7823. IEEE, 2022.
[29] Yilin Liu, Fuyou Xue, and Hui Huang. simulator. 2021.
Urbanscene3d: A large scale urban scene dataset and
[30] David G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal Computer Vision, 60(2):91-100, 2004.
[31] Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold- gs: Structured 3d gaussians for view-adaptive rendering, 2023.
[32] Ricardo Martin BreralaBrrraala... Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, and Daniel Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In IEEE Conference on Computer Vision and Pattern Recognition, pages
7210-7219 2021.
[33] Hidenobu Matsuki, Riku Murai, Paul H. J. Kelly, and Andrew J. Davison. Gaussian Splatting
SLAM. 2024.
[34] Quan Meng, Anpei Chen, Haimin Luo, Minye Wu, Hao Su, Lan Xu, Xuming He, and Jingyi Yu. Gnerf: Gan-based neural radiance field without posed camera. In 2021 IEEE/CVF International Conference on Computer Vision, pages 6331 IEEE, 2021.
[35] Zhenxing Mi and Dan Xu. Switch-nerf: Learning scene decomposition with mixture of experts for large-scale neural radiance fields. In The Eleventh International Conference on Learning Representations, 2023.
[36] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In Computer Vision ECCV 2020 16th European Conference, volume 12346, pages 405-421, 2020.
[37] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., (12) 1500::15. 2022.
[38] KL Navaneet, Kossar Pourahmadi Meibodi, Soroush Abbasi Koohpayegani, and Hamed Pirsi- avash. Compact3d: Compressing gaussian splat radiance field models with vector quantization. arXiv preprint arXiv:2311.1815, 2023.
12

---

[39]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward 7. Yang. Zachary De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, pages 8024-8035, 2019.
[40] Konstantinos Rematas, Andrew Liu, Pratul P. Srinivasan, Jonathan T. Barron, Andrea Tagliasac- chi, Thomas A. Funkhouser, and Vittorio Ferrari. Urban radiance fields. In /EEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 12922-12932. 2022.
[41] Kerui Ren, Lihan Jiang, Tao Lu, Mulin Yu, Linning Xu, Zhangkai Ni, and Bo Dai. Octree-gs: Towards consistent real-time rendering with lod-structured 3d gaussians, 2024.
[42] Barbara Roessle, Jonathan T. Barron, Ben Mildenhall, Pratul P. Srinivasan, and Matthias NieBner.
Dense depth priors for neural radiance fields from sparse input views. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
[43] Johannes L. Schönberger and Jan-Michael Frahm. Structure cenccture-rro---o from-motion revisited. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, pages 4 4104-4113, 2016. [44] Qing Shuai, Haoyu Guo, Zhen Xu, Haotong I Lin, Sida Peng, Hujun Bao, and Xiaowei Zhou. Real-time view synthesis for large scenes with millions of square meters. 2024.
[45] Noah Snavely, Steven M. Seitz, and Richard Szeliski. Photo tourism: exploring photo collections in 3d. ACM Trans. Graph., 25(3):835-88) 2006.
[46] Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. Light field neural rendering. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8259-8269. IEEE, 2022.
[47] Cheng Sun, Min Sun, and Hwann Tzong Chen. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 5449-5459, 2022.
[48] Teppei Suzuki. Fed3dgs: Scalable 3d gaussian splatting with federated learning. CORR, abs/2403.1 14600, 2024.
[49] Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben P. Mildenhall, Pratul P. Srinivasan, Jonathan T. Barron, and Henrik Kretzschmar. Block-nerf: Scalable large scene neural view synthesis. In /EEE/CVF Conference on Computer Vision and Pattern Recognition,
pages 2238-8248, 2022.
[50] Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich- Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. In Advances in Neural Information Processing Systems, 2020.
[51] Haithem Turki, Deva Ramanan, and Mahadev Satyanarayanan. Mega-nerf: Scalable construc- tion of large-scale nerfs for virtual fly- throughs. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22912-12922 2022.
[52] Haithem Turki, Jason Y. Zhang, Francesco Ferroni, and Deva Ramanan. SUDS: scalable urban dynamic scenes. In IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages
1 12375-12385 2023.
[53] Evangelos Ververas, Rolandos Alexandros Potamias, Jifei Song, Jiankang Deng, and Stefanos Zafeiriou. Sags: Structure-aware 3d gaussian splatting. arXiv:2404 19149, 2024.
[54] Qiangian Wang, Zhicheng Wang. Kyle Genova, Pratul P. Srinivasan. Howard Zhou, Jonathan T. Barron, Ricardo Martin-Brualla, Noah Snavely, and Thomas A. Funkhouser. Ibrnet: Learning multi-view image-based rendering. In IEEE Conference on Computer Vision and Pattern Recognition, pages 4690-4699 Computer Vision Foundation IEEE, 2021.
[55] Zhou Wang. Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600-612 612, 2004.
[56] Yi Wei, Shaohui Liu, Yongming Rao, Wang Zhao, Jiwen Lu, and Jie Zhou. Nerfingmvs: Guided optimization of neural radiance fields for indoor multi-view stereo. In 2021 IEEE/CVF International Conference on Computer Vision, pages $590-5599. IEEE, 2021.
[57] Linning Xu, Yuanbo Xiangli, Sida Peng, Xingang Pan, Nanxuan Zhao, Christian Theobalt, Bo Dai, and Dahua Lin. Grid-guided neural radiance fields for large urban scenes. In IEFF/CVF Conference Computer Vision and Pattern Recognition, pages 2296-8306 2023.
13

---

[58]
Zhiwen Yan, Weng Fei Low, Yu Chen, and Gim Hee Lee. Multi-scale 3d gaussian splatting for anti-aliased rendering. CoRR, abs/23 1.17089, 2023.
[59] Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng. and Angjoo Kanazawa. Plenoctrees for real-time rendering of neural radiance fields. In 2021 IEEE/CVF International Conference on
Computer Vision, pages 5732-5741, 2021.
[60]
Zehao Yu, Anpei Chen, Binbin Huang. Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splatting. arXiv:2311, 122311,1649,, 2023.
[61] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features perceptual metric. In 2018 IEEE Conference Computer Vision and Pattern Recognition, pages 586-595, Computer Vision Foundation IEEE Computer Society, 2018
[62] Runze Zhang. Siyu Zhu, Tian Fang, and Long Quan. Distributed very large scale bundle adjustment by global camera consensus. IEEE International Conference on Computer Vision,
pages 29-38, 2017.
[63] Siyu Zhu, Runze Zhang, Lei Zhou, Tianwei Shen, Tian Fang, Ping Tan, and Long Quan. Very large-scale global sfm by distributed motion averaging. In IEEE Conference on Computer Vision and Pattern Recognition, pages
2018.
[64] Zehao Zhu, Zhiwen Fan, Yifan Jiang, and Zhangyang Wang. Fsgs: Real-time few-shot view synthesis using gaussian splatting, 2023.
14

---

A Appendix
A.1 Derivation of Eq. 7
Given Eq. 5 Lx(x, 2,y) Ei-1 (Ji(x)) -x:- - X; and u; y.y.
we have
y. (x: z) + 6 ||x; 211
=yTr, ? |1ri112
f + 23ill2
yill
2
2p
P r + uul12
P u 1
(11)
2
Substitute Eq. 11 into Eq.
5
we have the scaled form ADMM:
Lx( Σ (f((s;) + P [1x; u:l12
- (1.1113)
-
A.2
Proof of Eq.
(8b)
The complete form of Eq. (8b) of
1
u!).
(12)
K
We denote the average of vector with an overline. Then the consensus step for Eq. (bb can be rewritten by:
2 x n².
(13)
Moreover, averaging both sides of Eq. (8 for the dual variables gives:
n' x
2
(14)
Substituting Eq. (13) into Eq. (14) gives that i'+1 O, which shows that the dual variables have
average value zeros after the first iteration. Therefore, proved zf := R Σi=1 (xx+
A.3 Detailed form of Eq.
We provide the detailed form of Eq. (8) in this section. Note that each 3D Gaussian can be represented by X, = {u,, qi S; ), we need to apply different penalty terms to different properties of 3D) GS. Specifically, the dual variables correspond different properties of 3D GS are denoted as:
u, {up uy Us Uy, u.)
(15)
where Ups Ug: Us: Uf; U, are respectively denote the dual variable corresponds to the mean p, the quaternion Ug, the scaling matrix u, the feature vectors to encode color information uy, and the
15

---

opacity u. Therefore, we can expand Eq. 8 as:
(x+) = argmin (f(x+) + PD (p.)) )
2 + (u
+ Pg ((aa))) o + )91
2
2
+ Pe
((($))* - 2 + ( (1((())) 2 1
2
+
Po
|[(ot)²
2 + (u,,))
(16a)
2
k
1 K
1
1
1-1
q
2+1
(s))
7
K
K k=1
K 1=1
k=1
K
K Σ (o))
8
1-1 =
1
(1) 20 =
(16b)
K
K
k=1
1=1
(uf.r)" ( (u (p:) )- 1 - zt (u )(1 (u + (9 z²+1
(u=,)) ) = (u + (s/) - =ztt1, (u),) |41 = (u )² + (1,) z'+1,
(u",,) ( = ( + (ot) 1+1
(16c)
A.4 Algorithm for Recursive Scene Splitting
We present the algorithm for the recursive scene splitting in Alg. !
Algorithm Recursive Scene Splitting Algorithm
Input: 3D points (X.), number of blocks K.
Output: Local 3D points in each block X' (X,,*)
1: Estimate a bounding box aabb which tightly covers all 3D points.
2: Initialize cells C = {aabb), local 3D points X' 0
3: while [C] < K do
4: Let current cells Cour C
5: while |Cour| > 0 do
6: Pop a bounding box in current cells aabb := (our-pop(0)
7:
Remove the bounding box from cells C := C - {aabb}
8:
Split the bounding box into two sub-cells aabbı, aabb; along the longer axis Group points into two blocks (X1, X²}
9:
Re-estimate two tighter bounding boxes aabbi, aabbi, for (X1, X2) Push bounding boxes aabb,, aabb, into cells: = C + faabb,, aabb;}
11:
12:
for block & |C| do
13:
Group points X₂ located the same cell Cs into a same block
14:
Update X' by X' := X' + Xx
At line 5, | denotes the capacity of a set. Algorithm is adopted to split both the training views and sparse point clouds from StM.
A.5 Algorithm for Distributed Training of 3D GS
We present the algorithm for the distributed training of 3D Gaussian Splatting in Alg 2
Note that we adopt Eq. (16) in our implementation.
A.6 More Implementation Details
All experiments are conducted on the Nvidia RTX 6000 GPUs with 48 GB memory. For our method, we initialize the dual variables Up: My Us Uy, u, to zeros. For the penalty parameters, we set Ppr Pq Ps. Po to le4 and Py to 1e3 empirically. Though there are other choices of the initial values
16

---

Algorithm 2 Distributed 3D Gaussian Training Algorithm
Input: Initial 3D Gaussians in each block (Xxl k E [1, K]}. consensus interval intv Output: Global 3D Gaussians z = {ui, 9i 81, fi, 0:)
1: Initialize u, as 0
2: for < T do
3:
for block k <. K distributedly do
4: {u [++ 1++1 1++1 p1++1
1+1 1,6 1.k ,0 == arg min (fi(xi,)) 1x k z' + +
9i,k
5: if t mod intv = 0 then
6:
for block K distributedly do
7:
Send local 3D Gaussians X the master node
8:
Apply the consensus step z'+1 & > (x(x²)
Broadcast the global 3D Gaussians = all slave nodes 10: for block & < K distributedly do
9:
Update the dual variables u = u + x z11
that could improve the results of our method, we found this set of values is good enough for all scenes in our experiments and we did not do more ablations on the initial values. Due computational resources limitation, we test our method on only 5 GPU nodes, where one is the master node that maintains the global model and the others are slave nodes for training local 3D Gaussians. The performance of our method can be improved further with more GPU nodes.
A.7 More Qualitative Results
We present more qualitative results of our method in Fig. and Fif. 10 We provide qualitative comparisons with VastGaussian in areas where blocks overlap in Fig. TO consistent results. However, our method presents higher fidelity rendering results than VastGaussian near the splitting boundary, which also validated the effectiveness and importance the consensus step.
Both methods produce fairly
A.8 Limitations and Future Work
Our method can distributedly train 3D GS on large-scale scenes. However, it brings additional communication overheads to the system. Fortunately, we found the communication overhead did not slow down the training performance. This is due to our balanced splitting algorithm minimizing synchronization time when there a need to consensus the local 3D Gaussians from all blocks. Moreover, we pruned unnecessary small 3D Gaussians to further reduce the number of 3D GS, which also reduced the communication overhead.
Though our method can train large-scale scenes efficiently, the GPU memory requirement can still be an issue, This is because when we zoom out to capture larger areas, more 3D Gaussians included in the rasterization step, which consumes more GPU memory. Our future work will consider introducing the lefel-of-details (LOD) technique into our distributed training pipeline. Similar to existing LOD GS methods [1 44, LOD can be utilized to reduce the number of 3D Gaussians that are far away from the cameras.
A.9 More Discussions
Training and Waiting Time of Each Block. We tested the time cost from transferring the data to the master node to receiving data from the master node for each slave node on the Campus dataset. The mean and variance of time are respectively 5.63 seconds and 0.75 seconds each time. The low variance indicates that our method can balance the training time well. We argue that the data transfer time our method can kept constant since we can always control the number of local Gaussians to constant number (e.g. <= 6000, 000 3D Gaussian primitives) with enough GPUs, matter how large the scene is, since the data transfer between different slave nodes and the master node exccuted distributedly instead of sequentially.
17

---

Ground Truth
Mega-NeRF
Switch-NeRF
3D GS
Ours
Figure 9: More qualitative results on the mill19 dataset and the UrbanScene3D dataset. Zoom in for the best view.
FONE 33 1PPS
PAR JA
PONE TE LIPS
PONK 31 76 LIPSS
SHIM 1PORS
SHIDE BISS
CORSS LPMESIENZET
LIPES UUIEO
SIIMA SSIM-LATT
ESINE 0 TOE
SODE CO20
SODM 1
SCIME USA
SRIN 0373
0MS 0221
SalM: 3.200
Ground Truth
Megn-NeRF
Switcb-NeRF
3D GS
YastGausian
Ours
Figure 10: More qualitative comparisons of method and state-of-the-art methods.
18

---

Figure 11: Qualitative results near scene boundary on the UrbanScene3D dataset. Top-Right: VastGaussian. Bottom-Right: our method.
Implementation Flexibility. One another issue with our method is that the implementation of our method not as flexible as VastGaussian. Our method relies on an RPC module for the data transmission and communication between the master node and the slave nodes. On the other hand, VastGaussian can be implemented decentralized without the master node. However, our method can render higher-quality images and the communication overheads can be neglected compared to the long training time of the original 3D GS. Therefore, this is not a limitation of our work.
Shift Other 3D GS Representations. Many follow-up works improved the original 3D GS. Some works focus on compressing the size of 3D GS [T6] 38 and some of the other works focus on These methods can also be applied to the training of each block in our method. Some other works also adopted intermediate representations to improve the original 3D iS, OctreeGS [1 decoded the properties
constraining the training without changing the optimization parameters [64 23
of 3D GS from the feature embedding in each anchor node, and SAGS 53 adopts hash encodings for each 3D Gaussian, a GNN encoder, and a corresponding decoder to generate the properties of 3D Gaussians. In this case, we can change the optimization parameters and the corresponding penalty parameters and dual variables to these intermediate feature embeddiingseencodnng. In our future work, we will consider more consolidated implementation that can easily shift to these 3D GS representations.
Comparison to Concurrent Works. 1) VastGaussian 26 focuses mostly on the data splitting strategy and guaranteeing the consistency of different 3D Gaussians can also be a challenge in different scenes. However, our method ensured the consistency of the shared 3D Gaussians through the 3D Gaussian consensus with only a quite simple splitting approach. Nonetheless, the data splitting approach in VastGaussian can be introduced into our framework to enhance the robustness. Moreover, the decoupled appearance encoding can also be applied to our method to reduce floaters. Our work and VastGaussian thus complementary to each other. 2) Hierarchy-GS [22) also trains 3DGS distributedly in different grid cells. However, Hierarchy-Gi focuses more on the rendering speed by designing hierarchical tree structure for 3D Gaussians. During training, the generated hierarchy is loaded and optimized by randomly selecting a training view and target granularity. After training all individual chunks, Hierarchy- first runs a coarse initial optimization on the full dataset and adds an auxiliary skybox. This coarse model serves as a minimal basis for providing backdrop details for parts of the scene that are outside of a chunk. When consolidating different chunks, a 3D Gaussian primitive is deleted if it is associated to-but outside-chunk and is closer to another chunk to maintain the consistency 3D Gaussians. The naive pruning approach neglects the gradient flow the 3D Gaussians to other 3D Gaussians that are inside only in an associated chunk. Therefore, its rendering quality can be worse than VastGaussian and our method. Nonetheless, hierarchical LOD approach can still serve good complement to our method and VastGaussian.
Social Impact. DOGS, VastGaussian and Hierarchy-GS focus on different parts of training 3D GS large-scale scenes and therefore are complementary to each other. These methods can be consolidated into a more robust and scalable system to open a new world in city-scale 3D reconstruction. Our method, however, requires an additional master node for controlling the training of the 3DGS, which consumes more GPU resources.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Refer to the abstract and the last paragraph of Sec. [oor our contribution.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to Sec. A.8 in the appendix for the limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
20

---

Justification:
Proofs can be found in Sec. A.1 and Sec. A.2 in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced
statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Implementation details are included in Sec. S the main paper. Additional implementation details are included in Sec. Sec.[A.6]in the supplementary. Pseudo algorithms are included Sec. A.44 and Sec. A.5 in the supplementary for better understanding and reproducing the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
21

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We will release related code and checkpoints upon the acceptance of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //iips.cc/pill wttpcccppppcccceeeeeeeeee..c.ett httaei/ggiieee/eeeeebbissssssss /Cc////oddiiisssseeeeeee ssiiiiiiccyyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: Training and testing details are included in Sec. 4in the main paper. Additional details are included in Sec. in the supplementary.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[No]
Justification:
We follow related works for the setting of error bar.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
22

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute resources and running time are included in Sec. in the main paper and Sec. A.6 the supplementary.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http://wwwwrp.cccoo/./..//ccccca
Answer: [Yes]
Justification:
We strictly follow the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Societal impacts is included in Sec. A.9 in the supplementary.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
23

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper has such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We included the urls and cited the papers for code and datasets used in this paper.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
24

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We use and cite existing datasets in this work. Other assets including related code/models will be released upon acceptance.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
25

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "JjlAaBNhGc", "reviewer": "Reviewer_Y2Rh", "summary": "The paper presents DoGaussian, a novel method for accelerating the training of 3D Gaussian Splatting models for large-scale 3D scene reconstruction. It introduces a distributed training approach using scene decomposition and the ADMM, resulting in a 6x faster training time while maintaining high-quality rendering. The method involves maintaining a global 3DGS model on a master node and local models on slave nodes, with a consensus mechanism ensuring consistency across models. Experiments on standard large-scale datasets confirm the effectiveness of DoGaussian.", "questions": "Can you provide more qualitative results and comparisons?", "limitations": "Yes", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The use of the ADMM ensures that the global and local models reach a consensus on the shared 3D Gaussians. This guarantees convergence and stability during the training phase, which is crucial for maintaining the integrity of the model.\n2. Despite the focus on efficiency, the paper does not compromise on the quality of the output. It demonstrates state-of-the-art rendering quality, which is a testament to the robustness of the proposed method.\n3. By introducing a distributed training methodology for 3DGS, the paper significantly accelerates the training process on large-scale scenes. This is achieved by decomposing scenes into manageable blocks, allowing for parallel processing on multiple nodes.", "weaknesses": "1. The qualitative comparisons are not that convincing, especially compared with the original 3DGS and vastGaussian. The authors are encouraged to provide more results in the rebuttal. I will raise the score accordingly.\n2. The training and inference speed of DoGaussian is slightly slower than vastGaussian.\n3. More baselines should be included, such as [Fed3DGS](https://arxiv.org/pdf/2403.11460).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- ***Q1: Provide more qualitative results in the rebuttal (compared with the original 3DGS and VastGaussian).***\n- **A1:** We include more qualitative results in Fig.1 in the attached PDF.\n\n- ***Q2: More baselines should be included, such as Fed3DGS.***\n- **A2:** We include the results of Fed3DGS in our updated Table. We can observe that **the rendering quality of Fed3DGS is far below 3D-GS-based methods**. Note that though Fed3DGS also maintains a global model in a central server, it requires optimizing the opacities and appearance of the global model with all local models, which is computationally inefficient. Moreover, not like our method, which adopts ADMM to ensure the training convergence, the global model optimization in Fed3DGS is only used to prune Gaussians to prevent the monotonically increased number of global Gaussians, which is intuitively designed without theoretical convergence guarantee.\n\n| **Scenes** | **Building** | **Building**  | **Building**  | **Rubble** | **Rubble** | **Rubble** | **Campus** | **Campus** | **Campus** | **Residence** | **Residence** | **Residence** | **Sci-Art** | **Sci-Art** | **Sci-Art** |\n|------------|-------------------------------|------------------------------|---------------------------------|----------------------------|----------------------------|-------------------------------|----------------------------|----------------------------|--------------------------------|-------------------------------|-------------------------------|--------------------------------|-----------------------------|-----------------------------|--------------------------------|\n|  | PSNR$\\uparrow$ | SSIM $\\uparrow$ | LPIPS $\\downarrow$ | PSNR $\\uparrow$ | SSIM $\\uparrow$ | LPIPS $\\downarrow$ | PSNR $\\uparrow$ | SSIM $\\uparrow$ | LPIPS $\\downarrow$ | PSNR $\\uparrow$ | SSIM $\\uparrow$ | LPIPS $\\downarrow$ | PSNR $\\uparrow$ | SSIM $\\uparrow$ | LPIPS $\\downarrow$ |\n| Mega-NeRF  | 20.92                         | 0.547                        | 0.454                           | 24.06                      | 0.553                      | 0.508                         | 23.42                      | 0.537                      | 0.636                          |  22.08 | 0.628                         | 0.401                          | 25.60 | 0.770                         | 0.312                          |\n| Switch-NeRF| 21.54                         | 0.579                        | 0.397                           | 24.31                      | 0.562                      | 0.478                         | 23.62                      | 0.541                      | 0.616                          | 22.57 | 0.654                         | 0.352                          | 26.51 | 0.795 | 0.271                          |\n| $\\text{3D-GS}$ | 22.53 | 0.738 | 0.214 | 25.51 | 0.725 | 0.316 | 23.67 | 0.688 | 0.347 | 22.36 | 0.745 | 0.247 | 24.13 |0.791 | 0.262 |\n| $\\text{Fed3DGS}$ | **18.66** | **0.602** | **0.362** | **20.62** | **0.588** | **0.437** | **21.64** | **0.635** | **0.436** | **20.00** | **0.665** | **0.344** | **21.03** | **0.730** | **0.335** |\n| $\\text{VastGaussian}^{\\dagger}$ | 21.80 | 0.728 | 0.225 | 25.20 | 0.742 | 0.264 | 23.82 | 0.695 | 0.329 | 21.01 | 0.699 | 0.261 | 22.64 | 0.761 | 0.261 |\n| Hierarchy-GS | 21.52 | 0.723 | 0.297 | 24.64 | 0.755 | 0.284 | -- | -- | -- | -- | -- | -- | -- | -- | -- |\n| DoGaussian | 22.73 | 0.759 | 0.204 | 25.78 | 0.765 | 0.257 | 24.01 | 0.681 | 0.377 | 21.94 | 0.740 | 0.244 | 24.42 | 0.804 | 0.219 |\n\n- ***Q3: More qualitative results***\n- **A3:** See the attached file for more qualitative results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hhmeQueb4v", "reviewer": "Reviewer_zf9w", "summary": "This paper introduces traditional ADMM to Gaussian Spaltting and achieves distributed Gaussian Splatting training. The proposed distributed approach reduces training time and guarantes training convergence and stability. Experiments demonstrates both effectivenss and efficiency of this method.", "questions": "NA", "limitations": "NA", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces powerful ADMM to large-scale 3D reconstruction, which is an effective and elegant approach to distributed optimization.\n2. The compact optimization scheme help the method achieves high quality reconstruction.\n3. The idea of the proposed distributed optimization could inspire research in other tasks, such as full-body reconstruction.\n4. The paper is well written.", "weaknesses": "1. More experiments are needed to prove the superiority of the consensus step. I hope to see more qualitative and quantitative comparisons with VastGaussian in areas where blocks overlap.\n2. The authors claim that proposed split method can balance the training time of each block. However, there is a lack of experiments showing the training and waiting time of each block.\n3. The authors enable consensus and sharing when every block reaches 100 iterations. Is it possible to allow the blocks to be at different iterations? It can be more efficient to avoid blocking. However, there is no discussion about the difference between synchronous and asynchronous methods.\n4. There is no discussion about the memory consumption of master node. How much GPU memory is needed for the master node? Will its memory consumption be linearly increase as the total number of Gaussians increases?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **Q1: More experiments are needed to prove the superiority of the consensus step. More qualitative and quantitative comparisons with VastGaussian in areas where blocks overlap.**\n- **A1:** We include more qualitative results in Fig.2 in the attached PDF to show the importance of the consensus step. We can observe that the distributed training presents noisy results without the consensus step. From the two bottom-right figures, we can observe obvious artifacts along the block boundary without the consensus step. \nAs is shown in Fig.4 of the attached file, both methods produce fairly consistent results. However, our method presents higher fidelity rendering results than VastGaussian near the splitting boundary, which also validated the effectiveness and importance of the consensus step.\n\n- ***Q2: The authors claim that proposed split method can balance the training time of each block. However, there is a lack of experiments showing the training and waiting time of each block.***\n- **A2:** We tested the time cost from transferring the data to the master node to receiving data from the master node for each slave node on the Campus dataset. The mean and variance of time are respectively $5.63$ seconds and $0.75$ seconds each time. The low variance shows that our method can balance the training time very well.\n\n- ***Q3: The authors enable consensus and sharing when every block reaches 100 iterations. Is it possible to allow the blocks to be at different iterations? It can be more efficient to avoid blocking. However, there is no discussion about the difference between synchronous and asynchronous methods.***\n- **A3:** The consensus and sharing steps can be done asynchronously in implementation. However, by doing so, the convergence of ADMM is not guaranteed and the results can be sub-optimal. To validate this, (1) we first transfer the local Gaussians to the master node and then optimize them without blocking; (2) we trigger the consensus and sharing step once the master node receives local Gaussians from all blocks; (3) the local Guassians are regularized with the newly updated global Gaussians once the slave nodes received the data. As a result, the PSNR, SSIM, LPIPS on the building scene are respectively $18.73, 0.441, 0.673$, which dropped significantly.\n    We argue that **the data transfer time of our method can be kept constant** since we can always control the number of local Gaussians to a constant number (e.g. $<= 6000,000$ 3D Gaussian primitives) with enough GPUs, no matter how large the scene is, since the data transfer between different slave nodes and the master node is executed distributedly instead of sequentially.\n\n- ***Q4: memory consumption of master node***\n- **A4:** See **A1** in the common questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2UDI7kg800", "reviewer": "Reviewer_tFmz", "summary": "This paper proposes a distributed training strategy for 3dgs in large-scale scenes. The scene is evenly splitted into K blocks, but also maintain the global scene representation. Then the optimization of the scene is transferred to a condition optimization solved by the classic   ADMM. The results demonstrate their effectiveness and efficiency.", "questions": "1. How to load the global gaussians? Does the method work when a single GPU can not load the global scene (main node)?\n\n2. As the different splitting could lead to different overlapping regions and different ADMM problems, does it matter?", "limitations": "As discussed in the Question 1.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It is interesting to convert the distributed training of 3dgs into a condition optimization problem. It could boost the rendering in overlapping region significantly.\n\n2. The experiments are sufficient.", "weaknesses": "1. The scene splitting is somehow tailored for bird view images. As for the extending scenes with surround views, such as autonomous driving, a training image in one block could see a large ratio of point cloud in other blocks.\n\n2. It would be better to modify some representations in the paper. \n\na. line 82, cone sampling of mip-nerf is not designed for the representation enhancement for outdoor scenes.\n\nb. line 98, \"leveraging xxx performance for xxx reconstruction\" seems confusing.\n\nc. line 4 of the fig.2, there are a redundant 'the' in sentence \"a copy the the global 3D Gaussians\".\n\n3. It would be better to highlight the best performance in Table 2, 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- ***Q1: applicability on autonomous driving scenes***\n- **A1:** See **A4** in the common questions.\n\n- ***Q2: It would be better to modify some representations in the paper***\n- **A2:** Thanks for the suggestions. We revised the paper accordingly as suggested:\n***(a)*** line 82, we revised it to \"To address the aliasing issue in vanilla NeRF, Mip-NeRF[1] proposed to use Gaussian to approximate the cone sampling, ...\".\n***(b)*** line 98, we revised it as \"or leveraging the efficient rasterizer of point rendering for real-time indoor reconstruction\".\n***(c)*** line 4 of the fig.2, we revised it as \"a copy of the global 3D Gaussians\".\n\n- ***Q3: It would be better to highlight the best performance in Table 2, 4.***\n- **A3**: Thanks for the suggestion. We will highlight the best results in Tables 2 and 4 in our revised paper as is done in Table 1, 3.\n\n- ***Q4: How to load the global gaussians? Does the method work when a single GPU can not load the global scene (main node)?***\n- **A4:** See **A2** in the common questions.\n\n- ***Q5: As the different splitting could lead to different overlapping regions and different ADMM problems, does it matter?***\n- **A5:** It is possible that better splitting methods can lead to better results and boost the convergence of the consensus ADMM problems. From Eq.8(b) of the main paper, the global Gaussians are updated by averaging the values from local Gaussians. Though local Gaussians converge towards the global Gaussians during training, a high variance of the shared local Gaussians (it occurs when Gaussians are not well constrained) can decrease the convergence of ADMM. Our method does not suffer from this issue since we adopt a confidential scale factor to ensure the scene boundaries have enough overlapping areas.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p3am2A9uya", "reviewer": "Reviewer_eg7b", "summary": "DoGaussian incorporates the 'divide-and-conquer' approach and introduces the ADMM algorithm into the 3DGS training process for large-scale 3D reconstruction tasks, reducing training time by 6+ times compared to the original 3DGS. Specifically, DoGaussian first splits the scenes into K local blocks of similar sizes and then maintains a global 3DGS node to ensure consistency through consensus on the K shared blocks. During the inference stage, DoGaussian moves all shared blocks and queries only the global block to improve rendering efficiency.", "questions": "There are three questions regarding the weaknesses mentioned above:\n\n1. The authors need to explain the artifacts in the teaser.\n2. It would be better to conduct experiments on street datasets and larger datasets.\n3. I’m also curious to know if the final results would be better using the visibility-aware splitting strategy of VastGaussian.\n\nI would be happy to raise my score if the authors can address all my questions.", "limitations": "The authors discuss the limitation in the appendix A.8.", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. DoGaussian proposes a novel distributed training strategy, achieving a significant acceleration of the 3DGS training process without sacrificing rendering visual quality.\n2. DoGaussian functions more like a plugin and can be applied to any GS representation work.\n3. The paper is well-written and easy to follow, and the supplementary video is exceptionally well made.", "weaknesses": "1. **Artifacts in Teaser.** There are noticeable artifacts in the teaser, particularly in the picture located in the bottom right corner. It would be beneficial for the authors to provide an explanation of these artifacts and address them accordingly.\n2. **Suitable for Street Scenes.** In the experiments section, all experiments are conducted on aerial datasets, but reconstructing street data is also an important problem. Can the authors demonstrate DoGaussian's applicability using the San Francisco Mission Bay dataset from Block-NeRF[1] or Block_A from the MatrixCity[2] dataset?\n3. **GPU Memory Problem.** DoGaussian appears to be a system paper that utilizes a distributed approach to improve training efficiency by placing the global 3DGS on a main node and some local 3DGS on other nodes. One potential reason other works do not use this strategy could be GPU memory limitations. All datasets used by the authors are processed by 3DGS on a single GPU, which does not truly qualify as 'large-scale.' It would be beneficial for the authors to demonstrate that DoGaussian can be applied to larger datasets, such as the aerial and street data of big city in MatrixCity, which is used in NeRF-XL[3].\n\n[1] Block-nerf: Scalable large scene neural view synthesis.\n\n[2] Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond\n\n[3] NeRF-XL: Scaling NeRFs with Multiple GPUs", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- ***Q1: Noticeable artifacts in the teaser, particularly in the picture located in the bottom right corner.***\n- **A1:** From Fig.3 in the attached PDF, we can observe that the artifacts in the teaser appear to be near the scene boundary, which is a common problem for 3DGS-based methods and not a particular issue caused by our method.\n\n- ***Q2: Suitable for Street Scenes.***\n- **A2:**: See **A4** in the common questions.\n\n- ***Q3: GPU Memory Problem***\n- **A3:** See **A1**, **A2** and **A3** in the common questions.\n\n- ***Q4: It would be better to conduct experiments on street datasets and larger datasets, such as the aerial and street data of Big City in MatrixCity, which is used in NeRF-XL.***\n- **A4:** Since we have only a limited number of GPUs with 48GB memory for each, we did not test our method on the Big City scene in MatrixCity. However, to further show the applicability of our method to larger-scale scenes, we evaluated our method on the $2.7 \\text{km}^2$ Small City scene in the MatrixCity dataset, which contains $5,620$ training views and $741$ validation views. We early terminated the training of the original 3D-GS since it did not finish the training within two days. From the table below, our method achieved the best results in rendering quality. The visual qualitative results are shown in Fig.5 in the attached PDF. \n\n*Table. Quantitative results of our method on the Small City aerial scene of the MatrixCity dataset.*\n|            | PSNR $\\uparrow$ | SSIM $\\uparrow$ | LPIPS $\\downarrow$ | Time $\\downarrow$ | Points | Mem $\\downarrow$ | FPS $\\uparrow$ |\n|------------|-----------------|--------------|-----------------|--------------|--------|--------------|-------------|\n| 3D-GS      | 27.36 | 0.818 | 0.237 | 47:40 | 11.9 | 6.31 | 45.57 |\n| VastGaussian$^{\\dagger}$ | 28.33 | 0.835 | 0.220 | 05:53 | 12.5 | 6.99 | 40.04 |\n| DoGaussian | **28.58** | **0.847** | **0.219** | 06:34 | 10.3 | 5.82 | 48.34 |\n\n- ***Q5: If the final results would be better using the visibility-aware splitting strategy of VastGaussian.***\n- **A5:** The visibility-aware splitting strategy of VastGaussian is utilized to add more redundant training views to each block to ensure well-constrained scene boundaries. However, this strategy also imbalances the size of each block. In our experiments, we find this strategy improves little of our method in the rendered image quality but increases the training time. On the Campus scene, when adopting the visibility-aware strategy in our method, the PSNR is increased by 0.09db, but the training time is increased by 25 minutes. While visibility-awareness is a good strategy for improving the rendering quality of the model, some post-processing steps may be required to ensure balanced block-splitting results, which can be a future work of our method.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all reviewers for their detailed comments and suggestions. Some **common questions are answered below**, and **more qualitative results** are provided in the attached PDF.
- ***Q1 How much GPU memory is needed for the master node? Will its memory consumption be linearly increase as the total number of Gaussians increases?***
- **A1**: In our experiments, the peak GPU memory of the master node is below 32 GB for all the datasets. The memory consumption increases linearly with the total number of Gaussians. However, we emphasize that **the global Guassians on the master node do not produce gradients** because we just use the value of local Gaussians in the consensus step to update the global Gaussians (we do not need the backpropagation for the optimizer to update the global Gaussians).
In implementation, we update the global Gaussians with torch's no gradient mode. We do not need to maintain a computational graph as we train the local Gaussians. Therefore, the memory consumption of maintaining the global Gaussians on the master node is much less than training the original 3DGS that has the same number of Gaussians on a single GPU.
    
- ***Q2: How to load the global gaussian? Does the method work when a single GPU cannot load the global scene (main node)?***
- **A2:** During training, the global Gaussians are maintained on the GPU of the master node. During consensus and sharing, local Gaussians can be mapped to their corresponding global Gaussians by indexing. During inference, the global Gaussians are used normally just as the original 3D-GS. We emphasize that **our method can still be applied to such scenes when a single GPU cannot load the global scene**: ***(1)*** As is explained in ***A1***, the memory requirement of the master node is not as large as training the same number of global Gaussians on a GPU. ***(2)*** In extremely large scenes where a single GPU cannot hold all global Gaussians (this is also an issue for all other large-scale 3DGS methods. For example, VastGaussian needs the final fused model to do inference), we can cache the global Gaussians on RAM and transfer them to GPU memory in a block-wise manner whenever the consensus and sharing step are triggered.
***In brief***, this is an implementation philosophy that can be solved as explained above and we will leave this as our future work.
    
- ***Q3: One potential reason other works do not use this strategy could be GPU memory limitations. All datasets used by the authors are processed by 3DGS on a single GPU, which does not truly qualify as 'large-scale.'***
- **A3:** The Mill19 and UrbanScene3D datasets are widely adopted in evaluating large-scale NeRF/3D-GS methods, and we follow previous methods (Mega-NeRF, Switch-NeRF, VastGaussian) to evaluate our method on these most commonly evaluated large-scale scenes. We emphasize that how we qualify a scene as `large-scale` is not simply by whether it can be trained on a single GPU (***e.g.*** if we have an 80GB GPU, we can train 3DGS on any existing datasets), but also on how long it takes to train a NeRF/3D-GS model on such scenes. We can train any scene on a single GPU by controlling the number of model parameters, but it can take a very long time for the model to converge to render high-fidelity images.

- ***Q4: In the experiments section, all experiments are conducted on aerial datasets, but reconstructing street data is also an important problem. Can the authors demonstrate DoGaussian's applicability using the San Francisco Mission Bay dataset from Block-NeRF or BlockA from the MatrixCity dataset?***
- **A4:** See the table below for the results on the BlockA scene of the MatrixCity dataset. VastGaussian failed on this dataset since two blocks produce no 3D Gaussian primitives due to the imbalanced splitting. Neither 3D-GS nor our method produces satisfactory results with the default parameters used to train 3D-GS, this may be because the BlockA scene contains too many narrow views and needs careful fine-tuning of the training parameters. However, our method still produces better results than the original 3D-GS. An exhaustive evaluation of our method on street view scenes can be a future work.

|          | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | Time $\downarrow$ | Points | Mem $\downarrow$ | FPS $\uparrow$ |
|------------|-----------------|--------------|-----------------|--------------|--------|--------------|-------------|
| 3D-GS      | 20.03 | 0.643 | 0.650 | 14:24 | 1.85 | 2.33 | 193.32 |
| VastGaussian$^{\dagger}$ | - | - | - | - | - | - | - |
| DoGaussian | 21.61 | 0.652 | 0.649 | 02:33 | 2.37 | 2.89 | 180.51 |

Meta-review: The paper received good scores, and the authors-reviewers engaged in a productive discussion during the rebuttal phase. The consensus is to accept the paper. 

The paper presents an ADMM-based solution to split any large scene reconstruction into multiple small/local GS reconstruction tasks while maintaining consensus using a global GS solution using ADMM for synchronization. All the reviewers liked the approach and appreciated the 6x improvement in computation. Questions around the role of the global node and comparisons with alternatives (VastGS, etc.) were resolved in the rebuttal phase. Authors are encouraged to add additional clarifications while preparing the final version of the submission.
Decision: Accept (poster)