On Scaling Up 3D Gaussian Splatting Training
Authors: Hexu Zhao
Haoyang Weng
Daohan Lu
Ang Li
Jinyang Li
Aurojit Panda
Saining Xie
Venue: ICLR 2025 Oral (2025)
Keywords: Gaussian Splatting
Machine Learning System
Distributed Training
Primary area: infrastructure, software libraries, hardware, systems, etc.
TL;DR: We describe a scalable distributed training system for 3D Guassian Splatting.

Abstract: 3D Gaussian Splatting (3DGS) is increasingly popular for 3D reconstruction due to its superior visual quality and rendering speed. However, 3DGS training currently occurs on a single GPU, limiting its ability to handle high-resolution and large-scale 3D reconstruction tasks due to memory constraints. We introduce Grendel, a distributed system designed to partition 3DGS parameters and parallelize computation across multiple GPUs. As each Gaussian affects a small, dynamic subset of rendered pixels, Grendel employs sparse all-to-all communication to transfer the necessary Gaussians to pixel partitions and performs dynamic load balancing. Unlike existing 3DGS systems that train using one camera view image at a time, Grendel supports batched training with multiple views. We explore various optimization hyperparameter scaling strategies and find that a simple sqrt(batch-size) scaling rule is highly effective. Evaluations using large-scale, high-resolution scenes show that Grendel enhances rendering quality by scaling up 3DGS parameters across multiple GPUs. On the 4K ``Rubble'' dataset, we achieve a test PSNR of 27.28 by distributing 40.4 million Gaussians across 16 GPU, compared to a PSNR of 26.28 using 11.2 million Gaussians on a single GPU. Grendel is an open-source project available at: https://github.com/nyu-systems/Grendel-GS

Published as a conference paper at ICLR 2025
ON SCALING UP 3D GAUSSIAN SPLATTING TRAINING
Hexu Zhao', Haoyang Weng'; Daohan Lu"; Li?, Jinyang Li', Aurojjt Panda', Saining Xie NNww York University
Paciffc Northwest National Laboratory
ABSTRACT
3D Gaussian Splatting (3DGS) increasingly popular for 3D reconstruction due to its superior visual quality and rendering speed. However, 3DGS training currently occurs on single GPU, limiting its ability to handle high-resolution and large-scale 3D reconstruction tasks due to memory constraints. We introduce Grendel, distributed system designed to partition 3DGS parameters and parallelize computation across multiple GPUs. As each Gaussian affects a small, dynamic subset of rendered pixels, Grendel employs sparse all-to-all communication to transfer the necessary Gaussians pixel partitions and performs dynamic load balancing. Unlike existing 3DGS systems that train using one camera view image time, Grendel supports batched training with multiple views. We explore various optimization hyperparameter scaling strategies and find that a simple sqri(batch__ee _size) scaling rule is highly effective. Evaluations using large-scale, high-resolution scenes show that Grendel enhances rendering quality by scaling up 3DGS parameters across multiple GPUs. On the 4K "Rubble" dataset, we achieve test PSNR of 27.28 by distributing 40.4 million Gaussians across 16 GPUs, compared to a PSNR of 26.28 using 11.2 million Gaussians on single GPU. Grendel is open-source project available at: https: / gi thub. com/ nyu systems/Gres Peesteml/Grenel--ie
Dataset: Rubble
Dataset: MatrixCity
Resolution:
Resolution: 1886P
aGausstans: 48,008, eee
#caussians: 28,080, 880
Training: 16 GPUs/BS-16
Training: 16 GPUs/BS-16
Figure 1: Two large-scale, high-resolution scene reconstructions using Grendel, our distributed 3D Gaussian rendering system. Both images are rendered using 16 GPUs. The left and right images are represented using 40 million and 24 million gaussians respectively. Grendel achieves state of the art quality (PSNR) for both scenes.
INTRODUCTION
3D Gaussian Splatting (Kerbl et al. 2023 (3DGS) has emerged as a popular technique for 3D novel view synthesis, primarily due to its faster training and rendering compared to previous approaches such as NeRF (Mildenhall et al. 2020). However, most existing 3DGS pipelines are constrained to using single GPU for training, creating memory and computation bottlenecks when applied
Daohan and Haoyang contributed equally this work.

---

Published as a conference paper at ICLR 2025
to high-resolution larger-scale scenes. For example, the standard Rubble dataset Turki et al.) 2022) contains 1657 images, each with a 4K resolution. A single A100 40GB GPU can hold up to I1.2 million Gaussians well below the quality saturation point for 3DGS. As demonstrate in Section 5.2.1 increasing the number of Gaussians continues to improve reconstruction quality. Therefore, in order scale up 3DGS training in terms of parameter count and speed, we develop the Grendel system, which distributes 3DGS training across multiple GPUs and uses an empirical rule to automatically adapt training hyperparameters based on the batch size.
Existing work on scaling 3DGS to large-scale scenes have explored approaches such as dividc-and- conquer Lin al.] (2024); Liu et al. (2024); Kerbl et al. (2024). level-of-detall representation and rendering (Kerbl et al.) 2024 Ren et al. 2024|] Lu et al., 2024). These methods are complementary to the system-level parallelization approach that we adopt to scale 3DGS beyond the memory and
compute limitations of a single GPU.
As distributed training frameworks have become widely used for many state-of-the-art DNN models such as LLMs (Shoeybi et al.) 2020 Rajbhandari et al. 2020), it is tempting to also use them to distribute 3DGS. However, although 3DGS uses gradient-based optimization, it is not based on neural networks. Specifically, it features unique computation pipeline with dynamic and imbalanced workload patterns. Consequently, existing DNN training frameworks (Shoeybi et al., 2020; Rajbhandari al. 2020; et et al., 2020 ZZhao et al., 2023), which assume consistent and balanced workload with regular dense tensor operations, would not work well for 3DGS.
In this paper, we present several key observations on scaling up 3DGS that inform the design of our distributed training pipeline. For instance, we note that each stage of the 3DGS training pipeline in an iteration can be effectively parallelized, but the axes of parallelization differ across stages, resulting in mixed parallelism. More concretely, in 3DGS, some computations operate on individual output pixels (allowing for pixel-wise parallelism), while others operate on individual 3D Gaussians (allowing for Gaussian-wise parallelism). Mixed parallelism necessitates data shuffling between stages. To minimize communication, we also observe that 3DGS exhibits spatial locality, where only a small number of Gaussians affect the rendering of each output image patch. Finally, the computational intensity of rendering an output pixel changes as training progresses. Such dynamic and unbalanced workloads will cause any static workload partitioning strategy to become suboptimal.
In this paper, we describe Grendel, a distributed 3DGS training framework designed to leverage distribution-thar is, it distributes Gaussians
our above observations. Grendel uses Gaussian across GPUs-for steps in a training iteration that exhibit Gaussian-wise parallelism, and pixel-wise distribution for other steps. It minimizes the communication overhead when switching between Gaussian-win and pixel-wise distribution by assigning contiguous image areas to GPUs during pixel-wise distribution and exploiting spatial locality to minimize the number of Gaussians transferred among GPUs. Finally, Grendel employs a dynamic load balancer that uses previous training iterations to distribute pixel-wise computations to minimize workload imbalance.
Grendel additionally scales up training by batching multiple images. This differs from conventional 3DGS training that exclusively uses a batch size of 1, which would lead to reduced GPU utilization in our distributed framework. To maintain data efficiency and reconstruction quality with larger batches, one needs to re-tune optimizer hyperparameters. To this end, we introduce an automatic hyperparameter scaling rule for batched 3DGS training based on heuristical independent gradients hypothesis. We empirically validate the effectiveness of our proposed approach - Grendel supports distributed training with large batch sizes (we test up to 32) while maintaining reconstruction quality and data efficiency compared to batch size- = 1.
In summary, work makes the following contributions:
We describe the design and implementation of Grendel, a scalable, memory-efficient, adaptive distributed training system for 3DGS. Grendel allows batched 3DGS training be scaled up and run on up to 32 GPUs.
We explore the large-batch training dynamics of 3DGS to identify simple sqri(batch_siaze learning rate scaling strategy, which enables efficient, hyperparameter-tune free training for batch sizes beyond one.
We show that Grendel enables high-resolution large scale scene rendering: we use 16 GPUs and render 4K images for large-scale Rubble scene from MegaNERR (Turki et al. 2022). For this scene, Grendel uses 40.4 million Gaussians to achieve PSNR of 27.28. outperforming the current oate-of-theeart The memory required exceeds a single GPU's capacity, making it difficult to render this scene at this quality without Grendel's techniques.
2

---

Published as a conference paper at ICLR 2025
Particon
Aender
Portttoon spussara
GPU-0
allZall
GPU-1
GPU11
Single GRU
GPJ-
GPU-1
Batch Images
(a)
(b)
Figure 2: (a) Traditional 3DGS training pipeline using a single GPU vs. (b) Our Grendel system that distributes 3D Gaussians across multiple GPUs alleviate the GPU memory bottleneck. We also partition the computation in the pixel and batch dimensions to for further speedup. Every square represents a 16 x 16 block of pixels.
GAUSSIAN SPLATTING: BACKGROUND, OPPORTUNITIES AND CHALLENGES
3D Gaussian Splatting (Kerbl et al. 2023) (3DGS) is rendering method that represents 3D scenes using a (potentially large) set of anistropic 3D Gaussians. Each 3D Gaussian is represented by four learnable parameters: (a) its 3D position 2; E R3; (b) its shape described by a 3D covariance matrix computed using the Guassian's 's scaling vector s; E R and rotation vector 4 R$; (c) its opacity 0; E IR; and (d) its spherical harmonics shy E IR448 The color contribution of each Gaussian is detemined by these parameters and by the viewing-direction
2.1 BACKGROUND ON 3D GAUSSIAN TRAINING
To train 3DGS, the user provides an initial point cloud (may be random or estimated) for scene and a set of posed images from different angles. The training process initializes Gaussians using the point cloud. Each training step selects random camera view and uses the current Gaussian parameters to render the view. It then computes loss by comparing the rendered image to the ground truth, and uses back- propppppppgttion to update the Gaussian parameters. The training process also uses an adaptive densification mechanism to add Gaussians to under-reconstruuted areas, by cloning or splitting existing ones based on their position variance and scale threshold, with more details in A.
Concretely, the training pipeline consists of four steps: Gaussian transformation, image rendering, loss calculation, and backpropagation. Standard approaches to backpropagation are used in this setting, and we detail the remaining three steps below:
Gaussian transformation: Given a camera view and the associated screen space, each Gaussian is transformed and projected determine its position Ze IR² on screen, its distance depthe,s E IR. from the screen, and its coverage (or footprint radius) radius,,; E TR. Additionally, the color of each Gaussian C,, is determined according to the viewing direction using its learnable spherical harmonics coefficients sh; R848
2. Rendering: After Gaussian transformation, the image is rendered by computing each pixel's color. To do so, for a given pixel p. 3DGS first finds all Gaussians that intersect with p. We say that a Gaussian i intersects with p if p lies within radiuse,, of the Gaussian i's projected center T.,i. Then 3DGS iterates over intersecting Gaussians in increasing depth (i.c. in increasing depth..:) and uses alpha-composition to combine their contributions until threshold opacity has been reached.
3. Loss calculation: Finally, the 3DGS computes the L1 and SSIM loss by comparing the rendered image to the ground truth image. The LI loss measures absolute difference between pixel colors, while the SSIM loss measures the similarity between pixel windows. Both metrics are computed per-pixel for both forward and backward implementations.
2.2 OPPORTUNITIES AND CHALLENGES DISTRIBUTING 3DGS
In designing Grendel for scaling up 3D Gaussian Splatting training, exploit the following opportu- nities the above-described training process and address several challenges:
Opportunity: mixed parallelism. Each of the steps described above is inherently parallel but requires different kinds of work partitioning. particular, the Gaussian transformation step operates on individual Gaussians and thus should be partitioned by Gaussians. On the other hand, the rendering and loss calculation steps operate on individual pixels(or pixels windows for SSIM loss) and thus should be partitioned by pixel.
3

---

Published as a conference paper at ICLR 2025
Opportunity: spatial locality. Most Gaussians intersect small contiguous area of the rendered image due to their typically small radius. As illustrated in Figure(3) 90% of the 3D Gaussians in three scenes (Rubble. Bicycle, and Train) have radius < 2% of image width. Consequently, a pixel is affected by a small subset of the scene's 3D Gaussians, with significant overlap among neighboring pixels' Gaussians.
Challenge: dynamic and unbalanced workloads. Different image areas intersect varying quantities of Gaussians, as shown in Fig 4 For instance, an image region containing the sky likely corresponds to fewer Gaussians than a region with a person. Additionally, the density, position, shape, and opacity of Gaussians change throughout training. Therefore, the number of Gaussians and their mapping to pixels evolve over time, leading to computational workload imbal- ances across different image regions and over the training period. Fixed partitioning schemes thus suffer from load imbalance. Refer to the Appendix $A.4foo for further details.
Carrudativ Porcert Curve Racius Haptilde
Go hocarrss Racios
Figure 3: Cumulative percents of Radius; relative to image width
Challenge: absence of batching. Current 3DGS systems process images one at a time, which suffices for single GPU training. However, as shown in $ this approach is inefficient in distributed setting with multiple GPUs. Effective training with larger batch sizes necessitates an understanding of the unique optimization dynamics of 3DGS, which may differ from those of conventional neural networks.
3 SYSTEM DESIGN
Here, we describe how Grendel exploits the mixed parallelism and spatial locality of 3DGS ($3.1) to address the challenge of dynamic and unbalanced workloads ($3.2
3.1 MIXED PARALLELISM TRAINING
Figure b)) provides an overview of Grendel's design. Grendel distributes work according to 3DGS' mixed parallelism: it uses Gaussian-wise distribution-wherr each GPU operates on disjoint subset of Gaussians- for the Gaussian transformation step, and pixel-wise Distribution-wher each GPU operates on a disjoint subset of pixels- for the image rendering and loss computation step. The sputial locality characteristic allows Grendel to benefit from sparse all-to-all communication when transitioning between these stages.
Gaussian-wise Distribution. Grendel partitions the Gaussians, including their parameters and optimizer states, and distributes them uniformly across GPUs. Then, each GPU independently computes the Gaussian transformation for the set of 3D Gaussians assigned to it. We found that the amount of computation required does not significantly vary across Guassians, and thus evenly distributing Gaussians across GPUs allows to fit the maximal number of Gaussians while speeding up computation linearly for this step.
Pixel-wise Distribution. We distribute contiguous image areas across GPUs for the image rendering and loss computation steps. Distributing contiguous areas allows us to exploit spatial locality and reduce the number of Gaussians transferred among GPUs. In our implementation, we partition each image in a batch by dividing it into 16 x 16-pixel blocks, serializing the blocks, and then distributing consecutive subsequences of blocks different GPUs using an adaptive strategy ($3.2 For batching, each GPU can be assigned blocks from different images in batch, as shown in Figure 2)b). Transferring Gaussians with sparse all-to-all communication. To render an image pixel, a GPU needs access to Gaussians that intersect the pixel, which cannot be pre-determined as they are view- dependent and change during training. Therefore, Grendel includes a communication step after the Gaussian transformation. As 3DGS exhibits spatial locality, each pixel partition only requires a small subset of all 3D Gaussians. We leverage this to reduce communication: each GPU first decides the set of intersecting Gaussians for rendering a pixel partition (Figure 5) before using a sparse all all-to-all communication to retrieve Gaussians intersecting with any pixels in the partition. A reversed all-to-all communication is done during the backward pass.
Although Grendel's 's design bears some resemblance to FSDP (Zhao et al. 2023 used for distributed neural network training, there are important differences. Firstly, unlike weight sharding in FSDP, Gaussian-wis distribution in Grendel is not merely for storage but for also for computation (the Gaus- sian transformation). Secondly, unlike FSDP which transfers weight shards using the dense all-gather communication, Grendel transfers only relevant Guassians using sparse all-to-all communication.
4

---

Published as a conference paper at ICLR 2025
Intersection Gaussions
1000
930
6000
Splatted
Non-Intersecting Gousslans (Ignored)
Figure 4: We present a heatmap of the per-tile imbalance in the number of rendered Gaussians the Train Dataset (Knapitsch et al., (017). considers Gaussians whose Redder tiles indicate more Gaussian splats. Distant, low-detail areas footprints intersect with its like the sky need fewer Gaussians than detailed foreground regions like assigned pixel render area. the train, highlighting an imbalance in rendering intensity.
Figure 5: Each GPU only
3.2 ITERATIVE WORKLOAD REBALANCING
Pixel-wise Distribution Rebalancing. As discussed in $22 the computational load of rendering pixel varies across space (different pixels) and time (different training iterations). Thus, unlike in distributed neural network training, a uniform or fixed distribution cannot guarantee balanced workloads,
adaptive pixel distribution strategy is needed.
We record the rendering time of each pixel of each training image during every epoch after the first few. Since the scene generally changes smoothly between consecutive epochs during training, the rendering time of each pixel also changes slowly. Therefore, the rendering times from previous epochs form a good estimate adaptively assign pixels to different GPUs such that the workloads are approximately balanced.
pixel's rendering time the current epoch. Based on this estimate, we can
Specifically, Grendel measures the running time (including image rendering, loss computation, and the corresponding backward computation) of each block of pixels assigned to GPU, computes average per-pixel computation time for the GPU, and uses this average to approximate the computation time for any pixel y assigned to the GPU. For example, if a GPU is assigned pixels Po through Pa. and takes time for all of these pixels, then Grendel assumes that pixel p3 where i [0, ni requires time for computation. In subsequent iterations, the image is re-split that the sum of the computation time for pixels assigned to all GPUs are equal. our implementation, we use 16 16 pixel blocks as the split granularity. We show the pseudocode (Algorithm) I for calculating the Division Points to split an image into load- balanced subsequences blocks.
Algorithm 1 Calculation of Division Points
Require: B(number of pixel blocks), G(number of GPUs), ET (Estimated runtime per pixel block)
Ensure: DP (division points)
1: CT TUUCT.CCCUUUMM (EE)
Cumulative sum ET b Estimated runtime per GPU b Thresholds for Division Points
2: ET gpa t CT - 1)/G
3: TH < TRRCH.ARANGE(O, G) ET ET
4: DP < TORCH. SEAACCCHORTDDDCCTT.T.
> Division Points
5: return DP
wiuss Distribution Rebalancing, When training starts, we distribute 3D Gaussians uni- formly among the GPUs. As training progresses, new Gaussians added by cloning and splitting existing ones( @..T.) Newly added Gaussians make the distribution imbalanced as different Gaussians densify at different rates that depend the scene's local details. Therefore, redistribute the 3I) Gaussians after every few densification steps to restore uniformity.
4 SCALING HYPERPARAMETER FOR BATCHED TRAINING
To efficiently scale to multiple GPUs, Grendel increases batch size beyond one, enabling parti- tioning of both images and pixels within each image, shown in Figure b).
However, increasing the batch size without adjusting hyperparameters, particularly the learning rate, can result in unstable and inefficient training (Goyal et al. [017] Qiao et al.] 2021), and hyperparameter tuning is often tedious. Though some methods simplify learning-rate tuning for deep neural networks, they either build SGD (Goyal et al. 2017) (we Adam) they leverage
5

---

Published as a conference paper at ICLR 2025
the layer-wise structure of neural networks Ginsburg et al. 2018 You al. 2020) (3DGS is not neural network). Our result is driven by the Independent Gradients Hypothesis for 3DGS training. Inspired by (Malladi et al. 2022), suggests the same learning rate scaling as recent works (Malladi et al. 2022 Granziol et al. 2022)
derive scaling rule for the hyperparameters of Adam, which
but a different scaling for 31 and f3 that works better for 3DGS.
We propose to scale Adam's 's learning rate and momentum based on batch size as follows:
X' = À x hattcbhhsiiae size Bi 1/2 = shatch_size @shatc__sin size
(2)
where is the original learning rate, and $1, 32 are the original first and second moments in Adam. X' 11 (82 are the adjusted hyperparameters to work with greater batch size. We refer to these as the square-root learning rate scaling and the exponential momentum scaling rules.
Independent Gradients Hypothesis. To derive these scaling rules, we first consider 3D GS training simplified setting, assuming that gradients calculated from each camera view are independent of those induced from other views. Consequently, if we are given batch of b camera views, taking b sequential gradient descent steps for each view in the batch is equivalent to taking one bigger step where the gradients are summed together. If we were using the vanilla gradient descent algorithm and averaging the gradients in a batch, setting the learning rate to scale linearly with the batch size achieves this equivalence. However, 3D GS uses Adam, an adaptive learning rate optimizer that (1) divides the gradients by the square root of the per- per-paraeeter second moment estimate, and (2) uses momentum combine current gradients and past gradients in an coanentiallmovig--aaee making bigger update different from simply summing up smaller batch-size-ond one updates. Under the independent gradients hypothesis, we derive the following corrections to Adam hyperparameters to approximate batch-size-one training with a larger batch:
fashion,
Let us denote 9k as the gradient of some parameter evaluated at view k. and g Σier 92 as the [V
full-batch gradient (mean of gradients across views), where is the set of all views. Let us further assume [[gh] = 0 for all k. By the independence assumption: Cov(ge, 93) = B[(gx 0) 0)]
0 when 1 + j and [[( when k = j.
Then, parameter update from batch-size-1 Adam step (without momentum) on view le is:
(N)
ge
[
VEIV
VIVIVEL
However, the parameter update from one Adam step (without momentum) on batch of views B S V of size b is:
(0)
ΣLEn 9x/D
1 Σxen
Eargy (ESE (1))
1 1
V
E
Vb VIVEET
Thus, setting the learning rate X' = - Vo allows the batch update A AB to match with the total individual updates ΣLEB A (*), Alongside the square-root learning rate scaling (Eq Lp.
we also
propose an exponential momentum scaling accommodate larger batches (Eq 2 Imtially used
by (Busbridge et al. (2023), this rule scales the momentum parameters with B = , bizcc_ssize which
exponentially decreases the influence of past gradients when the batch size increases.
We wish to stress that in the real world, even though some cameras share similar poses, a set of random cameras generally observe different parts of scene, hence the gradients in batch are mostly sparse and can be thought as roughly independent. We empirically study the independent gradient hypothesis and evaluate our proposed scaling rules.
4.1 EMPIRICAL EVIDENCE OF Independent Gradients
To see if the Independent Gradients Hypothesis holds in practice, we analyze the average per- parameter variance of the gradients in real-world settings. We plot the sparsity and variance of the gradients of the diffuse color parameters starting at pre-trained checkpoints
the "Rubble" dataset (Turki et al. 2022) against the batch size in Figure 6 We find that the inverse of the variance increases roughly linearly, then transitions into plateau. We find this behavior in all three checkpoint iterations, representing early, middle, and late training stages. The initial linear increase of the precision suggests that gradients are roughly uncorrelated at batch sizes used in this work (up to 32) and supports the

---

Published as a conference paper at ICLR 2025
independent gradients hypothesis. While single image may have sparse gradients; in large batch, gradients overlap and become less sparse. They also grow more correlated, as camera with similar poses expected to offer similar gradients.
Batch size vs Grad Sparsiyy
Batch sizo vs Grad Vartance
Botch sizo vs Grad Grad Procision
24
Parmmeer
Cocerra Gind
47
Batch size S0g scale)
Batch
19930
50000
Figure 6: Gradients are roughly uncorrelated in practice. On the "Rubble" dataset Turki et al. 2022). ) the inverse of the average parameter variance increases linearly, then rises to a plateau, suggesting that the gradients are roughly uncorrelated initially but become less so the batch size becomes large. Averaged over 32 random trials.
4.2 EMPIRICAL. TESTING OF PROPOSED SCALING RULES
To empirically validate the proposed learning rate and momentum scaling rules, we train the "Rubble" scene up to iteration 15,000 using a batch size of Then, we reset the Adam optimizer states and continue training with different batch sizes. We compare how well different learning rate and momentum scaling rules maintain a similar training trajectory when switching to larger batch sizes in Figure 12 Appendix B.3). Without loss of generality, we focus on the diffuse color parameters for this ysis. Figure 2a compares three different learning rate scaling rules E [constant, sqrt, linear] where only our proposed "sqrt" holds high update cosine similarity and similar update magnitude across different training batch sizes. Similarly, shows our proposed exponential momentum scaling rule keeps update cosine similarity higher than the alternative which leaves the momentum coefficients unchanged.
5 EVALUATION
Our evaluation aims to demonstrate Grendel's scalability, showing both that it can render high- resolution images from large scenes, and that its performance scales with additional hardware resources. We also compare Grendel's system-level parallelization with CityGaussian' ' (Liu et al. 2024) divide-and-conquer approach [5.3] The ablation study on dynamic load balancing and learning rate scaling strategies is presented in Appendix C.I
5.1 SETTING AND DATASETS
Experimental Setup. We conducted our evaluation in the Perlmutter GPU cluster NERSC Each node we used was equipped with 4 A100 GPUs with 40GB of GPU memory, and interconnected with each other using 25GB/s NVLink per direction. Servers were connected to each other using 200Gbps Slingshot network.
Datasets. We evaluate Grendel using the datasets and corresponding resolution settings shown in T Of these, Rubble and MatrixCity Block_All represent the large scale datasets that are out of reach for most existing 3DGS systems, while other datasets commonly used in 3DGS papers. These datasets vary in area size and resolution to comprehensively test our system.
Evaluation Metries. We report image quality using SSIM, PSNR and LPIPS values. and throughput in training images per second. We take both forward and backward time into consideration of throughput, And note that throughput images per second may differ from throughput in iterations per second, as one iteration includes the batch size number of images.
5.2 PERFORMANCE AND MEMORY SCALING
We start by evaluating Grendel's 's scaling, and how additional GPUs impact computation performance and memory.
7

---

Published as a conference paper at ICLR 2025
Dataset
Scenes
Resolutions
Images
Test Set Setting
Tanks & Temple (Knapitsch ct al. 2017)
2
1K
251 to 301
1/8 all images
DeepBlending (Hedman al. 2018
2
1K
225 263
1/8 of all images
Mip-NeRF 360 [Barron et al. 2022
9
1080P
100 to 330
1/8 all images
Rubble (Turki al. 2022)
4591
3436
1657
official test set
MatrixCily Block All Li et al.
2023)
1080P
5620
official test set
Table 1: Scenes used in our evaluation: We cover scenes of varying sizes and resolutions.
30
Speedup PSNR
5.33x
64
00M
38.03
Training homugghputtin (imga/st(mgg//ae
4.6x
4.83x
4.36x
32
25. 18
39.12
30
Training Speedup
4-
25
Batch Size
16
DOM
24.75
36.10
3.25x
PSNR
3
21.84
22.0
22 0909
21.73
22.06
21.76
0.0
13.74
22.45
20
2-
-0
10
12.56
1
1.0x
00M
00M
5.55
15
1
2
8
166
32
1GPU
4GPU
8GPU
8GPU
16GPU BS-32
GPUs
BS-16
Figure 7: To avoid OOM, GPUs are needed to train the large 4K "Rubble" scene. We fur- ther improve throughput by distributing across even more GPUs and increasing the batch size.
Figure 8: Even for the small 'Train' scene, our distributed training with larger batch sizes achieves speedup without compromising test PSNR. All configs train for 30K total images.
Computation. We evaluated how additional GPUs impact Grendel's performance using both large- scale (Rubble) and small-scale (Train and Mip-Nerf360) datasets.
We used the Rubble scene to evaluate the training throughput. For this experiment we used 35 million Gaussians which have been trained convergence. Because the time required to render 4K images for this scene, we measured throughput for training over another 10,000 images times, and in Figure we report throughput (in images per second) as we vary the number of GPUs (x-axis) and batch size (y-axis). We observe that we cannot render this scene with a single GPU (regardless of batch size) because of its memory requirements. Furthermore, both increasing the number of GPUs and increasing batch size yield performance improvements: performance increases from 5.55 images per second (4 GPUs, batch size 1) to 38.03 images per-second (32 GPUs, batch size 64).
Next, the 980 x 545 resolution Train scene to evaluate both throughput and image quality during scaling. Due to its small, low-resolution nature, it can be trained from scratch for each experiment. Our results in Figure show image quality when trained with the same total number of images. Notably, our 16-GPU setup with batch size of 32 completes training on 30K images in just 2 minutes and 42.97 seconds, representing the waae-oof-theaar training speed to the best of our knowledge.
show that additional GPUs improve throughput while maintaining
As shown in Figure 9 we also achieve 3x to 4x speed up using GPU and batch size of without PSNR degradation across 13 scenes from the Mip-Nerf360 dataset (first half) and the Tanks Temple and Deep Blending datasets (second half). We use default hyperparameters from the 3DGS repository (Kerbl al. 2023). We train the same number of images: 50k for Mip-NeRF 360 and 30k for the slightly smaller TT & DB datasets, to ensure convergence and a fair comparison.
Memory Scaling, Scaling the number of GPUs increases memory, allowing more Gaussians to represent scene. We tested this by adding Gaussians through densification until we ran out of memory. Figure 14 (In Appendix CC3) illustrates the number of Gaussians that Grendel can accommodate (in millions) with batch sizes of 1,4, and 16, as the number of GPUs increases. The results demonstrate linear scaling. In $.2.1 we show the utility of using additional Gaussians.
We provide additional details about experiments from this section in Appendix C.3
8

---

Published as a conference paper at ICLR 2025
40
TT & DB (4GPU B5-4 speedup over 1GPU 85-11
Mip360 (4GPU 45 speediup 1GPU BS=1)
35
Training Throughput(ing/eee (ings//g//)
70-
1GHU B5-1
4GPU BSe1 Loadbalance
85-1 wio Loadba
4GPJ BB5- wis ioadialence
4GPu BSod Loasbslance
60-
Training Speedup
50
40-
25
30-
20
20
12
1GPU B5-1 (Tast PSNRI
-15
10.
4GPU BS-4 flest PSNR)
counter
kitchen
roo stump
bicycle
garden
bortaal
flowers
treehill
train
truck
playroom
drjohnson
counter
kitchen
room
stump
bicyde
garden
bonsal
flowers
treehill
Figure 9: Training Speedup and PSNR on Mip- NeRF360 and Tanks& T Deep Blend-
Figure 10: Speedup from Iterative Load Bal- ancing and increased batch sizes Mip- NeRF360.
ing.
Rubble
MatrixCity Block_All
Bicycle
One CPU Wemmory Limit
-02
SSIM LAPSS
PSNR
One GPU Nimorry
-65
262+
442
-65
354
201
19.0
8 Gaussians (Millions)
of Gaussians (Millions)
of Causslans (Millions)
Figure 11: Scalability Statistics: Gaussian Quantity vs. Reconstruction Quality
Using more Gaussians results in better test metries for reconstruction. The red line indicates the number of Gaussians single GPU can handle, which is insufficient for achieving high-quality results.
5.2.1 GAUSSIAN QUANTITY VS. RECONSTRU QUALITY
Scaling to multiple GPUs allows Grendel to use a larger number of Gaussians to represent scenes. larger number of Gaussians can capture fine grained scene details, and should thus be able to better reconstruct large-scale, high-resolution scenes. We evaluated this effect using three scenes: Rubble, MatrixCity Block All, and Bicycle. and varied the number of Gaussians used by changing densification settings: we lowered the gradient norm threshold to initiate densification and reduced the threshold for splitting Gaussians instead of cloning until the densification mechanism produced the target number of Gaussians without manual interference. We rendered Rubble and Matrix City Block All using 16 GPUs and a batch size of 16, while we used GPUs and a batch size of 4 for Bicycle. The difference in number of GPUs and batch sizes is due to differences in scene sizes: bicycle much smaller than the other two datasets.
In Figure II we show that image quality metrics (PSNR, SSIM and LPIPS) improve more Gaussians. The red line in the Rubble and Matrix City Block_Al graphs shows the number of Gaussians that fit on single GPU (Bicycle, being smaller, can be rendered on single GPU). Figure B shows the rendered images as we scale Gaussians quantity, and demonstrates the quality improvements are human visible. These results demonstrate benefits of using more Gaussians, and demonstrate the necessity of multi-GPU 3DGS training systems like Grendel.
add
5.3 COMPARISON TO CITYGAUSSIAN
We evaluate Grendel's system-level parallelization against CityGaussian et al. 2024), the open- source 3DGS divide-and-conquee solution. Since CitySaus is not designed for high-resolution reconstruction, we perform these experiments on downsampled scenes: Rubble (4x downsampled) (Turki et al., 2022), Building (4x downsampled) Turki et al.] 2022), and MatrixCity Block All (downsampled to a width of 1600 pixels) (Li et al.) 2023). All experiments are conducted on A A100s. We provide comparison points for CiyGaassian. CityGS Official uses the authors'

---

Published as a conference paper at ICLR 2025
official script [ yielding PSNR numbers comparable to those reported in (Liu et al. 2024 CityGS Official experiments train Rubble, Building, and Matr Block All on a total of 300000, 630000, and 11 1110000 images, respectively. CityGS 100K-imager experiments use the same configuration as CayGS Official, except for training each scene with a total of only 200000 images, in order to compare the convergence rate with Grendel. In Grendel 200K- each scene is trained using Grendel for a total of 200000 images, matching the count in CityGS 200K-images.
Method
Building
ManrisCity Block_All
PSNR
SSIM
LPIPS
Total Time
PSNR
SSIM
LPIPS
Total Time
PSNR
SSIM
LPIPS
Total Time
CiyCS Official
25. 55
0.811
0.231
2,88 hms
22. 14
0.784
0.341
4.57 hms
27.41
0.864
0,205
8.25 hrs
Ciycs 200KK-images
25.40
0.796
0.249
2.18 hrs
20.3
0.725
0.299
2.22 brg
23.68
0.701
D.422
3,00 hrs
Grendel 200K-imagus (ours)
0.859
22.69
0.12
22.33
0.205
Table 2: Quantitative evaluation of Grendel compared CityGaussian(Liu al., 2024).
We report
PSNR1, SSIM1, and LPIPS. on test views, along with Total Training Time. The best and second best results are highlighted. Time Decomposition is provided in Table IO Appendix (5).
As shown in Table 2 Grendel achieves test PSNR comparable to, or surpassing those of CityGS Official. Grendel is much accieving 3x-6.7x speed improvements over CityGS Official. Com- pared to CityGS 200K -inggemgees, Grendel demonstrates greater training efficiency in both convergence rate and training time. Our experience also shows that Grendel is simpler to use. CityGS requires running several separate procedures each of which requires hyperparameter tuning. By contrast, running Grendel over multiple GPUs requires similar efforts as the original 3DGS.
RELATED WORKS
Large-scale scene reconstruction. Prior works have proposed the dovide-and-conuee approach to scale 3DGS work with large scenes. VastGaussian (Lin al. (0024), CityGaussian (Liu al.) 2024), and Hierarchical Gaussian (Kerbl et al. 2024) divide large scenes into small regions and train each region separately, and then merge the resulting sub-models. Hierarchical Gaussian (Kerbl et al.
2024), Octree-GS (Ren et al. 2024) and City Gaussisan (Liu et al. (202)) describe level-of-detal based approaches to adaptively reduce the number of Gaussians considered for distant objects. Grendel's system-level parallelization is complementary to these algorithmic innovations. For example, the initial coarse training step in both CityGaussisan (Liu et al., 2024), and Hierarchical Gaussian (Kerbl) et al. 2024) can utilize Grendel for multi-GPU acceleration. Similar methods (Turki et al., 2022
Yuanbo et al. 2022 Li et al.] 2024b) have been employed to scale NeRF, but they are not directly applicable to 3DGS due to its distinct computation pattern, discussed in Section I
Distributed Training for 3DGS. DOGS (Chen & Lee (024) modifies training with ADMM distributed optimization, averaging Gaussians shared across partition boundaries every 100 iterations. This method enables asynchronous training, which can potentially impact convergence rate. In contrast, Grendel preserves the original 3DGS algorithm, maintaining the same convergence characteristies as the single-GPU version. RetinaGS (Li et al. 2024a) also retains the original 3DGS algorithm but employs a distinct parallclization strategy. In RetinaGS, each GPU renders the entire image using its local partition of Gaussians, with the rendered outputs subsequently merged. However, this approach results in redundant computations, as many GPUs render pixels beyond the opacity saturation depth unnecessarily. Distributed training for neural networks is further discussed in Appendix D
Large Batch Size Training. Large batch training has been widely adopted to improve the ML training performance and efficiency, but it has also been recognized by prior work (Keskar et al.) 2017) that increasing batch size can adversely impact model performance. This has led to the development of empirical rules for neural networks training, including linear scaling and learning rate warmum for SGD (Goyal al. (001)). square root scaling for Adam (Malladi et al. [022] and liyer-wised adaptive rate scaling You et al. 2020; Ginsburg et al., 2018). The scaling rules in our work are inspired by these, but focuses on batch size scaling for 3DGS.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
We provide hyperparameter details in Appendix C.3 We will also release code for training, rendering. and evaluation. We will release pre-trained model checkpoints for Rubble and MatrixCity datasets to reproduce the results reported in the paper.
ACKNOWLEDGEMENTS
We thank Xichen Pan and Youming Deng for their help on paper writing, We thank Matthias Niessner for his insightful and constructive feedback on our manuscript. We thank Yixuan Li and Lihan Jiang from the MatrixCity team for their assistance in providing initial data points of their dataset. We thank Kaifeng Lyu for discussions on Adam training dynamics analysis. This research used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility located at Lawrence Berkeley National Laboratory, operated under Contract No. DE-ACC2-0SCHI1234
REFERENCES
Jonathan Barron, Ben Mildenhall, Dor Verbin, Pratul Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. CVPR, 2022. doi: 10.1 N0.11099CVPSS2288.2222 00539.
Dan Busbridge, Jason Ramapuram, Pierre Ablin, Tatiana Likhomanenko, Eeshan Gunesh Dhekane, Xavier Suau, and Russell Webb. How to scale your EMA. In NeurIPS, 2023. URL https //openreview. net forum? hett/oormmidd-eeeeeeeeeee Dikeevaayy
Yu Chen and Gim Hee Lee. Dogs: Distributed-oriente gaussian splatting for scale 3d recon- struction via gaussian consensus, 2024. URL httpt::t/t//attt http:/iw..org/////200000 13943)
Boris Ginsburg, Igor Gitman, and Yang You. Large batch training of convolutional networks with layer-wise adaptive rate scaling, 2018. URL http:s://peeneev.ee..ete/////
IJ4uaX2aW
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint
026777 2017.
Diego Granziol, Stefan Zohren, and Stephen Roberts. Learning rates as function of batch size: A random matrix theory approach neural network training. Journal of Machine Learning Research, 23(173): 2022.
Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Brostow. Deep blending for free-viewpoint image-based rendering. ACM Transactions Graphics (Proc. SIGGRAPH Asia), 2018
Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, Hy- oukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, and zhifeng Chen. Gpipe: Efficient training of giant neural networks using pipeline parallelism. In NeurIPS. 2019. URL
wwwrips.cc/papp... files/ F11ees/ppper2211/ Fil Filee/993663e080229538888 (b1006666656110
Paper.
Berharr Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023. URL
hittps://rege sam. inria. fr htt////ugga/////a/ http:h//dddduuuuuuuuoeaa sp www.stpaatt..... atting//
Berhaard Kerbl, Andreas Meuleman, Georgios Kopanas, Michael Wimmer, Alexandre Lanvin, and George Drettakis. A hierarchical 3d gaussian representation for real-time rendering of very large datasets. ACM Transactions on Graphics, 2024. URL https: repo- sam, inria. fr/ Fungraph/hierarchh ca wtw.rrrchchaaa---ssssssss-ee

---

Published as a conference paper at ICLR 2025
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On training for deep learning: Generalization gap and sharp minima. In ICLR, 2017. http::
net wtw.et//oorummmndlllol......
Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics, 2017.
Bingling Li, Shengyi Chen, Luchao Wang, Kaimin Liao, Sijie Yan, and Yuanjun Xiong. Retinags: Scalable training for dense scene rendering with billion-scale 3d gaussians, 2024a. httpts
7/arxiv. htt.//oggaabbb//ppbb 24tp://bs/2222.11133
Ruilong Li. Sanja Fidler, Angjoo Kanazawa, and Francis Williams. Nerf-xl: Scaling nerfs with
multiple gpus, 2024b.
Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, and Soumith Chintala. Pytorch distributed: Experiences on accelerating data parallel training. In VLDBB, 2020.
Yixuan Li, Lihan Jiang, Linning Xu, Yuanbo Xiangli, Zhenzhi Wang, Dahua Lin. and Bo Dai. Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond. In ICCV, 2023,
Jiaqi Lin, Zhihao Li, Xiao Tang, Jianzhuang Liu, Shiyong Liu, Jiayue Liu, Yangdi Lu, Xiaofei Wu, Songcen Xu, Youliang Yan, and Wenming Yang. Vastgaussian: Vast 3d gaussians for large scene reconstruction. CVPR, 2024.
Yang Liu, He Guan, Chuanchen Luo, Lue Fan, Junran Peng, and Zhaoxiang Zhang. Citygaussian: Real-time high-quality large-scale scene rendering with gaussians. In CVPR, 2024.
Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang. Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering, In CVPR, 2024.
Sadhika Malladi, Kaifeng Lyu, Abhishek Panigrahi, and Sanjeev Arora. On the SDEs and scaling rules for adaptive gradient algorithms. NeurlPS, 2022. BtRthtttss////eneeet
net/
forum?id=F2mi HKQQP.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Deepak Narayanan, Aaron Harlap, Amar Phanishayee, Vivek Seshadri, Nikhil R Devanur, Gregory R Ganger, Phillip Gibbons, and Matei Zaharia. Pipedream: generalized pipeline parallelism for dnn training. In SOSP, 2019.
Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vi- jay Anand Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, and Matei Zaharia. Efficient large-scale language model training on gpu clusters using megatron-Im. In SOSP, 2021.
NERSC. Perlmutter architecture. per_mmutterarrciteetuue
URL Accessed: 05-2-.
Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R. Ganger, and Eric P. Xing. Pollux: Co-adaptive cluster scheduling for goodput- optimized deep learning. In OSDI, 2021.
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. In SC, 2020.
Kerui Ren, Lihan Jiang, Tao Lu, Mulin Yu, Linning Xu, Zhangkai Ni, and Bo Dai. Octree-gs: Towards consistent real-time rendering with lod-structured 3d gaussians. arXiv preprint arXiv:2403. 17898, 2024.
Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catan- zaro. Megatron-Imm Training multi-billion parameter language models using model parallelism. In SC, 2020.
12

---

Published as a conference paper at ICLR 2025
Haithem Turki, Deva Ramanan, and Mahadev Satyanarayanan. Mega-nerf: Scalable construction of large-scale nerfs for virtual thyouhs. In CVPR, 2022.
Minjie Wang, Chien-chin Huang, and Jinyang Li. Supporting very large models using automatic dataflow graph partitioning. In EuroSys, 2019.
Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Blake Hechtman, Yanping Huang, Rahul Joshi, Maxim Krikun, Dmitry Lepikhin, Andy Ly, Marcello Maggioni, Ruoming Pang, Noam Shazeer, Shibo Wang, Tao Wang, Yonghui Wu, and Zhifeng Chen. Gspmd: General and scalable paral- lelization for ml computation graphs. In arX1v:2105.00663, 2021.
Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. Large batch optimization for deep learning: Training bert in 76 minutes. In ICLR, 2020. URL httpt::tpt///ttttt httpss//ppeeeeeeewww http://wwppeeeeeeeeeeeeeeeee/eea net/ forum Parum?id=Syxe 4wnEtvH.
Xiangli Yuanbo, Xu Linning, Pan Xingang, Zhao Nanxuan, Rao Anyi, Theobalt Christian, Dai Bo, and Lin Dahua. Bungeenerf: Progressive neural radiance field for extreme multi-scale scene rendering. In ECCV, 2022.
Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, and Shen Li. Pytorch fsdp: Experiences scaling fully sharded data parallel, 2023.
Lianmin Zheng, Zhuohan Li, Hao Zhang, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo Eric Xing, et al. Alpa: Automating inter-and intra-operator parallelism for distributed deep learning. In OSDI, 2022.
13

---

Published as a conference paper at ICLR 2025
A ADDITIONAL PRELIMINARIES & OBSERVATIONS DETAILS
This appendix provides additional information about 3DGS, beyond what was covered in
A.1 DENSIFICATION PROCESS
Densification the process by which 3DGS adds more Gaussians to improve details in particular region. A Gaussian that shows significant position variance across training steps, might either be clones or split. The decision on whether to clone or split depends on whether their scale exceeds threshold. Hyperparameters determine the start and stop iteration for densification, its frequency, the gradient threshold for initiating densification, and the scale threshold that determines whether to split clone. To create decrease the gradient threshold for densification. If we aim to capture more details using smaller Gaussians, we should lower the scale threshold split more Gaussians. The training process also includes pruning strategies such as eliminating Gaussians with low opacity and using opacity reset techniques remove redundant Gaussians.
Gaussians, need increase the stop iteration and frequency, and
A.2 Z-BUFFER
The indices of intersecting gaussians for each pixel are stored a 7.-buffer, used both forward and backward. This Z-buffer is the switch between Vienddepe Gaussian Transformation and Pixel Render. Since a single gaussian can project onto multiple pixels within its footprint, the total size of all pixels' Z-buffers exceeds both the count of 3DGS and pixels. The Z-buffer itself, along with auxiliary buffers needed for sorting it, etc, consumes significant activation memory. This can also lead to out-of-memory (OOM) errors if the resolution, scene size, or batch size is increased.
A.3 MIXED PARALLELISM
In the main text, some steps of 3DGS are not mentioned, but these steps can also be parallelized. The Gaussian transformation backward and gradient updates by the optimizer are also Gaussian-wise computations and will be distributed the same way as the Gaussian transformation forward. Similarly, the Render Backward and Loss Backward computations are pixel-wise and will be distributed just like the Render Forward.
Regarding the memory aspect, each Gaussian has independent transformed states, gradients, optimizer states, and parameters. Therefore, we save these states together on the corresponding GPU that contains their parameters. And activation states like significant Z-buffers, auxiliary buffers for sorting and other functions, loss intermediate activations are managed pixel-wise along with the image distribution.
Regarding densification mechanism, since we clone, split or prune Gaussians independently based on their variance, perform this process locally the GPU that stores them.
A.4 DYNAMIC UNBALANCED WORKLOADS
Physical scenes are naturally sparse on global scale. Different areas have different densities of 3D gaussians (i.e sky and a tree). Thus, the intensity of rendering not only varies from pixel to pixel within an image but also differs between various images, leading to workloads unbalance. Figure shows the differences in render intensity across the image.
Besides, during the training, gaussians parameters are continuously changing. More precisely, the change of 3D position parameters and co- co-variance parameters affect each gaussian's 's coverage of pixels on the screen. The change of opacity parameters affect the number of gaussians that contribute to each pixel. Both of them lead to render intensity change. The densification process targets areas under construction. During training, simpler scene elements are completed first, allowing more complex parts be progressively densified. This means Gaussians from different regions densify varying rates. The dynamic nature of the workloads is more pronounced at the beginning of training, as it initially focuses constructing the global structure before filling in local details.
14

---

Published as a conference paper at ICLR 2025
The different computational steps have distinct characteristics in terms of workload dynamicity. Even though, the rendering computation is dynamic and unbalanced; computation intensity for loss calculation remains consistent across pixels, and the view-dependent transformation maintains a uniform computational intensity across gaussians. Actually, render forward and backward have different patterns of unbalance and dynamicity. The computational complexity for the forward process scales with the number of 3DGS intersecting the ray. In contrast, the complexity of the backward process depends on Gaussians that contributed to color and loss before reaching opacity saturation, typically those on the first surface. Then, running time for render forward and backward, loss forward and backward have different dominating influence factors, and every step takes a significant amount of time.
B ADDITIONAL DESIGN DETAILS
B.1 SCHEDULING GRANULARITY: PIXEL BLOCK SIZE
In our design, we organize these pixels from all the images in a batch into single row. Then, we divide this row into parts, and each GPU takes care of one part. However, if there are a lot of pixels, the strategy scheduler computation overhead will be very large. So we group the pixels into blocks of 16 by 16 pixels, put these blocks in row and allocate these blocks instead. The size of block is essentially the scheduling granularity, which a trade-off between scheduler overhead and uneven workloads due to additional blocks. After scheduling, we will have 2D boolean array, compute lorauteeooollllllllllll indicating whether the pixel block at i-th row and j-th column should computed by the local GPU. We will then render only the pixels within the blocks where compute compute_loclly locally is true.
B.2 GAUSSIAN DISTRIBUTION REBALANCE
An important observation that distributing pixels balance runtime doesn't 't necessarily balance the number of Gaussians each GPU touches in rendering; So, to minimize total communication volume, GPUs may need to store varying quantity of Gaussians based on the formula above. Specifically, only the forward runtime correlates directly with the number of touched 3DGS; however, the time it takes for pixel-wise loss calculations and rendering backward depends on the quantity of pixels and the count of gaussians that indeed contributed to the rendered pixel color, respectively. In our experiments, random redistribution leads to fastest training here, even if its overall communication volume is not the minimum solution. Because in our experiment setting, we use NCCL all2all as the underlying communication primitive, which prefers the uniform send and receive volume among different GPU. If we change to communication primitive that only cares about the total communication volume, then we may need to change to other redistribution strategy.
B.3 SCALING RULE HYPERPARAMETER ABLATION
The effectiveness of our automatic hyperparameter scaling rules is demonstrated in the ablation study, as shown in Figure
C ADDITIONAL EXPERIMENTS SETTING AND STATISTICS
C.I ABLATION STUDY
Figure i0jllustrates that our load balancing techniques and increased batch size significantly improve training throughput on 1080p Mip-NeRF360 dataset, compared to the one GPU baseline and our straightforward distributed system with conventional batch size of one and no load balancing.
Similar results are observed with the 4K Rubble Dataset, as shown in Figure 15] Although good speed can be achieved without load balancing, load balancing allows to consistently achieve
higher throughput across various types and scales of scenes. The ablation study for the learning rate scaling strategies have already been discussed in 4.2 along with our analysis.
15

---

Published as a conference paper at ICLR 2025
Tran Ihaget
Toar Urggos
vf
2011
(a) Learning rate scaling rules vs. BS invariance.
(b) Momentum scaling rules vs. BS invariance.
We plot the training trajectories of the diffuse color parameters on "Rubble", when training with batch size E [4, 16, 32] using different learning rate and momentum scaling strategies. Cumulative weight updates using the square-root learning rate scaling rule (a, curves) and exponential momentum scaling rule (b, red curves) maintain high cosine similarity to batch-size updates and have norms that are roughly invariant to the batch size. All trajectories in (a) employ our proposed exponential momentum scaling but differing learning rate scaling; while all trajectories in (b) employ our proposed square-root learning rate scaling but differing momentum scaling.
Figure 12:
C.2 STATISTICS FOR MIP-NERF 360,
AND DEEPBLENDING DATASETS
We provide full statistics of training results on Mip-NeRF 360, Tank& Temples and DeepBlending datasets in Table
Dataset
Scene
GPU (bsz=l)
4 GPU (bsz=4)
PSNR
Throughput
PSNR
Throughput
Mip-NeRF360
counter
29.16
16.25
29.19
56.24
kitchen
31.49
14.24
31.40
49.16
room
31.51
15.82
31.18
53.36
stump
26.19
14.95
26.19
54.53
bicycle
24.63
12.01
24.69
44.44
garden
26.82
12.10
26.86
45.83
bonsai
32.34
17.87
32.23
61.88
flowers
21.11
14.47
21.10
53.94
treehill
22.38
14.78
22.43
55.31
Tank&Templer
train
21.84
34.72
21.75
101.69
truck
25.44
27.55
25.42
95,85
DeepBlending
playroom
30.11
21.98
30.22
75.38
drjohnson
29.15
17.74
29.19
62.11
Table 3: Performance Comparison Between Non-Distribution and GPU Distribution
C.3 SCALABILITY
Table 4 and 6 show the increased reconstruction quality with more gaussians. While many hyperpa- rameters influence the number of Gaussians created by densification, we focused on adjusting three key parameters: (1) the stop iteration for densification, (2) the threshold for initiating densification. and (3) the threshold for deciding whether to split or clone Gaussian. Initially, we gradually increased the densification stop iteration 5,000 iterations. However, due to the pruning mechanism, this adjustment alone proved insufficient. Consequently, we also lowered the two thresholds to gener- ate more Gaussians. For fair comparison, other densification parameters- as the interval, start iteration, and opacity reset interval-went were kept constant. For the Rubble scene, each experiment run for the same 125 epochs, exposing models to 200,000 images, ensuring consistency. Although training larger models for longer durations and lowering the positional learning rate improved results in my observations, we maintained consistent training steps and learning rates across experiments ensure fairness.
16

---

Published as a conference paper at ICLR 2025
Table show the Throughput Scalability by Increasing batch size and leveraging more GPUs, for Rubble and Train scene, respectively. Essentially, more GPUs and larger batch size give higher throughput. More GPUs provide more computational power while larger batch size utilze these GPUs better.
Table demonstrates that additional GPUs increase available memory for more Gaussians, evaluated on the Rubble scene with various batch sizes reflecting different levels of activation memory usage. We achieve linear scaling illustrated in Figure [4] Essentially, more GPUs provide additional memory to store Gaussians, while a larger batch size increases activation memory usage, leaving less memory available for Gaussians.
Bicycle
4JN
MatrisCity
Rubbie
Greeal
Tet
TFM Cranders
Figure 13: Visualization: Gaussian Quantity vs. Reconstruction Quality
Results SSIM
Densification Settings
Experiment
n3dgs
PSNR
LPIPS
Stop Iter
Thresholds
EXP
2114045
24.84
0.70
0.48
5000
(0.0002, 0.01)
EXP 2
5793396
25.85
0.75
0.42
15000
(0.0002, 0.01)
EXP
9173931
26.14
0.77
0.38
50000
(0.0002, 0.01)
EXP 4
11168630
26.28
0.78
0.37
50000
(0.00018, 0.008)
EXP
15754744
26.61
0.79
0.35
50000
(0.00015, 0.005)
EXP
21177774
26.91
0.80
0.33
50000
(0.00013, 0.003)
EXP
30474202
27.06
0.82
0.31
50000
(0.0001, 0.002)
EXP
40397406
27.28
0.82
0.29
50000
(0.00008, 0.0016)
Table 4: Scalablity on Rubble: Gaussian Quantity, Results and Hyperparameter Settings
Results SSIM
Densification Settings
Experiment
n3dgs
PSNR
LPIPS
Start Points
Densify Iter
EXP 1545568
24.41
0.73
0.41
1545568
EXP
3867136
25.36
0.77
0.36
3867136
EXP
9485755
26.6
0.82
0.27
7743616
5000
EXP
14165332
26.78
0.83
0.25
15540941
5000
EXP
24355726
27.0
0.84
0.23
15540941
30000
EXP
30074630
26.96
0.84
0.22
15540941
40000
Table 5; MatrixCity Block All Statistics: Gaussian Quantity, Results and Hyperparameter Settings
17

---

Published as a conference paper at ICLR 2025
Results SSIM
Densification Settings
Experiment
n3dgs
PSNR
LPIPS
Stop Iter
Thresholds
EXP
2185112
24.09
0.66
0.35
5000
(0.0002, 0.01)
EXP 2
3035508
24.28
0.68
0.32
7000
(0.0002, 0.01)
EXP
4154806
24.59
0.70
0.29
10000
(0.0002, 0.01)
EXP 4
5272686
24.71
0.71
0.28
15000
(0.0002, 0.01)
EXP
6579244
24.76
0.72
0.27
15000
(0.00018, 0.008)
EXP
9636072
24.85
0.73
0.25
15000
(0.00015, 0.005)
Table 6: Bicycle Statistics: Gaussian Quantity, Results and Hyperparameter settings
Loadbalance
Max of (Millions)
35-
350
bsz=1
Throughput (Imgsises
300
bsz-4
b5z=16
250
20-
150
10
100
50
15-4
85-0
DS-16
BS-
12
16 GPU Count
32
Figure 15: Load balancing and larger batch accelerate training on 4K Rubble Scene
Figure 14: More GPUs provide additional memory to support more Gaussians before encountering OOM.
C.4 RENDER SPEEDUP
We compare Grendel's rendering speeds single-GPU and 4-GPU setups for the Rubble, MatrixCity Block-All, and Bicycle scenes, using our state-of-the-ard trained Gaussian models corresponding to Figure [1] We experiment with various resolutions and scene scales, rendering image time for fair comparison. The 4-GPU setup achieves a speedup of 1.88x to 2.63x, as shown in Table
D ADDITIONAL RELATED WORKS
Distributed training for neural networks. Existing work have exploited various types of parallelism to train neural networks across GPUs. These include data parallelism (Li et al.) 2020), tensor parallelism (Shoeybi et al. 2020; Narayanan al. (222I). pipeline parallelism (Huang al. 2019
Narayanan et al. 2019 9) and FSDP (Zhao et al.] |2023] Rajbhandari et al.) 2020). Several systems also support multiple types parallelism and/or aim automatically partition the workload to optimize speed (Zheng et al. 2022 Xu et al. 2222]] Wang et al.) 2019). However, as we discussed earlier ($T).
aal.[0099
neural network and 3DGS have very different computation patterns. The former performs repeated layer-wise computation dominated by dense matrix multiply operations while the latter's 3 stages
GPU Count
bsz=1
bs7=2
bs7=4
bs7=8
bsz=16
bsz=32
bsz=64
I GPU
00M
2 GPU
00M
4 GPU
5.55
6.52 12.56
7.28 12.55
0OM
8 GPU
13.74
OOM
16 GPU
22.45
24.75 36.10
25.18 39.12
OOM 38.03
32 GPU
Table 7: Scalability Rubble: Speed up from More GPU and Larger Batch Size
18

---

Published as a conference paper at ICLR 2025
Experiment
# GPU
Batch Size
Throughput
PSNR
EXP 1
34.72
21.84
EXP 2
4
16
112.78
22.01
EXP 3
8
16
151.52
22.09
EXP 4
8
32
159.57
21.73
EXP 5
16
16
167.60
22.06
EXP 6
32
185.19
21.76
Table 8: Scalability on Train: Speed up from More GPU and Larger Batch Size
GPU Count
bsz=1
bsz=4
bsz=16
1 GPU
12.71 M
7.10 M
OOM
2 GPU
31,40 M
21.80 M
3.91 M
4 GPU
63.44 M
43.48 M
19.55 M
8 GPU
116.85 M
82.31 M
36.44 M
16 GPU
230.41 M
169.37 M
74.98 M
32 GPU
354.46 M
313.10 M
150.21 M
Table 9: Scalability on Rubble: More Available memory with more GPU
training process is irregular and sparse. As a result, although Grendel's distribution strategy may resemble those seen in existing work (e.g., FSDP Zhao al. (2023)) the details quite different.
19

---

Published as a conference paper at ICLR 2025
Method
Rubble
Building
MarraCity BlockAll
Tooal Time
Time Decompocitin
Total Time
Time Decompesition
Total Tine
Time Decomposiion
sourse: 43minl4s
caure
tun couns:
Odiica Odiicia
colle
A.I0 SaSmmin
8.25
56
9h5Tmin
59
cloud
2mis27
trin, course: 43min14s
traie couse. 44mia3s dhis partion:: 17mint7
man cours: 47min31s
CiyyS 200K inages
2,18 les
dain zertitu:: bminss
9 cells tran ce A100: 1h20kniz
2.22
cells cain ea A,00 Iallmin
3,00 hrs
del perlioon: H120mmn22
20 cells tran on A100: 1h20mas
Marge poine cload)
Mange poin.
Mazg puint cloud:
2000-imagg
51min
insived
Table 10: Time Decomposition for experiments in Table 2 CityGaussian employs a divide-and- conquer approach with four steps: coarse training, partitioning the scene into cells, training each cell independently, and merging the resulting point clouds. In contrast, 3DGS on multiple GPUs with Grendel be in the same way as the original single-GPU training, simply by allocating additional GPUs.
Scene (Resolution) Rubble (3436x4591)
GPU
GPU
8.8 img/s
21.7 img/s
Matrixcity Block-all (1080x1920)
29.2 img/s
76.8 img/s
Bicycle (1275x1920)
42.6 img/s
80.3 img/s
Table 11: 4-GPU Render Speedup with Grendel Compared Single-GPU Rendering
20


Reviews:
[{"review_id": "xXHag3Zl0t", "reviewer": "Reviewer_VE39", "summary": "This paper, named Grendel, presents a system to enable the distributed training of 3DGS. By leveraging the spatial locality and discovering the Gaussian intersection on image pixels, Grendel enables transferring Gaussians with sparse all-to-all communication while balancing the communication overloads. The load balancing can be simply adjusted according to the time estimation of rendering a pixel in previous iterations. While increasing the batch size needs adjusting the hyperparameters for optimizers, Grendel proposed an automatic hyperparameter scaling rule to enable the hyperparameter-tuning-free batched 3DGS training. Experiments conducted on the small-scale Mip-NeRF360, Tansk-and-Temples, and DeepBlending dataset, the large-scale Mega-NeRF dataset, and the large-scale MatrixCity data validated the effectiveness of the proposed method.", "questions": "(1) At line 206, the method splits an image into 16x16 pixel blocks. I'm curious whether the patch size of image blocks can influence the performance of image rendering quality.\n\n(2) Since this method enables training 3DGS from multiple views, it can gather gradients from multiple views and then better constrain the training of 3DGS (as has been shown in the paper *MVGS: Multi-view-regulated Gaussian Splatting for Novel View Synthesis*). However, I saw no performance gains using the distributed training. In lines 312-323, the paper claimed that \"even though some cameras share similar poses, a set of random cameras generally observe different parts of a scene, hence the gradients in a batch are mostly sparse and can be thought of as roughly independent.\". I think that might be the reason that the distributed training does not improve the rendering quality. Can the authors try to explain/discuss this further? For example, are there any non-random batch-splitting methods that can be applied to improve the performance?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "(1) This work enables the distributed training of 3DGS, which can be very useful to accelerate the training of 3DGS.\n\n(2) The paper is well-written and easy to understand.\n\n(2) This work did exhaustive experiments on very large-scale datasets (e.g., the MatrixCity dataset)", "weaknesses": "(1) The related works are put into the last section of the paper, which is weird to me. It lacks discussion of some existing distributed methods, such as **DOGS** (Chen and Lee, NeurIPS 2024), and RetinaGS(Li, et, al, arXiv 2024).\n\n(2) In Sec.6 Related Works, The discussion of VastGaussian, CityGaussian, and Hierarchical Gaussian is wrong. They do not merge the resulting images. Instead, they merge the sub-models. And the authors also claimed that \"None of these systems can consider a full large-scale scene directly and achieve the same quality as ours\". This is ***over-claimed and does not respect existing works since they did not compare to any of these methods*** but only tried different configurations of their Grendel methods.\n\n(3) Baselines are missing in the current version. It is a good system work to me. However, over-claiming something in the paper won't make it a better work. I would like to see a fair comparison of the method to more baselines (original 3DGS, VastGaussian, CityGaussian, DOGS, etc.) and raise my score if they do so.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FmEU7ietHv", "reviewer": "Reviewer_smer", "summary": "This paper proposes a parallel training pipeline for 3D Gaussian Splatting to be trained on up to 32 GPUs, instead of 1 GPU only. This significantly improves the limit of 3DGS in terms of primitive count, scene scale, scene detail, and training speed. This paper splits the 3DGS training process into per-Gaussian stage and per-pixel stage. In per-Gaussian stage, each GPU has its own Gaussians to calculate the Gaussian transformations, mostly 3D to 2D transformation and color evaluation. In per-pixel stage, each GPU is in charge of a set of pixels for rendering, and performs a sparse communication to fetch the required Gaussians from other GPUs. Lastly, a backward propagation is carried out in a similar manner to pass back the gradients. As a result, the proposed method demonstrates a significantly higher number of Gaussians supported, much faster training speed, and a moderate increase in rendering quality.", "questions": "1. Can the proposed method be applied to 3DGS with LOD design?\n2. How does the training across different machines instead of different GPUs affect the performance of the proposed paper?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "**Motivation**\n* This paper has a strong motivation of expanding the training of 3DGS into multiple GPUs/machines. The vanilla 3DGS has a limited number of primitives supported due to the GPU memory limit, but the exceptional performance of 3DGS indicates its great usefulness in large-scale reconstruction. This paper is monumental in providing support in terms of parallel training.\n\n**Method**\n* This paper correctly identifies the per-Gaussian and per-pixel stages of the training pipeline of 3DGS. Since it split the set of Gaussians onto each GPU, the first per-Gaussian stage is relatively simple. For the per-pixel stage, the paper provides intuitive and empirical support for the locality property of 3DGS for each pixel. The paper cleverly leverages this property to design a sparse communication to transport the 3DGS across GPU at a low speed and bandwidth sacrifice. This enables the distributed training of 3DGS.\n* This paper rigorously analyzed the scaling property of the 3DGS training process, and carefully designed a hyper-parameter setting paradigm based on the independent gradient assumption and empirical support. \n* To further improve the utilization of all GPUs, the load balancing algorithm is introduced to ensure each GPU can adaptively adjust their workload during training, because the number of Gaussians required for each region of the image cannot be predetermined.\n\n**Experiments**\n* This paper conducts extensive experiments across various scales of scenes under different experiment settings. The experiment result are well presented to demonstrate the improvement in primitive count, training speed, and rendering quality.", "weaknesses": "**Method**\n* The paper follows the vanilla 3DGS design and does not consider the Level of Detail support. Since this paper focuses on the large-scale reconstruction, LOD is essential for downstream applications. It is not clear how the proposed method can support LOD.\n\n**Experiment**\n* The quality improvement is not as significant as the number of primitives or the training speed. However, I would like to argue that this is because of the lack of an appropriate dataset to evaluate a very large-scale reconstruction, which limit the performance of this paper. \n* Although the training speed improvement is very significant with this parallel training, large scenes require very time-consuming pose estimation to be done first. The overall training time including the pose estimation might not be improving as much as the reported improvement. This is not the problem of this paper, but does weaken the impact of this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R5crdXEJ3q", "reviewer": "Reviewer_e5kX", "summary": "This paper proposes Grendel, a parallelized computation system for 3D Gaussian Splatting. Grendel introduces two parallelization strategies designed for different stages of the 3DGS rendering pipeline: partition by Gaussians and partition by pixels. The partition by Gaussians method distributes Gaussians within the point cloud across multiple GPUs, while partition by pixels assigns pixels across GPUs. Communication between these parallelization schemes is conducted through NCCL all2all. To enhance batch optimization, a learning rate scaling rule is proposed. Experiments demonstrate performance improvements through the utilization of more Gaussians compared to 3DGS approaches in Large scene datasets.", "questions": "1. How much does the rendering speed change?\n\n2. By how much does the actual GPU memory requirement per GPU reduce when using batch processing?\n\n3. What is the storage requirement based on the number of Gaussians?\n\n$~$\n\nMinor comments:\n\nThe figure reference is inconsistent in line 173.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Proposal of parallelization with respect to different stages of the rendering pipeline of 3DGS is interesting.\n\n2. With the proposed method, training 3DGS on high-resolution images is possible.\n\n3. Batch processing can significantly increase the training speed.", "weaknesses": "1. This paper lacks a conclusion and limitations, making it difficult to understand its overall contribution contribution and appear incomplete.\n\n2. Similar to the proposed learning rate scaling, RAdam [1] also suggests a method for variance rectification. It also rectifies the learning rate using variance to enable adaptive learning and, I believe, takes batch size into account. However, the advantage of the proposed method remains unclear.\n\n3. The rendering speed is expected to be slow due to the time required for Gaussian transfers between GPUs, however, the paper does not provide details regarding rendering speed.\n\n4. Based on Fig. 14, it appears that increasing batch size does not decrease the maximum number of Gaussians significantly.\n\n5. As shown in Fig. 11, for datasets that are not high-resolution like Rubble, the performance gain becomes marginal even if the number of Gaussians exceeds what can be handled by a single GPU.\n\n[1] Liu et al., *On the Variance of the Adaptive Learning Rate and Beyond*, ICLR 2020", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hIjNpgLMxE", "reviewer": "Reviewer_sLkS", "summary": "This paper introduces GrendelGS, which eliminates the bottleneck of 3DGS in parallel training and scaling up. GrendelGS mixes Gaussian-wise and pixel-wise parallelism, so as to fit computation requirements of different stages. It also makes an effort to minimize communication among GPUs and workload imbalance. And it reveals hyperparameter scaling law for 3DGS parallel training.", "questions": "1. The legends in Figure 1 are put in inversed order. A modification seems to be required.\n2. In Algorithm 1, the meaning of ET_j is confusing and I didn't find a clear definition. An annotation in the graph can optimize the reading experience.\n3. In the \"Independent Gradients Hypothesis\" section, why can we assume E(g_k)=0 over all views? This hypothesis is not that obvious.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 4, "strengths": "1. Originality.  A reasonable parallel training strategy with novel modification for 3DGS based on valuable insights.\n2. Quality. Sufficient experimental validation on effectiveness of the method.\n3. Significance. Pioneering and well-optimized parallel training implementation for 3DGS, resolving critical challenges in synchronized parallel 3DGS training. On the one hand, GrendelGS avoids tedious model pertaining or data and primitive partitioning in prior works like VastGaussian or CityGaussian. On the other hand, this work enables efficient training when multiple GPUs are available, especially under instant requirements or large-scale scenes.", "weaknesses": "1. While the superior efficiency over the original 3DGS is clearly and sufficiently illustrated through experiments, it would be beneficial to show if this synchronized parallel training mechanism shows higher efficiency over existing open-sourced asynchronized paradigms, such as HierarchyGS, OctreeGS, or CityGS. Note that for a fair comparison, the batch size and overall iterations should be aligned. Considering the limited time for rebuttal, a comparison with one existing method on 2-3 representative large-scale scenes is enough.\n2. The Independent Gradients Hypothesis is introduced to support the hyperparameter scaling law. It's reasonable that the hypothesis is true on Rubble or even other large-scale scenes, since the views are extremely sparse and approximately independent. However, for condensed views over small scenes or single objects, the hypothesis may not well suit the reality, especially when the batch size grows.  It seems more reasonable to limit the hypothesis to cases with significant view sparsity, or check if the scaling law brings worse results on small scenes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)