RetinaGS: Scalable Training for Dense Scene Rendering with Billion-Scale 3D Gaussians

## Abstract

Abstract In this work, we explore the possibility of training high-parameter 3D Gaussian splatting (3DGS) models on large-scale, high-resolution datasets. We design a general model parallel training method for 3DGS, named RetinaGS, which uses a proper rendering equation and can be applied to any scene and arbitrary distribution of Gaussian primitives. It enables us to explore the scaling behavior of 3DGS in terms of primitive numbers and training resolutions that were difficult to explore before and surpass previous state-of-the-art reconstruction quality. We observe a clear positive trend of increasing visual quality when increasing primitive numbers with our method. We also demonstrate the first attempt at training a 3DGS model with more than one billion primitives on the full MatrixCity dataset that attains a promising visual quality.

## 1 Introduction

Figure: Figure 1: Left: Different sizes of datasets require varying levels of computational power and numbers of 3DGS Primitives. Larger and higher-resolution datasets can no longer be trained using just a single GPU, which limits the pursuit of scale and fidelity in 3DGS reconstruction. Right: The billion-level model bring better visual experience than million-level model on MatrixCity-ALL dataset, which is trained via our distributed modeling with 64 GPUs.
Refer to caption: extracted/5685481/figures/Teaser_300dpi_crop.png

3D scene reconstruction with Gaussian Splatting (GS) [] has drastically improved rendering quality and rendering speed over previous neural 3D representation [, , ]. However, this success has been largely limited to reconstructing scenes with limited image or video resolution (typically $<=1600$ pixels wide), data volume, and view distance. Viewing the scene at high resolution or close range remains an unsolved challenge. To achieve imaging effects flawless to the human retina, which we refer to as the goal of retina-level reconstruction, we would desire to train the GS models with higher spatial resolution, larger datasets, and more varying viewing distances.

However, all these demands lead to more compute and memory costs. Fig. 1 illustrates the compute cost and memory footprint when training GS models across different rendering resolutions and different data volume on several datasets.
We can observe that high-resolution training indeed improves the rendered views’ visual quality. It also inevitably requires more splats, which means higher parameter counts. As the total compute of training grows roughly by the product of the number of pixels per view and the number of splats, training time on a single GPU will quickly become unacceptable. The increased splat numbers also lead to a growing memory footprint that could become infeasible for even the best single modern GPU. To enable effective scaling, we need a method for distributed GS training.

In this paper, we present a method to achieve efficient distributed training of GS models while retaining equivalence to the existing single GPU training scheme. We start by dividing the model space into a set of convex subspaces and identify one subset of splats for each subspace so that all the subsets collectively form an overlapping cover of the set of all splats. We then show that, with a proper subset assignment strategy, each ray’s original iterative alpha-blending process can be transformed into a hierarchical form, in which we can first compute partial color and alpha values for each subset and then obtain the rendered pixel color value by orderly merging these partial values.

We derive our training method by exploiting this property and designating one worker for each subset. In the forward process, all workers compute subset-level color and alpha values for all rays in parallel. The partial values, sufficient to compute rendered pixel values, can then be merged through cross-worker communication with a minimal message size. After computing the reconstruction loss and its gradients w.r.t. the partial values, we can distribute the corresponding gradients to each worker and run the backward process in parallel. We further improve training efficiency by utilizing KD-tree to produce subspaces that induce subsets with more even cardinality. This method allows us to efficiently train arbitrarily large GS models on multiple GPUs without any assumption on the reconstructed scene.

We validate the training method on multiple 3D reconstruction datasets with high training resolutions. The trained GS models outperform 3DGS[] on Mip-NeRF360[], Mega-NeRF [], ScanNet++[] and MatrixCity[] datasets. Through scaling study, we observe a clear tendency to improve visual quality when the GS models grow in splat numbers. Finally, we demonstrate to our knowledge the first attempt at training GS models with billion-scale primitives on city-level dataset with 140k images, resulting in unprecedented visual experience using million-scale primitives.

## 2 Related Work

##### Distributed Neural Radiance Fields.

NeRF [] has revolutionized 3D scene reconstruction and novel viewpoint generation with its photorealistic rendering capabilities, sparking a series of subsequent innovations focused on enhancing various aspects of the technology. These advancements strive to improve rendering quality [, , , ], optimize training speed [, , , ], increase memory efficiency [, , ] and expand the scale of reconstructed scenes [, , , , , , , ]. While these fields might benefit from larger models or enhanced computational power, it is primarily the pursuit of scaling scene sizes that has driven researchers to explore ways to speed up or scale up NeRF models. Switch-NeRF [] utilizes a Mixture of Experts (MoE) to increase the capacity of NeRF models, enabling them to represent urban-scale scenes effectively. Bungee-NeRF [] employs a hierarchical assembly of submodels, granting the model extensive multiscale representational capabilities. Recognizing the limitations imposed by GPU computational power and memory on model scale and training velocity, some researchers have begun deconstructing models into smaller components, adopting distributed approaches to address these challenges. Block-NeRF [] segments cities into multiple overlapping blocks, each represented by its own NeRF model, and uses neural networks to fuse the outputs of multiple NeRF models in image space, achieving seamless visual results. Mega-NeRF [] introduces a simple geometric clustering algorithm and partitions training pixels into various NeRF submodules, which can then be trained in parallel.

##### Distributed Point-Based Representation.

NeRF utilizes a volumetric rendering approach that inherently limits their inference speed, which makes it challenging to achieve real-time performance. Several studies [, , , ] have focused on optimizing inference speeds, yet achieving both real-time performance and high-quality rendering remains elusive. In contrast, the 3D Gaussian Splatting (3DGS) [] method, which employs a point-based scene representation, achieves state-of-the-art rendering quality and speed and has significant advantages in training speed. However, the "shallow and wide" structure of the 3DGS model and its explicit representation lead to a larger parameter count and increased training memory footprint. VastGaussian [] is the first attempt to large-scale scene reconstruction with 3DGS. It divides the ground plane into a series of 2D cells with one splat subset for each cell. Unlike our method, all subsets are mutually exclusive and considered as independent 3DGS models in training. To mitigate the deviation from the original 3DGS rendering equation at subspace boundaries, VastGaussian introduces additional nearby cameras and Gaussian primitives to assist in training at the cost of increased compute and memory. CityGaussian [] uses a similar partitioning strategy but trains an additional coarse GS model as the baseline to regularize submodels.
However, artifacts caused by deviation from the proper rendering equation could still intensify at certain view angles and overcome the mitigations. As a result, these approaches mostly assume bird-eye-view urban scenarios due to their simplicity in view angles and splat distribution. Our method is based on an equivalent form of the 3DGS rendering equation. Thus it can be applied to any scene with arbitrary splat distribution.

##### Distributed Deep Learning.

Our work is also related to distributed deep learning, which aims to scale up the training system for deep neural network models [, , ]. Early approaches revolve around training multi-GPU convolutional neural networks (CNNs) [] on image [] and video [] datasets, with data parallelism being highly efficient thanks to CNNs’ high compute to bandwidth ratio. Recent development of large language models presented new challenges in distributing model parameters and larger clusters, resulting in dedicated model parallelism [] approaches for Transformers [] models, such as tensor-parallelism [], pipeline parallelism [], or hybrid parallelism [, , ]. Fully sharded data parallelism [], a type of redundancy-free data parallelism [], also works great with LLM training. Our method is best described as model parallelism as it partitions GS model parameters onto multiple workers to distribute workload and incurs communication cost only for Gaussian on the subspace boundaries. This is well suited to GS models’ relatively low compute-to-parameter ratio(^1^11also known as operational intensity [].), which makes data parallelism less efficient.

## 3 Approach

### 3.1 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) utilizes a series of anisotropic 3D Gaussian primitives to explicitly characterize scenes. Each Gaussian primitive, known as a splat, is defined by its central position $\mu\in\mathbb{R}^{3}$, and a covariance matrix $\Sigma\in\mathbb{R}^{3\times 3}$. A splat’s influence at any given point $x$ within the scene’s world coordinate system is attenuated by the Gaussian function

$$ $G(x)=e^{-\frac{1}{2}(x-\mu)^{T}\Sigma^{-1}(x-\mu)}.$ (1) $$

In practice, the function is truncated to save computation.
Each splat also carries an opacity $\alpha\in\mathbb{R}$. Its color attributes $F\in\mathbb{R}^{C}$ are expressed through spherical harmonics (SH) $c\in\mathbb{R}^{3}$ to allow view-dependent textures. A view is rendered through rasterization the Gaussian primitives onto a 2D imaging plane, during which the 3D Gaussians are projected to 2D Gaussians $G^{\prime}(x^{\prime})$ through Jacobian linearization as described in [].
As a result, the influence weight of each 3D primitive on a given ray $l$ needs to be computed through path integration, whereas a 2D primitive only requires a single sampling, that is, $g(l)=G^{\prime}(x^{\prime})$, where $x^{\prime}$ is the intersection point of the ray $l$ with the 2D imaging plane.
By employing alpha-blending, these primitives are rendered with the following rendering equation

$$ $C(l)=\sum_{g_{i}\in{N_{l}}}c_{i}\sigma_{i}\prod_{g_{j}\in N_{l},~{}j<i}(1- \sigma_{j}),\quad\sigma_{i}=\alpha_{i}g_{i}(l),$ (2) $$

where ${N_{l}}$ denotes the set of Gaussian primitives that contribute to ray $l$, arranged in order of their depth.

### 3.2 Distributed Training of 3DGS

Given the set of all splats $N=\{g_{i}\}$ and $K$ workers $\{w_{1},\ldots,w_{K}\}$, we aim to devise a distributed training method with minimal communication overhead that still conforms to the rendering equation in Eq. 2. We first divide the scene space into a set of convex subspaces $S_{1},\ldots,S_{K}$. One worker is expected to only work on a subspace and incur minimal communication needs. To achieve this, we generate $K$ subsets $N_{1},\ldots,N_{K}$ and $N_{1}\cup N_{2}\cup...\cup N_{K}=N$, which are allowed to overlap. Below we show how to create $N_{k}$ and manipulate Eq. 2 for distributed computation for the workers.

Given a ray $l$ and a subspace $S_{k}$, we can always obtain a subset of $N$ as $N_{lk}$ which denotes all splats that, when projected to a truncated 2D Gaussian according to $l$, intersects with $l$ within $S_{k}$. Note since the Gaussians are truncated, not all splats will intersect with $l$.
Since $N_{lk}$ is ray dependent, we define $N^{*}_{k}=\cup_{l}N_{lk}$ as the union of $N_{lk}$ for all possible rays.
Going through every possible ray to obtain $N^{*}_{k}$ is infeasible.
However, we know that any intersection point must reside on its corresponding 3D ellipse. So, we can instead define $N_{k}$ as the set of splats whose corresponding 3D ellipsoid intersects with $S_{k}$. Then it is obvious that $N^{*}_{k}\subseteq N_{k}$ and $N_{k}$ is not dependent on any specific ray.
To render the color for a given ray, We can first calculate the partial color $C_{k}(l)$ and the partial opacity $T_{k}(l)$ on $N_{k}$ as

$$ $\displaystyle C_{k}(l)$ $\displaystyle=\sum_{g_{i}\in N_{k}}c_{i}\sigma_{i}\mathbbm{1}(g_{i}\in N_{lk}) \prod_{g_{j}\in N_{k},~{}j<i}(1-\sigma_{j}\mathbbm{1}(g_{j}\in N_{lk})),$ $\displaystyle=\sum_{g_{i}\in N_{lk}}c_{i}\sigma_{i}\prod_{g_{j}\in N_{lk},~{}j <i}(1-\sigma_{j})$ (3) $\displaystyle T_{k}(l)$ $\displaystyle=\prod_{g_{i}\in{N}_{k}}(1-\sigma_{i}\mathbbm{1}(j\in N_{lk}))$ $\displaystyle=\prod_{g_{i}\in{N}_{lk}}(1-\sigma_{i}),$ (4) $$

where $\sigma_{i}=\alpha_{i}g^{k}_{i}(l)$ and $\mathbbm{1}(\cdot)$ denotes the indicator function which equals to 1 when the condition specified is true and otherwise equals to 0.
Note splats in $N_{k}$ are ordered by their distance to the origin of $l$ to attain the index $i,~{}j$ in Eq. 3 and Eq. 4. The introduction of the indicator function allows the accumulation to be carried on the entire $N_{k}$ for any ray $l$.
The partial values can then be merged in the order that the ray passes through them, which can be represented as a permutation $o_{l}$ of size $K$:

$$ $C(l)=\sum_{k\in o_{l}}C_{k}(l)\prod_{m\in o_{l},~{}m<k}T_{m}(l).$ (5) $$

Substituting Eq. 3 and 4 into Eq. 5, we can observe that Eq. 2 is equivalent to Eq.  5 for any $l$. The relevant proof process has been placed in the appendix.
Therefore we have transformed the rendering equation of 3DGS into the independent computation of subset-level partial colors and opacities and the subsequent merging. All workers can compute their corresponding partial values in parallel for each subset $N_{k}$ and perform the merge step through cross-worker communication. The result will be identical to when rendered on a single worker.

Evaluating the condition of $g_{i}\in N_{lk}$. Since the subspaces $S_{1},S_{2},\ldots,S_{K}$ seamlessly partition the entire space and each subspace is convex, it necessitates that all dividing surfaces are planar. Each subspace $S_{k}$ can be represented by a set of plane constraints:
$S_{k}=\{x\in\mathbb{R}^{3}:n^{\top}_{km}\cdot x+d_{km}\leq 0,\text{ for all }m\}$
, which allows us to transform the indicator function $\mathbbm{1}$ in Eq 3 into a form more amenable to computation:

$$ $\displaystyle\mathbbm{1}(g_{i}\in N_{lk})=\mathbbm{1}(x_{i}\in S_{k})=\prod_{m }\mathbbm{1}(n^{\top}_{km}\cdot x_{i}+d_{km}\leq 0),$ (6) $$

where $x_{i}$ represents the world coordinates of the intersection between the ray $l$ and the 2D Gaussian primitive $G^{\prime}_{i}$. Expressing the ray $l$ as the equation $l(t)=o+td$, which is defined by the camera center $o\in\mathbb{R}^{3}$ and the unit ray direction $d\in\mathbb{R}^{3}$, then $x_{i}$ precisely equals the projection center point $u_{i}$ of $g_{i}$ onto $l$, which can be represented as:
$x_{i}=u_{i}+(d^{\top}\cdot(o-u_{i}))d$.

### 3.3 Distributed Training with Sub-models

To balance each subset’s size and each worker’s workload, we employ a KD tree to determine the partition of the subspaces $\{S_{k}\}$. Initially, we construct a three-dimensional KD tree with a depth of $L$ using the center coordinates of Gaussian primitives. The KD tree recursively uses hyperplanes perpendicular to the X, Y, and Z axes to bisect the space and equally divided primitives, ultimately resulting in a series of rectangular subspaces $S_{1},S_{2},\ldots,S_{K}$, where $K=2^{L}$. Then, the corresponding subsets for each worker can be derived as:
$N_{k}=\{i:n^{\top}_{km}\cdot u_{i}+d_{km}\leq D_{i},\text{ for all }m\}$, where $u_{i}$ is the center of primitives and $D_{i}$ is the truncation threshold for $g_{i}$. In our implementation, we set $D_{i}$ to be three times the length of the major axis of the Gaussian ellipsoid.
Appendix.Fig 10 provides an intuitive illustration of how the primitives near the subspace boundary work.

### 3.4 The Complete Training Pipeline

Figure: Figure 2: By employing planes generated using KD-Tree, we spatially partitioned the initial 3DGS model to a set of sub-models. These sub-models share certain primitives only when these primitives cross boundaries. The rendering results of sub-models are then merged to form the final rendered image. After the loss is computed uniformly, the corresponding gradients are returned to each sub-model to update their primitive parameters.
Refer to caption: extracted/5685481/figures/pipeline_1500dpi.png

The actual training pipeline is shown in Figure 2. We denote each subset as a sub-model and assign it to a separate GPU, while a central manager is responsible for managing the KD-Tree and the subspaces $\{S_{n}\}$. This manager also handles the parsing of incoming rendering requests and distributes rendering tasks to the relevant sub-models. The computational results from all sub-models, $\{T_{k}\}$ and $\{C_{k}\}$, are sent back to the central manager. These results can be represented in a 2D map with the same resolution as the target image, which consumes only minimal communication bandwidth. The central manager then completes the final rendering based on Eq 5, calculates the loss, and sends the gradient back to each sub-model for model parameter updates. After a predetermined number of training epochs, we repeat the partitioning process to accommodate any significant shift of primitive centers. Note that primitives belonging to intersections of multiple subsets require gradient synchronization after each training step. Empirically, we found this could be omitted without significantly affecting reconstruction quality.

#### 3.4.1 Primitive Initialization

In the original 3D Gaussian Splatting (3DGS) approach, initialization and densification processes involved numerous heuristic strategies and hyperparameters to ensure that Gaussian primitives were appropriately positioned. Such strategies made it difficult to control the number of primitives, impeding our ability to effectively scale the model size and leverage the advantages of distributed modeling to enhance rendering fidelity. Furthermore, we found that the densification process does not always promote an increase in primitives in high-resolution training, which makes it counterproductive.

In this paper, we adopt a simple yet effective strategy for initialization as shown in Fig 2 and do not adjust the number afterwards. We performed Multi-View Stereo (MVS) on all training data to obtain depth estimations for all training viewpoints, which were then transformed into dense 3D point clouds. These point clouds could be flexibly sampled and initialized as Gaussian primitives. This streamlined approach allowed us to control the number of primitives as desired. It is also easier to balance workers’ workloads thanks to a predefined number of primitives.

## 4 Experiments

##### Datasets.

Our performance evaluation spanned 4 datasets that comprising indoor and outdoor scenes. These datasets include all scenes from full-resolution MipNeRF-360 []. We extended our analysis to high-resolution scenarios based on the ScanNet++ dataset [], with a focus on scenes labeled 108ec0b806 and 8133208cb6. After distortion correction, two scenes provided 863 and 476 high-resolution images respectively (8408 pixels wide).
Furthermore, we conducted tests in large-scale environments, including the Residence, Building, and Rubble datasets from Mega-NeRF [], as well as the entire MatrixCity Small City Aerial dataset (referred to as MatrixCity-Aerial, 1920$\times$1080), consisting of 6,362 images. We also utilized the MatrixCity Small City Dense Street dataset, sampling 5 angles every meter along the centerline of streets to gather 135,290 images (1000$\times$1000). From the Small City Dense Street dataset, we selected a focused test set of 2,480 images (referred to as MatrixCity-M). By combining all images from Aerial and Dense Street (referred to as MatrixCity-ALL), we conducted a comprehensive Billion GS level reconstruction. We followed the official Train/Test splits for Mega-NeRF and MatrixCity-Aerial, and used every eighth image for testing in other datasets as recommended by MipNeRF-360.
To obtain superior MVS initial points, we reran Colmap’s sparse reconstruction based on full-resolution images of MipNeRF-360 and ScanNet++ to obtain poses (using official provided poses of Mage-NeRF and MatrixCity), and subsequently performed dense reconstruction [] on all datasets except MatrixCity-ALL. For MatrixCity-ALL, we replaced depth estimations of MVS with official provided depth estimations to avoid super long time cost of Colmap’s dense reconstruction (using MVS for MatrixCity-M and MatrixCity-Aerial).

##### Baselines and Metrics.

Our comparisons include 3DGS and NeRF-related works. For fair comparisons, results from an equal number of iterations from our own 3DGS runs are also presented. We primarily assessed the rendered image quality using three metrics consistent with 3DGS: PSNR, SSIM, and LPIPS.

##### Implementation Details.

Our method is based on 3DGS. We extended the number of training iterations to 60k on MipNeRF-360, ScanNet++ and Maga-NeRF and 20 epochs on all MatrixCity datasets for both 3DGS and ours to ensure adequate convergence. We do not adjust the number of primitives during training. Since the primitives are initialized with relatively accurate position parameters from MVS, we reduce the learning rate for the position parameters in all primitives from from $1.6\times 10^{-4}$ to $1.6\times 10^{-6}$ with a exponential decay function, which is 1/100th of the original setting in 3DGS. All experiments are conducted on NVIDIA A100 GPUs.

### 4.1 Experimental Results

Figure: (a) Garden
Refer to caption: extracted/5685481/figures/convergence_curve_garden_nearest_neighbor_interpolation.png

Fig 3presents our quantitative results on the MipNeRF-360 (Garden), ScanNet++ (108ec0b806), and MatrixCity-M with various training resolution. To standardize the metrics and align them more closely with direct visual perception, we upsample all rendered results from different resolutions to the full resolution using nearest-neighbor upsampling and evaluate on the full resolution. Utilizing MVS for initializing Gaussian points and disabling densification allows us to easily control the number of model points. Furthermore, our distributed training approach enables the use of a large number of Gaussian primitives. We observed a strong positive correlation between the number of Gaussian primitives and the final model’s PSNR. When the number of primitives is similar, our PSNR closely matches the quality of 3DGS trained on a single GPU; however, we can effectively achieve higher PSNRs by simply increasing the number of primitives. Table 1 shows results on more datasets, uniformly showing that our method consistently performs better across all datasets, especially on high-resolution and large-scale datasets.

Figure: Figure 4: Visualization of models and with various number of primitives and training resolution on Garden and ScanNet++ dataset (top and bottom metrics: training resolution/splats count/PSNR). As we get close to objects or zoom into camera, higher training resolutions and more primitives help maintain rendering clarity and reveal more details, which bring better visual experience and better quantitative results than the 3DGS baseline.
Refer to caption: extracted/5685481/figures/cases2_750dpi.png

We further analyze the relationship between the number of Gaussian points and subjective visual effects, as illustrated in Fig. 4. As revealed in Fig. 3, at a fixed number of GS primitives, higher resolutions yield better image quality; similarly, at a fixed high resolution, an increased number of primitives enhances image quality. We believe that the number of primitives determines the capacity of the 3DGS model, while higher image resolutions bring a greater amount of information, necessitating a larger model capacity to achieve adequate fitting. Therefore, to ensure a Retina quality effect, both high resolution and high-quality rendering are essential, which in turn imposes substantial demands on the number of GS primitives.

**Table 1: PSNR vs. #GS analysis. * indicates that the results are average values across multiple scenes. See Appendix for complete results.**
| Datasets | MatrixCity-Aerial | MatrixCity-M | Mega-NeRF* | ScanNet++* | MipNeRF-360* |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Pixels | 13.19B | 2.48B | 2.25B | 33.44B | 2.32B |  |  |  |  |  |
| Metrics | PSNR | #GS | PSNR | #GS | PSNR | #GS | PSNR | #GS | PSNR | #GS |
| 3DGS^† | 26.56 | 25.06M | 27.81 | 1.53M | 22.56 | 6.57M | 28.42 | 1.87M | 27.33 | 3.02M |
| Ours | 27.70 | 217.30M | 31.12 | 62.18M | 23.03 | 35.57M | 28.91 | 39.89M | 27.78 | 27.79M |

### 4.2 Exploration study

Initialization and Densification. In practice, we observed issues with the original 3DGS’s point-growing strategy, as shown in Appendix.Fig 11. Excessive iterations lead to deteriorating results. Using an aggressive densification strategy in 3DGS did not yield better outcomes, and excessive point splitting made the training unstable. In Fig 5, we show that initialization via MVS results in a more stable training run and better model quality. By simply initializing using MVS with more primitives, our model surpasses the original 3DGS even with careful tuning.

Figure: (a) Garden
Refer to caption: extracted/5685481/figures/PM_Garden.png

Validity of Distribute Rendering we devise a simple test to validate the correctness of the underlying rendering equation in our method. We precisely positioned the camera’s optical axis on the dividing plane between two subspaces sharing this plane, ensuring that each ray passes through only one subspace. In this case, each image pixel should only be rendered by primitives on one side of the plane. If we do not perform the step, we would expect a crisp boundary between color pixels and completely dark pixels on the partially rendered images from the two subspaces. As shown in Fig. 6, our method exhibits the expected behavior while baseline partition approaches [, ] that deviates from the proper rendering equation fails the test.

Figure: Figure 6: Our two submodels produced distinctly bounded outputs, and result in better PSNR results. In contrast, spatial-based partitions exhibited blurry boundaries that could not be entirely eliminated by comprehensive end-to-end refinement, demonstrating the superiority of our approach.

##### KD-Tree Partition vs. Fixed-size Partition.

We compare our KD-Tree partitioning approach to a naive grid division strategy that uniformly divides the scene into blocks of fixed size in Table 2. The KD-Tree effectively balances the number of primitives in each sub-model, resulting in an optimal balance in peak memory usage and the best training speed. KD-tree is used throughout all experiments work without further notice.

**Table 2: Partition efficiency. Range of primitive count, memory usage, total time, and communication time of submodels under different partition strategies in distributed training. The fixed-size division resulted in sub-models with significantly varied number of primitives and computation times, which increased time spent blocking on waiting, thereby reducing the overall system efficiency.**
| Partition | Batch Size | GPU | Primitives (M) | Mem.(GiB) | Total Time (s) | Comm. (s) |
| --- | --- | --- | --- | --- | --- | --- |
| KD-Tree | 2 | 2 | 2.87$\sim$2.88 | 6.26$\sim$6.32 | 26.74$\sim$26.76 | 4.39$\sim$6.04 |
| Fixed-size | 2 | 2 | 0.81$\sim$4.93 | 3.57$\sim$9.01 | 32.45$\sim$32.52 | 1.04$\sim$23.83 |
| KD-Tree | 4 | 4 | 1.45$\sim$1.47 | 5.26$\sim$5.40 | 18.87$\sim$18.90 | 5.33$\sim$7.34 |
| Fixed-size | 4 | 4 | 0.05$\sim$4.79 | 3.08$\sim$10.95 | 32.26$\sim$32.41 | 4.77$\sim$26.83 |

##### Model parallelism vs. Data parallelism.

We compare data parallel training and training with our method, which is model parallelism, on the MipNeRF-360-garden dataset.
As shown in Table 4, our method achieve lower peak memory usage and higher training throughput. It is evident that even with such a small GS model, our method maintains its advantages over DP and single GPU training. However, as the size of submodels decreases and their quantity increases, it becomes increasingly challenging for MP to achieve workload balance. As shown in Table 4, a straightforward solution is to increase the batch size, which statistically leads to a more balanced workload.

**Table 3: Parallel mechanism efficiency. Efficiency of data parallel(DP) and our model parallel(MP) with various setting.**
| Parallel | GPU | Batch Size | Mem. | Time | Comm. |
| --- | --- | --- | --- | --- | --- |
| - | 1 | 1 | 8.33 | 32.35 | 0 |
| DP | 2 | 2 | 9.66 | 45.36 | 29.22 |
| MP | 2 | 2 | 6.26 | 26.76 | 6.038 |
| DP | 4 | 4 | 9.66 | 30.62 | 21.56 |
| MP | 4 | 4 | 5.26 | 18.90 | 7.34 |
| DP | 8 | 8 | 9.66 | 20.38 | 16.51 |
| MP | 8 | 8 | 5.05 | 17.738 | 12.73 |

## 5 Training Billion-Scale 3DGS

Our distributed modeling method enables the GS model to scale up to extremely large sizes. Typically, scenes in previous work are trained on a single GPU using 10 to 200 million pixels with a GS model with millions of primitives. In this work, we use 64 A100 GPUs for 10 days to train a 3DGS model with a billion primitives on the MatrixCity-ALL dataset, which includes 141,652 images containing over 148 billion pixels. There have been no previous reports of successful training at this scale of training set or model size.
After training the model with our method for 20 epochs, billion-scale model achieve superior quantitative results compared to million-scale model as shown in Table. 5.
We provide more visualization results in the Fig 7.

Figure: Figure 7: 1 Billion-Scale vs. 100 Million-Scale vs. 10 Million-Scale 3DGS Model on Maxtricity-ALL Dataset.
Refer to caption: extracted/5685481/figures/city/5_GT.png

**Table 5: Qualitative results on MatrixCity-All. More primitives can effectively improve performance, although it requires more GPUs.**
| Datasets | MatrixCity-All |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Pixels | 148.24B |  |  |  |  |
| Metrics | SSIM$\uparrow$ | PSNR$\uparrow$ | LPIPS$\downarrow$ | #GS | GPU |
| Ours-10M | 0.608 | 16.53 | 0.536 | 10.00M | 1 |
| Ours-100M | 0.761 | 23.07 | 0.397 | 100.00M | 8 |
| Ours-1B | 0.815 | 25.50 | 0.282 | 1023.13M | 64 |

## 6 Conclusion and Limitation

In this paper, we study the problem of distributed training of 3DGS models. We devise a model parallelism-based training method that utilizes a proper rendering equation to avoid artifacts. This allows us to significantly expand the model scale in terms of primitive numbers and seamlessly support large-scale scene reconstruction and detailed rendering. We also explore training 3DGS model at unprecedented scales. Despite the promising results of the method, load balancing among sub-models remains a challenging problem when the worker number increases. Additionally, the MVS used for primitive initialization has a relatively low throughput, warranting a study of a better initialization method.

## 7 Appendix

### 7.1 More Visualizations

Figure: Figure 8: Comparisons of high-primitive-number RetinaGS models and 3DGS baseline on ScanNet++ dataset. Note the superior rendering quality, especially on high-frequency textures like text and leaves.
Refer to caption: extracted/5685481/figures/scannet/108ec/DSC07681_gt.png

Figure: Figure 9: Blending results at the sub-space boundaries for different partition approaches. Approximate methods leak through the boundaries in partial rendering, resulting in obvious artifacts after merging. RetinaGS uses the equivalent form the 3DGS rendering equation, so it does not have this issue and shows no artifacts in the final rendering.

Figure: Figure 10: In RetinaGS, one primitive may be involved in partial value computation of more than one overlapping subsets, but the indicator function in Eq 6 ensures sure its color and opacity will only take effect once. For each ray, it can be illustrated as each subspace only possesses a fraction of the Gaussian ellipsoid, and the fragments collaboratively and distributively accomplish the task of a single primitive.
Refer to caption: extracted/5685481/figures/disorder3.png

Figure: Figure 11: In 3DGS, as the densification threshold decreases, the growth rate of GS increases. However, due to the rapid growth rate, floaters are generated in the model that cannot be eliminated through training (marked with red rectangle), which deteriorates the reconstruction effect. The MVS initialization strategy introduced in RetinaGS will mitigate this issue.
Refer to caption: extracted/5685481/figures/PM_Failed_crop.png

### 7.2 Equivalence Derivation

Starting from Eq. 3 for partial color $C_{k}(l)$:

$$ $C_{k}(l)=\sum_{g_{i}\in N_{lk}}c_{i}\sigma_{i}\prod_{g_{j}\in N_{lk},~{}j<i}(1 -\sigma_{j}),$ $$

Eq. 4 for partial opacity $T_{k}(l)$:

$$ $T_{k}(l)=\prod_{g_{i}\in{N}_{lk}}(1-\sigma_{i}),$ $$

and Eq. 5 for merging formula:

$$ $C(l)=\sum_{k\in o_{l}}C_{k}(l)\prod_{m\in o_{l},~{}m<k}T_{m}(l),$ $$

we substitute the expressions for partial color and partial opacity into the merging formula:

$$ $C(l)=\sum_{k\in o_{l}}\left(\sum_{g_{i}\in N_{lk}}c_{i}\sigma_{i}\prod_{g_{j} \in N_{lk},~{}j<i}(1-\sigma_{j})\right)\prod_{m\in o_{l},~{}m<k}\left(\prod_{g _{j}\in{N}_{lm}}(1-\sigma_{j})\right).$ $$

Now we expand this expression step by step. First, applying the distributive property of multiplication, we can pull out the inner summation symbol:

$$ $C(l)=\sum_{k\in o_{l}}\sum_{g_{i}\in N_{lk}}c_{i}\sigma_{i}\left(\prod_{g_{j} \in N_{lk},~{}j<i}(1-\sigma_{j})\right)\left(\prod_{m\in o_{l},~{}m<k}\prod_{g _{j}\in{N}_{lm}}(1-\sigma_{j})\right).$ $$

Then, utilizing the commutative property of multiplication, we can rewrite the above equation as:

$$ $C(l)=\sum_{k\in o_{l}}\sum_{g_{i}\in N_{lk}}c_{i}\sigma_{i}\left(\prod_{g_{j} \in N_{l1}}(1-\sigma_{j})\right)...\left(\prod_{g_{j}\in N_{l(k-1)}}(1-\sigma_ {j})\right)\cdot\left(\prod_{g_{j}\in N_{lk},~{}j<i}(1-\sigma_{j})\right).$ $$

Since $N_{l}$ in Eq 2 and $N_{lk}$ in Eq 3 follow a consistent rule for the ordering of elements, we have $N_{l}=(N_{l1},N_{l2},...N_{lK})$. This allows us to reduce the product symbols to one:

$$ $C(l)=\sum_{k\in o_{l}}\sum_{g_{i}\in N_{lk}}c_{i}\sigma_{i}\left(\prod_{g_{j} \in N_{l},~{}j<i}(1-\sigma_{j})\right).$ $$

Similarly, nested summations can also be simplified into one:

$$ $C(l)=\sum_{g_{i}\in N_{l}}c_{i}\sigma_{i}\left(\prod_{g_{j}\in N_{l},~{}j<i}(1 -\sigma_{j})\right),$ $$

which has the exact same form as Eq 2. Now we prove the original rendering equation of 3DGS is equivalent to the hierarchical form underlying the distributed training framework of RetinaGS.

### 7.3 More Experimental Results

In the main text, for the sake of brevity, we did not present the experimental results for each individual scene on several datasets. Instead, we provided the average results across all scenes contained in each dataset (as highlighted with * in the experimental tables of the main text). Here, we present the detailed experimental data for each individual scene.

**Table 6: Experimental results(60K iterations) on MatrixCity. R & P means resolution and training pixels.**
| Datasets | MatrixCity-Aerial | MatrixCity-M |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| R & P | 1920$\times$1080 & 13.19B | 1000$\times$1000 & 2.48B |  |  |  |  |  |  |
| Metrics | SSIM$\uparrow$ | PSNR$\uparrow$ | LPIPS$\downarrow$ | #GS | SSIM$\uparrow$ | PSNR$\uparrow$ | LPIPS$\downarrow$ | #GS |
| 3DGS | 0.833 | 26.56 | 0.244 | 25.06M | 0.851 | 27.81 | 0.271 | 1.53M |
| Ours | 0.840 | 27.70 | 0.177 | 217.3M | 0.932 | 31.12 | 0.110 | 62.18M |

**Table 7: Experimental results(60K iterations) on Mega-NeRF.**
| Datasets | Residence | Rubble | Building |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Metrics | SSIM$\uparrow$ | PSNR$\uparrow$ | LPIPS$\downarrow$ | SSIM$\uparrow$ | PSNR$\uparrow$ | LPIPS$\downarrow$ | SSIM$\uparrow$ | PSNR$\uparrow$ | LPIPS$\downarrow$ |
| Mega-NeRF | 0.628 | 22.08 | 0.489 | 0.553 | 24.06 | 0.516 | 0.547 | 20.93 | 0.504 |
| Switch-NeRF | 0.654 | 22.57 | 0.457 | 0.562 | 24.31 | 0.496 | 0.579 | 21.54 | 0.474 |
| GP-NeRF | 0.661 | 22.31 | 0.448 | 0.565 | 24.06 | 0.496 | 0.566 | 21.03 | 0.486 |
| 3DGS | 0.751 | 21.43 | 0.274 | 0.709 | 24.47 | 0.337 | 0.723 | 21.74 | 0.302 |
| Ours | 0.781 | 21.87 | 0.217 | 0.760 | 25.09 | 0.234 | 0.754 | 22.14 | 0.227 |

**Table 8: PSNR vs. Primitive numbers(60K iterations) on Mega-NeRF.**
| Datasets | Residence | Rubble | Building |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| R & P | 1368$\times$912 & 3.19B | 1152$\times$864 & 1.64B | 1152$\times$864 & 1.91B |  |  |  |
| Metrics | PSNR | #GS | PSNR | #GS | PSNR | #GS |
| 3DGS | 21.43 | 6.42M | 24.47 | 4.7M | 21.44 | 8.6M |
| Ours | 21.87 | 51.41M | 25.09 | 27.9M | 22.14 | 27.4M |

**Table 9: Experimental results(20 epochs) on MatrixCity. ScanNet++.**
| Datasets | 108ec0b806 | 8133208cb6 |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| R & P | 8408$\times$5944 & 43.13B | 8408$\times$5935 & 23.75B |  |  |  |  |  |  |
| Metrics | SSIM$\uparrow$ | PSNR$\uparrow$ | LPIPS$\downarrow$ | #GS | SSIM$\uparrow$ | PSNR$\uparrow$ | LPIPS$\downarrow$ | #GS |
| 3DGS | 0.881 | 28.95 | 0.408 | 2.65M | 0.908 | 27.89 | 0.355 | 1.09M |
| Ours | 0.883 | 29.71 | 0.395 | 47.59M | 0.908 | 28.11 | 0.340 | 32.19M |

**Table 10: #GS for Mip-NeRF360 scenes under 1.6K resolution and full resolution. The full marks the results obtained in full resolution. The dagger †marks the results obtained in our own experiments (60K iterations).**
| Method | Scenes | bicycle | garden | flowers | stump | treehill | room | counter | kitchen | bonsai |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Resolution-Wide | 4946 | 5187 | 5025 | 4978 | 5068 | 3114 | 3115 | 3115 | 3118 |  |
| Pixels-full | 3.15B | 3.22B | 2.87B | 2.05B | 2.37B | 2.00B | 1.55B | 1.80B | 1.89B |  |
| 3D-GS^† | 7.03M | 6.92M | 4.11M | 5.34M | 4.17M | 1.80M | 1.27M | 1.95M | 1.05M |  |
| Ours | 31.67M | 62.94M | 20.53M | 15.32M | 22.75M | 22.41M | 22.83M | 28.31M | 23.43M |  |
| 3D-GS^†-full | 5.04M | 7.39M | 3.11M | 3.30M | 2.61M | 1.70M | 1.20M | 1.91M | 0.97M |  |
| Ours-full | 31.67M | 62.94M | 20.53M | 15.32M | 22.75M | 22.41M | 22.83M | 28.31M | 23.43M |  |

**Table 11: SSIM scores for Mip-NeRF360 scenes under 1.6K resolution and full resolution. The full marks the results obtained in full resolution. The dagger †marks the results obtained in our own experiments (60K iterations).**
| Method | Scenes | bicycle | garden | flowers | stump | treehill | room | counter | kitchen | bonsai |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 3D-GS | 0.771 | 0.868 | 0.605 | 0.775 | 0.638 | 0.914 | 0.905 | 0.922 | 0.938 |  |
| 3D-GS^† | 0.770 | 0.866 | 0.623 | 0.771 | 0.641 | 0.931 | 0.919 | 0.933 | 0.950 |  |
| Mip-NeRF360 | 0.685 | 0.813 | 0.583 | 0.744 | 0.632 | 0.913 | 0.894 | 0.920 | 0.941 |  |
| iNPG | 0.491 | 0.649 | 0.450 | 0.574 | 0.518 | 0.855 | 0.798 | 0.818 | 0.890 |  |
| Plenoxel | 0.496 | 0.606 | 0.431 | 0.523 | 0.509 | 0.841 | 0.759 | 0.648 | 0.814 |  |
| Ours | 0.776 | 0.870 | 0.624 | 0.777 | 0.650 | 0.935 | 0.927 | 0.941 | 0.958 |  |
| 3D-GS^†-full | 0.732 | 0.817 | 0.604 | 0.795 | 0.696 | 0.922 | 0.917 | 0.926 | 0.941 |  |
| Ours-full | 0.751 | 0.827 | 0.635 | 0.800 | 0.703 | 0.927 | 0.925 | 0.932 | 0.947 |  |

**Table 12: PSNR scores for Mip-NeRF360 scenes under 1.6K resolution and full resolution. The full marks the results obtained in full resolution. The dagger †marks the results obtained in our own experiments (60K iterations).**
| Method | Scenes | bicycle | garden | flowers | stump | treehill | room | counter | kitchen | bonsai |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 3D-GS | 25.25 | 27.41 | 21.52 | 26.55 | 22.49 | 30.63 | 28.70 | 30.32 | 31.98 |  |
| 3D-GS^† | 25.33 | 27.58 | 21.85 | 26.74 | 22.46 | 31.93 | 29.54 | 31.52 | 33.10 |  |
| Mip-NeRF360 | 24.37 | 26.98 | 21.73 | 26.40 | 22.87 | 31.63 | 29.55 | 32.23 | 33.46 |  |
| iNPG | 22.19 | 24.60 | 20.34 | 23.63 | 22.36 | 29.27 | 26.44 | 28.55 | 30.34 |  |
| Plenoxel | 21.91 | 23.49 | 20.09 | 20.66 | 22.24 | 27.59 | 23.62 | 23.42 | 24.67 |  |
| Ours | 25.41 | 27.74 | 21.94 | 26.86 | 22.67 | 32.86 | 29.91 | 32.49 | 34.09 |  |
| 3D-GS^†-full | 24.47 | 26.67 | 20.78 | 26.23 | 22.34 | 31.57 | 29.33 | 31.72 | 32.92 |  |
| Ours-full | 24.86 | 27.06 | 21.58 | 26.58 | 22.42 | 31.65 | 29.90 | 32.23 | 33.76 |  |

**Table 13: LPIPS scores for Mip-NeRF360 scenes under 1.6K resolution and full resolution. The full marks the results obtained in full resolution. The dagger †marks the results obtained in our own experiments (60K iterations).**
| Method | Scenes | bicycle | garden | flowers | stump | treehill | room | counter | kitchen | bonsai |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 3D-GS | 0.205 | 0.103 | 0.336 | 0.210 | 0.317 | 0.220 | 0.204 | 0.129 | 0.205 |  |
| 3D-GS^† | 0.200 | 0.107 | 0.320 | 0.223 | 0.317 | 0.186 | 0.172 | 0.112 | 0.168 |  |
| Mip-NeRF360 | 0.301 | 0.170 | 0.344 | 0.261 | 0.339 | 0.211 | 0.204 | 0.127 | 0.176 |  |
| iNPG | 0.487 | 0.312 | 0.481 | 0.450 | 0.489 | 0.301 | 0.342 | 0.254 | 0.227 |  |
| Plenoxel | 0.506 | 0.386 | 0.521 | 0.503 | 0.540 | 0.418 | 0.441 | 0.447 | 0.398 |  |
| Ours | 0.181 | 0.100 | 0.316 | 0.218 | 0.287 | 0.160 | 0.139 | 0.099 | 0.134 |  |
| 3D-GS^†-full | 0.375 | 0.268 | 0.438 | 0.386 | 0.447 | 0.277 | 0.261 | 0.188 | 0.263 |  |
| Ours-full | 0.290 | 0.195 | 0.379 | 0.334 | 0.362 | 0.233 | 0.212 | 0.163 | 0.223 |  |

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 5
presentation: 3
questions: (1) In the experiments, RetinaGS uses dense point clouds for initialization. Do the other methods use the same dense point clouds for a fair comparison? If not, the author should provide their results with sparse/dense point clouds.

(2) The author provides training time for their method in Table 2 and Table 3. While the training time of other methods is missing. I wonder would RetinaGS would be slower/faster than CityGaussian/VastGaussian/DOGS?
soundness: 3
strengths: (1) This paper did very exhaustive experiments on publicly available large-scale datasets, such as Mega-NeRF and MatrixCity.

(2) This paper can train 3DGS up to billion-scale 3D Gaussian primitives, which is very impressive.

(3) This paper is a good engineering work, though it is not the only work that can scale up the training of 3D GS to billion-scale data.
summary: This paper, named RetinaGS, aims to train 3DGS in a distributed way. It first divides the scene space into a set of convex subspaces, each subspace contains a subset of 3DGS and can thus be distributed trained. For each subspace, the proposed method calculates a partial color and partial opacity, and the final color is obtained by a weighted sum of all subspaces. This paper also did exhaustive experiments to show the effectiveness of their method on the MipNeRF360 dataset, Mega-NeRF dataset, and the MatrixCity dataset. The proposed method can achieve comparable rendering quality to existing NeRF/3DGS methods (GP-NeRF, 3DGS, and CityGaussian), while with many more model parameters.
weaknesses: (1) This method lacks discussion of more existing distributed methods, such as Hierarchical-GS (Kerbl, *el,al*, SIGRRAPH 2024), DOGS (Chen and Lee, NeurIPS 2024), and City-on-Web (Song *et,al*, ECCV 2024).

(2) To me, this method is quite similar to 1) **DOGS**, which also adopts a central manager to manage sub-models, and 2) **City-on-Web**, which also adopts a similar partial color and partial opacity rendering strategy (while City-on-Web uses volume rendering instead of point rendering) to compute the final color. Therefore, discussions and comparisons to DOGS and City-on-Web are necessary while missing in the current version.

(3) The proposed method requires a gradient computation step and a gradient synchronization step on the central manager, which can consume more time than other distributed methods (VastGaussian, DOGS). The time for these steps should be clarified in the paper.

(4) This method requires running MVS on all training data. However, running MVS on large-scale datasets is time-consuming, and can even require much more time than training 3DGS on the same dataset.

(5) This paper cares about the distributed training of 3DGS, and how the method can scale up the training of 3DGS to a billion scale. The improvements in visual quality are only based on the improved number of 3DGS. e.g. in Table 1, the PSNR is 27.70 with 217.30M 3D Gaussians, which has only marginal improvement compared to CityGaussian with 23.7M 3D Gaussians.

(6) I appreciate the authors' effort in running exhaustive experiments on many existing large-scale datasets. However, the paper is more like a system work, and the novelty of the paper is limited, especially since the partial color and partial opacity strategy are already proposed in City-on-Web and there is lack of discussion of this paper with related works.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: Insufficient Scenes in Mega-NeRF Experiments: The paper uses only three of the six scenes from the Mega-NeRF dataset for experiments, without explaining the rationale for this choice. This may affect the comprehensiveness of the experimental results. It would be beneficial to include results from all scenes or provide a clear explanation for the selected subset.

Some minor issues:
1. The word “distirbuted” should be corrected to “distributed” in Line 97.
2. In Table 4 on page 9, “Bacth Size” should be corrected to “Batch Size.”
soundness: 3
strengths: 1. Clear Writing: The paper is well-organized, with logically structured sections and clear language. It effectively presents the problem background, research motivation, methods, and experimental results, making it easy for readers to understand the design principles and implementation of RetinaGS.
2. Engineering Contributions of Distributed Training: The proposed distributed training framework is technically sound and addresses high-performance computing needs. The framework also makes notable engineering contributions, including the use of KD-tree for load balancing across GPUs and multi-view stereo (MVS) for initializing Gaussian splats, which improve the efficiency and stability of distributed training.
3. Broad Experimentation: The method was tested on multiple datasets, including Scannet++, MipNeRF-360, Mega-NeRF, and MatrixCity, showcasing its applicability and performance across varying scene scales. The MatrixCity dataset especially highlights the scalability and rendering quality of the method.
summary: This paper introduces a distributed training framework, RetinaGS, for training large-scale 3D Gaussian Splatting (3DGS) models to achieve high-definition 3D scene reconstruction. To overcome the limitations of single-GPU training, RetinaGS employs a precise distributed rendering equation, uses KD-tree partitioning for load balancing, and initializes Gaussian splats through multi-view stereo, enabling parallel training of 3DGS across multiple GPUs. Experiments were conducted on datasets including Scannet++, MipNeRF-360, Mega-NeRF, and MatrixCity, demonstrating superior rendering quality and efficiency of RetinaGS on large-scale datasets. Notably, RetinaGS achieved the training of over a billion splats on the MatrixCity dataset, reaching unprecedented visual quality.
weaknesses: 1. Limited Scale of Experimented Scenes: Although the method was tested on four datasets, Scannet++ and MipNeRF-360 have relatively small scene scales, limiting the demonstration of its potential for large-scale scenes. Moreover, while MatrixCity is a large-scale dataset, it is synthetic and lacks validation on real-world scenes. Testing on more large-scale datasets, especially with real-world data, would better illustrate the practical applicability of RetinaGS.
2. Lack of Comparison with Other Methods: The paper does not include comparisons with methods like VastGaussian or Hierarchical 3D Gaussians, which are also significant contributions in distributed 3D reconstruction. Comparing RetinaGS with these methods would provide a more comprehensive view of its advantages and limitations.
3. Lack of Quantitative Results in Exploration Study: In the first two sections of the EXPLORATION STUDY, the lack of quantitative results weakens support for the effectiveness of the method. Adding quantitative analysis would provide a clearer demonstration of the impact of different settings on model performance.
4. Limited Novelty: The core idea of this paper is relatively straightforward, with the primary contribution being in the engineering implementation rather than in methodological innovation. Consequently, RetinaGS demonstrates moderate novelty in terms of the underlying approach.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: This work is developed upon 3DGS, can it be compatible with derivatives of 3DGS, e.g. 2DGS or scaffoldgs?
soundness: 3
strengths: + Significance: this paper is well motivated and tackles a critical gap in existing GS-based methods by addressing the need for higher resolution and the capacity to handle larger datasets and more complex scenes. By moving towards scalable, high-resolution 3D reconstruction, this work could influence a range of fields such as virtual reality, autonomous driving, and simulation. 
+ Quality: the distributed training of 3DGS is well supported by theory and proved efficacy in the experiments. Illustrations are clear and straightforward, with abundant experiments on diverse scenes. Supplemantary material is informative and solid.
summary: The authors aim to achieve exceptional rendering visual fidelity, which requires training GS models with higher spatial resolution, larger datasets, and diverse viewing perspectives. However, current 3DGS training struggles in these settings. A core issue identified is that 3DGS training remains constrained by single-GPU setups, which become infeasible for handling even moderately sized scenes due to time and memory demands. While recent distributed training approaches split the scene data into subspaces processed independently on multiple GPUs, they rely on fixed data layouts like bird's-eye views that minimize ray overlap between subspaces. This partitioning strategy, however, does not generalize well to more complex 3D scenes where ray paths do not align with predefined cells, leading to rendering artifacts or training challenges.
This paper proposed a novel strategy to overcome these limitations, by introducing a more flexible KD-tree partitioning method and optimized multi-GPU training architecture to enhance the applicability of GS models across varied 3D scenes.
weaknesses: - Missing related work: the proposed partitioning and ray segment rendering reminds me of an existing work NeRF-XL. Perhaps these are concurrent works, but authors should at least discuss it in related work.
- Practical usage of KD-tree: it seems that KD-tree partition makes sense when the scene or scene points are evenly distribuuted. Will it still maintain a balanced workload between workers when the scene is not evenly distributed. 
- MVS points for initialization: I suppose that authors need to work from a quite dense and envenly distributed point initialization so they need to do MVS first. This is not necessary a weakness but maybe constraining its application e.g. when views are sparse or inconsistent to get an ideal MVS result, which is quite common in large-scale scene capturing.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 1
questions: 1. Does *k* represent the *k*-th index?

2. During rendering, is it necessary to explore all subspaces along the ray path? On average, how many subspaces does each ray hit?

$~$

Minor comments:

There is an inconsistency in figure references on line 269.

Dots are missing in abbreviations on lines 233, 252, 274, 347, 417, 419, and 879.

A dot is missing in the sentence "Validity of Distribute Rendering" on line 422.
soundness: 2
strengths: 1. RetinaGS is a 3DGS model capable of training on large-scale datasets through parallelization.

2. The subspace-level rendering pipeline parallelization using a KD-Tree is intriguing.

3. RetinaGS allows for a controllable number of Gaussians.
summary: This paper presents RetinaGS, which utilizes a subspace-based parallelization method as a learning approach for Gaussian Splatting. Each subspace is handled by a single process and rendered using a rendering function with indicators. Instead of the standard splitting algorithm in Gaussian Splatting, MVS primitives are used to control the total number of Gaussians.
weaknesses: 1. In Table 1, the number of Gaussians is more than five times that of 3DGS. In Figures 5 and 6, reducing the number of Gaussians to 1/10 lowers the PSNR by about 2. Compared to 3DGS, if RetinaGS has the same number of Gaussians, its PSNR would likely be even lower. Therefore, the proposed method does not appear effective relative to the increasing number of Gaussians.

2. It appears that RetinaGS depends on the accuracy of the MVS algorithm.

3. I recommend a comparison with Hierarchical3DGS[1], as both methods present chunk-based Gaussian Splatting approaches.

4. Missing a comparison of rendering speeds.

5. Fig. 5 is difficult to understand. First, the resolution/splat, count, and PSNR are not clearly visible. Additionally, the images without labels are ambiguous in meaning. For instance, the images in the bottom row all seem to represent GT.

[1] Kerbl et al., *A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets*, ACM Transactions on Graphics, 2024

Meta-review: This paper receives ratings of 5,5,6,5. The AC follows the recommendations of the reviewers to reject the paper. The main concerns of this paper are: 1) The lack of discussion on more existing distributed methods. 2) The proposed method is very similar to existing works such as DOGS and City-on-web. 3) Performance and efficiency might not be as good as other existing methods. 4) Limited scale of experimented scenes and lack of ablations. Although the authors tried to provide their rebuttals to address the issues, the reviewers remained concern over the proposed method and experiments.

Decision: Reject