Sort-free Gaussian Splatting via Weighted Sum Rendering

## Abstract

Abstract Recently, 3D Gaussian Splatting (3DGS) has emerged as a significant advancement in 3D scene reconstruction, attracting considerable attention due to its ability to recover high-fidelity details while maintaining low complexity. Despite the promising results achieved by 3DGS, its rendering performance is constrained by its dependence on costly non-commutative alpha-blending operations. These operations mandate complex view dependent sorting operations that introduce computational overhead, especially on the resource-constrained platforms such as mobile phones. In this paper, we propose Weighted Sum Rendering, which approximates alpha blending with weighted sums, thereby removing the need for sorting. This simplifies implementation, delivers superior performance, and eliminates the “popping” artifacts caused by sorting. Experimental results show that optimizing a generalized Gaussian splatting formulation to the new differentiable rendering yields competitive image quality. The method was implemented and tested in a mobile device GPU, achieving on average 1.23 × 1.23\times 1.23 × faster rendering.

## 1 Introduction

Photo-realistic 3D view synthesis has very wide use in graphics applications like video games, virtual reality, and 3D scene modeling techniques based on learnable appearance and transparency have achieved remarkable success thanks to their capability for consistently generating image details. Neural Radiance Fields (NeRF) by Mildenhall et al. () is one seminal work using this approach, where Multi-Layer Perceptron (MLP) networks and positional encoding are used to create models representing a scene’s radiance field and opacity.

More recently, 3D Gaussian Splatting (3DGS) was proposed by Kerbl et al. (), and rapidly gained popularity (Fei et al., ; Chen & Wang, ; Wu et al., ). It replaces the MLP networks used by NeRF with a sparse scattering of anisotropic 3D Gaussian “splats” that have view-dependent appearance. These splats are efficiently rendered using rasterization instead of costly volumetric ray-marching, while maintaining its rendering differentiability. Nevertheless, while 3DGS produces high quality views with significantly lower complexity compared to NeRF, it is still challenging for resource-constrained platforms like mobile phones.

An important complexity factor results from the fact that splatting volumetric elements with transparency requires sorting before $\alpha$-blending, and it is difficult to combine non-commutative blending operations—which are intrinsically sequential—with the parallelization required for fast rendering. Furthermore, approximating ray integrals with sorting can create visible “popping” transitions due to temporal changes in rendering order, that can only be alleviated with more complex sorting and rendering (Radl et al., ).

The original 3DGS proposal achieves impressive rendering performance with a compute-based rasterizer implemented in CUDA, enabled by a tiled-based GPU radix sort that ensures front-to-back $\alpha$-blending, but incurring memory overhead. Subsequent implementations have leveraged other APIs such as Vulkan or WebGL kishimisu (); Kwok (), either choosing a similar compute-based implementation, or leveraging the graphics hardware (GPU) to varying degrees. Unfortunately, there is no mechanism to feed the hardware rasterizer in a tile-wise fashion, so such implementations must either perform a costly global-sort prior to submitting work to the hardware rasterizer, or maintain a compute-based rasterizer, sacrificing the benefits of hardware execution. In the case of WebGL, implementations often rely on an asynchronous CPU-sort that runs at a lower-frequency than the frame rate, which degrades view quality and exacerbates temporal “popping” artifacts.

Although the sorting stage in Gaussian Splatting appears inevitable, it results from using approximations to physical processes. In reality, a general scheme for learning scene representations, with components and parameters optimized for the best view reproductions, is bound only by the mathematical constraints of the scene model. This observation motivates us to pursue alternative methods that reduce rendering complexity while incurring minimal degradation in view quality.

We are further encouraged by a widely adopted empirical method for rendering transparent media utilizing the traditional rendering pipeline — Order Independent Transparency (OIT)  Meshkin (); McGuire & Bavoil (), representing a class of techniques in rasterization-based computer graphics for rendering transparency without sorting. By eliminating the sorting step from the rasterization pipeline, OIT maintains rendering speed and has been incorporated into commercial tools such as Blender, Vulkan, Unity, and Unreal Engine.

In this paper we exploit those approaches to develop a novel view synthesis method combining learning with commutative blending operations. It extends the definition of learned Gaussian Splatting, and integrates it with a new method for volumetric rendering, called Weighted-Sum Rendering (GS-WSR), still relying on differentiable rendering and machine learning techniques. To render an image, our approach employs the depth of each Gaussian to compute weights for each Gaussian splat using learned functions. These weighted Gaussians are then splatted onto the image plane using a straightforward summation operation, thereby obviating the necessity for sorting. As a further enhancement, we found that introducing view-dependent opacity greatly improved image quality for sort-free Gaussian Splatting.

This paper contributes to Gaussian Splatting as follows. First, we present the first sort-free Gaussian Splatting technique which is compatible with the graphics pipeline, and allows us the benefit of hardware rasterization. Second, we introduce Weighted Sum Rendering as well as view-dependent opacity that can effectively learn the model parameters and produce high-quality rendered images. Third, our experiments show that our method accelerates Gaussian Splatting with comparable visual performance on mobile phones.

## 2 Related Work

Neural Scene Representations. Recent advancements in neural scene representations for novel view synthesis have led to significant progress, with techniques assigning neural features to structures like volumes Lombardi et al. (); Sitzmann et al. (), textures Chen et al. (); Thies et al. (), or point clouds Aliev et al. (). The pioneering NeRF (Neural Radiance Fields) approach Mildenhall et al. (), revolutionized the field by using Multi-Layer Perceptrons (MLPs) to encode 3D density and radiance without proxy geometry, achieving photorealistic renderings from sparse 2D images. Subsequent research has aimed to enhance NeRF’s quality and efficiency through optimized sampling strategies Neff et al. (), light field-based formulations Li et al. (), compact representations Müller et al. (), tensor decomposition Chen et al. (), or leveraging the polygon rasterization pipeline for efficient rendering on mobile devices such as MobileNeRF Chen et al. (). However, the high computational and memory requirement still puts a challenge on mapping these methods on resource constrained platforms.

Order Dependent Transparency 3DGS The novel 3D Gaussian Splatting (3DGS) and its numerous follow-up works primarily utilize anisotropic 3D Gaussians for scene representation and an efficient differentiable rasterizer to project these Gaussians onto the image plane, either in pixel or feature representation Hamdi et al. (); Lin et al. (); Yu et al. (); Liang et al. (); Yan et al. (); Lu et al. (); Bulò et al. (); Fan et al. (); Fu et al. (); Zhang et al. (); Zou et al. (); Jiang et al. (); Qin et al. (); Li et al. (). These methods enable fast, high-resolution rendering while maintaining excellent quality. For a comprehensive overview, please refer to recent survey papers such as Chen & Wang (). A crucial requirement in these methods for proper blending and rendering is the order of the Gaussians, which are typically sorted by their depth using a tile-based sorting algorithm. However, this sorting requirement introduces several challenges in practical implementation and visual quality, such as sudden changes in the appearance of object parts or “popping” artifacts, as recently addressed in Radl et al. (). While their results show a limited increase in computational complexity, the necessary sorting modifications are significantly more involved.

Order Independent Transparency. Modeling partial coverage has a long history in computer graphics. It’s an essential problem as we need to render fine non-opaque structure or elements such as flames, smoke, clouds, hair, etc. In Porter and Duff’s seminal work Porter & Duff (), they formulate transparency as the OVER operator as

$$ $\mathbf{C}=\alpha_{0}\mathbf{c}_{0}+(1-\alpha_{0})\mathbf{c}_{1},$ (1) $$

where $\alpha$, $\mathbf{c}$ indicate alpha and color, respectively. As the OVER operator is not commutative, it requires back-to-front order for composition. Traditional accelerated either by successively “peeling” depth layers Everitt () or accumulating list for sorting like A-buffers Carpenter (). These methods introduce time and memory overhead.

To avoid sorting, there are many methods that have been proposed to approximate compositing, namely Order-Independent Transparency (OIT). For instance, $k$-buffer methods work like depth peeling methods, but only storing and accumulating first $k$-layers in a single pass Bavoil et al. (). Alternatively, stochastic transparency methods in Monte Carlo rendering samples the fragments according to the opacity and depth, which can generate promising results for a large sampling rate Enderton et al. (). A survey on these methods can be found in Wyman ().

The weighted blended OIT proposed by McGuire & Bavoil () is the most relevant to our method. There are several OIT variants listed in  McGuire & Bavoil (), the most general of which is to replace the OVER operator with the following commutative blending operator

$$ $\mathbf{C}=\prod_{i=1}^{\mathcal{N}}\left(1-\alpha_{i}\right)\mathbf{c}_{0}+ \left(1-\prod_{i=1}^{\mathcal{N}}\left(1-\alpha_{i}\right)\right)\frac{\sum_{i =1}^{\mathcal{N}}\mathbf{c}_{i}\alpha_{i}w\left(d_{i},\alpha_{i}\right)}{\sum_ {i=1}^{\mathcal{N}}\alpha_{i}w\left(d_{i},\alpha_{i}\right)}$ (2) $$

where $d_{i}$ is the distance to camera, $\mathbf{c}_{0}$ is the background color, $w\left(\cdot\right)$ is a function that decreases with distance, so that objects nearer to the camera are assigned larger weights. In the OIT rendering Eq. 2 there are two sums defining pixel values, and since addition is commutative, they can be computed in any order. The final pixel value can only be computed after the two sums are known, which can be implemented as a single pass post-process over the image.

## 3 Preliminaries: Gaussian Splatting

The 3DGS scene model ${\cal G}=\{({\mathbf{p}}_{i},t_{i},{\mathbf{q}}_{i},{\mathbf{s}}_{i},{\mathbf{
H}}_{i})\}_{i=1}^{\mathcal{N}}$ represents a scene of $\mathcal{N}$ Gaussians with center locations ${\mathbf{p}}\in\mathbb{R}^{3}$, maximum opacity $t_{i}\in\left[0,1\right]$, orientation ${\mathbf{q}}\in\mathbb{R}^{4}$, scale ${\mathbf{s}}\in\mathbb{R}^{3}$, and spherical harmonics coefficients ${\mathbf{H}}$ following an equation similar to Gaussian probability distributions. The opacity of each element at a position ${\mathbf{x}}$ in 3D space is defined according to

$$ $\alpha_{i}\left({\mathbf{x}}\right)=t_{i}\exp\left(-\frac{\left({\mathbf{x}}-{ \mathbf{p}}_{i}\right)^{t}\left[\mathbf{\Sigma}\left({\mathbf{q}}_{i},{\mathbf {s}}_{i}\right)\right]^{-1}\left({\mathbf{x}}-{\mathbf{p}}_{i}\right)}{2} \right),\quad i=1,2,\cdots,\mathcal{N}$ (3) $$

and considering a camera with focal point at position ${\mathbf{f}}$, the view-dependent color is defined by

$$ $\mathbf{c}_{i}=Y\left(\lVert{\mathbf{f}}-{\mathbf{p}}_{i}\rVert,{\mathbf{H}}_{ i}\right),\quad i=1,2,\cdots,\mathcal{N},$ (4) $$

where $Y(\cdot)$ indicates the spherical harmonic function and $\lVert\cdot\rVert$ indicates the vector norm.

To render an image, each 3D Gaussian is mapped to 2D as approximated by a 2D Gaussian with the following $2\times 2$ covariance matrix

$$ $\mathbf{\Sigma}_{\text{2D}}=\mathbf{J}\mathbf{W}\mathbf{\Sigma}_{\text{3D}} \mathbf{W}^{t}\mathbf{J}^{t}$ (5) $$

where $\mathbf{W}$ is a matrix defined by the camera’s image-generation transformation, and $\mathbf{J}$ is the Jacobian matrix defined by an affine approximation of the projective camera transformation.

Critically, due to it’s non-commutative blending operations requiring a strict depth ordering, it is necessary to sort Gaussians according to their distance to the camera position $\mathbf{f}$. Formally, it is necessary to determine a sorting sequence $s[k]$ of indexes according to increasing depth such that

$$ $k\geq l\Rightarrow d_{s\left[k\right]}\geq d_{s\left[l\right]},k,l=1,2,\cdots, \mathcal{N}$ (6) $$

After sorting Gaussians in depth order, the final pixel color obtained by Gaussian splatting can be computed according to

$$ $\mathbf{C}=\sum_{i=1}^{\mathcal{N}}\mathbf{c}_{i}\alpha_{i}\prod_{j=1}^{i-1}(1 -\alpha_{j}),$ (7) $$

which corresponds to the well-known computer graphics technique of alpha-blending (Marschner & Shirley, ).

## 4 Sort-free Gaussian Splatting via Weighted Sum Rendering

Figure: Figure 1: The architecture of sort-free Gaussian Splatting via Weighted Sum Rendering. 3DGS needs to tiling, replication, sorting, and rendering. Our method only needs to calculate the weight for each Gaussian, and independently sum their contributions per-pixel.
Refer to caption: x1.png

### 4.1 Sort-free Gaussian Splatting

As discussed in Sec. 3, 3DGS relies on non-commutative alpha blending, which requires a depth order sorting operation before rendering. To recover high-fidelity of the rendered images, 3DGS models typically include a substantial number of Gaussians, which can escalate to millions in complex scenes. The overhead associated with these sorting operations poses a challenge in splatting Gaussians onto an image without effective optimization.

To accelerate rendering, 3DGS incorporates several optimizations designed to leverage parallel computation capabilities in CUDA. As shown in Figure 1, 3DGS splits the rendered image into multiple non-overlapping tiles, with Gaussians that span across multiple tiles being duplicated accordingly. Each Gaussian is assigned a tile ID and sorted by depth on a per-tile basis. Once sorted, all tiles are rendered in parallel, significantly improving computational efficiency. Additionally, 3DGS employs early termination to further optimize performance: if the alpha value of the frontmost Gaussians is sufficiently high, the rendering process will skip subsequent Gaussians, reducing unnecessary computations and memory usage.

While 3DGS achieved promising results, its CUDA based implementation restricts its portability, and its sorting requirement limits its performance. Their usage of an efficient tile-based radix sort forces them maintain a fully compute-based render pipeline, including compute-based rasterization. On the other hand, current APIs fail to expose efficient mechanisms to submit geometry to the hardware rasterizer in a tiled fashion, thereby requiring that implementations leveraging the hardware rasterizer execute a more costly global sort over all visible Gaussians. 3DGS chose to pursue the fully compute-based approach with its CUDA implementation, which denies them some of the benefits of the GPU’s fixed function hardware. Their tile-based sort incurs some overhead in its duplication of Gaussians at tile boundaries, which is exacerbated in instances where there are a large number of Gaussians, and finally, as noted by prior work Radl et al. (), sorting Gaussians by their centers can introduce “popping” artifacts during view transformation, further affecting visual quality.

These limitations motivate the elimination of the sorting phase in Gaussian Splatting, which not only significantly simplifies the implementation but also enhances compatibility with hardware graphics pipelines. Inspired by Order Independent Transparency (OIT), we propose a novel approach that modifies the volumetric rendering process in 3DGS to a more efficient representation termed Weighted Sum Rendering (WSR).

### 4.2 Weighted Sum Rendering

Figure: Figure 2: Three variants of Weighted Sum Rendering with different weight calculations, namely Direct Weighted Sum Rendering (DIR-WSR), Exponential Weighted Sum Rendering (EXP-WSR), and Linear Correction Weighted Sum Rendering (LC-WSR).
Refer to caption: x2.png

Figure 1 compares the architectures of the original sort-based 3DGS model, with our proposed sort-free Gaussian Splatting framework. Unlike the volumetric rendering approach employed by 3DGS, our method estimates the transparency of each Gaussian solely based on its depth and learnable parameters, thereby eliminating the need for depth ordering. The final image can be rendered using

$$ $\mathbf{C}=\frac{\mathbf{c}_{B}w_{B}+\sum_{i=1}^{\mathcal{N}}\mathbf{c}_{i} \alpha_{i}w(d_{i})}{w_{B}+\sum_{i=1}^{\mathcal{N}}\alpha_{i}w(d_{i})},$ (8) $$

where $\mathbf{c}_{B}$ and $w_{B}$ indicate the color and learnable weight of the background, respectively. $d$ indicates the depth. $w(\cdot)$ indicates the learnable weight function. Our network learns the Gaussian’s parameters during training. Rendering with Eq. 8 in WSR corresponds to computing weighted sums. As the addition is commutative, these sums can be computed in any order, overcoming the constraints of depth sorting. Unlike traditional OIT methods Meshkin (); McGuire & Bavoil (), which rely on predefined parameters, our method optimizes the parameters of this new representation during training.

3DGS produces novel views based on a physics-based blend model, whereas GS-WSR demonstrates the benefits of departing from a phsyically-based model, and using machine learning to train the parameters for a non-physically based model.
In volumetric rendering Eq. 7, it is necessary to have $\alpha_{i}\in\left[0,1\right]$ to guarantee that all terms are positive. However, these constraints are not necessary in WSR Eq. 8 as $\alpha_{i}$ serves as a learnable parameter in a radiance field model. Removing such constraints can potentially result in better approximations. Similarly, our view-dependent opacity may not correspond to optical laws, but it is in practice useful for minimizing the limitations of the WSR Eq. 8 compared to volume rendering Eq. 7.

Direct Weighted Sum Rendering (DIR-WSR). A straightforward method of rendering is to sum all contributions directly, allowing the network to learn specific opacities. In DIR-WSR, the weight is defined as a constant as follows

$$ $w(d_{i})=1,\quad i=1,2,\cdots,\mathcal{N}.$ (9) $$

However, DIR-WSR does not work well for the complex scenes, and often introduces blurring artifacts in areas where Gaussians overlap. We attribute the poor visual performance due to the lack of depth information, which prevents accurate estimation of transparency. A visual example is provided in Figure 2, illustrating two non-transparent Gaussians with $\alpha=1$ with different color(^1^11Please note that in WSR, $\alpha$ can be larger than 1. We assume $\alpha=1$ as non-transparency for illustrative purposes.). Since DIR-WSR uses a constant weight of 1, it is unable to correctly handle this situation, leading to an incorrect estimation of the resulting color as purple.

Exponential Weighted Sum Rendering (EXP-WSR). To better capture the effects of occlusion, we introduce EXP-WSR to assign larger weights to Gaussians closer to the camera. The weight is defined as

$$ $w(d_{i})=\exp\!\left(-\sigma d_{i}^{\beta}\right),\quad i=1,2,\cdots,\mathcal{ N},$ (10) $$

where $\sigma$, $\beta$ are learnable parameters. In this manner, the Gaussians closer to the viewer would have a higher weight, thus contributing more to the final rendered image. Although EXP-WSR can effectively reduce artifacts and generate better results compared to DIR-WSR, it is not entirely free from visual distortions, as distant Gaussians still contribute to the rendered image to some extent. As shown in Figure 2, the red Gaussian, being closer to the viewer, has a larger weight than the blue Gaussian, yielding results that are closer to the ground truth. However, some artifacts remain visible in the final output.

Linear Correction Weighted Sum Rendering (LC-WSR). Inspired by the deformable convolution  Dai et al. () and KPConv Thomas et al. (), we use linear correction to estimate the weight from depth by

$$ $w(d_{i})=\max\left(0,1-\frac{d_{i}}{\sigma}\right)v_{i},\quad i=1,2,\cdots, \mathcal{N},$ (11) $$

where $\sigma$, $v_{i}$ are learnable parameters. This formulation will assign a relatively larger weight for the Gaussians closer to the camera. For distant Gaussians, the weight may be reduced to 0, depending on $\sigma$ or $v_{i}$. As shown in Figure 2, the rendered result is closest to the ground truth, since this model can set weights to zero it can more accurately model occlusions. Additionally, compared to EXP-WSR, LC-WSR is cheaper to compute.

### 4.3 View-dependent opacity

Figure: Figure 3: View dependent opacity. In 3DGS, the accumulated $\hat{\alpha}$ changes depending on the viewer’s direction, which motivates us to assign view-dependent opacities in sort-free Gaussian Splatting.
Refer to caption: x3.png

In Sec  4.2, we proposed WSR to assign different weights to Gaussians according to their depth. Moreover, the order of Gaussians would also change depending on the viewer’s direction in the original 3DGS. Figure 3 illustrates this phenomenon: two viewers observe two Gaussians from different directions. The left viewer assigns a larger weight to the red Gaussian, while the right viewer assigns a smaller weight to the same Gaussian. This observation motivates our approach to assigning view-dependent opacities.

We substitute our opacity values for an additional set of spherical harmonic coefficients for view-dependency. It’s another mechanism by which we mitigate the contribution of occluded Gaussians. We modify Eq 3, which defines the sort-free Gaussian’s opacity, replacing the 3DGS element’s maximum opacity $t_{i}\in\left[0,1\right]$ with an unconstrained value $u_{i}$ as

$$ $u_{i}=Y\left(\lVert{\mathbf{f}}-{\mathbf{p}}_{i}\rVert,{\mathbf{H}}_{i}\right) ,\quad i=1,2,\cdots,\mathcal{N},$ (12) $$

where $Y(\cdot)$ indicates the spherical harmonic function and $\lVert\cdot\rVert$ indicates the vector norm. ${\mathbf{H}}_{i}$ indicates the learnable spherical harmonics coefficients for opacity. Thus, the maximum opacity $u_{i}$ depends on view direction $\lVert{\mathbf{f}}-{\mathbf{p}}_{i}\rVert$ according to learned spherical harmonics parameter vector $\mathbf{H}_{i}$. There is a slight difference to the SH evaluation for the RGB channels and the opacity in our implementation. The maximum opacity $u_{i}$ is not clamped. Please note that the view-dependent opacity may not correspond to optical laws, but it is in practice useful for minimizing the limitations of OIT rendering Eq. 8 compared to volume rendering Eq. 7.

### 4.4 Implementation details

Loss function. We optimize our sort-free Gaussian Splatting using the same rendering loss from 3DGS Kerbl et al. (), which contains the $\ell_{1}$ loss and D-SSIM loss with a factor of 0.2.

Training and Evaluation. We trained our sort-free Gaussian Splatting method using PyTorch. We implemented the custom CUDA kernels for the PyTorch version. For EXP-WSR, we initialize the $\sigma=0.1$ and $\beta=0.8$. For LC-WSR, we initialize $\sigma=10$ and $v_{i}=0.1$. For convenience, we also evaluate the quality of our method with a PyTorch inference simulation.

Testing on mobile devices. To measure model efficiency on mobile devices, we implemented our method and the competitive 3DGS method in Vulkan (^2^22https://www.vulkan.org), which is a cross-platform graphics and compute API created and maintained by the Khronos Group (^3^33https://www.khronos.org).

3DGS’s rendering on mobile device contains four steps: pre-processing, lighting, sorting, and rasterization. The pre-processing step projects the 3D Gaussian into 2D based on the camera view. The lighting stage calculates per-Gaussian color based on the camera view and the spherical harmonics (SHs). The sorting step sorts the Gaussians front to back, and the rasterization step rasterizes all Gaussians within each tile to output the final image.

To make a fair comparison, we implemented two versions of 3DGS, namely 3DGS-Compute and 3DGS-Graphics.
3DGS-Compute attempts to faithfully replicate the approach from 3DGS Kerbl et al. () in Vulkan compute shaders. It follows a similar scheme of replicating Gaussians per-overlapped tile, followed by compute-based rasterization. The most notable difference is that the optimized radix-sort from the NVIDIA CUB libraries NVIDIA () had to be replaced with a Vulkan equivalent.
By contrast, the 3DGS-Graphics approach allows for more mobile friendly execution. It’s implementation globally sorts the Gaussians, eliminating per-tile replication, and allowing it to easily submit a global list of Gaussian to the hardware graphics pipeline. Gaussian color and opacity are evaluated in a fragment shader. These modifications allow Gaussians to be efficiently rendered by the fixed-function rasterizer, while their contributions are accumulated by hardware blending operations.

For both our method and the 3DGS methods, the pre-processing and lighting steps remain the same as the original 3DGS implementation, except that our method’s opacity is now calculated in the lighting stage based on the camera view and SHs. Our method removes the entire sorting step, and rasterizes Gaussians in a fragment shader. Besides, our method contains an extra subpss to perform the final normalization step. With our method, all Gaussians are rendered using a single instanced draw call instead of the per-tile fashion used by 3DGS. We verified the consistency between our on mobile implementation versus the Pytorch reference implementation. Please refer to our supplementary for more details about our on mobile implementation.

## 5 Experimental Results

Figure: Figure 4: Visual comparison on the Mip-NeRF 360 dataset. Our OIT method achieves similar visual performance compared to 3DGS. Please note that our method doesn’t require the order of Gaussians.
Refer to caption: extracted/5952245/Figs/comp_wsr/5_rec.png

Datasets. To ensure a fair comparison, we followed the evaluation setting of 3DGS Kerbl et al. () and conducted our experiments on 13 real-world scenes. Specifically, they include the complete set of scenes from the Mip-Nerf360 dataset Barron et al. (), two scenes from the Tanks & Temples dataset Knapitsch et al. (), and two scenes from the Deep Blending dataset Hedman et al. (). These scenes encompass various challenging scenarios, including both indoor and outdoor environments.

Evaluation metrics. We evaluate reconstruction fidelity using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS) metrics. For model efficiency, we report run times on a Qualcomm^® Adreno^TM GPU from Snapdragon 8 gen 3 chipset. To enable comparison, we also re-implemented our method and the official 3DGS (Kerbl et al., ) in Vulkan for this setting.

**Table 1: PSNR scores of our method on the Mip-NeRF360 dataset, the Tanks & Temples dataset, and the Deep Blending dataset. Our method achieved comparable results with 3DGS.**
| Method | Mip-NeRF 360 | Tanks & Temples | Deep Blending |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ |  |
| Plenoxels | 23.08 | 0.626 | 0.463 | 21.08 | 0.719 | 0.379 | 23.06 | 0.795 | 0.510 |
| INGP-Base | 25.30 | 0.671 | 0.371 | 21.72 | 0.723 | 0.330 | 23.62 | 0.797 | 0.423 |
| INGP-Big | 25.59 | 0.699 | 0.331 | 21.92 | 0.745 | 0.305 | 24.96 | 0.817 | 0.390 |
| M-NeRF360 | 27.69 | 0.792 | 0.237 | 22.22 | 0.759 | 0.257 | 29.40 | 0.901 | 0.245 |
| 3DGS | 27.21 | 0.815 | 0.214 | 23.14 | 0.841 | 0.183 | 29.41 | 0.903 | 0.243 |
| LC-WSR | 27.19 | 0.804 | 0.211 | 23.61 | 0.842 | 0.177 | 29.63 | 0.902 | 0.229 |

**Table 2: Runtime (ms) comparion using a Snapdragon 8 Gen 3 GPU on the Mip-NeRF360, Tanks & Temples and Deep Blending datasets. The images are rendered at a resolution of $1920\times 1080$. When the mobile resources are exhausted, it would significantly slow down the rasterization (labeled with “*”).**
| Method | Task | Mip-NeRF360 | Tanks&Temples | Deep Blending |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| bicycle | flowers | garden | stump | treehill | room | counter | kitchen | bonsai | truck | train | drjohnson | playroom |  |  |
| 3DGS-Compute | Pre-processing | 12.84 | 7.05 | 10.15 | 6.94 | 7.53 | 3.99 | 3.02 | 4.46 | 3.12 | 6.29 | 3.03 | 7.50 | 6.04 |
| Lighting | 4.57 | 3.26 | 5.19 | 3.43 | 3.37 | 1.84 | 1.55 | 3.07 | 1.40 | 3.53 | 2.41 | 2.61 | 2.40 |  |
| Sorting | 246.65 | 122.68 | 210.35 | 180.23 | 139.98 | 70.83 | 50.57 | 89.96 | 50.63 | 105.83 | 60.83 | 140.64 | 110.97 |  |
| Rasterization | 1239.12* | 115.18 | 608.19* | 265.27 | 147.08 | 140.16 | 85.94 | 177.01 | 73.43 | 165.64 | 130.23 | 212.10 | 167.09 |  |
| Total | 1511.96 | 253.60 | 842.15 | 463.06 | 303.68 | 219.31 | 143.03 | 277.35 | 130.58 | 285.11 | 198.01 | 367.93 | 290.32 |  |
| 3DGS-Graphics | Pre-processing | 13.27 | 7.82 | 13.73 | 10.46 | 8.01 | 3.40 | 2.62 | 4.09 | 2.62 | 5.84 | 2.35 | 7.22 | 5.57 |
| Lighting | 5.32 | 3.66 | 5.67 | 3.68 | 3.80 | 2.00 | 1.61 | 3.55 | 1.51 | 4.03 | 2.68 | 2.94 | 2.71 |  |
| Sorting | 40.26 | 24.79 | 38.25 | 31.09 | 25.47 | 11.23 | 9.47 | 13.76 | 9.17 | 19.16 | 7.43 | 23.13 | 18.43 |  |
| Rasterization | 618.83^∗ | 246.47^∗ | 697.81^∗ | 344.72^∗ | 208.28^∗ | 32.50 | 28.22 | 52.84 | 30.86 | 78.24 | 39.70 | 164.25* | 82.45 |  |
| Total | 678.08 | 283.11 | 755.89 | 390.33 | 245.95 | 49.46 | 42.25 | 74.56 | 44.47 | 107.64 | 52.48 | 197.86 | 109.46 |  |
| LC-WSR | Pre-processing | 6.83 | 3.92 | 5.53 | 6.46 | 4.63 | 1.94 | 1.67 | 2.74 | 1.64 | 3.42 | 1.61 | 4.61 | 2.48 |
| Lighting | 4.07 | 2.81 | 4.36 | 3.67 | 3.23 | 1.41 | 1.35 | 2.98 | 1.16 | 3.21 | 2.31 | 2.91 | 1.70 |  |
| Rasterization | 67.84 | 28.30 | 57.84 | 57.25 | 52.99 | 45.44 | 32.82 | 68.51 | 37.52 | 52.89 | 48.98 | 78.11 | 49.87 |  |
| Total | 78.99 | 35.28 | 67.90 | 67.63 | 61.11 | 49.02 | 34.06 | 74.47 | 40.63 | 59.75 | 53.13 | 85.85 | 54.28 |  |

### 5.1 Comparison with state-of-the-art methods

We compare our method with state-of-the-art techniques, including Plenoxels Fridovich-Keil et al. (), INGP Müller et al. (), M-NeRF360 Barron et al. (), and 3DGS Kerbl et al. (). The results in Table 1 demonstrate that our method, while avoiding any sorting, achieves comparable results to the baseline 3DGS. In terms of PSNR, our method outperforms 3DGS by 0.47dB and 0.22dB on the Tanks & Temples and Deep Blending datasets, respectively, while averaging only 0.02dB less on the Mip-NeRF360 dataset. For visual comparison, Figure 4 shows that our LC-WSR excellently recovers challenging details of the fireplace in the “Dr Johnson” scenes. In other scenes, WSR particularly recovers better fidelity in areas with strong illumination and reflection.

### 5.2 Computational complexity study

In this study, we examine the running time and memory consumption of our methods on the mobile devices, using an Snapdragon 8 Gen 3 GPU, and with an implementation based on Vulkan. To make a fair comparison, we also re-implement the 3DGS in Vulkan. Please refer Section  4.4 for implementation details. We conduct our experiments on the Mip-Nerf360, Tanks & Temples, and Deep Blending datasets, where each image is rendered at a resolution of $1920\times 1080$.

Table 2 reports the runtime comparison of 3DGS-Compute, 3DGS-Graphics, and our LC-WSR on the Mip-NeRF360, Tanks&Temples, and Deep Blending datasets. Compared to 3DGS-Compute, 3DGS-Graphics is more efficient as it eliminates the replication steps and thus reducing the number of Gaussians for sorting. Besides 3DGS-Graphics can also better utilize the graphic fixed-functions in the hardware-supported graphic rasterization pipeline. Our method is in most cases faster than the 3DGS-Graphics method due to the elimination of the sorting step and fewer Gaussians. Note that for certain scenes using the 3DGS-Graphics method, we can exhaust our edge device’s resources due to the sorting step and high Gaussian count, which will drastically slow down the rasterization step (labeled with “*”). In contrast, our method is fully capable of running all scenes in real-time. In theory, our method’s lighting pass should be slightly slower because opacity is calculated using SH, and our rasterization step should be slower due to the weight calculations in the fragment shader and the extra subpass for weight normalization. However, we observed that WSR tends to generate models with fewer Gaussians (scene average 2.88M) compared to the original 3DGS (scene average 3.98M), which is consistent with prior work  Radl et al. () that suggests modifying opacity leads to a reduction in the number of Gaussians. This also contributes to faster lighting and rasterization steps for us compared with 3DGS-Graphics. For certain scenes (kitchen and train) that have a similar number of Gaussians, our total time is close to or slightly worse compared to the original model because the overhead added in the rasterization step is not fully compensated by the removal of the sorting step, which may indicate further optimizations are needed for our method’s rasterization implementation. Overall, the proposed method is on average 1.23× faster in total time when the mobile resources are not exhausted.

Table 3 compares the memory of our method with 3DGS-Graphics and 3DGS-Compute. Compared to 3DGS-Compute, 3DGS-Graphics utilizes less memory as it performs global sorting thereby eliminating the replication step. Our method only requires around 63% memory of 3DGS-Graphics. This memory reduction can be attributed to removing the sorting operations as well as fewer total Gaussians. Specifically, removing sorting reduces around 17% memory, while the reducing the number of Gaussians contribute the remaining.

**Table 3: Memory (MB) comparison on the Mip-NeRF360, Tanks & Temples and Deep Blending datasets. The images are rendered at a resolution of $1920\times 1080$.**
| Method | Mip-NeRF360 | Tanks & Temples | Deep Blending |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| bicycle | flowers | garden | stump | treehill | room | counter | kitchen | bonsai | truck | train | drjohnson | playroom |  |
| 3DGS-Compute | 2017 | 1205 | 1920 | 1636 | 1253 | 542 | 421 | 626 | 429 | 850 | 358 | 1131 | 851 |
| 3DGS-Graphics | 1520 | 911 | 1448 | 1235 | 947 | 413 | 322 | 476 | 328 | 644 | 274 | 855 | 645 |
| LC-WSR | 881 | 526 | 681 | 853 | 618 | 276 | 241 | 367 | 251 | 439 | 217 | 615 | 344 |

### 5.3 Ablation study

Figure: Figure 5: Our method eliminates the “popping” artifacts during view transformation.
Refer to caption: x4.png

Comparison of WSR variants. We compare the variants of WSR in Table 4 on Mip-NeRF360, Tanks & Temples, and Deep Blending datasets. LC-WSR achieved the best performance on Mip-NeRF360 and Tanks & Temples datasets, while EXP-WSR provides the best performance on Deep Blending dataset. See Figure 4 for visual examples that demonstrate LC-WSR performs best at recovering fine details.

View dependent opacity. We examine how our method works with view dependent opacity. Experimentally comparing our method with the view independent opacity method used in 3DGS, we demonstrate view dependent opacity can significantly improve the results, shown in Table 5.

Popping artifacts. As a side effect of not relying on sorting, our methods also eliminates “popping” artifacts. Figure 5 shows an experiment in which we slightly rotate the camera, causing the black Gaussian to suddenly “pop” out in 3DGS, generating noticeable temporal artifacts, whereas our method produces temporally consistent results.

Learnable parameters. Traditional OIT methods rely on predefined parameters, while our method optimizes the parameters during training. We compare fixed parameters vs. our learnable parameters in Table 5, which shows that our learnable parameters significantly improve the performance.

**Table 4: Comparison between WSR methods on the Mip-NeRF360, Tanks & Temples and Deep Blending datasets.**
| Method | Mip-NeRF 360 | Tanks&Temples | Deep Blending |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ |  |
| DIR-WSR | 25.99 | 0.778 | 0.262 | 22.80 | 0.821 | 0.218 | 28.85 | 0.899 | 0.254 |
| EXP-WSR | 26.97 | 0.801 | 0.216 | 23.32 | 0.833 | 0.183 | 29.77 | 0.902 | 0.229 |
| LC-WSR | 27.19 | 0.804 | 0.211 | 23.61 | 0.842 | 0.177 | 29.63 | 0.902 | 0.229 |

**Table 5: Ablation study on the Mip-NeRF360, Tanks & Temples and Deep Blending datasets.**
| Method | Mip-NeRF 360 | Tanks & Temples | Deep Blending |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ |  |
| Ours | 27.19 | 0.804 | 0.211 | 23.61 | 0.842 | 0.177 | 29.63 | 0.902 | 0.229 |
| w.o. learnable parameters | 23.19 | 0.711 | 0.318 | 21.55 | 0.788 | 0.245 | 27.92 | 0.893 | 0.260 |
| w.o. view-dependent opacity | 25.88 | 0.784 | 0.251 | 21.83 | 0.798 | 0.237 | 29.27 | 0.902 | 0.249 |

## 6 Conclusions

This paper introduces a novel sort-free Gaussian Splatting method, enabled by its use of learned non-commutative blend weight functions, a technique we term Weighted Sum Rendering. We presented three variants of Weighted Sum Rendering: DIR-WSR, EXP-WSR, and LC-WSR. These variants effectively eliminate the need for the sort operation in 3DGS. Additionally, we developed a view-dependent opacity technique that significantly enhances reconstruction fidelity in sort-free Gaussian Splatting. Our experimental results demonstrate that WSR not only achieves competitive visual performance compared to 3DGS but also operates 1.23 times faster on mobile devices.

Limitations and Future Work. Early-Termination. In 3DGS a pixel can be easily terminated by monitoring if its accumulated alpha has been saturated. Alternately, we could implement heuristic to halt further evaluation of Gaussians whose opacity falls below some threshold, though our current WSR implementation does not support such an optimization. We implemented our method using the traditional graphics pipeline to make efficient use of mobile GPUs, especially the hardware rasterizer. In our implementation, opacity and weight computations are performed in the fragment shader, while blending is handled by the hardware, since every fragment operates independently and does not have access to the accumulated opacity, we are unable to easily implement early termination. This would require reading the render target’s content within the invoking draw pass, which necessitates costly read-modify-write operations and requires Vulkan extensions that many mobile devices do not currently support. Additionally, since the screen-space location of each Gaussian is determined only after the vertex shader, a pixel can only be discarded very late in the pipeline. Compact Gaussians. Recently, several compact Gaussian methods have been introduced to significantly reduce the number of Gaussians while maintaining visual performance Jo et al. (); Lee et al. (); Fan et al. (); Niedermayr et al. (); Chen et al. (); Morgenstern et al. (); Navaneet et al. (). We believe that our sort-free method can benefit from the rapid pace of research on compact Gaussians.

## Appendix A Implementation details on mobile phones

For the 3DGS-Graphic method, the pre-processing and the lighting steps remain the same as the original 3DGS implementation except for that they are now written in GLSL compute shaders. We changed the rasterization step from a compute based pipeline to traditional graphic pipeline using instanced draw, where each Gaussian treated as an instance and covered by a triangle pair in vertex shader. The fragment shader then colors the pixels within a Gaussian’s radius. The alpha blending is handled by the hardware automatically at the rendering back-end. With the graphic pipeline, all Gaussians are rendered using a single instanced draw call instead of the per-tile fashion used by 3DGS. The sorting step is also changed from a tile-based sort to global sort, which is implemented using GLSL compute shader.

For our method, the pre-processing and the lighting stages remain the same as the 3DGS method in Vulkan except for that now opacity is calculated in the lighting stage based on camera view and SHs. Sorting step is removed and we also pass depth for each Gaussian to the graphic pipeline to calculate the per-Gaussian weight. The alpha blending operation is changed from using the blending factor to simple summation of the color and weight. We added an extra graphic pass using the Vulkan subpass feature to handle the final normalization. To make the 3DGS-Graphic method and our method a fair comparison, the render target for both is set to use 16-bit RGBA. Note that the original 3DGS can be implemented using 8-bit RGBA with a slightly lower image quality and better rendering speed, but because our method requires the color and weight to be accumulated, we used 16-bit to avoid overflow issue.

## Appendix B Per Scenes Evaluation Metric

Table 6,  7, and  8 present the PSNR, SSIM, and LPIPS metrics for each scene within Mip-NeRF360, Tanks & Temples, and Deep Blending datasets, respectively.

**Table 6: PSNR scores of our method on the Mip-NeRF360 dataset, the Tanks&Temples dataset, and the Deep Blending dataset.**
| Method | Mip-NeRF360 | Tanks&Temples | Deep Blending |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| bicycle | flowers | garden | stump | treehill | room | counter | kitchen | bonsai | Avg | truck | train | Avg | drjohnson | playroom | Avg |  |
| Plenoxels | 21.91 | 20.10 | 23.49 | 20.66 | 22.25 | 27.59 | 23.62 | 23.42 | 24.67 | 23.08 | 23.22 | 18.93 | 21.08 | 23.14 | 22.98 | 23.06 |
| INGP-Base | 22.19 | 20.35 | 24.60 | 23.63 | 22.36 | 29.27 | 26.44 | 28.55 | 30.34 | 25.30 | 23.26 | 20.17 | 21.72 | 27.75 | 19.48 | 23.62 |
| INGP-Big | 22.17 | 20.65 | 25.07 | 23.47 | 22.37 | 29.69 | 26.69 | 29.48 | 30.69 | 25.59 | 23.38 | 20.46 | 21.92 | 28.26 | 21.67 | 24.96 |
| Mip-NeRF360 | 24.31 | 21.65 | 26.88 | 26.18 | 22.93 | 31.47 | 29.45 | 31.99 | 33.40 | 27.69 | 24.91 | 19.52 | 22.22 | 29.14 | 29.66 | 29.40 |
| 3DGS | 25.25 | 21.52 | 27.41 | 26.55 | 22.49 | 30.63 | 28.70 | 30.32 | 31.98 | 27.21 | 25.18 | 21.09 | 23.14 | 28.77 | 30.04 | 29.41 |
| Ours-LC | 24.20 | 20.45 | 27.78 | 25.39 | 22.01 | 31.93 | 29.53 | 31.38 | 32.05 | 27.19 | 25.28 | 21.93 | 23.61 | 29.25 | 30.00 | 29.63 |

**Table 7: SSIM scores of our method on the Mip-NeRF360 dataset, the Tanks&Temples dataset, and the Deep Blending dataset.**
| Method | Mip-NeRF360 | Tanks&Temples | Deep Blending |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| bicycle | flowers | garden | stump | treehill | room | counter | kitchen | bonsai | Avg | truck | train | Avg | drjohnson | playroom | Avg |  |
| Plenoxels | 0.496 | 0.431 | 0.606 | 0.523 | 0.509 | 0.8417 | 0.759 | 0.648 | 0.814 | 0.626 | 0.774 | 0.663 | 0.719 | 0.787 | 0.802 | 0.795 |
| INGP-Base | 0.491 | 0.450 | 0.649 | 0.574 | 0.518 | 0.855 | 0.798 | 0.818 | 0.890 | 0.671 | 0.779 | 0.666 | 0.723 | 0.839 | 0.754 | 0.797 |
| INGP-Big | 0.512 | 0.486 | 0.701 | 0.594 | 0.542 | 0.871 | 0.817 | 0.858 | 0.906 | 0.699 | 0.800 | 0.689 | 0.745 | 0.854 | 0.779 | 0.817 |
| Mip-NeRF360 | 0.685 | 0.583 | 0.813 | 0.744 | 0.632 | 0.913 | 0.894 | 0.920 | 0.941 | 0.792 | 0.857 | 0.660 | 0.759 | 0.901 | 0.900 | 0.901 |
| 3DGS | 0.771 | 0.605 | 0.868 | 0.775 | 0.638 | 0.914 | 0.905 | 0.922 | 0.938 | 0.815 | 0.879 | 0.802 | 0.841 | 0.899 | 0.906 | 0.903 |
| Ours | 0.744 | 0.580 | 0.872 | 0.728 | 0.614 | 0.925 | 0.909 | 0.923 | 0.938 | 0.804 | 0.882 | 0.802 | 0.842 | 0.898 | 0.906 | 0.902 |

**Table 8: LPIPS scores of our method on the Mip-NeRF360 dataset, the Tanks&Temples dataset, and the Deep Blending dataset.**
| Method | Mip-NeRF360 | Tanks&Temples | Deep Blending |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| bicycle | flowers | garden | stump | treehill | room | counter | kitchen | bonsai | Avg | truck | train | Avg | drjohnson | playroom | Avg |  |
| Plenoxels | 0.506 | 0.521 | 0.386 | 0.503 | 0.540 | 0.4186 | 0.441 | 0.447 | 0.398 | 0.463 | 0.335 | 0.422 | 0.379 | 0.521 | 0.499 | 0.510 |
| INGP-Base | 0.487 | 0.481 | 0.312 | 0.450 | 0.489 | 0.301 | 0.342 | 0.254 | 0.227 | 0.371 | 0.274 | 0.386 | 0.330 | 0.381 | 0.465 | 0.423 |
| INGP-Big | 0.446 | 0.441 | 0.257 | 0.421 | 0.450 | 0.261 | 0.306 | 0.195 | 0.205 | 0.331 | 0.249 | 0.360 | 0.305 | 0.352 | 0.428 | 0.390 |
| Mip-NeRF360 | 0.301 | 0.344 | 0.170 | 0.261 | 0.339 | 0.211 | 0.204 | 0.127 | 0.176 | 0.237 | 0.159 | 0.354 | 0.257 | 0.237 | 0.252 | 0.245 |
| 3DGS | 0.205 | 0.336 | 0.103 | 0.210 | 0.317 | 0.220 | 0.204 | 0.129 | 0.205 | 0.214 | 0.148 | 0.218 | 0.183 | 0.244 | 0.241 | 0.243 |
| Ours | 0.205 | 0.342 | 0.097 | 0.235 | 0.311 | 0.197 | 0.191 | 0.125 | 0.199 | 0.211 | 0.136 | 0.219 | 0.177 | 0.233 | 0.225 | 0.229 |

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: In the paper, the author points out that for some complex scenes, since a large number of 3D Gaussians are needed for representation, the sorting process will exhaust the resources of mobile devices, making real-time rendering impossible, such as the bicycle scene in Mip-NeRF 360 dataset. However, I tried to visualize the bicycle scene using a gsplat-based webgl viewer on my phone (iPhone 15 Pro Max), and was surprised to find that it achieves real-time rendering, about 20-30fps. This seems to indicate that some optimizations to the 3DGS rendering pipeline (gsplat uses some tricks) can increase the rendering speed on mobile devices without more complex designs. I would like to know the author's opinion on my point of view and more analyses of the key differences that enable real-time performance on mobile devices. And they can test the performance of gsplat on Snapdragon® 8gen3 chipset and compare with their proposed method.

Here is the link of gsplat webgl-viewer https://gsplat.tech
soundness: 3
strengths: - The author provides a clear analysis of the tile-based sort in the vanilla 3DGS, explains its impact on rendering speed, and explains why it is difficult to implement on other devices.
- The proposed method speeds up rendering while ensuring rendering quality, and has been verified on consumer mobile devices.
- The proposed weighted sum approximation alleviates the *pop* phenomenon caused by inconsistent sorting results under different view-directions in the vanilla 3DGS.
summary: This paper proposed a sort-free Gaussian Splatting pipeline, which reduces the computational overhead of rendering to adapt to scenarios with limited computing resources. The key insight of this paper is that the view dependent sorting in the vanilla 3DGS involves a lot of computational overhead and is difficult to port to mobile devices. Therefore, the authors proposed to use weighted sum and view-dependent opacity to approximate alpha-blending. Experiments demonstrate that proposed method achieves comparable rendering quality and faster rendering speed than the original 3DGS on mobile devices.
weaknesses: - The proposed method gets rid of the costly sorting, but a more complex weight and view-dependent opacity are used. Although the author points out in the paper that a general scheme for learning scene representations is bound only by the mathematical constraints of the scene model. According to the experimental results in the paper, the use of simple weighted-sum and view-independent opacity cannot achieve satisfactory rendering results. Does this mean that for rendering processes that are not physically and optically, more complex models must be used, such as introducing more learnable parameters as in the paper?
- In the conclusion part of the paper, the author mentioned some recent research on compact Gaussian Splatting. According to my understanding of these works, since the methods proposed in these works obtain more compact 3D Gaussian representations, they can reduce the sorting overhead and speed up rendering to a certain extent. Therefore, I think the author should add a comparison of rendering quality, speed and memory usage between the proposed method and these works, such as Compact 3DGS[1], LightGaussian[2] and Compressed 3DGS[3] in the experimental evaluation section.

[1] Compact 3D Gaussian Representation for Radiance Field https://openaccess.thecvf.com/content/CVPR2024/papers/Lee_Compact_3D_Gaussian_Representation_for_Radiance_Field_CVPR_2024_paper.pdf

[2] LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS https://arxiv.org/pdf/2311.17245

[3]  Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis https://openaccess.thecvf.com/content/CVPR2024/papers/Niedermayr_Compressed_3D_Gaussian_Splatting_for_Accelerated_Novel_View_Synthesis_CVPR_2024_paper.pdf

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: - Does the method exhibit color shift artifacts during camera movement along the z-direction, as suggested by the concerns regarding linear weighting?
- Since opacity is not capped at 1, how did you enforce the data range of final color, simply clamp?
- How do other Gaussian attributes maintain their geometric meaning in this framework? Specifically: Does the mean of Gaussians still correspond to point cloud locations? How about depth extraction?
- While traditional 3DGS implementations employ culling to manage computational complexity, does this approach require weighted summation of ALL Gaussians? What are the implications for scalability?
- Please clarify the memory usage analysis: does it encompass only Gaussian attributes or include runtime memory requirements?
What is the memory overhead of spherical harmonics coefficients for view-dependent opacity compared to vanilla 3DGS?
- Regarding Table 2, could you elaborate on the starred data points showing significantly higher values? How does resource exhaustion due to sorting impact subsequent pipeline stages?
- Given the potential for hardware-accelerated sorting methods specifically designed for 3DGS, possibly through hardware-software co-design, how might this work adapt to or complement such future developments?
soundness: 2
strengths: - The paper aims to address a fundamental bottleneck in 3DGS rendering by proposing a sort-free pipeline, distinguishing itself from previous approaches that primarily focused on reducing Gaussian counts.
- The investigation of various weight derivation functions (direct sum-up, exponential weighted, and linear correction weighted) demonstrates a relative comprehensive experimental analysis effort.
- Implementation and evaluation on mobile devices effectively demonstrates the practical benefits of the sort-free approach in resource-constrained environments.
- The method appears to mitigate "pop" artifacts, a persistent challenge in volumetric rendering, providing an additional benefit beyond computational efficiency.
summary: The authors propose a novel rasterization backend for 3D Gaussian splatting (3DGS) that addresses the computational and memory overhead associated with depth sorting. Their primary contribution is replacing traditional alpha blending with a weighted sum rendering approach. The rendering is achieved through direct weighted summation of Gaussians, where weights are determined by a parameterized truncated linear function applied to the product of color and opacity. Additionally, they introduce view-dependent optimizable opacity for individual Gaussians, modeled using spherical harmonics.
Their weighted sum rendering (WSR) technique demonstrates comparable rendering quality to conventional 3DGS. To validate the advantages of their sort-free approach, the authors implement both vanilla 3DGS and WSR on the Vulkan API and evaluate performance on mobile GPU hardware, achieving a 1.23x speedup.
weaknesses: - The theoretical foundation for replacing alpha blending with an algebraic function requires stronger justification. While the proposed representation achieves good results, it lacks the physical insights inherent in traditional alpha blending, which models light obstruction during propagation. The introduction of view-dependent opacity appears necessary for acceptable rendering quality, but the paper would benefit from a more rigorous analysis of why this combination works, rather than empirical validation alone.
- The preference for linear weights over exponential weights raises questions about robustness. When considering camera movement along viewing rays, exponential weights (with β ≈ 1) should theoretically provide more consistent mixing ratios between Gaussians. Linear weights may introduce color shift artifacts due to distance-dependent mixing ratio variations. While the authors claims "some artifacts remain visible" then selected linear weight rather than exponential weight, these issues warrant more detailed examination.
- The method's generalizability is limited by the transformation of opacity into a view-dependent "black box" variable. This fundamentally alters the role of opacity, which traditionally serves as a crucial signal for pruning and densification in many 3DGS applications, potentially limiting compatibility with existing and future research in this domain.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: 1. Can the authors provide some qualitative results on the Bicycle scene?
2. Can the authors provide some videos showing the removal of the popping effect?
3. Can the authors show some failure cases of the proposed approximation?

I really want to give a positive rating to this paper, because I know it is very difficult to make a sort-free rendering to achieve the same quality as the sort-based version. However, the marginal speed improvement diminishes the significance of this paper. Furthermore, I hope the authors can demonstrate some failure cases of such approximation methods, as they are quite common in traditional sort-free algorithms. Doing this could give the reader a more comprehensive understanding of the proposed method.
soundness: 3
strengths: **Motivation**
* The motivation of this paper is very clear. It clearly illustrates the drawbacks of the sorting requirement of 3DGS and the possible benefit of removing it. Sort-free transparent object rendering is also a well-studied direction in traditional rendering. Combining the algorithms from traditional computer graphics with 3DGS is well-motivated.

**Method**
* The proposed method transformed the traditional depth-based weighted rendering to a learnable version to better suit the per-scene optimization used in 3DGS, which provides significant performance improvement. 
* The proposed view-dependent opacity is an innovative approach that mimic the occlusion effect. Because this SH parameters are optimized together, it compensates for some performance loss from incorrect occlusion estimation without sorting.

**Experiments**
* The performance on normal desktop GPU demonstrates a small improvement in rendering speed and comparable rendering speed. In fact, it is already remarkable that a sort-free rendering algorithm can match the performance of a sorting-based rendering algorithm in a scene full of semi-transparent primitives.
* The performance improvement on the edge device shows a more significant improvement because of the demanding sorting step.
summary: This paper proposes a sort-free variant to the 3DGS algorithm. Because 3DGS is a semi-transparent representation, it requires sorting of all primitives which leads to higher running time, more hardware requirements, and lower compatibility on edge devices. Learning from the traditional sort-free rendering algorithm which replaces the sorting with an estimated weight from depth, this paper proposes to eliminate the sorting process of 3DGS completely. 

This paper further proposes a view-dependent opacity to better simulate the occlusion effect. By learning spherical harmonics to represent the opacity, it allows the 3D Gaussians to appear and disappear at different angles, which could further mimic the occlusion effect.

As a result, this paper achieves a slightly higher rendering speed and lower GPU memory footprint, better compatibility on edge devices, and comparable rendering quality than 3DGS.
weaknesses: ** Motivation **
* As shown in the paper, the sorting step is usually considered slow, but actually contributes to only 25% of the total rendering time. The real problem preventing 3DGS based method from matching the rendering speed of traditional methods is the alpha blending step. Although this paper removes the sorting step, it still needs the per-pixel alpha blending of Gaussians of calculated weights. As a result, the speed improvement is rather insignificant. 
* Rather than removing the sorting step, many works have proposed to removing the redundant Gaussians directly. These methods seem to provide much more significant speed boost. Of course these two method can be run in parallel, but it still weaken the importance of the proposed method. 

** Method **
* The proposed method is largely borrowed from the traditional sort-free methods. I think this is acceptable because of the long history of sort-free rendering research in traditional computer graphics, it does negatively affect the novelty of the proposed method slightly.

** Experiments**
* As mentioned above, the improvement on standard desktop GPU is rather insignificant. Since sort-free rendering is essentially an approximation of the sort-based rendering, it might not be a good idea to choose this method for such a small improvement while there is a possibility of very bad performance on some scenes. 
* Continuing from the last point, sort-free approximations are usually inaccurate on small and thin structures. For example, I noticed that the proposed method on the bicycle scene in MipNeRF-360 dataset is somewhat lower than the original 3DGS method. This scene contains very thin and fine structures on the bicycle which could be a better illustration of whether the proposed method suffers from similar artifacts like the traditional methods. 
* Despite the significant rendering speed improvement on the edge devices, the paper lacks a detailed explanation of how the edge device running out of resources leads to such a big decrease in rendering speed.

---

confidence: 4
contribution: 4
initial_rating: 8
presentation: 4
questions: 1. In section 4.3 the authors reveal that view-dependent opacity is obtained by the addition of a new set of spherical harmonics related to the depth weighting introduced by the authors. In the experimental section, the authors show that this doesn't impact memory usage during rendering and a rationale given in the removal of the sorting step and the potential reduction in the number of Gaussians. However, this appears to be referring to instantaneous memory requirements. How does the introduction of the new set of spherical harmonics for each Gaussian affect post-training storage requirements of the radiance field? Is a greater storage footprint needed for the radiance fields produced by the proposed methods?
2. Line 53 "maintain rendering differentiable" should be "maintain rendering differentiability".
3. Line 141 "Chris Wyman provide a good survey on these methods" should be "Chris Wyman provides a good survey on these methods"
4. Line 273, "However, DIR-WSR does not work well for the complex scenes, " should be "However, DIR-WSR does not work well for complex scenes, "
5. Line 307 ends with an additional full stop.
soundness: 3
strengths: This work solves a crucial problem in real-time rendering of Gaussian Splatting based radiance fields. By removing the need for depth-based sorting of the Gaussians before rendering, critical memory requirements are reduced enabling the implementation of the methodology on mobile devices or other applications with limited computational capacity. So I feel that the work represents a strong contribution to the field. Other strengths include:
1. Clear and easy to read figures and tables clearly detailing the benefits of the proposed method.
2. A detailed and clear explanation of the proposed method. It is clear what has been done, how this is motivated by previous literature.
3. A strong experimental validation. Comparisons are made against well-recognized metrics and a the latest state-of-the-art methods. The experimental results support well the conclusions reached.
summary: This work attempts to solve a key problem with current Gaussian Splatting techniques, the computational load that occurs during rendering as a result of the non-commutativity of the opacity calculation needed during the rendering process. The authors achieve this by a novel weighting of term added to each Gaussian to replace the traditional opacity term. The weighting term allows for an order independent formulation of the computation of the final colour of the a rendered ray through the field. View-dependence is obtained by the introduction of a spherical harmonic based formulation to the new weighting term similar to that currently used in prior-art Gaussian Splatting methods for view-dependent colour. Performance is evaluated against prior-art techniques.
weaknesses: This work is of quite a high quality. I do not see any specific weaknesses in the methodology or the experimental results. There is one specific minor weakness in the experimental section wherein I would like further information on the memory requirements of the proposed method. I detail the exact question I have in the "Questions" section below.
One final minor issue is the presence of minor grammar errors in the text as I mention in the "Questions" section below.

Meta-review: The paper presents a novel sort-free rendering pipeline for 3D Gaussian Splatting (3DGS). Inspired by order-independent transparency, the method approximates alpha blending using weighted sums, thereby eliminating the need for sorting and reducing rendering time, particularly on mobile devices. Additionally, the proposed approach mitigates pop artifacts typically caused by sorting. 

The motivation is clear, the idea is novel, and the experiments are compelling. The primary weakness of the initial submission, its lack of comparisons with compact 3DGS representations, was effectively addressed in the rebuttal. 

Overall, the paper makes a valuable contribution by presenting a novel sort-free rendering pipeline for 3DGS. The implementation and evaluation on mobile devices convincingly demonstrate the practical advantages of this approach in resource-constrained environments.

Decision: Accept (Poster)