6DGS: Enhanced Direction-Aware Gaussian Splatting for Volumetric Rendering
Authors: Zhongpai Gao
Benjamin Planche
Meng Zheng
Anwesa Choudhuri
Terrence Chen
Ziyan Wu
Venue: ICLR 2025 Poster (2025)
Keywords: 3D Gaussian splatting
6D Gaussian splatting
volumetric rendering
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: The paper introduces 6D Gaussian Splatting (6DGS) for real-time radiance field rendering. It achieves up to a 15.73 dB PSNR boost while using 66.5% fewer Gaussian points over 3DGS.

Abstract: Novel view synthesis has advanced significantly with the development of neural radiance fields (NeRF) and 3D Gaussian splatting (3DGS). However, achieving high quality without compromising real-time rendering remains challenging, particularly for physically-based rendering using ray/path tracing with view-dependent effects. Recently, N-dimensional Gaussians (N-DG) introduced a 6D spatial-angular representation to better incorporate view-dependent effects, but the Gaussian representation and control scheme are sub-optimal. In this paper, we revisit 6D Gaussians and introduce 6D Gaussian Splatting (6DGS), which enhances color and opacity representations and leverages the additional directional information in the 6D space for optimized Gaussian control. Our approach is fully compatible with the 3DGS framework and significantly improves real-time radiance field rendering by better modeling view-dependent effects and fine details. Experiments demonstrate that 6DGS significantly outperforms 3DGS and N-DG, achieving up to a 15.73 dB improvement in PSNR with a reduction of 66.5\% Gaussian points compared to 3DGS. The project page is: https://gaozhongpai.github.io/6dgs/.

Published as a conference paper at ICLR 2025
6DGS: ENHANCED DIRECTION-AWARE GAUSSIAN SPLATTING FOR VOLUMETRIC RENDERING
Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri, Terrence Chen, Ziyan Wu United Imaging Intelligence, Boston MA, USA (first. last] @uu - com
6DGS (Ours) FPS: 315.1
6DGS. (Ows) PSNR. 33.56
6DGS (Ours) FPS: 367.7
6DGS (Ours) PSNR: 42.48
3DGS PSNR: 25.71
FPS: 0.035
FPS: 0.028
3DGS (Fails) PSNR: 26.75
Volumetric Ray Trading (GT)
Volumetric Ray Tracing
Figure Visualizations
volumetric rendering. Top-left: our 6DGS rendering: bottom-right: physically-basee rendering using ray/path tracing; right: comparison with 3DGS over the red regions.
ABSTRACT
Novel view synthesis has advanced significantly with the development of neural ra- diance fields (NcRF) and 3D Gaussian splatting (3DGS). However, achieving high quality without compromising real-time rendering remains challenging, particularly for physically-bare based rendering using ray/path tracing with view-dependent effects. Recently, N-dimensional Gaussians (N-DG) introduced 6D opatial-angulan repre- sentation to better incorporate view-dependent effects, but the Gaussian representa- tion and control scheme sub-optimal. In this paper, we revisit 6D Gaussians and introduce 6D Gaussian Splatting (6DGS), which enhances color and opacity representations and leverages the additional directional information the 6D space for optimized Gaussian control. Our approach is fully compatible with the 3DGS framework and significantly improves real-time radiance field rendering by better modeling view-dependent effects and fine details. Experiments demonstrate that 6DGS significantly outperforms 3DGS and N-DG, achieving up to a 15.73 dB improvement in PSNR with reduction of 66.5% Gaussian points compared 3DGS. The project page is: https:
/ /tlEp:://gaozhonnee
github. 10/ 6dqs/
INTRODUCTION
Novel view synthesis enables the generation of new viewpoints of scene from limited images, underpinning applications in virtual reality, augmented reality, and realistic rendering for films and games. A significant milestone was the development of neural radiance fields (NeRF) (Mildenhall et al. 2020), representing scenes as continuous volumetric functions that map 3D coordinates and viewing directions color and density values. While NeRF captures intricate details and complex lighting effects, its reliance computationally intensive neural networks makes real-time rendering challenging. To address this, 3D Gaussian splatting (3DGS) (Kerbl al. (023) was introduced, using
"Corresponding author

---

Published as a conference paper at ICLR 2025
3D Gaussians represent scenes. By projecting these Gaussians onto the image plane and aggregating their contributions, 3DGS significantly accelerates rendering, achieving real-time performance while maintaining high visual fidelity.
However, both NeRF (Mildenhall et al. 2020) and 3DGS (Kerbl ct al. 2023) face limitations when dealing with view-dependent effects caeisss-pheeoooee phenomena where the appearance of scene changes with the viewing direction due to reflections, refractions, and complex material properties. NeRF addresses view dependency by conditioning the radiance on viewing direction, but capturing high-frequency specular highlights and anisotropic reflections remains challenge. Similarly, 3DGS, constrained by its purely spatial (3D) representation, struggles to model these effects accurately, especially in scenes with glossy surfaces, transparency, or significant anisotropy.
To overcome these challenges, recent work by Diolatzis al. (2024) introduced N-dimensional Gaussians (N-DG), extending the Gaussian representation into higher dimensions by incorporating additional variables such as viewing direction, leading to a 6D spatial-ange -nngular representation. This approach allows for a more expressive model that can capture view-dependent effects, enabling more accurate rendering of complex visual phenomena like specular reflections and refractions. The 6D representation combines position and direction, effectively modeling how light interacts with surfaces from different viewpoints.
Despite these advancements, N-DG presents certain drawbacks. The 6D Gaussian representation and its optimization scheme are suboptimal, resulting in rendering inefficiencies. Specifically, the method tends allocate a significantly higher number of Gaussian points in scenes with strong view dependency to achieve acceptable rendering quality. Conversely, in scenes without substantial view-dependent effects, it may under-utilize resources, leading fewer Gaussian points and potential loss of detail. This imbalance affects both the efficiency and the scalability of the method, making it less practical for a wide range of applications.
In this paper, we propose 6D Gaussian Splatring (6DGS), novel method that integrates the strengths of both 3DGS and N-DG while addressing their respective limitations. First, we enhance the handling of color and opacity within the Gaussians. By refining the representation of these properties, our method more effectively captures the view-dependent effects. This leads to more accurate depiction of scenes with transparent or glossy materials and intricate lighting conditions. Additionally, we develop an optimization scheme that leverages the additional directional information available the 6D representation, which allows for better adaptive control of Gaussian distribution across the scene.
Our 6DGS method is designed to be fully compatible with the existing 3DGS framework. This compatibility ensures that applications and systems currently using 3DGS can adopt method with minimal modifications, allowing for seamless integration and immediate benefits in rendering quality and performance. Furthermore, we provide a comprehensive theoretical analysis of the conditional Gaussian parameters derived from the 6D representation, elucidating their physical significance in the context of rendering and offering insights into how our method models the interaction of light with surfaces from different viewing directions.
We validate method through extensive experiments on two datasets: custom dataset with physically based rendering using ray tracing. named the 6DGS-PBR dataset, and public dataset without strong view-dependent effects, i.e., the Synthetic NeRF dataset (Mildenhall al. (020)) The 6DGS-PBR dataset contains scenes with complex geometries, materials, and lighting conditions that exhibit strong view-dependent effects, making it suitable for testing our method's 's capabilities in handling complex light interactions. Our results demonstrate that 6DGS significantly outperforms existing methods in both rendering quality and efficiency on the 6DGS-PBR dataset and achieves comparable performance on the Synthetic NeRF dataset. Specifically, we achieve up to a 15.73 dB improvement in peak signal-to-noise ratio (PSNR) while using only 33.5% of the Gaussian points compared to 3DGS. These improvements underscore the effectiveness of our approach in capturing fine details and complex view-dependent effects in real-time rendering scenarios.
Our contributions can be summarized as follows:
Enhanced 6D Gaussian Representation and Optimization: We propose 6D Gaussian Splatring (6DGS), which advances the 6D Gaussian representation by improving color and opacity modeling and introducing optimization strategy that leverages directional information. These enhancements result in superior rendering quality with fewer Gaussian points, particularly in scenes exhibiting complex view-dependent effects.
2

---

Published as a conference paper at ICLR 2025
Validation: We validate 6DGS method on a custom physically -based ray tracing dataset (6DGS-PBR), demonstrating its superiority in both image quality and rendering speed compared to existing approaches. Additionally, its generalizability to scenes without strong view-dependent effects.
evaluation
public dataset showcases
Compatibility with the 3DGS Framework: Our 6DGS method is compatible with the existing 3DGS optimization framework, allowing applications to adopt our method with minimal modifications and benefit from enhanced performance without extensive changes. Theoretical Analysis: We provide a theoretical analysis of the conditional Gaussian parame- ters derived from the 6D representation, highlighting their physical significance in rendering and how they contribute to modeling view-dependent effects.
RELATED WORK
The field of novel view synthesis has seen significant advancements in recent years, with various techniques developed to enhance rendering quality and efficiency. This section reviews the most relevant works on 3D Gaussian splatting, N-dimensional Gaussians, and Gaussian-based ray tracing.
3D Gaussian Splatting. 31D Gaussian splatting (3DGS) (Kerbl et al.) 2023) has emerged
significant advancement in computer graphics and 3D vision, achieving high- high-fidelity rendering quality while maintaining real-time performance. Numerous works have been proposed improve rendering quality (Yu et al. [202]] Lu et al. 20044), rendering efficiency (Lee et al. [2024] Bagdasarian jt al.) (2044), and training optimization KKhrradmand al. [2024] (Höllein et al.) (024). as well as to
|224|HHlllin
explore applications (Kocabas et al. | et al. 2024b. Niedermayr et al.] 2024) and extensions
(Charatan et al.] [024]] Luiten et al. [0244] [ et al. 2024 Tang et al. [2024] Gao et al., 2024) of
3DGS. For example, Mip-Splatting Yu al. (0244) introduces Gaussian low- ppas filter based on
Nyquist's ' theorem to address aliasing and dilation artifacts by matching the maximal sampling rate across all observed samples. Compact3D (Lee et al.
2024 applies vector quantization to compress different attributes into corresponding codebooks, storing the index of each Gaussian to reduce storage overhead. 3DGS-MCMC (Kheradmand et al.] (0224) proposes densification and pruning strategies in 3DGS as deterministic state transitions of Markov Chain Monte Carlo (MCMC) samples instead of using heuristics. Many of these advancements and applications of 3DGS can potentially be applied to our 6DGS, which extends 3DGS by incorporating an additional directional component.
N-dimensional Gaussians. To enhance the 3D Gaussian representation, researchers have introduced other Gaussian representations for rendering. 2DGS (Huang et al.) 2024) introduces perspective- accurate 2D splatting process utilizing ray-splat intersection and rasterization to enhance geometry reconstruction. 3D-HGS (Li et al. 2024) proposes a 3D
kernel to improve performance without compromising rendering speed. 4DGS (Yang et al.] (022)) proposes to approximate the underlying spatio-temporal 4D volume of a dynamic scene by optimizing collection of 4D primitives. N-DG (Diolatzis et al.] 2024) introduces N-dimensional Gaussians (N-DG) along with a high- dimensional culling scheme inspired by locality-sensitive sensitive hashing. Specifically, N-DG introduces 10-dimensional Gaussian (10-DG) that incorporates geometry and material information such as world position, view direction, albedoo, and roughness, as well as 6-dimensional Gaussian (6-DG) that includes world position and view direction. Our 6DGS combines the strengths of 3DGS and N-DG for better representation and better adaptive control of Gaussians.
Gaussian-based Ray Tracing. Unlike most 3DGS methods that render Gaussians via rasterization, some approaches have proposed Gaussian-based ray tracing, generally the cost of slower rendering speeds or even lower quality. For instance, [ondor et al. (2024) models scattering and emissive media using mixtures of simple kernel-based volumetric primitives but achieve lower quality and slower speeds compared to 3DGS. Blanc et al. (2024) enables differentiable ray casting of irregularly distributed Gaussians using a BVH structure, but their rendering is slower than rasterization- based methods. Moenne-Loccoz et al. (2024) performs ray tracing with BVH for secondary lighting effects such as shadows and reflections, but this approach is approximately three times slower than rasterization. [Zhou et al. (2024a) proposes a unified rendering primitive based on 3D Gaussian distributions, enabling physically based scattering for accurate global illumination but do not achieve real-time performance. In contrast, our 6DGS method slices 6D Gaussians into conditional 3D Gaussians and renders via rasterization, approximating physically based may-traced images with high fidelity while achieving real-time performance.
3

---

Published as a conference paper at ICLR 2025
3 PRELIMINARY
3.1 6D GAUSSIAN REPRESENTATION
N-dimensional Gaussian (N-DG) (Diolatzis et al. 2024) extends the 3DGS (Kerbl et al. 2023) approach by introducing 6D Gaussian representation. which includes both position and directional information. Specifically, each Gaussian is defined by the following parameters: position (Hp E RD), direction (J4z E R2), covariance matrix (E E 6x6))), opacity (a E R RR),, and color (c E IR$).
The covariance matrix E is modeled 6D matrix that includes both spatial and directional variances. To ensure stability and positive definiteness, we utilize a Cholesky decomposition, parameterizing Σ with lower triangular matrix L LL Diagonal elements are ensured be positive using an exponential activation function, while the off-diagonal elements are constrained within [-1, 1] using sigmoid function.
3.2 SLICE 6D GAUSSIAN TO CONDITIONAL 3DGS
The slicing Gaussians technique is adopted to render 6D Gaussians efficiently. For a given viewing direction d, we compute conditional 3D Gaussian that represents the slice of our 6D Gaussian in that direction. Let X = [Xp. Xa] be the 6D Gaussian random variable, where X, represents the position and Xa represents the direction. The joint distribution is:
(
Up pd
Σ Σ
Edd Σd
X N~N
(1)
where End represents the cross-covariance between position and direction.
For rendering, compute the conditional 3D Gaussian distribution p(XpIXa d), using the properties multivariate Gaussians, as follows:
p(Xp/-Xa = d) N (Moondd Econd),
(2)
where
Moond Mpp + Epd (d pa),
(3)
Econd Ep EpdE I Epd
(4)
The opacity of each Gaussian also depends on the view direction as:
foond exp (-(d- ua)
(d pa))
(5)
Olcond = a fcomd:
(6)
where foond represents the conditional probability density function (PDF) of the directional component, evaluated at the current viewing direction d, and accond is the modulated opacity based on foond-
4
METHOD
4.1
THEORETICAL ANALYSIS CONDITIONAL GAUSSIAN
This section provides theoretical analysis of the conditional Gaussian parameters derived from the 6D Gaussian representation, highlighting their physical meanings in Gaussian splatting.
Conditional Mean (Hcond). The conditional mean Aeond can be interpreted as the Best Linear Unbiased Estimator (BLUE) for the position component Xp, as in Equation linear estimator å BLUE if it is unbiased (E 8) and has the smallest variance among all linear unbiased estimators. The expression for Peond given earlier ensures that the position mean is unbiased and minimizes variance according the Schur complement. The conditional mean Meond represents the expected position of the Gaussian splat in 3D space, adjusting dynamically based on the viewing direction to capture non-planar geometry and parallax effects.
4

---

Published as a conference paper at ICLR 2025
Slice 6DGS to conditional 3DGS
6DGS definition
Conditioned 3DGS
Position: Hp
Mp
Position: Hoond view-dependent
Direction: Ad
Opacity:
Compatible
Meomir
Moonttz
Opacity: Rcond view-dependent
6D covariance: Σ
3D covariance: Eeond enw-independent
View1
View2
3DGS Pipeline
Adaptive density control
3DGS
Differentiable rasterizer
Initialization
Image
Figure 2: Proposed method of direction aware 6DGS compatible with the existing 3DGS pipeline. The position and opacity of the conditional 3D Gaussian are adjusted according to the view direction.
Conditional Covariance (Econd). The conditional covariance Doond is derived as the Schur com- plement of En in the joint covariance matrix Σ, as in Equation This covariance matrix represents the residual uncertainty in X, after accounting for the correlation with Xx. Notably, Ecand remains constant regardless of the viewing direction d. The conditional covariance Exond describes the shape and orientation of the Gaussian, encoding the local surface geometry and the uncertainty Xp after accounting for the correlation with X
Conditional Opacity (Acond). The function feood is derived from the conditional probability density function (PDF) of the directional component in the 6D Gaussian. Specifically, fcond represents the likelihood of the viewing direction d aligning with the mean direction Sud given the directional vari- ance Σg. Equation is standard form the exponent a Gaussian PDF, reflecting the Mahalanobis distance. The opacity Acoond is scaled by foond in Equation O introduce view-dependency. based on the principle that visibility may vary with viewing direction due to anisotropic properties. By modulating the base opacity with fcond, ensure that Okcond dynamically adjusts to reflect the directional characteristics of the Gaussian splat. This approach leverages the mathematical relation- ship between conditional probabilities and Gaussian distributions, enabling realistic and physically plausible rendering of view-dependent effects.
Together, these parameters form robust framework that integrates view-dependent and view- independent characteristics, enabling efficient and realistic rendering, particularly effective for scenes with varied geometries and materials.
4.2 ENHANCED 6D GAUSSIAN REPRESENTATION
In N-DG (Diolatzis et al. 2024). the color is defined by learnable RGB values. To introduce view-dependent effects, we adopt the spherical harmonics representation (e(d)= R + R$, as used in 3DGS. This representation captures the variation in color based the viewing direction. The spherical harmonics functions Y (d) of order are parameterized by the coefficients 8 E R8 The view-dependent spherical harmonics representation
be expressed
Frex
(d) =
Σ
Σ
3 (
Ł= m=-É
where f is the sigmoid function used to normalize the colors.
To better control the effect of the view direction on opacity, we refine the conditional PDF of the directional component as follows:
foond exp (-Nopa (d- Ha) T Ei (d Ha)))
(8)
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 Slice 6DGS to 3DGS, In inference, we pre-compute Econd, and the scale S and rotation R are not required. Only Hoood and Ocond (highlighted blue) need be computed for each rendering.
Input: Lower triangular L, position Mp, direction μd- opacity a, color c, view direction d Output:
Conditional position, Meend, covariance, Econd. opacity Acond, scale S, rotation R 1: Compute covariance matrix as: Σ
LL
Σ
Edd
2: Partition Σ into blocks: Σ
Σ
Ed
3: Calculate the conditional covariance, mean, and opacity:
Econd 2p Ed Epd
Meond = My + Ep (d (ta)
Ocood 02 Scund, where fcund exp (-Xopa (d- (d-vaa) [nd)==/l - Ha))
4: Perform SVD as: Econd UDUT
5: Extract rotation matrix and scale: R = U, S = v diag(D) DD), R.3 = R. sign(det( R))
where 0 Aopa < is a hyper-parameter or per-Gaussian learnable parameter that controls the influence of the view direction on the opacity. When adjusting Aopa typee-parameter, we can modulate the resulting conditional opacity Okcond, thereby controlling the density of the Gaussian during optimization; when treating Aupu per- per-Gaussian learnable parameter, it adapts the level of view-dependency for each Gaussian through training. This allows for finer control over the rendering process, enabling the representation of more complex visual effects.
4.3 IMPROVED CONTROL OF GAUSSIANS
To enhance the control of Gaussians, adapt the explicit adaptive control mechanism from 3DGS, leveraging the additional directional information available in our 6l) Gaussian representation. Instead of relying the high- nigh-dimensional culling scheme used in N-DG, our approach focuses refining Gaussian placement and density based on the scene's 's geometry.
In 3DGS, the adaptive Gaussian densification scheme involves two primary operations: cloning and splitting. Cloning is employed when small-scale geometry is insufficiently covered by existing Gaussians, while splitting is used to divide a large Gaussian into smaller ones when it encompasses fine details the geometry. This scheme requires the scale and rotation of each Gaussian, which are not directly provided in our 6D Gaussian representation.
To extract the necessary scale and rotation information from our 6D representation, we utilize the conditional covariance matrix Econd- By performing Singular Value Decomposition (SVD) Econd- we can decompose the matrix into its principal components, revealing both the rotation and scale of the Gaussian. The decomposition is given by Econd DD where U is orthogonal matrix, and D is diagonal matrix containing the singular values. From this decomposition: the rotation matrix R is derived from the left singular vectors as U, and the scale vector S is obtained by taking the square root of the diagonal elements of D as / diag D). To ensure that the rotation matrix R forms a hight-haneed coordinate system, we adjust its last column based on the sign of its determinant: R_3 R.a sign(det(
This SVD-based decomposition allows to represent the conditional Gaussian as an oriented ellipsoid, with clearly defined rotation and scale. By extracting these components, we can directly apply the adaptive Gaussian densification scheme from 3DGS (Kerbl al.) (023), thereby improving the coverage of small-scale geometry and enhancing the overall quality of the rendered scene.
In 3DGS, Gaussians are pruned when their opacity falls below a minimum threshold Tmin (e,g., Tmin = 0.005 by default) or when they become excessively large. In our 6DGS approach, the conditional opacity acond is also influenced by the opacity parameter λopa- By incorporating Aopas can fine-tune the density Gaussians with greater precision, allowing for more granular control over which Gaussians retained or pruned based their opacity.

---

Published as a conference paper at ICLR 2025
4.4 COMPATIBILITY WITH 3DGS
Algorithm ooutlines the implementation details for converting (i.e., slicing) our 6DGS representation into a subsequent implementation remains identical to that of 3DGS.
format in a single function. Once this slicing operation is performed, the
The proposed 6DGS seamlessly integrates with the existing training framework of 3DGS, including the use of the same loss functions, optimizers, and training hyperparameters (except for the minimum opacity threshold where we set Tmin 0.01). By slicing 6DGS into a conditional 3DGS format, we can directly utilize the adaptive density control and the differentiable rasterization method employed in 3DGS. This compatibility ensures that many downstream applications can effortlessly switch from 3DGS our 6DGS, resulting in enhanced performance without the need for extensive modifications.
Note that the scale S and rotation R are only required during refinement iterations (e.g.. every 100 iterations) for the adaptive density control. Additionally, during inference, Ecand can pre-computed. Therefore, only /Acodd and Oroond need to be computed for each rendering. To further enhance rendering efficiency, the slice operation can be implemented in CUDA.
5 EXPERIMENTS
5.1 EXPERIMENTAL PROTOCOL
Datasets. We evaluate 6DGS on two datasets: the public Synthetic NeRF dataset (Mildenhall et al. 2020) and a custom dataset using physically-based rendering (PBR), which we refer to as the JDGS-PBR dataset. The 6DGS-PBR dataset consists six scenes: 1) oud from the Walt Disney Animation Studios volumetric cloud dataset
| 2) bunny-cl oud, exp os on, and smoke from OpenVDB volumetric model: 3) suzanne, the standard Blender test mesh with a "Glasss BSDF" translucent material applied to it; 4) scan, prepared from real CT scan.
We rendered these scenes in Blender using its PBR engine "Cycle". At the image sizes (with width equal to height) as listed in Table 2 on an NVIDIA Tesla V100 GPU, the rendering times per view were as follows: bunny-cloud took 504.0 seconds per view, cloud 838.6 seconds, explos ion on 35.5 seconds, smoke 71.5 seconds, suzanne 9.1 seconds, and ct-s 28.5 seconds. For the ct-scan object, we generated 360 views with corresponding camera poses and randomly selected 324 images for training and 36 for testing. For each of the other objects, we rendered 150 views, randomly selecting 100 images for training and 50 for testing. We will make our 6DGS-PBR dataset publicly available the community.
Evaluation Metries. We evaluate method's performance using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) for image quality, number of Gaussian points (# point) for size, and Frames Per Second (FPS) for rendering speed.
Implementation. In our experiments, we set Agpa 0.35 and the minimum opacity threshold T = 0.01. For learnable Aopa. we initialize Aopa = 0.35 and make it trainable only during the iterations of 15,000 28,0000. All other parameters are set to their default values in 3DGS (Kerbl et al.) (023). For the ct-scan object, we initialize the point cloud using the marching cubes algorithm as in DDGS (Gao et al. 2024). For the other objects and the Synthetic NeRF dataset (Mildenhall et al. 2020 we randomly initialize the point cloud with 100,000 points within a cube encompassing the scene. Training is performed on single NVIDIA Tesla V100 GPU with 16 GB of memory, using the Adam optimizer (Kingma & Ba, 2014). We set the learning rate to 1 x 10-2 for the 6D covariance parameters and 10 3 for the direction component pda The default learning rates from 3DGS are applied to the remaining parameters.
5.2
COMPARISON WITH
comppares our 6DGS method with 3DGS and N-DG on the 6DGS-PBR dataset. Our 6DGS achieves significantly better image quality, with an average improvement of +10.08 dB PSNR.
https://ww..opevvbbb.om /oddouuu/
7

---

Published as a conference paper at ICLR 2025
3DGS
N-D6
6DGS (Ours)
GT
3DGS
N-DG
6DGS (Ours)
Figure 3: Qualitative comparison of methods on the 6DGS-PBR dataset (zoom in for details).
Table 1: Comparison of methods on the 6DGS-PBR dataset.
3DGS SSIM
N-DG SSIM
6DGS (Ours)
PSNR
points.
PSNR
# points
PSNR
SSIM
points.
bunny cloud
30.75
0.988
21,074
35.48
0.990
530,711
41.57
0.993
6,660
cloud
29.70
0.972
58,233
42.40
0.991
98.149
40.41
0.991
12,657
explosion
26.75
0.953
51.140
40.16
0.989
207.778
42.48
0.991
17.133
smake
28.55
0.969
60,533
41.61
0.992
212,050
40.61
0.992
10,762
suzanne
23.70
0.901
270,001
26.00
0.921
232,145
27.03
0.928
174,746
ct-scan
25.71
0.917
229.683
30.96
0.952
1,073.082
33.56
0.965
182,981
avg
0.950
115,111
36.10
0.973
392,319
37.61
0.977
67,490
while using significantly fewer Gaussian points (a reduction of 41.4% on average) compared to 3DGS. Compared to N-DG, our 6DGS also achieves higher image quality (+1.51 dB on average) and uses significantly fewer Gaussian points (a reduction of 82.8% % on average).
Table 2 presents the rendering speed comparison. In the FPS comparison, we calculated the average FPS from 20 test images, where each image's ' FPS value is the mean of 500 repeated measurements. Because our 6DGS uses fewer Gaussian points, achieves faster rendering speeds compared to both 3DGS and N-DG. We further integrate FlashGS (Feng et al. 2024), open-source CUDA Python library for efficient differentiable rasterization of 3D Gaussian splatting, into our 6DGS rendering pipeline, referred to 6DGS-flast With 6DGS-flashe we achieve average rendering speed of 326.3 FPS, which is sufficient for many real-time applications. Note that, we can further improve our rendering speed by implementing 6DGS slicing algorithm in CUDA.
8

---

Published as a conference paper at ICLR 2025
Table 2: Rendering speed (FPS) comparison of methods the 6DGS-PBR dataset.
bunny cloud
cloud
explosion
smoke
auzanne
ct scan
avg
Image size
1408
1408
1024
1536
1408
1024
N/A
3DGS
49.1
74.4
161.9
96.7
42.0
259.7
114.0
N-DG
27.0
90.7
50.4
46.9
13.3
49.0
6DGS (Ours)
178.4
178.2
120.2
138.6
34.7
276.8
154.5
6DGS-flash (Ours)
315.3
318.0
367.7
345.5
295.9
315.1
326.3
Table 3: Comparison of methods on the Synthetic NeRF dataset.
3DGS
N-DG
6DGS (ours)
PSNR
SSIM
points.
PSNR
SSIM
# points.
PSNRT
SSIM
# points..
chair
35.91
0.987
272.130
30.87
0.956
108,091
35.49
0.986
223,747
drums
26.15
0.955
346.245
24.37
0.927
106,756
26.45
0.953
250.267
ficus
34.49
0.987
295,997
29.82
0,965
59,052
33.45
0.984
197,741
hotdog
37.72
0.985
147,098
33.89
0.971
82,261
37.90
0.985
102,451
lego
35.79
0,983
322,704
29.85
0.948
151,291
35.25
0.980
233,227
materials
29.98
0,960
282,334
26.86
0.938
77,206
30.71
0.967
222.209
mic
35.47
0,992
310,608
29.99
0.968
40,848
36.13
0.992
272.052
ship 30.52
0,905
328,053
26.35
0.862
337,294
30.72
0.903
270,163
avg
33.25
0.969
288,146
29.00
0.942
120,350
33.26
0.969
221,482
In Figure 3] compare our 6DGS method with 3DGS and N-DG in terms of rendering image quality and visualization of the generated point clouds. We observe that 3DGS fails on the bunny-ce c clou, cloud, expl 1 osioi on, and smoke datasets, exhibiting strong visible artifacts. N-DG tends to produce artifacts in the background and generates blurrier images away from the center. In contrast, our 6DGS achieves the best visual quality, closely matching the ground-truth images.
Furthermore, the visualization of the point clouds reveals that both 3DGS and N-DG struggle to generate faithful shapes of the objects. Specifically, 3DGS fails capture the shapes in the bunny-cloud, cloud, explosion. and smoke datasets. N-DG tends to over-generate point clouds from the initialization, resulting shapes resembling the initial marching-cube shape for ct-scan and randomly sampled cube for other objects. In contrast, our 6DGS successfully reconstructs the shapes of the objects.
To evaluate the generalizability of our 6DGS method to scenes without significant view-dependent effects, we conducted experiments on the public Synthetic NeRF dataset (Mildenhall et al. 2020), as shown in Table 3 Our results demonstrate that 6DGS achieves image quality comparable to 3DGS while using significantly fewer Gaussian points (a reduction of 23.1 % on average). In contrast, N-DG produces considerably lower image quality and uses fewer Gaussian points because it is specifically designed to model scenes with view-dependent effects and tends to underperform in scenes without strong view dependency.
5.3 ABLATION STUDY
We conduct ablation experiments on both the 6DGS-PBR dataset and Synthetic NeRF dataset (see Appendix Table 8) to investigate the effects of various components in our 6DGS method. Specifically, we examine the impact of the parameter Aopa in the conditional probability density function (PDF) of the directional component (see Appendix Figure 5) defined as feond = exp(->upa D), where D (d pa) "d - (aa) represents the Mahalanobis distance between the viewing direction and the Gaussian mean direction μas and Aopa is scalar between 0 and that controls the influence of the view direction on opacity. Our ablation experiments include several settings related to Aopa:
No- Setting Aopa
0 results in found exp(O) 1, meaning opacity is not modulated by the view direction.
No-dopa:
Setting Aspa 1 uses the default foond in N-DG, i.e., foond exp(-D). App = 0.35: We set > = 0.35, so found = exp(- -0.35 35 D), providing a balance between modulation and full modulation.

---

Published as a conference paper at ICLR 2025
Table 4: Ablation study the 6DGS-PBR dataset.
bunny-clas cloud
cloud
explosion
suzanne
ct-8can
avg
38.13
37.39
41.05
38.52
26.76
33.27
35.85
No-fapa
38.15
36.16
35.49
29.65
33.31
No-Agu
39.12
39,86
40.55
36.88
26.92
32.99
36.05
PSNR
0.005
40.15
40.46
43.14
41.04
27.09
33.45
37.56
Aupu
0.35
40.47
40.73
42.69
40.45
27.15
33.42
37.49
learnable-Age
41.57
40.42
42.48
40.61
27.03
33,56
37.61
No-SH
6,196
11,356
16,465
9,929
171,472
177,204
65,437
No-food
11,698
31,999
39,736
35,023
349,716
320,879
131,509
No-Aopo 0.005
4,860
11,736
13,582
10,540
158,041
154,997
58,959
points
63,042
51,883
61,507
49,422
301,614
405,332
155,467
Apps 0.35
6,830
12,454
17.051
10,570
172,373
181,539
66,803
learnable-Agp
6,660
12,657
17,133
10,762
174,746
182,981
67,490
Learnable Aopa: Aapa is treated as a trainable parameter between 0 and for each Gaussian, allowing the model to learn the optimal degree of view-dependent opacity.
Additionally, we include ablations of other components:
No-SH: Color is treated learnable RGB parameter per Gaussian, as N-DG, instead of using spherical harmonics (SH) to represent view-dependent color.
T 0.005: The default minimum opacity threshold used 3DGS, whereas our default choice is 7 = 0.01.
These ablation studies help us understand the contribution of each component to the overall perfor- mance of our 6DGS method.
Table presents the results of our ablation studies. Compared to our model with Aopa 0.35, setting No-Seoud (i.e., Aopa = 0) significantly degrades the image quality while increasing the number of Gaussian points. This highlights the importance of incorporating view dependency into the opacity function. Conversely, when set No- (i.e., Aopa 1), the image quality also degrades. but the number of Gaussian points decreases. This suggests that adjusting Aup allows us to balance the trade-off between image quality and the number of Gaussian points, which directly affects rendering speed. When we make Aops a learnable parameter, the model achieves improved image quality with comparable number of Gaussian points.
Furthermore, the No-SII setting, where spherical harmonics (SH) are not used for color representation, results in degraded image quality. This is because SH provides powerful representation for view- dependent color effects. Setting the minimum opacity threshold to 5 0.005 (the default value in 3DGS) achieves similar image quality but requires significantly more Gaussian points. This is because introducing view-dependent opacity decay via the found function necessitates a larger minimum opacity threshold.
CONCLUSION
In this work, introduce 6D Gaussian splatting (6DGS) that builds on the foundations laid by 3DGS and N-dimensional Gaussians (N-DG). By improving the handling of color and opacity within the 6D spatial-angular framework and optimizing the adaptive control of Gaussians using additional directional information, we develop method that not only maintains compatibility with the 3DGS framework but also offers superior rendering capabilities, particularly in modeling complex view- dependent effects. Our extensive experiments the custom physically-basee rendering dataset (6DGS-PBR) demonstrate the effectiveness of 6DGS in achieving higher image quality and faster rendering speed compared to 3DGS and N-DG.
Looking ahead, 6DGS opens avenues for more accurate and efficient real-time volumetric rendering in virtual and augmented reality, gaming, and film production. Future research will explore further optimizations and extensions to enhance the scalability and robustness of the 6DGS framework, as well as its application dynamic scenes and integration with advanced lighting models.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Milena T Bagdasarian. Paul Knoll, Florian Barthel, Anna Hilsmann, Peter Eisert, and Wieland Morgenstern. 3dgs. zip: survey on 3d gaussian splatting compression methods. arXiv preprint arXiv:2407.09510 2024.
Hugo Blanc, Emmaanelll Deschaud, and Alexis Paljic. Raygauss: Volumetric gaussian-basee ray casting for photorealistic novel view synthesis. arXiv preprint arXiv:2408.033: 2024.
David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp- 19457-19467, 2024.
Jorge Condor, Sebastien Speierer, Lukas Bode, Aljaz Bozic, Simon Green, Piotr Didyk, and Adrian Jarabo. Don't I splat your gaussians: Volumetric ray-traced primitives for modeling and rendering 15425
scattering and emissive media, 2024. UtLhttts:://axiii
Stavros Diolatzis, Tobias Zirr, Alexander Kuznetsov, Georgios Kopanas, and Anton Kaplanyan. N-dimensional gaussians for fitting of high dimensional functions. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024.
Guofeng Feng, Siyan Chen, Rong Fu, Zimu Liao, Yi Wang, Tao Liu, Zhilin Pei, Hengjie Li. Xingcheng Zhang, and Bo Dai. Flashgs: Efficient 3d gaussian splatting for large-scale and high-resolution rendering. arXiv preprint arXiv:2408.00 07966, 2024.
Zhongpai Gao, Benjamin Planche, Meng Zheng, Xiao Chen, Terrence Chen, and Ziyan Wu. Ddgs-et: Direction-diseetangiee gaussian splatting for realistic volume rendering. Advances in Neural Information Processing Systems, 2024.
Peter Hedman, Pratul P Srinivasan, Ben Mildenhall, Jonathan T Barron, and Paul Debevec. Baking neural radiance fields for real-time view synthesis. Proceedings of the IEEE/CVF international conference on computer vision, pp. 5875-5884, 2021.
Lukas Höllein, Aljaž Božič, Michael Zollhöfer, and Matthias Nicßner. 3dgs-Im: Faster gaussian- splatting optimization with levenberg-marquarde arXiv preprint arXhv:2409.12822 2024.
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(():133911 2023.
Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Jeff Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 3d gaussian splatting markov chain monte carlo. arXiv preprint arXiv:2404.0995 2024.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1422,694, 2014.
Arno Knapitsch, Jaesik Park, Qian-Yi Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 1614) 2017.
Muhammed Kocabas, Jen- Hao Rick Chang, James Gabriel, Oncel Tuzel, and Anurag Ranjan. Hugs: Human gaussian splats. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 505-515, 2024.
Joo Chan Lee, Daniel Rho, Xiangyu Sun, Jong Hwan Ko, and Eunbyung Park. Compact 3d gaussian representation for radiance field. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21719-21728. 2024.
Haolin Li, Jinyang Liu, Mario Sznaier, and Octavia Camps. 3d-hgs: 3d half- galfsiansni splatting, arXiv preprint XXi:22406.027200 2024.
11

---

Published as a conference paper at ICLR 2025
Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs gs: Structured 3d gaussians for new-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Viston and Pattern Recognition, pp. 20654-20664 2024.
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 3DV, 2024.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Computer Vision-ECCV 2020, pp. 405-421, 2020.
Nicolas Moenne-Loccoz, Ashkan Mirzaei, Or Perel, Riccardo de Lutio, Janick Martinez Esturo, Gavriel State, Sanja Fidler, Nicholas Sharp, and Zan Gojeic. 3d gaussian ray tracing: Fast tracing of particle scenes. arXiv preprint arX 2024.
Simon Niedermayr, Christoph Neuhauser, Kaloian Petkov, Klaus Engel, and Rüdiger Westermann. Application of 3d gaussian splatting for cinematic anatomy on consumer class devices. 2024.
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang. Gang Zeng, and Ziwei Liu. Lgm: Large vie gaussian model for high-resolution 3d content creation. arXiv preprint arXiv:2402.05054, 2024.
Suttisak Wizadwongsa, Pakkapon Phongthawee, Jiraphon Yenphraphai, and Supasom Suwajanakom. Nex: Real-time view synthesis with neural basis expansion. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 8543, 2021.
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20310-20320, 2024.
Zeyu Yang, Hongye Yang, Zijie Pan, Xiatian Zhu, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. International Conference on Learning Representations (ICLR), 2024.
Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19447-19456, 2024.
Yang Zhou, Songyin Wu, and Ling-Qi Yan. Unified gaussian primitives for scene representation and rendering. arXiv preprint arXiv:2406.09733 09733, 2024a.
Zhenglin Zhou, Fan Ma, Hehe Fan, and Yi Yang. Headstudio: Text to animatable head avatars with 3d gaussian splatting. arXiv preprint arXiv:2402.06149 2024b.
12

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 REAL-WORLD SCENES
Table 5: Comparison of methods on real-world datasets: Deep Blending (Hedman et al. 2021 and
Tanks & Temples (Knapitsch et al. (017).
3DGS
N-DG
6DGS (Ours)
Dataset
Scene
PSNR
SSIM
# pponnts
PSNR
SSIM
points
PSNR
SSIM
# points
Deep Blending
dr johnson playroom
29.22
0.898
3,276,989
26.31
0.828
494,494
28.12
0.883
2,074,490
29.74
0.900
2,332,830
27.74
0.866
258,999
29.06
0.892
1,685,626
avg
29.48
0.899
2.804,910
27.03
0.847
376,747
28.59
0.888
1,880,058
Tanks& Temples
train
21.75
0.803
1,100,525
13.26
0.440
996,826
21.95
0.787
839,408
truck
25.08
0.869
2,606,855
13.65
0.462
663,616
25.05
0.859
2,050,162
avg
23.42
0.836
1,853,690
13.46
0,451
830,221
23.50
0.823
1,444.785
Table 6: Comparison of methods on the real-world dataset: Shiny (Wizadwongsa al.
2021
3DGS
6DGS (Ours)
Scene
PSNR
SSIM
# points
PSNR
SSIM
# points
cd
25.51
0.843
1,128,098
28.39
0.895
596,789
crest
18.80
0.622
4.611.724
19.35
0.648
3,144.178
food
18.39
0,500
2,362,888
17.97
0.475
1.190,056
glants
24.24
0.844
2,341,337
24.15
0.826
1,635,315
lab
24.69
0.836
843.202
27.66
0.903
490,878
pasta
15.45
0.373
2,287,582
14.90
0.349
978,188
seasoning
26.25
0.823
1,085,732
26.36
0,811
524,848
tools
26.20
0.908
1,180,973
25.28
0.884
593,190
avg
22.44
0.719
1,980,192
23.01
0.724
1,144,180
We evaluate 6DGS on three real-world datasets: Deep Blending (Hedman et al., (021), Tanks &
Temples (Knapitsch et al. (017). and Shiny (Wizadwongsa et al. 2021) to validate its robustness and effectiveness in practical scenarios, shown in Table 5 and 6 These datasets present diverse challenges, including scenes with varying levels of complexity, strong view-dependent effects, and detailed geometric structures.
In the Deep Blending dataset (Hedman et al. 202I). 6DGS achieves comparable quality to 3DGS, with average PSNR of 28.59 versus 29.48, while reducing the number of Gaussian points by 33%. This demonstrates that 6DGS effectively maintains high-quality rendering even with significantly fewer points, thereby improving computational efficiency without largely compromising accuracy.
On the Tanks & Temples dataset (Knapitsch et al. 2017). which features large-scale outdoor scenes with intricate geometric details, 6DGS outperforms 3DGS with slight improvement in PSNR (23.50 vs. 23.42) and 22% reduction in the number of points used. These results highlight the ability of 6DGS generalize effectively to outdoor scenes and achieve improved efficiency.
The Shiny dataset (Wizadwongsa et al. 2021 known for its challenging scenes with strong view- dependent effects, offers rigorous test for 6DGS's directional modeling capabilities. Following the settings in NeX (Wizadwongsa et al. 2021), we resize the images to 1008 x 567 for ed and 1ab and 1008 x 756 for other scenes. In this dataset, 6DGS excels, particularly in scenes such as ed and lab, where it achieves PSNR gains of +2.88 and +2.97, respectively, compared to 3DGS. Additionally, these gains are achieved while utilizing approximately 50% fewer Gaussian points. These results underscore the strength of 6DGS in handling view-dependent phenomena.
Across all datasets, the improvements in PSNR and point efficiency demonstrate the robustness and versatility of 6DGS in real-world settings. By reducing the number of Gaussian points while maintaining or exceeding rendering quality, 6DGS offers a significant advancement in computational efficiency and scalability for practical applications. This ability to model strong view-dependent effects and detailed geometry efficiently positions 6DGS as a valuable tool for various rendering tasks, from real-time applications high-fidelity visualizations.
13

---

Published as a conference paper at ICLR 2025
3DGS
N-DG
6DGS (Ours)
GT
Figure 4: Qualitative comparison of methods on the subsurface scatter (SSS) dragon scene.
A.2 SUBSURFACE SCATTERING SCENE
Table 7: Comparison of methods on the scene of subsurface scattering (SSS) dragon.
SSS Dragon
PSNR
SSIM
# points
FPS
Train (min)
3DGS
26.57
0.813
269,250
104.6
24
N-DG
33.19
0.936
196,645
86.0
50
6DGS (Ours)
35.00
0.937
128,748
111.5
33
6DGS-Flash (Ours)
324.3
To further validate the versatility of our method, we have added new dragon scene that exhibits both subsurface scattering (SSS) and surface scattering effects. Following the same evaluation protocol as other scenes (except for ct-sc we rendered 1,50 views and randomly selected 100 images for training and 50 for testing. As shown in Table 6DGS significantly outperforms both baselines, achieving the best PSNR (35.00) and SSIM (0.937) while using substantially fewer Gaussian points. Our method also maintains competitive training efficiency (33 minutes 24 minutes for 3DGS and 50 minutes for N-DG) while achieving real-time rendering performance (115.5 FPS, further accelerated to 324.3 FPS with EDGS-Flash) Figure demonstrates that 6DGS achieves superior visual quality and more faithful geometry reconstruction, further validating our method's effectiveness in handling diverse scenarios involving both volumetric and surface-based light transport phenomena.
A.3 VIEW-DEPENDENT OPACITY
Figure illustrates the effect of adjusting the parameter Aupa the conditional probability density function (PDF) fcoma of the directional component, plotted as a function of the Mahalanobis distance D between the viewing direction d and the Gaussian mean direction pag- The Mahalanobis distance is
14

---

Published as a conference paper at ICLR 2025
1.2
Conditinal PDF (/cond)
1.0
0.8
0.6
0.4
exp(-1_0DD
0.2
exp(-0.35:D)
exp(-0-D)
0.0
0.0
0.4
0.8
1.2
1.6
2.0
2.4
Direction Mahalanobis distance (D)
Figure Effect of adjusting Aopa on the conditional probability density function (PDF) of the directional component foond. plotted as the functions of the Mahalanobis distance D between the view direction and the Gaussian mean direction pd-
defined as:
D (d- pa)) T (d pa),
(9)
and the conditional PDF is given by:
Scond exp (-Xopa D).
(10)
When Aopa the Mahalanobis distance has the maximum influence the opacity, meaning that even small deviations in the viewing direction d from the Gaussian mean direction μı significantly reduce feond. Conversely, when Aopa D. resulting in constant foond = 1 regardless of the viewing direction. When Aop 0.35, the Mahalanobis distance D) has moderate influence on the opacity, providing balance between sensitivity to the viewing direction and maintaining opacity over a wider range of directions.
the opacity remains unaffected by the Mahalanobis distance
A.4 MORE ABLATION STUDY
Table 8: Ablation study the Synthetic NeRF dataset.
chair
drums
ficus
hotdog
loga
materials
mic
ship
avg
No-SH
34.87
26. 10
31.91
37.50
34.78
30.13
35.49
30.34
32.64
Noof
35.91
35.18
34.82
No-Auaa
35,53
26.29
33.40
37.88
35.26
30.45
36.03
30.86
33.21
PSNR
0.005
35.22
26.36
33.07
37.52
35.03
30.60
35.57
30.49
32.98
Auga
0.35
35.51
26.37
33.39
35.22
35.98
30.60
33.20
leamable-Xgo
35.49
26.45
33.45
37.99
35.25
30.71
36.13
30.72
33.26
No-SH
204,009
240,153
146,698
99,599
221,974
201,124
245,754
256,769
202,010
No-
217,898
304,075
265.681
128,762
261 2,06 000
273,947
310.472
305.298
258.392
No-Ayp
204,335
242.526
188,295
101,039
223,815
210.190
265,266
271.282
213,344
points
0.005
341,765
374,158
316,975
197,901
344,362
325,540
391,525
370,648
332,859
Aupa 0.35
233.148
247.822
195,205
103,263
274,525
219,917
275.292
264,650
226,728
leamable-Xxo.
223.227
250.267
197.741
102,451
233,807
222.209
272.052
270.163
221,482
Table g presents the results of our ablation studies on the Synthetic NeRF dataset (Mildenhall et al. 2020p. Since this dataset lacks strong view-dependent effects, the influence of Auga less significant compared to the 6DGS-PPR PBR dataset (see 4). Compared to our model with Xuga 0.35, setting No (i.c., Aspa = 0) slightly degrades image quality and increases the number of Gaussian points. Conversely, when set No-Àppa (i.e., Appa = 1), both the image quality and the number of Gaussians remain comparable. Notably, when we make Aopa a learnable parameter, the model achieves improved image quality with comparable number of Gaussian points. Furthermore, the No-SH setting results in degraded image quality. Setting the minimum opacity threshold to - = 0.005 (the default value in 3DGS) also degrades the performance and requires more Gaussian points.
15

---

Published as a conference paper at ICLR 2025
Table 9: Training time (min) comparison of methods on the 6DGS-PBR dataset.
Dunny-clour cloud
explosion
smoke
suzanne
ct-scan
avg
3DGS
29
38
30
N-DG
184
57
68
131
96
6DGS (Ours)
33
35
31
40
55
20
36
A.5 TRAINING TIME COMPARISON
As shown in Table 9 our 6DGS demonstrates competitive training efficiency, requiring only 20% more time than 3DGS (36 vs 30 minutes) while being significantly faster than N-DG (96 minutes). Our training time could be further reduced by implementing Algorithm I in CUDA.
A.6 MORE QUALITATIVE RESULTS
Figure showcases additional qualitative results from the 6DGS-PBR dataset, highlighting the supe- rior visual quality achieved by our 6DGS method compared 3DGS and N-DG. These visualizations emphasize the ability of 6DGS capture fine details and complex view-dependent effects while maintaining high level of geometric fidelity.
Figures 7 and 8 provide qualitative results three real-world datasets: Deep Blending (Hedman itt al., 202T). Tanks & Temples (Knapitsch et al., 2017), and Shiny (Wizadwongsa et al. (021)) Across these datasets, 6DGS demonstrates comparable superior visual quality 3DGS. These results further validate the robustness and versatility of 6DGS in handling diverse and challenging scenarios, both synthetic and real-world.
Finally, we provide the Python implementation code for the 6DGS slicing algorithm in Listing
import torch
def 6dgs_to_3dgs (ly alpha, mu_P, mu_d, d, iteration, refine_ refine_ iterstons, lambda lambda_opa=0) opa=0.35).
Compute the covariance matrix Sigma
Sigma torch.matmul (I, L.T)
Partition Sigma into blocks
Sigma_p = Sigma [;, :3, :3]
sigma_pd = Sigmal:, :3, 3:]
Sigma_d Sigmal: 3:, 3:]
10
Calculate the conditional mean, covariance, opacity
11
Sigma___inv = forch.inverse (Sigma_d)
12
Sigma_regr
(Sigma_ sigma_d_inv)
cond mu torch (Sigma, regr, x.unsqueeze (-1)) squueee
14
Sigma_ Sigma
forch.matmus (Sigma_regs: regr, sigma_pd.I] pd.I)
15
f_cond torch. exp (-1 (-1ambdaa_ppa * torch. ('bi, bij, bj-bbb' x,
$igma_d_inv, x) unsqueeze (-1))
alpha_cond alpha f_cond
18
s, None, None
19
if iteration in refine refine_iterations iterations:
20
# Porform SVD on the conditions covariance matrix
U, S, V torch, svd (Sigma
22
Extract the rotation matrix and scale U
23
24
R[:, 2] *- torch. sign (torch. (R) unsqueeze (-1))
terch. sqrt (S) return cond, Sigma_come cond, aipha_cond, s, R
Listing I: Python code for slicing 6DGS to conditional 3DGS
16

---

Published as a conference paper at ICLR 2025
3DGS
N-DG
6DGS (Ours)
GT
Figure 6: More qualitative comparison of methods on 6DGS-PBR dataset (zoom in for details).
17

---

Published as a conference paper at ICLR 2025
3DGS
N-DG
6DGS (Ours)
GT
Figure 7: Qualitative comparison of methods on the Deep Blending (Hedman et al. 202 and Tanks
& Temples (Knapitsch al. 2017) datasets (zoom for details).
18

---

Published as a conference paper at ICLR 2025
3DGS
6DGS (Ours)
GT
Figure 8: Qualitative comparison of methods on the Shiny dataset (Wizadwongsa et al. 2021
19


Reviews:
[{"review_id": "qVtnyH6Lyx", "reviewer": "Reviewer_3cPv", "summary": "This paper presents 6D Gaussian Splatting(6DGS), an advanced method for volumetric rendering that enhances traditional 3D Gaussian splatting (3DGS) by incorporating view-dependent information through a 6D spatial-angular Gaussian representation.\n\nThe authors address key limitations in 3DGS, such as handling specular reflections and anisotropic effects, by leveraging a 6D Gaussian model that adapts both color and opacity based on viewing angles. Experimental results demonstrate significant improvements in both image quality (up to a 15.73 dB PSNR increase) and rendering efficiency, achieving real-time speeds on complex datasets.", "questions": "- Could the authors provide more insights into how the choice of λopa influences performance across diverse datasets?\n\n- How would the model adapt or perform in dynamic scenes with frequent view changes?\n\n- Would implementing 6DGS directly in CUDA offer significant speed improvements for real-time applications, and if so, are there plans for such an implementation?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The paper presents a novel 6D Gaussian Splatting (6DGS) method, which effectively models view-dependent effects through spatial and directional information.\n\nBy improving color and opacity handling, 6DGS produces high-quality, realistic renderings, particularly useful in scenes with complex reflections and lighting. Its compatibility with the 3DGS framework enhances its practical relevance, allowing integration with minimal adjustments. The experiments are well-designed and validate the method's efficiency, achieving significant reductions in Gaussian points without compromising image quality.", "weaknesses": "The method is primarily tested on synthetic datasets, lacking real-world scene evaluations that would confirm its robustness in practical applications. The new parameter λopa adds complexity, as it requires careful tuning. Additionally, while efficient for static scenes, the model’s performance on dynamic, interactive scenes remains untested, limiting its demonstrated versatility. Finally, the initialization phase can be computationally intensive, posing a potential challenge for scaling to large or complex scenes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AC77nvRDyR", "reviewer": "Reviewer_n9Tc", "summary": "This paper proposes an extension of the existing 3D Gaussian Splatting technique to a six-dimensional Gaussian Splatting method. 6DGS combines 3D positional information with 3D view direction information using a six-dimensional Gaussian, enabling it to precisely capture view-dependent variations in color and transparency. To enhance computational efficiency, a restricted covariance method is introduced, which selectively incorporates only the necessary correlations, and spherical harmonics are employed to accurately model view-dependent color changes. Additionally, a new synthetic dataset called 6DGS-PBRT was constructed for performance evaluation, with experimental validation of view-dependent effects. This dataset, comprising synthetic scenes with various lighting and material properties, demonstrates that 6DGS can approximate complex visual effects effectively.", "questions": "1. Why does the proposed 6DGS outperform Ray Tracing-based Gaussian models on PBRT-rendered datasets? Intuitively, Ray Tracing-based models might be expected to perform better. Could it be due to incomplete implementation or insufficient optimization of the Ray Tracing-based model?\n\n2. Although 6DGS is not Ray-Tracing-based, it appears to approximate the complex rendering effects and physical phenomena modeled by PBRT effectively. Could it also achieve high performance on more complex effects, such as subsurface scattering involving multiple scattering events?\n\n3. While the synthetic dataset results already establish the value of this paper’s contributions, is there a specific reason or technical difficulty that prevented experiments on real-world datasets?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The fact that the proposed 6DGS is compatible with the existing 3D Gaussian Splatting (3DGS) framework and requires minimal modification appears to be a significant strength. This compatibility suggests that 6DGS can be effectively integrated into various existing datasets and application environments, potentially achieving impressive performance with minimal adjustments.\n\n2. The theoretical explanations, figures, and experimental results are intuitively presented, making it easy to understand the contribution and performance of the proposed approach. This strong presentation quality greatly enhances readability.\n\n3. The authors constructed a new 6DGS-PBRT dataset specifically to evaluate view-dependent effects, which seems to add considerable value. This dataset enables a more precise validation of 6DGS’s performance and appears to be an effective foundation for demonstrating its utility.\n\n4. The approach provides important insights for the optimization of multi-dimensional Gaussian splatting techniques. By introducing restricted covariance modeling to refine view-dependent effects, this work offers valuable clues for future optimization of higher-dimensional Gaussian models.", "weaknesses": "1. Although experiments on synthetic datasets are meaningful, the lack of experiments on real-world captured datasets may limit the understanding of the model’s performance in real environments.\n\n2. While the authors’ 6DGS-PBRT dataset seems well-suited for view-dependent effects, the limited variety of classes may fall short in representing a wider range of optical phenomena.\n\n3. Additional explanation about the visual characteristics of each dataset would enhance clarity. For instance, clarifying that “cloud” captures absorption and scattering effects, or “glass” represents refraction effects, could make the paper more accessible. It also seems that adding reflective objects or other classes could provide a more comprehensive evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5I3CBVF5JS", "reviewer": "Reviewer_zNRe", "summary": "This paper aims to address the challenge of handling view-dependent effects in scenes, a common issue encountered in both NeRF and 3DGS. Inspired by N-DG, this paper further enhances the 6D spatial-angular representation to better model scene details, particularly in contexts exhibiting complex view-dependent effects. By leveraging additional directional information, the proposed method achieves an improved representation of opacity and color while utilizing fewer Gaussians, thereby enhancing rendering speed. In addition, this paper offers a theoretical analysis of the conditional Gaussian parameters derived from the 6DGS representation, revealing its physical significance and contributions to enhancing rendering quality.", "questions": "Based on the aforementioned weaknesses, the following issues also require discussion:\n- Given that N-DG demonstrates rendering results on real data, and as mentioned in line 62 of the paper, the proposed method can handle scenes with glossy surfaces and transparency, can the method achieve better modeling and rendering results when applied to the real-world dataset collected by NeX? As NeX proposes view-dependent pixel representation that can handle non-Lambertain surfaces and model fine details, the author should conduct comparative experiments with NeX.\n\n[1] NeX: Real-time View Synthesis with Neural Basis Expansion, CVPR 2021.\n\n- Since N-DG can be extended to a higher-dimensional representation, was the dimensional setting for this method in the comparative experiments set as 6D?\n\n- As the method proposed in the paper is based on per-scene optimization, the authors should consider providing a comparison of training times across different scenes. This would help understand the contributions of the method better.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The method proposed in this paper enhances the modeling of opacity and color while reducing the scale of Gaussians. By slicing the 6DGS to conditional 3DGS, it further improves rendering speed.\n- This paper offers a theoretical analysis of 6DGS that enhances rendering quality, aiding in the understanding of its contribution to modeling complex scenes, particularly the scenes with view-dependent effects.\n- This paper clearly defines the problem to be addressed and proposes the 6DGS representation, which is beneficial for overcoming the limitations of previous representations in applications. Additionally, it provides some validation across multiple datasets.", "weaknesses": "- The experimental validation is insufficient. (1) The method proposed in this paper emphasizes its capability to model complex scenes exhibiting view-dependent effects; however, the qualitative experiments are limited to results derived from the 6DGS-PBRT dataset proposed in this paper. The authors should provide additional visualization results, such as those derived from the Synthetic NeRF dataset to strengthen the effectiveness of the proposed method. (2) The paper only presents results for two synthetic datasets, which does not adequately demonstrate the robustness and applicability of the proposed method in real-world scenarios.\n\n- Some important analyses are missing. Due to the challenges of camera pose estimation in real-world scenes with view-dependent effects, and considering that camera pose influences Gaussian density control, the authors should consider conducting further analysis on this issue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uawESFyiGs", "reviewer": "Reviewer_cn4m", "summary": "3DGS excels at synthesizing novel views of scenes, but it struggles with showing effects that depend on the viewer's perspective, like reflections, refractions, and scattering caused by complicated light transport. To solve this issue, the paper introduces a method called 6D-GS, which is based on N-dimensional GS. This method represents a scene as a 6D signal to capture both the spatial position and view direction at the same time. The paper enhances N-dimension GS by incorporating a view-dependent opacity function and an optimized density control. Extensive experiments on two synthetic datasets demonstrate the effectiveness of the proposed method.", "questions": "I am interested in learning more about the potential applications of the algorithm. Can the algorithm be used for general radiance field reconstruction  instead of using synthetic data in dense view settings? If so, can you provide results from real-world datasets such as NeX? \nIf not, it would be helpful to understand the specific scenarios where algorithms can be applied and to discuss any limitations.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The paper is well-written and easy to follow. The details are clearly described, and I believe the results can be reproduced by an expert in this field.\n\n- The method is simple and easy to implement. It allows slicing the 6D Gaussian given a view direction into a 3D Gaussian, which can then be directly fed into the existing 3D Gaussian rasterizer.\n\n- Experiments show impressive results in the 6DGS-PBRT datasets, achieving 10dB over 3DGS and 1.5dB over N-D Gaussian. The results may interest research that aims to overfit high-dimension data.", "weaknesses": "The main weakness is the absence of supplemental video, which makes it difficult to access the novel view synthesis results. \n\nFrom my understanding, allocating more capacity to model view dependency of a scene might hinder the geometry and novel view synthesis. For instance, early NeRF uses a relatively small MLP with fewer Fourier features to strike a balance. \n\nFurthermore, the introduction of view-dependent opacity also complicates the results. For instance, while making the opacity view-dependent allows for easy fitting of high specular reflection, it is possible that when you view the specular spot from a slightly different angle, it becomes transparent. Since the PSNR metric may not provide a clear indication, it is recommended to include a video demonstration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)