6DGS: Enhanced Direction-Aware Gaussian Splatting for Volumetric Rendering

## Abstract

Abstract Novel view synthesis has advanced significantly with the development of neural radiance fields (NeRF) and 3D Gaussian splatting (3DGS). However, achieving high quality without compromising real-time rendering remains challenging, particularly for physically-based ray tracing with view-dependent effects. Recently, N-dimensional Gaussians (N-DG) introduced a 6D spatial-angular representation to better incorporate view-dependent effects, but the Gaussian representation and control scheme are sub-optimal. In this paper, we revisit 6D Gaussians and introduce 6D Gaussian Splatting (6DGS), which enhances color and opacity representations and leverages the additional directional information in the 6D space for optimized Gaussian control. Our approach is fully compatible with the 3DGS framework and significantly improves real-time radiance field rendering by better modeling view-dependent effects and fine details. Experiments demonstrate that 6DGS significantly outperforms 3DGS and N-DG, achieving up to a 15.73 dB improvement in PSNR with a reduction of 66.5% Gaussian points compared to 3DGS. The project page is: https://gaozhongpai.github.io/6dgs/

## 1 Introduction

Novel view synthesis enables the generation of new viewpoints of a scene from limited images, underpinning applications in virtual reality, augmented reality, and realistic rendering for films and games. A significant milestone was the development of neural radiance fields (NeRF) (Mildenhall et al., ), representing scenes as continuous volumetric functions that map 3D coordinates and viewing directions to color and density values. While NeRF captures intricate details and complex lighting effects, its reliance on computationally intensive neural networks makes real-time rendering challenging. To address this, 3D Gaussian splatting (3DGS) (Kerbl et al., ) was introduced, using 3D Gaussians to represent scenes. By projecting these Gaussians onto the image plane and aggregating their contributions, 3DGS significantly accelerates rendering, achieving real-time performance while maintaining high visual fidelity.

However, both NeRF (Mildenhall et al., ) and 3DGS (Kerbl et al., ) face limitations when dealing with view-dependent effects—phenomena where the appearance of a scene changes with the viewing direction due to reflections, refractions, and complex material properties. NeRF addresses view dependency by conditioning the radiance on viewing direction, but capturing high-frequency specular highlights and anisotropic reflections remains a challenge. Similarly, 3DGS, constrained by its purely spatial (3D) representation, struggles to model these effects accurately, especially in scenes with glossy surfaces, transparency, or significant anisotropy.

To overcome these challenges, recent work by Diolatzis et al. () introduced N-dimensional Gaussians (N-DG), extending the Gaussian representation into higher dimensions by incorporating additional variables such as viewing direction, leading to a 6D spatial-angular representation. This approach allows for a more expressive model that can capture view-dependent effects, enabling more accurate rendering of complex visual phenomena like specular reflections and refractions. The 6D representation combines position and direction, effectively modeling how light interacts with surfaces from different viewpoints.

Despite these advancements, N-DG presents certain drawbacks. The 6D Gaussian representation and its optimization scheme are suboptimal, resulting in rendering inefficiencies. Specifically, the method tends to allocate a significantly higher number of Gaussian points in scenes with strong view dependency to achieve acceptable rendering quality. Conversely, in scenes without substantial view-dependent effects, it may under-utilize resources, leading to fewer Gaussian points and potential loss of detail. This imbalance affects both the efficiency and the scalability of the method, making it less practical for a wide range of applications.

In this paper, we propose *6D Gaussian Splatting (6DGS)*, a novel method that integrates the strengths of both 3DGS and N-DG while addressing their respective limitations. First, we enhance the handling of color and opacity within the Gaussians. By refining the representation of these properties, our method more effectively captures the view-dependent effects. This leads to a more accurate depiction of scenes with transparent or glossy materials and intricate lighting conditions. Additionally, we develop an optimization scheme that leverages the additional directional information available in the 6D representation, which allows for better adaptive control of Gaussian distribution across the scene.

Our 6DGS method is designed to be fully compatible with the existing 3DGS framework. This compatibility ensures that applications and systems currently using 3DGS can adopt our method with minimal modifications, allowing for seamless integration and immediate benefits in rendering quality and performance. Furthermore, we provide a comprehensive theoretical analysis of the conditional Gaussian parameters derived from the 6D representation, elucidating their physical significance in the context of rendering and offering insights into how our method models the interaction of light with surfaces from different viewing directions.

We validate our method through extensive experiments on two datasets: a custom dataset with physically based rendering using ray tracing, named the *6DGS-PBRT* dataset, and a public dataset without strong view-dependent effects, *i.e*., the Synthetic NeRF dataset (Mildenhall et al., ). The 6DGS-PBRT dataset contains scenes with complex geometries, materials, and lighting conditions that exhibit strong view-dependent effects, making it suitable for testing our method’s capabilities in handling complex light interactions. Our results demonstrate that 6DGS significantly outperforms existing methods in both rendering quality and efficiency on the 6DGS-PBRT dataset and achieves comparable performance on the Synthetic NeRF dataset. Specifically, we achieve up to a 15.73 dB improvement in peak signal-to-noise ratio (PSNR) while using only 33.5% of the Gaussian points compared to 3DGS. These improvements underscore the effectiveness of our approach in capturing fine details and complex view-dependent effects in real-time rendering scenarios.

Our contributions can be summarized as follows:

- •
Enhanced 6D Gaussian Representation and Optimization: We propose *6D Gaussian Splatting (6DGS)*, which advances the 6D Gaussian representation by improving color and opacity modeling and introducing an optimization strategy that leverages directional information. These enhancements result in superior rendering quality with fewer Gaussian points, particularly in scenes exhibiting complex view-dependent effects.
- •
Validation: We validate our 6DGS method on a custom physically-based ray tracing dataset (*6DGS-PBRT*), demonstrating its superiority in both image quality and rendering speed compared to existing approaches. Additionally, our evaluation on a public dataset showcases its generalizability to scenes without strong view-dependent effects.
- •
Compatibility with the 3DGS Framework: Our 6DGS method is compatible with the existing 3DGS optimization framework, allowing applications to adopt our method with minimal modifications and benefit from enhanced performance without extensive changes.
- •
Theoretical Analysis: We provide a theoretical analysis of the conditional Gaussian parameters derived from the 6D representation, highlighting their physical significance in rendering and how they contribute to modeling view-dependent effects.

## 2 Related Work

The field of novel view synthesis has seen significant advancements in recent years, with various techniques developed to enhance rendering quality and efficiency. This section reviews the most relevant works on 3D Gaussian splatting, N-dimensional Gaussians, and Gaussian-based ray tracing.

#### 3D Gaussian Splatting.

3D Gaussian splatting (3DGS) (Kerbl et al., ) has emerged as a significant advancement in computer graphics and 3D vision, achieving high-fidelity rendering quality while maintaining real-time performance. Numerous works have been proposed to improve rendering quality (Yu et al., ; Lu et al., ), rendering efficiency (Lee et al., ; Bagdasarian et al., ), and training optimization (Kheradmand et al., ; Höllein et al., ), as well as to explore applications (Kocabas et al., ; Zhou et al., ; Niedermayr et al., ) and extensions (Charatan et al., ; Luiten et al., ; Wu et al., ; Tang et al., ) of 3DGS. For example, Mip-Splatting (Yu et al., ) introduces a Gaussian low-pass filter based on Nyquist’s theorem to address aliasing and dilation artifacts by matching the maximal sampling rate across all observed samples. Compact3D (Lee et al., ) applies vector quantization to compress different attributes into corresponding codebooks, storing the index of each Gaussian to reduce storage overhead. 3DGS-MCMC (Kheradmand et al., ) proposes densification and pruning strategies in 3DGS as deterministic state transitions of Markov Chain Monte Carlo (MCMC) samples instead of using heuristics. Many of these advancements and applications of 3DGS can potentially be applied to our 6DGS, which extends 3DGS by incorporating an additional directional component.

#### N-dimensional Gaussians.

To enhance the 3D Gaussian representation, researchers have introduced other Gaussian representations for rendering. 2DGS (Huang et al., ) introduces a perspective-accurate 2D splatting process utilizing ray-splat intersection and rasterization to enhance geometry reconstruction. 3D-HGS (Li et al., ) proposes a 3D half-Gaussian kernel to improve performance without compromising rendering speed. 4DGS (Yang et al., ) proposes to approximate the underlying spatio-temporal 4D volume of a dynamic scene by optimizing a collection of 4D primitives. N-DG (Diolatzis et al., ) introduces N-dimensional Gaussians (N-DG) along with a high-dimensional culling scheme inspired by locality-sensitive hashing. Specifically, N-DG introduces a 10-dimensional Gaussian (10-DG) that incorporates geometry and material information such as world position, view direction, albedo, and roughness, as well as a 6-dimensional Gaussian (6-DG) that includes world position and view direction. Our 6DGS combines the strengths of 3DGS and N-DG for better representation and better adaptive control of Gaussians.

#### Gaussian-based Ray Tracing.

Unlike most 3DGS methods that render Gaussians via rasterization, some approaches have proposed Gaussian-based ray tracing, generally at the cost of slower rendering speeds or even lower quality. For instance, Condor et al. () models scattering and emissive media using mixtures of simple kernel-based volumetric primitives but achieve lower quality and slower speeds compared to 3DGS. Blanc et al. () enables differentiable ray casting of irregularly distributed Gaussians using a BVH structure, but their rendering is slower than rasterization-based methods. Moenne-Loccoz et al. () performs ray tracing with BVH for secondary lighting effects such as shadows and reflections, but this approach is approximately three times slower than rasterization. Zhou et al. () proposes a unified rendering primitive based on 3D Gaussian distributions, enabling physically based scattering for accurate global illumination but do not achieve real-time performance. In contrast, our 6DGS method slices 6D Gaussians into conditional 3D Gaussians and renders via rasterization, approximating physically based ray-traced images with high fidelity while achieving real-time performance.

## 3 Preliminary

### 3.1 6D Gaussian Representation

N-dimensional Gaussian (N-DG) (Diolatzis et al., ) extends the 3DGS (Kerbl et al., ) approach by introducing a 6D Gaussian representation, which includes both position and directional information. Specifically, each Gaussian is defined by the following parameters: position ($\mu_{p}\in\mathbb{R}^{3}$), direction ($\mu_{d}\in\mathbb{R}^{3}$), covariance matrix ($\Sigma\in\mathbb{R}^{6\times 6}$), opacity ($\alpha\in\mathbb{R}^{1}$), and color ($c\in\mathbb{R}^{3}$).

The covariance matrix $\Sigma$ is modeled as a 6D matrix that includes both spatial and directional variances. To ensure stability and positive definiteness, we utilize a Cholesky decomposition, parameterizing $\Sigma$ with a lower triangular matrix $L$ as $\Sigma=LL^{\top}$. Diagonal elements are ensured to be positive using an exponential activation function, while the off-diagonal elements are constrained within $[-1,1]$ using a sigmoid function.

### 3.2 Slice 6D Gaussian to Conditional 3DGS

The slicing Gaussians technique is adopted to render 6D Gaussians efficiently. For a given viewing direction $d$, we compute a conditional 3D Gaussian that represents the slice of our 6D Gaussian in that direction. Let $X=[X_{p},X_{d}]$ be the 6D Gaussian random variable, where $X_{p}$ represents the position and $X_{d}$ represents the direction. The joint distribution is:

$$ $X\sim\mathcal{N}\left(\begin{bmatrix}\mu_{p}\\ \mu_{d}\end{bmatrix},\begin{bmatrix}\Sigma_{p}&\Sigma_{pd}\\ \Sigma_{pd}^{\top}&\Sigma_{d}\end{bmatrix}\right),$ (1) $$

where $\Sigma_{pd}$ represents the cross-covariance between position and direction.

For rendering, we compute the conditional 3D Gaussian distribution $p(X_{p}|X_{d}=d)$, using the properties of multivariate Gaussians, as follows:

$$ $\displaystyle p(X_{p}|X_{d}=d)$ $\displaystyle=\mathcal{N}(\mu_{\text{cond}},\Sigma_{\text{cond}}),$ (2) $$

where

$$ $\displaystyle\mu_{\text{cond}}$ $\displaystyle=\mu_{p}+\Sigma_{pd}\Sigma_{d}^{-1}(d-\mu_{d}),$ (3) $\displaystyle\Sigma_{\text{cond}}$ $\displaystyle=\Sigma_{p}-\Sigma_{pd}\Sigma_{d}^{-1}\Sigma_{pd}^{\top}.$ (4) $$

The opacity of each Gaussian also depends on the view direction as:

$$ $\displaystyle f_{\text{cond}}$ $\displaystyle=\exp\left(-(d-\mu_{d})^{\top}\Sigma_{d}^{-1}(d-\mu_{d})\right),$ (5) $\displaystyle\alpha_{\text{cond}}$ $\displaystyle=\alpha\cdot f_{\text{cond}},$ (6) $$

where $f_{\text{cond}}$ represents the conditional probability density function (PDF) of the directional component, evaluated at the current viewing direction $d$, and $\alpha_{\text{cond}}$ is the modulated opacity based on $f_{\text{cond}}$.

## 4 Method

Figure: Figure 2: Proposed method of direction-aware 6DGS compatible with the existing 3DGS pipeline. The position and opacity of the conditional 3D Gaussian are adjusted according to the view direction.
Refer to caption: x2.png

### 4.1 Theoretical Analysis of Conditional Gaussian

This section provides a theoretical analysis of the conditional Gaussian parameters derived from the 6D Gaussian representation, highlighting their physical meanings in Gaussian splatting.

#### Conditional Mean ( μ cond subscript 𝜇 cond \mu_{\text{cond}} italic_μ start_POSTSUBSCRIPT cond end_POSTSUBSCRIPT ).

The conditional mean $\mu_{\text{cond}}$ can be interpreted as the Best Linear Unbiased Estimator (BLUE) for the position component $X_{p}$, as in Equation 3. A linear estimator $\hat{\delta}$ is BLUE if it is unbiased ($E[\hat{\delta}]=\delta$) and has the smallest variance among all linear unbiased estimators. The expression for $\mu_{\text{cond}}$ given earlier ensures that the position mean is unbiased and minimizes variance according to the Schur complement. The conditional mean $\mu_{\text{cond}}$ represents the expected position of the Gaussian splat in 3D space, adjusting dynamically based on the viewing direction to capture non-planar geometry and parallax effects.

#### Conditional Covariance ( Σ cond subscript Σ cond \Sigma_{\text{cond}} roman_Σ start_POSTSUBSCRIPT cond end_POSTSUBSCRIPT ).

The conditional covariance $\Sigma_{\text{cond}}$ is derived as the Schur complement of $\Sigma_{d}$ in the joint covariance matrix $\Sigma$, as in Equation 4. This covariance matrix represents the residual uncertainty in $X_{p}$ after accounting for the correlation with $X_{d}$. Notably, $\Sigma_{\text{cond}}$ remains constant regardless of the viewing direction $d$. The conditional covariance $\Sigma_{\text{cond}}$ describes the shape and orientation of the Gaussian, encoding the local surface geometry and the uncertainty in $X_{p}$ after accounting for the correlation with $X_{d}$.

#### Conditional Opacity ( α cond subscript 𝛼 cond \alpha_{\text{cond}} italic_α start_POSTSUBSCRIPT cond end_POSTSUBSCRIPT ).

The function $f_{\text{cond}}$ is derived from the conditional probability density function (PDF) of the directional component in the 6D Gaussian. Specifically, $f_{\text{cond}}$ represents the likelihood of the viewing direction $d$ aligning with the mean direction $\mu_{d}$ given the directional variance $\Sigma_{d}$. Equation 5 is a standard form of the exponent in a Gaussian PDF, reflecting the Mahalanobis distance. The opacity $\alpha_{\text{cond}}$ is scaled by $f_{\text{cond}}$ in Equation 6 to introduce view-dependency, based on the principle that visibility may vary with viewing direction due to anisotropic properties. By modulating the base opacity $\alpha$ with $f_{\text{cond}}$, we ensure that $\alpha_{\text{cond}}$ dynamically adjusts to reflect the directional characteristics of the Gaussian splat. This approach leverages the mathematical relationship between conditional probabilities and Gaussian distributions, enabling realistic and physically plausible rendering of view-dependent effects.

Together, these parameters form a robust framework that integrates view-dependent and view-independent characteristics, enabling efficient and realistic rendering, particularly effective for scenes with varied geometries and materials.

### 4.2 Enhanced 6D Gaussian Representation

In N-DG (Diolatzis et al., ), the color $c$ is defined by learnable RGB values. To introduce view-dependent effects, we adopt the spherical harmonics representation $\phi_{\beta}(d):\mathbb{R}^{3}\xrightarrow{}\mathbb{R}^{3}$, as used in 3DGS. This representation captures the variation in color based on the viewing direction. The spherical harmonics functions $Y_{\ell}^{m}(d)$ of order $\ell=3$ are parameterized by the coefficients $\beta\in\mathbb{R}^{48}$. The view-dependent spherical harmonics representation can be expressed as:

$$ $\displaystyle\phi_{\beta}(d)=f\left(\sum_{\ell=0}^{\ell_{\text{max}}}\sum_{m=- \ell}^{\ell}\beta_{\ell}^{m}Y_{\ell}^{m}(d)\right),$ (7) $$

where $f$ is the sigmoid function used to normalize the colors.

To better control the effect of the view direction on opacity, we refine the conditional PDF of the directional component as follows:

$$ $\displaystyle f_{\text{cond}}$ $\displaystyle=\exp\left(-\lambda_{\text{opa}}\cdot(d-\mu_{d})^{\top}\Sigma_{d} ^{-1}(d-\mu_{d})\right),$ (8) $$

where $0<\lambda_{\text{opa}}<1$ is a hyper-parameter or per-Gaussian learnable parameter that controls the influence of the view direction on the opacity. When adjusting $\lambda_{\text{opa}}$ as a hyper-parameter, we can modulate the resulting conditional opacity $\alpha_{\text{cond}}$, thereby controlling the density of the Gaussian during optimization; when treating $\lambda_{\text{opa}}$ as a per-Gaussian learnable parameter, it adapts the level of view-dependency for each Gaussian through training. This allows for finer control over the rendering process, enabling the representation of more complex visual effects.

### 4.3 Improved Control of Gaussians

To enhance the control of Gaussians, we adapt the explicit adaptive control mechanism from 3DGS, leveraging the additional directional information available in our 6D Gaussian representation. Instead of relying on the high-dimensional culling scheme used in N-DG, our approach focuses on refining Gaussian placement and density based on the scene’s geometry.

In 3DGS, the adaptive Gaussian densification scheme involves two primary operations: cloning and splitting. Cloning is employed when small-scale geometry is insufficiently covered by existing Gaussians, while splitting is used to divide a large Gaussian into smaller ones when it encompasses fine details of the geometry. This scheme requires the scale and rotation of each Gaussian, which are not directly provided in our 6D Gaussian representation.

To extract the necessary scale and rotation information from our 6D representation, we utilize the conditional covariance matrix $\Sigma_{\text{cond}}$. By performing Singular Value Decomposition (SVD) on $\Sigma_{\text{cond}}$, we can decompose the matrix into its principal components, revealing both the rotation and scale of the Gaussian. The decomposition is given by $\Sigma_{\text{cond}}=USV^{\top}$, where $U$ and $V$ are orthogonal matrices, and $S$ is a diagonal matrix containing the singular values. From this decomposition: the rotation matrix $R$ is derived from the left singular vectors as $R=U$, and the scale vector $s$ is obtained by taking the square root of the diagonal elements of $S$ as $s=\sqrt{\text{diag}(S)}$. To ensure that the rotation matrix $R$ forms a right-handed coordinate system, we adjust its last column based on the sign of its determinant: $R_{:,3}=R_{:,3}\cdot\text{sign}(\text{det}(R)).$

Figure: Algorithm 1 Slice 6DGS to 3DGS. In inference, we pre-compute $\Sigma_{\text{cond}}$, and the scale $s$ and rotation $R$ are not required. Only $\mu_{\text{cond}}$ and $\alpha_{\text{cond}}$ (highlighted in blue) need to be computed for each rendering.

This SVD-based decomposition allows us to represent the conditional Gaussian as an oriented ellipsoid, with clearly defined rotation and scale. By extracting these components, we can directly apply the adaptive Gaussian densification scheme from 3DGS (Kerbl et al., ), thereby improving the coverage of small-scale geometry and enhancing the overall quality of the rendered scene.

In 3DGS, Gaussians are pruned when their opacity falls below a minimum threshold $\tau_{\text{min}}$ (*e.g*., $\tau_{\text{min}}=0.005$ by default) or when they become excessively large. In our 6DGS approach, the conditional opacity $\alpha_{\text{cond}}$ is also influenced by the opacity parameter $\lambda_{\text{opa}}$. By incorporating $\lambda_{\text{opa}}$, we can fine-tune the density of Gaussians with greater precision, allowing for more granular control over which Gaussians are retained or pruned based on their opacity.

### 4.4 Compatibility with 3DGS

Algorithm 1 outlines the implementation details for converting (*i.e*., slicing) our 6DGS representation into a 3DGS-compatible format in a single function. Once this slicing operation is performed, the subsequent implementation remains identical to that of 3DGS.

The proposed 6DGS seamlessly integrates with the existing training framework of 3DGS, including the use of the same loss functions, optimizers, and training hyperparameters (except for the minimum opacity threshold where we set $\tau_{\text{min}}=0.01$). By slicing 6DGS into a conditional 3DGS format, we can directly utilize the adaptive density control and the differentiable rasterization method employed in 3DGS. This compatibility ensures that many downstream applications can effortlessly switch from 3DGS to our 6DGS, resulting in enhanced performance without the need for extensive modifications.

Note that the scale $s$ and rotation $R$ are only required during refinement iterations (*e.g*., every 100 iterations) for the adaptive density control. Additionally, during inference, $\Sigma_{\text{cond}}$ can be pre-computed. Therefore, only $\mu_{\text{cond}}$ and $\alpha_{\text{cond}}$ need to be computed for each rendering. To further enhance rendering efficiency, the slice operation can be implemented in CUDA.

## 5 Experiments

### 5.1 Experimental Protocol

#### Datasets.

We evaluate 6DGS on two datasets: the public Synthetic NeRF dataset (Mildenhall et al., ) and a custom dataset rendered using physically-based ray tracing (PBRT), which we refer to as the 6DGS-PBRT dataset. The 6DGS-PBRT dataset consists of six scenes: 1) cloud from the Walt Disney Animation Studios volumetric cloud dataset (^1^11[https://disneyanimation.com/resources/clouds/](https://disneyanimation.com/resources/clouds/)); 2) bunny-cloud, explosion, and smoke from OpenVDB volumetric models(^2^22[https://www.openvdb.org/download/](https://www.openvdb.org/download/)); 3) suzanne, the standard Blender test mesh, with a “Glasss BSDF" translucent material applied to it; 4) ct-scan, prepared from a real CT scan.

We rendered these scenes in Blender using its PBRT engine “Cycle". At the image sizes (with width equal to height) as listed in Table 2 on an NVIDIA Tesla V100 GPU, the rendering times per view were as follows: bunny-cloud took 504.0 seconds per view, cloud 838.6 seconds, explosion 35.5 seconds, smoke 71.5 seconds, suzanne 9.1 seconds, and ct-scan 28.5 seconds. For the ct-scan object, we generated 360 views with corresponding camera poses and randomly selected 324 images for training and 36 for testing. For each of the other objects, we rendered 150 views, randomly selecting 100 images for training and 50 for testing. We will make our 6DGS-PBRT dataset publicly available to the community.

#### Evaluation Metrics.

We evaluate our method’s performance using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) for image quality, number of Gaussian points (# point) for size, and Frames Per Second (FPS) for rendering speed.

#### Implementation.

In our experiments, we set $\lambda_{\text{opa}}=0.35$ and the minimum opacity threshold $\tau=0.01$. For learnable $\lambda_{\text{opa}}$, we initialize $\lambda_{\text{opa}}=0.35$ and make it trainable only during the iterations of 15,000 - 28,000. All other parameters are set to their default values as in 3DGS (Kerbl et al., ). For the ct-scan object, we initialize the point cloud using the marching cubes algorithm as in DDGS (Gao et al., ). For the other objects and the Synthetic NeRF dataset (Mildenhall et al., ), we randomly initialize the point cloud with 100,000 points within a cube encompassing the scene. Training is performed on a single NVIDIA Tesla V100 GPU with 16 GB of memory, using the Adam optimizer (Kingma & Ba, ). We set the learning rate to $1\times 10^{-2}$ for the 6D covariance parameters and $1\times 10^{-3}$ for the direction component $\mu_{d}$. The default learning rates from 3DGS are applied to the remaining parameters.

Figure: Figure 3: Qualitative comparison of methods on the 6DGS-PBRT dataset (zoom in for details).
Refer to caption: x3.png

### 5.2 Comparison with State-of-the-Art

**Table 1: Comparison of methods on the 6DGS-PBRT dataset.**
|  | 3DGS | N-DG | 6DGS (Ours) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | PSNR$\uparrow$ | SSIM$\uparrow$ | # points$\downarrow$ | PSNR$\uparrow$ | SSIM$\uparrow$ | # points$\downarrow$ | PSNR$\uparrow$ | SSIM$\uparrow$ | # points$\downarrow$ |
| bunny-cloud | 30.75 | 0.988 | 21,074 | 35.48 | 0.990 | 530,711 | 41.57 | 0.993 | 6,660 |
| cloud | 29.70 | 0.972 | 58,233 | 42.40 | 0.991 | 98,149 | 40.41 | 0.991 | 12,657 |
| explosion | 26.75 | 0.953 | 51,140 | 40.16 | 0.989 | 207,778 | 42.48 | 0.991 | 17,133 |
| smoke | 28.55 | 0.969 | 60,533 | 41.61 | 0.992 | 212,050 | 40.61 | 0.992 | 10,762 |
| suzanne | 23.70 | 0.901 | 270,001 | 26.00 | 0.921 | 232,145 | 27.03 | 0.928 | 174,746 |
| ct-scan | 25.71 | 0.917 | 229,683 | 30.96 | 0.952 | 1,073,082 | 33.56 | 0.965 | 182,981 |
| avg | 27.53 | 0.950 | 115,111 | 36.10 | 0.973 | 392,319 | 37.61 | 0.977 | 67,490 |

Table 1 compares our 6DGS method with 3DGS and N-DG on the 6DGS-PBRT dataset. Our 6DGS achieves significantly better image quality, with an average improvement of +10.08 dB in PSNR, while using significantly fewer Gaussian points (a reduction of 41.4% on average) compared to 3DGS. Compared to N-DG, our 6DGS also achieves higher image quality (+1.51 dB on average) and uses significantly fewer Gaussian points (a reduction of 82.8% on average).

**Table 2: Rendering speed (FPS) comparison of methods on the 6DGS-PBRT dataset.**
|  | bunny-cloud | cloud | explosion | smoke | suzanne | ct-scan | avg |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Image size | 1408 | 1408 | 1024 | 1536 | 1408 | 1024 | N/A |
| 3DGS | 49.1 | 74.4 | 161.9 | 96.7 | 42.0 | 259.7 | 114.0 |
| N-DG | 27.0 | 90.7 | 65.6 | 50.4 | 46.9 | 13.3 | 49.0 |
| 6DGS (Ours) | 178.4 | 178.2 | 120.2 | 138.6 | 34.7 | 276.8 | 154.5 |
| 6DGS-flash (Ours) | 315.3 | 318.0 | 367.7 | 345.5 | 295.9 | 315.1 | 326.3 |

Table 2 presents the rendering speed comparison. In the FPS comparison, we calculated the average FPS from 20 test images, where each image’s FPS value is the mean of 500 repeated measurements. Because our 6DGS uses fewer Gaussian points, it achieves faster rendering speeds compared to both 3DGS and N-DG. We further integrate FlashGS (Feng et al., ), an open-source CUDA Python library for efficient differentiable rasterization of 3D Gaussian splatting, into our 6DGS rendering pipeline, referred to as 6DGS-flash. With 6DGS-flash, we achieve an average rendering speed of 326.3 FPS, which is sufficient for many real-time applications. Note that, we can further improve our rendering speed by implementing our 6DGS slicing algorithm in CUDA.

In Figure 3, we compare our 6DGS method with 3DGS and N-DG in terms of rendering image quality and visualization of the generated point clouds. We observe that 3DGS fails on the bunny-cloud, cloud, explosion, and smoke datasets, exhibiting strong visible artifacts. N-DG tends to produce artifacts in the background and generates blurrier images away from the center. In contrast, our 6DGS achieves the best visual quality, closely matching the ground-truth images.

Furthermore, the visualization of the point clouds reveals that both 3DGS and N-DG struggle to generate faithful shapes of the objects. Specifically, 3DGS fails to capture the shapes in the bunny-cloud, cloud, explosion, and smoke datasets. N-DG tends to over-generate point clouds from the initialization, resulting in shapes resembling the initial marching-cube shape for ct-scan and a randomly sampled cube for other objects. In contrast, our 6DGS successfully reconstructs the shapes of the objects.

To evaluate the generalizability of our 6DGS method to scenes without significant view-dependent effects, we conducted experiments on the public Synthetic NeRF dataset (Mildenhall et al., ), as shown in Table 3. Our results demonstrate that 6DGS achieves image quality comparable to 3DGS while using significantly fewer Gaussian points (a reduction of 23.1% on average). In contrast, N-DG produces considerably lower image quality and uses fewer Gaussian points because it is specifically designed to model scenes with view-dependent effects and tends to underperform in scenes without strong view dependency.

**Table 3: Comparison of methods on the Synthetic NeRF dataset.**
|  | 3DGS | N-DG | 6DGS (ours) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | PSNR$\uparrow$ | SSIM$\uparrow$ | # points$\downarrow$ | PSNR$\uparrow$ | SSIM$\uparrow$ | # points$\downarrow$ | PSNR$\uparrow$ | SSIM$\uparrow$ | # points$\downarrow$ |
| chair | 35.91 | 0.987 | 272,130 | 30.87 | 0.956 | 108,091 | 35.49 | 0.986 | 223,747 |
| drums | 26.15 | 0.955 | 346,245 | 24.37 | 0.927 | 106,756 | 26.45 | 0.953 | 250,267 |
| ficus | 34.49 | 0.987 | 295,997 | 29.82 | 0.965 | 59,052 | 33.45 | 0.984 | 197,741 |
| hotdog | 37.72 | 0.985 | 147,098 | 33.89 | 0.971 | 82,261 | 37.90 | 0.985 | 102,451 |
| lego | 35.79 | 0.983 | 322,704 | 29.85 | 0.948 | 151,291 | 35.25 | 0.980 | 233,227 |
| materials | 29.98 | 0.960 | 282,334 | 26.86 | 0.938 | 77,206 | 30.71 | 0.967 | 222,209 |
| mic | 35.47 | 0.992 | 310,608 | 29.99 | 0.968 | 40,848 | 36.13 | 0.992 | 272,052 |
| ship | 30.52 | 0.905 | 328,053 | 26.35 | 0.862 | 337,294 | 30.72 | 0.903 | 270,163 |
| avg | 33.25 | 0.969 | 288,146 | 29.00 | 0.942 | 120,350 | 33.26 | 0.969 | 221,482 |

### 5.3 Ablation Study

We conduct ablation experiments on both the 6DGS-PBRT dataset and Synthetic NeRF dataset (see Appendix Table 5) to investigate the effects of various components in our 6DGS method. Specifically, we examine the impact of the parameter $\lambda_{\text{opa}}$ in the conditional probability density function (PDF) of the directional component (see Appendix Figure 4), defined as $f_{\text{cond}}=\exp(-\lambda_{\text{opa}}\cdot D)$, where $D=(d-\mu_{d})^{\top}\Sigma_{d}^{-1}(d-\mu_{d})$ represents the Mahalanobis distance between the viewing direction $d$ and the Gaussian mean direction $\mu_{d}$, and $\lambda_{\text{opa}}$ is a scalar between 0 and 1 that controls the influence of the view direction on opacity. Our ablation experiments include several settings related to $\lambda_{\text{opa}}$:

- •
No-$f_{\text{cond}}$: Setting $\lambda_{\text{opa}}=0$ results in $f_{\text{cond}}=\exp(0)=1$, meaning opacity is not modulated by the view direction.
- •
No-$\lambda_{\text{opa}}$: Setting $\lambda_{\text{opa}}=1$ uses the default $f_{\text{cond}}$ as in N-DG, *i.e*., $f_{\text{cond}}=\exp(-D)$.
- •
$\lambda_{\text{opa}}=0.35$: We set $\lambda_{\text{opa}}=0.35$, so $f_{\text{cond}}=\exp(-0.35\cdot D)$, providing a balance between no modulation and full modulation.
- •
Learnable $\lambda_{\text{opa}}$: $\lambda_{\text{opa}}$ is treated as a trainable parameter between 0 and 1 for each Gaussian, allowing the model to learn the optimal degree of view-dependent opacity.

Additionally, we include ablations of other components:

- •
No-SH: Color is treated as a learnable RGB parameter per Gaussian, as in N-DG, instead of using spherical harmonics (SH) to represent view-dependent color.
- •
$\tau=0.005$: The default minimum opacity threshold used in 3DGS, whereas our default choice is $\tau=0.01$.

These ablation studies help us understand the contribution of each component to the overall performance of our 6DGS method.

Table 4 presents the results of our ablation studies. Compared to our model with $\lambda_{\text{opa}}=0.35$, setting No-$f_{\text{cond}}$ (*i.e*., $\lambda_{\text{opa}}=0$) significantly degrades the image quality while increasing the number of Gaussian points. This highlights the importance of incorporating view dependency into the opacity function. Conversely, when we set No-$\lambda_{\text{opa}}$ (*i.e*., $\lambda_{\text{opa}}=1$), the image quality also degrades, but the number of Gaussian points decreases. This suggests that adjusting $\lambda_{\text{opa}}$ allows us to balance the trade-off between image quality and the number of Gaussian points, which directly affects rendering speed. When we make $\lambda_{\text{opa}}$ a learnable parameter, the model achieves improved image quality with a comparable number of Gaussian points.

Furthermore, the No-SH setting, where spherical harmonics (SH) are not used for color representation, results in degraded image quality. This is because SH provides a powerful representation for view-dependent color effects. Setting the minimum opacity threshold to $\tau=0.005$ (the default value in 3DGS) achieves similar image quality but requires significantly more Gaussian points. This is because introducing view-dependent opacity decay via the $f_{\text{cond}}$ function necessitates a larger minimum opacity threshold.

**Table 4: Ablation study on the 6DGS-PBRT dataset.**
|  |  | bunny-cloud | cloud | explosion | smoke | suzanne | ct-scan | avg |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| PSNR | No-SH | 38.13 | 37.39 | 41.05 | 38.52 | 26.76 | 33.27 | 35.85 |
| No-$f_{\text{opa}}$ | 38.15 | 36.16 | 35.49 | 35.52 | 24.88 | 29.65 | 33.31 |  |
| No-$\lambda_{\text{opa}}$ | 39.12 | 39.86 | 40.55 | 36.88 | 26.92 | 32.99 | 36.05 |  |
| $\tau=0.005$ | 40.15 | 40.46 | 43.14 | 41.04 | 27.09 | 33.45 | 37.56 |  |
| $\lambda_{\text{opa}}=0.35$ | 40.47 | 40.73 | 42.69 | 40.45 | 27.15 | 33.42 | 37.49 |  |
| learnable-$\lambda_{\text{opa}}$ | 41.57 | 40.42 | 42.48 | 40.61 | 27.03 | 33.56 | 37.61 |  |
| # points | No-SH | 6,196 | 11,356 | 16,465 | 9,929 | 171,472 | 177,204 | 65,437 |
| No-$f_{\text{cond}}$ | 11,698 | 31,999 | 39,736 | 35,023 | 349,716 | 320,879 | 131,509 |  |
| No-$\lambda_{\text{opa}}$ | 4,860 | 11,736 | 13,582 | 10,540 | 158,041 | 154,997 | 58,959 |  |
| $\tau=0.005$ | 63,042 | 51,883 | 61,507 | 49,422 | 301,614 | 405,332 | 155,467 |  |
| $\lambda_{\text{opa}}=0.35$ | 6,830 | 12,454 | 17,051 | 10,570 | 172,373 | 181,539 | 66,803 |  |
| learnable-$\lambda_{\text{opa}}$ | 6,660 | 12,657 | 17,133 | 10,762 | 174,746 | 182,981 | 67,490 |  |

## 6 Conclusion

In this work, we introduce 6D Gaussian splatting (6DGS) that builds on the foundations laid by 3DGS and N-dimensional Gaussians (N-DG). By improving the handling of color and opacity within the 6D spatial-angular framework and optimizing the adaptive control of Gaussians using additional directional information, we develop a method that not only maintains compatibility with the 3DGS framework but also offers superior rendering capabilities, particularly in modeling complex view-dependent effects. Our extensive experiments on the custom physically-based ray tracing dataset (6DGS-PBRT) demonstrate the effectiveness of 6DGS in achieving higher image quality and faster rendering speed compared to 3DGS and N-DG.

Looking ahead, 6DGS opens avenues for more accurate and efficient real-time volumetric rendering in virtual and augmented reality, gaming, and film production. Future research will explore further optimizations and extensions to enhance the scalability and robustness of the 6DGS framework, as well as its application to dynamic scenes and integration with advanced lighting models.

## Appendix A Appendix

Figure: Figure 4: Effect of adjusting $\lambda_{\text{opa}}$ on the conditional probability density function (PDF) of the directional component $f_{\text{cond}}$, plotted as the functions of the Mahalanobis distance $D$ between the view direction $d$ and the Gaussian mean direction $\mu_{d}$.
Refer to caption: x4.png

**Table 5: Ablation study on the Synthetic NeRF dataset.**
|  |  | chair | drums | ficus | hotdog | lego | materials | mic | ship | avg |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| PSNR | No-SH | 34.87 | 26.10 | 31.91 | 37.50 | 34.78 | 30.13 | 35.49 | 30.34 | 32.64 |
| No-$f_{\text{cond}}$ | 35.91 | 26.25 | 34.09 | 37.70 | 35.18 | 30.73 | 34.82 | 30.27 | 33.12 |  |
| No-$\lambda_{\text{opa}}$ | 35.53 | 26.29 | 33.40 | 37.88 | 35.26 | 30.45 | 36.03 | 30.86 | 33.21 |  |
| $\tau=0.005$ | 35.22 | 26.36 | 33.07 | 37.52 | 35.03 | 30.60 | 35.57 | 30.49 | 32.98 |  |
| $\lambda_{\text{opa}}=0.35$ | 35.51 | 26.37 | 33.39 | 37.82 | 35.22 | 30.71 | 35.98 | 30.60 | 33.20 |  |
| learnable-$\lambda_{\text{opa}}$ | 35.49 | 26.45 | 33.45 | 37.90 | 35.25 | 30.71 | 36.13 | 30.72 | 33.26 |  |
| # points | No-SH | 204,009 | 240,153 | 146,698 | 99,599 | 221,974 | 201,124 | 245,754 | 256,769 | 202,010 |
| No-$f_{\text{cond}}$ | 217,898 | 304,075 | 265,681 | 128,762 | 261,006 | 273,947 | 310,472 | 305,298 | 258,392 |  |
| No-$\lambda_{\text{opa}}$ | 204,335 | 242,526 | 188,295 | 101,039 | 223,815 | 210,190 | 265,266 | 271,282 | 213,344 |  |
| $\tau=0.005$ | 341,765 | 374,158 | 316,975 | 197,901 | 344,362 | 325,540 | 391,525 | 370,648 | 332,859 |  |
| $\lambda_{\text{opa}}=0.35$ | 233,148 | 247,822 | 195,205 | 103,263 | 274,525 | 219,917 | 275,292 | 264,650 | 226,728 |  |
| learnable-$\lambda_{\text{opa}}$ | 223,227 | 250,267 | 197,741 | 102,451 | 233,807 | 222,209 | 272,052 | 270,163 | 221,482 |  |

Figure 4 illustrates the effect of adjusting the parameter $\lambda_{\text{opa}}$ on the conditional probability density function (PDF) $f_{\text{cond}}$ of the directional component, plotted as a function of the Mahalanobis distance $D$ between the viewing direction $d$ and the Gaussian mean direction $\mu_{d}$. The Mahalanobis distance is defined as:

$$ $D=(d-\mu_{d})^{\top}\Sigma_{d}^{-1}(d-\mu_{d}),$ (9) $$

and the conditional PDF is given by:

$$ $f_{\text{cond}}=\exp\left(-\lambda_{\text{opa}}\cdot D\right).$ (10) $$

When $\lambda_{\text{opa}}=1$, the Mahalanobis distance $D$ has the maximum influence on the opacity, meaning that even small deviations in the viewing direction $d$ from the Gaussian mean direction $\mu_{d}$ significantly reduce $f_{\text{cond}}$. Conversely, when $\lambda_{\text{opa}}=0$, the opacity remains unaffected by the Mahalanobis distance $D$, resulting in a constant $f_{\text{cond}}=1$ regardless of the viewing direction. When $\lambda_{\text{opa}}=0.35$, the Mahalanobis distance $D$ has a moderate influence on the opacity, providing a balance between sensitivity to the viewing direction and maintaining opacity over a wider range of directions.

Table 5 presents the results of our ablation studies on the Synthetic NeRF dataset (Mildenhall et al., ). Since this dataset lacks strong view-dependent effects, the influence of $\lambda_{\text{opa}}$ is less significant compared to the 6DGS-PBRT dataset (see Table 4). Compared to our model with $\lambda_{\text{opa}}=0.35$, setting No-$f_{\text{cond}}$ (i.e., $\lambda_{\text{opa}}=0$) slightly degrades image quality and increases the number of Gaussian points. Conversely, when we set No-$\lambda_{\text{opa}}$ (i.e., $\lambda_{\text{opa}}=1$), both the image quality and the number of Gaussians remain comparable. Notably, when we make $\lambda_{\text{opa}}$ a learnable parameter, the model achieves improved image quality with a comparable number of Gaussian points. Furthermore, the No-SH setting results in degraded image quality. Setting the minimum opacity threshold to $\tau=0.005$ (the default value in 3DGS) also degrades the performance and requires more Gaussian points.

Figure: Figure 5: More qualitative comparison of methods on the 6DGS-PBRT dataset (zoom in for details).
Refer to caption: x5.png

Figure 5 presents additional qualitative results on the 6DGS-PBRT dataset, illustrating that our 6DGS method consistently achieves superior visual quality compared to 3DGS and N-DG.

## Peer Reviews

confidence: 4
contribution: 4
initial_rating: 8
presentation: 3
questions: - Could the authors provide more insights into how the choice of λopa influences performance across diverse datasets?

- How would the model adapt or perform in dynamic scenes with frequent view changes?

- Would implementing 6DGS directly in CUDA offer significant speed improvements for real-time applications, and if so, are there plans for such an implementation?
soundness: 4
strengths: The paper presents a novel 6D Gaussian Splatting (6DGS) method, which effectively models view-dependent effects through spatial and directional information.

By improving color and opacity handling, 6DGS produces high-quality, realistic renderings, particularly useful in scenes with complex reflections and lighting. Its compatibility with the 3DGS framework enhances its practical relevance, allowing integration with minimal adjustments. The experiments are well-designed and validate the method's efficiency, achieving significant reductions in Gaussian points without compromising image quality.
summary: This paper presents 6D Gaussian Splatting(6DGS), an advanced method for volumetric rendering that enhances traditional 3D Gaussian splatting (3DGS) by incorporating view-dependent information through a 6D spatial-angular Gaussian representation.

The authors address key limitations in 3DGS, such as handling specular reflections and anisotropic effects, by leveraging a 6D Gaussian model that adapts both color and opacity based on viewing angles. Experimental results demonstrate significant improvements in both image quality (up to a 15.73 dB PSNR increase) and rendering efficiency, achieving real-time speeds on complex datasets.
weaknesses: The method is primarily tested on synthetic datasets, lacking real-world scene evaluations that would confirm its robustness in practical applications. The new parameter λopa adds complexity, as it requires careful tuning. Additionally, while efficient for static scenes, the model’s performance on dynamic, interactive scenes remains untested, limiting its demonstrated versatility. Finally, the initialization phase can be computationally intensive, posing a potential challenge for scaling to large or complex scenes.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 4
questions: 1. Why does the proposed 6DGS outperform Ray Tracing-based Gaussian models on PBRT-rendered datasets? Intuitively, Ray Tracing-based models might be expected to perform better. Could it be due to incomplete implementation or insufficient optimization of the Ray Tracing-based model?

2. Although 6DGS is not Ray-Tracing-based, it appears to approximate the complex rendering effects and physical phenomena modeled by PBRT effectively. Could it also achieve high performance on more complex effects, such as subsurface scattering involving multiple scattering events?

3. While the synthetic dataset results already establish the value of this paper’s contributions, is there a specific reason or technical difficulty that prevented experiments on real-world datasets?
soundness: 3
strengths: 1. The fact that the proposed 6DGS is compatible with the existing 3D Gaussian Splatting (3DGS) framework and requires minimal modification appears to be a significant strength. This compatibility suggests that 6DGS can be effectively integrated into various existing datasets and application environments, potentially achieving impressive performance with minimal adjustments.

2. The theoretical explanations, figures, and experimental results are intuitively presented, making it easy to understand the contribution and performance of the proposed approach. This strong presentation quality greatly enhances readability.

3. The authors constructed a new 6DGS-PBRT dataset specifically to evaluate view-dependent effects, which seems to add considerable value. This dataset enables a more precise validation of 6DGS’s performance and appears to be an effective foundation for demonstrating its utility.

4. The approach provides important insights for the optimization of multi-dimensional Gaussian splatting techniques. By introducing restricted covariance modeling to refine view-dependent effects, this work offers valuable clues for future optimization of higher-dimensional Gaussian models.
summary: This paper proposes an extension of the existing 3D Gaussian Splatting technique to a six-dimensional Gaussian Splatting method. 6DGS combines 3D positional information with 3D view direction information using a six-dimensional Gaussian, enabling it to precisely capture view-dependent variations in color and transparency. To enhance computational efficiency, a restricted covariance method is introduced, which selectively incorporates only the necessary correlations, and spherical harmonics are employed to accurately model view-dependent color changes. Additionally, a new synthetic dataset called 6DGS-PBRT was constructed for performance evaluation, with experimental validation of view-dependent effects. This dataset, comprising synthetic scenes with various lighting and material properties, demonstrates that 6DGS can approximate complex visual effects effectively.
weaknesses: 1. Although experiments on synthetic datasets are meaningful, the lack of experiments on real-world captured datasets may limit the understanding of the model’s performance in real environments.

2. While the authors’ 6DGS-PBRT dataset seems well-suited for view-dependent effects, the limited variety of classes may fall short in representing a wider range of optical phenomena.

3. Additional explanation about the visual characteristics of each dataset would enhance clarity. For instance, clarifying that “cloud” captures absorption and scattering effects, or “glass” represents refraction effects, could make the paper more accessible. It also seems that adding reflective objects or other classes could provide a more comprehensive evaluation.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Based on the aforementioned weaknesses, the following issues also require discussion:
- Given that N-DG demonstrates rendering results on real data, and as mentioned in line 62 of the paper, the proposed method can handle scenes with glossy surfaces and transparency, can the method achieve better modeling and rendering results when applied to the real-world dataset collected by NeX? As NeX proposes view-dependent pixel representation that can handle non-Lambertain surfaces and model fine details, the author should conduct comparative experiments with NeX.

[1] NeX: Real-time View Synthesis with Neural Basis Expansion, CVPR 2021.

- Since N-DG can be extended to a higher-dimensional representation, was the dimensional setting for this method in the comparative experiments set as 6D?

- As the method proposed in the paper is based on per-scene optimization, the authors should consider providing a comparison of training times across different scenes. This would help understand the contributions of the method better.
soundness: 2
strengths: - The method proposed in this paper enhances the modeling of opacity and color while reducing the scale of Gaussians. By slicing the 6DGS to conditional 3DGS, it further improves rendering speed.
- This paper offers a theoretical analysis of 6DGS that enhances rendering quality, aiding in the understanding of its contribution to modeling complex scenes, particularly the scenes with view-dependent effects.
- This paper clearly defines the problem to be addressed and proposes the 6DGS representation, which is beneficial for overcoming the limitations of previous representations in applications. Additionally, it provides some validation across multiple datasets.
summary: This paper aims to address the challenge of handling view-dependent effects in scenes, a common issue encountered in both NeRF and 3DGS. Inspired by N-DG, this paper further enhances the 6D spatial-angular representation to better model scene details, particularly in contexts exhibiting complex view-dependent effects. By leveraging additional directional information, the proposed method achieves an improved representation of opacity and color while utilizing fewer Gaussians, thereby enhancing rendering speed. In addition, this paper offers a theoretical analysis of the conditional Gaussian parameters derived from the 6DGS representation, revealing its physical significance and contributions to enhancing rendering quality.
weaknesses: - The experimental validation is insufficient. (1) The method proposed in this paper emphasizes its capability to model complex scenes exhibiting view-dependent effects; however, the qualitative experiments are limited to results derived from the 6DGS-PBRT dataset proposed in this paper. The authors should provide additional visualization results, such as those derived from the Synthetic NeRF dataset to strengthen the effectiveness of the proposed method. (2) The paper only presents results for two synthetic datasets, which does not adequately demonstrate the robustness and applicability of the proposed method in real-world scenarios.

- Some important analyses are missing. Due to the challenges of camera pose estimation in real-world scenes with view-dependent effects, and considering that camera pose influences Gaussian density control, the authors should consider conducting further analysis on this issue.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 4
questions: I am interested in learning more about the potential applications of the algorithm. Can the algorithm be used for general radiance field reconstruction  instead of using synthetic data in dense view settings? If so, can you provide results from real-world datasets such as NeX? 
If not, it would be helpful to understand the specific scenarios where algorithms can be applied and to discuss any limitations.
soundness: 3
strengths: - The paper is well-written and easy to follow. The details are clearly described, and I believe the results can be reproduced by an expert in this field.

- The method is simple and easy to implement. It allows slicing the 6D Gaussian given a view direction into a 3D Gaussian, which can then be directly fed into the existing 3D Gaussian rasterizer.

- Experiments show impressive results in the 6DGS-PBRT datasets, achieving 10dB over 3DGS and 1.5dB over N-D Gaussian. The results may interest research that aims to overfit high-dimension data.
summary: 3DGS excels at synthesizing novel views of scenes, but it struggles with showing effects that depend on the viewer's perspective, like reflections, refractions, and scattering caused by complicated light transport. To solve this issue, the paper introduces a method called 6D-GS, which is based on N-dimensional GS. This method represents a scene as a 6D signal to capture both the spatial position and view direction at the same time. The paper enhances N-dimension GS by incorporating a view-dependent opacity function and an optimized density control. Extensive experiments on two synthetic datasets demonstrate the effectiveness of the proposed method.
weaknesses: The main weakness is the absence of supplemental video, which makes it difficult to access the novel view synthesis results. 

From my understanding, allocating more capacity to model view dependency of a scene might hinder the geometry and novel view synthesis. For instance, early NeRF uses a relatively small MLP with fewer Fourier features to strike a balance. 

Furthermore, the introduction of view-dependent opacity also complicates the results. For instance, while making the opacity view-dependent allows for easy fitting of high specular reflection, it is possible that when you view the specular spot from a slightly different angle, it becomes transparent. Since the PSNR metric may not provide a clear indication, it is recommended to include a video demonstration.

Meta-review: This paper proposes a novel 3D Gaussians representation that can model view-dependent effects, and thus achieve better results on transparent or glossy materials. The proposed method achieves better performance compared to native 3DGS as well as N-D GS, the fact that it can be compatible with existing 3DGS pipeline also makes it more practical. The reviewers raised weaknesses such as missing experiments (e.g., testing on real-world data), robustness of newly introduced parameter, differentiation with Ray Tracing-based Gaussian models, missing supplementary videos, most of the issues are addressed during the rebuttal and reviewers all agree to accept this paper.

Decision: Accept (Poster)