OMG: Opacity Matters in Material Modeling with Gaussian Splatting
Authors: Silong Yong
Venkata Nagarjun Pudureddiyur Manivannan
Bernhard Kerbl
Zifu Wan
Simon Stepputtis
Katia P. Sycara
Yaqi Xie
Venue: ICLR 2025 Poster (2025)
Keywords: 3D Gaussian Splatting
Neural Rendering
Inverse Rendering
Visual Computing
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Inspired by radiative tranfer, we propose an addtional constraint and a physically correct activation function for inverse rendering models, resulting in significant improvements in terms of novel view synthesis and material modeling.

Abstract: Decomposing geometry, materials and lighting from a set of images, namely inverse rendering, has been a long-standing problem in computer vision and graphics. Recent advances in neural rendering enable photo-realistic and plausible inverse rendering results. The emergence of 3D Gaussian Splatting has boosted it to the next level by showing real-time rendering potentials.  An intuitive finding is that the models used for inverse rendering do not take into account the dependency of opacity w.r.t. material properties, namely cross section, as suggested by optics. Therefore, we develop a novel approach that adds this dependency to the modeling itself. Inspired by radiative transfer, we augment the opacity term by introducing a neural network that takes as input material properties to provide modeling of cross section and a physically correct activation function. The gradients for material properties are therefore not only from color but also from opacity, facilitating a constraint for their optimization. Therefore, the proposed method incorporates more accurate physical properties compared to previous works. We implement our method into 3 different baselines that use Gaussian Splatting for inverse rendering and achieve significant improvements universally in terms of novel view synthesis and material modeling.

Published as a conference paper at ICLR 2025
OMG: OPACITY MATTERS IN MATERIAL MODELING WITH GAUSSIAN SPLATTING
Silong Yong"
Venkata Nagarjun Pudureddiyur Manivannan"
Bernhard Kerbl
Zifu Wan
Simon Stepputtis
Katia Sycara
Yaqi Xie
Carnegie Mellon University
(s longy,, vpudured, bkerbl, zi fuw, sstepput, sycara, sycara,yaqix ya cmu. edu
ABSTRACT
Decomposing geometry, materials and lighting from a set of images, namely in- verse rendering, has been a long-standing problem in computer vision and graph- ics. Recent advances in neural rendering enable photo-realistic and plausible in- verse rendering results. The emergence of 3D Gaussian Splatting has boosted it the next level by showing real-time rendering potentials. An intuitive finding is that the models used for inverse rendering do not take into account the depen- dency of opacity w.r.t. material properties, namely cross section, suggested by optics. Therefore, we develop a novel approach that adds this dependency augment the opacity
the modeling itself. Inspired by radiative transfer, term by introducing neural network that takes as input material properties to provide modeling of cross section and a physically correct activation function. The gradients for material properties are therefore not only from color but also from opacity, facilitating a constraint for their optimization. Therefore, the pro- posed method incorporates more accurate physical properties compared to previ- ous works. We implement our method into different baselines that use Gaussian Splatting for inverse rendering and achieve significant improvements universally in terms of novel view synthesis and material modeling. Code is available at https://githuni com/ Low.ooogggo//oon
INTRODUCTION
Inverse rendering, long-sture standing problem in computer vision and graphics, aims to recover physical properties such as geometry, materials and lighting conditions from a set of images (Barron & Malik, 2013; Debevec, 2008). These properties as material editing, relighting and, from a broader perspective, real-to-sim transfer. However, is well-known to be challenging and ill-posed because of the inherent ambiguity of the task: Different combinations of illumination, materials and geometry may lead to similar rendering results. Recent success in novel view synthesis (Mildenhall et al., 2021; Barron al., 2021; Chen et al., 2022; Müller et al., 2022; Barron et al., 2023; 2022) inspired the use of implicit neural fields (NeRF-like methods) for inverse rendering (Zhang et al., 202 Yao al., 2022; Boss et al., 2021; Jin et al., 2023; Srinivasan et al., 2021; Zhang et al., 2021a). On the other hand, 3D Gaussian Splatting (3DGS) (Kerbl al., 2023) emerged as recent, high-performance alternative to these slower architectures for novel view synthesis by introducing much more compact scene representation and real- t-me rendering ability. Naturally, recent works combine 3DGS with inverse rendering (Liang et al., 2024; Gao et al., 2023; Jiang et al., 2024), taking advantage of its high-speed nature and compact geometry. By adding additional parameters that act as material properties for each Gaussian, they are able provide physically plausible material modeling and novel view synthesis results using physically-based rendering (PBR) thanks to the explicit nature of 3DGS. While it is intuitive take advantage of 3DGS for fast training and rendering with high quality, directly applying 3DGS for inverse rendering may result in sub-optimal modeling. Specifically, the correlation between the material attributes of each Gaussian (e.g., opacity, albedo, roughness, etc.)
essential downstream applications such
Joint collaborators in research inception

---

Published as a conference paper at ICLR 2025
remains under-explored, leading under-constrained modeling of the scene properties. Therefore, the inverse rendering results remain to be improved.
Observing the disentangled nature of 3DGS-basee methods, work, inspired by the Bouguer- Beer-Lambert law in radiative transfer, reveals the missing correlation between materials and opacity, two sets of parameters that are essential for physically-baset rendering using 3D Gaussian Splatting. By doing so, our work introduces constraints at model level to align the model better with the physical world. Specifically, we propose model opacity in 3DGS-based inverse rendering methods strictly following the form derived from the Bouguer-Beer-Lamee that the intensity of light should decrease exponentially when passing through an absorbing body. Intuitively speaking, the decrease rate depends not only on the path length that light travels, but also on the material and number density of the absorbing body. found in Fig. 1. Consider two different translucent materials glass and gas in Fig. 1. They react differently when hit by light. For instance, red light would be terminated by the gas while it's able to pass through the glass nearly unaffected as illustrated by Fig. 1. In other words, opacity varies between different materials. When designing the model, 3DGS-based inverse rendering methods (Liang al., 2024; Jiang et al., 2024; Gao al., 2023) overlook the fact that because of the disentangled representation, opacity is a standalone parameter that has no dependency on the material properties. As for NeRF-based methods that use alpha blending implicitly model the dependencies of opacity (volume density) and materials by using the same neural network to output different attributes. Therefore, for 3DGS-based methods, we propose to multiply the original opacity term with a material-dependeer give better modeling of the physical property opacity itself. The material properties therefore serve input for PBR as well as input for bipha-blending and receive gradients from both sides, facilitating a physically informed regularization to the model.
law, which states
illustration of the law can be
term and apply physically correct activation function to
To verify the correctness and effec- tiveness of the proposed formulation, we analytically conduct Taylor ex- pansion to our approach and compare the difference w.r.t the original way of computing opacity. The takeaway message is that the original way is actually an approximation of our approach. Empirically,
apply the modification to state of the art base- lines, namely GaussianShader (Jiang et al., 2024), GS-IR (Liang et al., 2024) and R3DG (Gao al., 2023)
and conduct experiments on both synthetic and real-world data. The experimental results thetic4Relight (Zhang et al., 2022),
An illustration of the motivation that opacity depends on materials. Consider two translucent materials gas and glass. Left: glass body that is hit by light lets the light pass through with little reduction in intensity. Right: a gas that hit by the same light make the light extinct inside it by absorbing it completely. This comparisons motivate our work. We perceive each Gaussian blob, paired with material properties, an absorbing body that has its own absorption coefficient represented by opacity, which should consider material properties influencing factors.
Figure 1:
Syn-
Shiny Blender (Verbin et al., 2022),
Glossy Synthetic (Liu et al., 2023)
and MIP-NeRF 360 (Barron et al., 2022) show that our approach enables universal performance improvement in terms of across different baselines and different data. The improve- ments on material modeling, i.e., albedo estimation and roughness estimation, lead to better novel view synthesis and relighting results in terms of PSNR, SSIM and LPIPS. Our experimental results indicate the significance of incorporating physically correct priors and constraints into the model.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
2.1 NEURAL SCENE REPRESENTATION
Recent trends in neural rendering (Mildenhall et al., 2021; Yu et al., 2021; Fridovich-Keil et al., 2022: Sitzmann et al., 2021; Yariv et al., 2021; Miller et al., 2024; Huang et al., 2024; Yong et al., 2024) have demonstrated impressive success in addressing visual computing problems like novel view synthesis. NeRF (Mildenhall et al., 2021) stands out as representative by modeling the scene using ML.P output volume density and color in a continuous space, which requires massive peated queries for volume rendering during training and inference to synthesize images. Efforts have been made to improve the efficiency NeRF by introducing additional data structures (Garbin al., 2021; Müller et al., 2022; Hedman et al., 2021; Yu et al., 2021; Fridovich- Keil et al., 2022; Chen al., 2022). However, the discretized nature of the data structures reduces the image quality some extent. Recently, 3D Gaussian Splatting (3DGS) (Kerbl et al., 2023) comes out and is able to achieve fast rendering speed while maintaining the rendering quality, therefore drawing the community's 's in- terest. Plenty of works focus on improving 3DGS by reducing the memory footprint (Fan et al., 2023; Navancet et al., 2023; Niedermayr et al., 2024), speeding up the training pipeline (Mallick et al., 2024; Höllein et al., 2024) and removing the heuristic designs for optimization (Kheradmand et al., 2024; Bulò et al., 2024). Another direction is on the application side, works have been done to apply 3DGS different domains such as semantic understanding (Qin et al., 2024; Guo al., 2024), time sequence modeling (Luiten et al., 2024; Lin et al., 2024) and robotic manipulation (Lu al., 2024; Shorinwa et al., 2024), etc. The wide range of domains for application encouraged the focus of our work on inverse rendering techniques that use 3DGS as the backbone.
2.2 INVERSE RENDERING
Inverse rendering aims to decompose image observations into geometry, material and lighting condi- tions (i.e., scene properties) that support myriad of downstream tasks, such material editing and relighting (Li et al., 2023; Zhang et al., 2022; Kanamori & Endo, 2019; Yang et al., 2022). Normally, is tackled by combining thysically-based rendering with a differentiable renderer for optimization- based decomposition of scene properties (Kajiya. 1986; Chen et al., 2019). While being an inher- ently ambiguous problem, many works assume different constraints input level, such known lighting conditions (Bi et al., 2020; Srinivasan et al., 2021; Zhang et al., 2022), fixed lighting (Dong al., 2014), unintended shadow (Verbin et al., 2024) absence of shadow simulation (Zhang al., rendering has
2021a). The combination of differentiable volume rendering and physically- enabled modeling more complex conditions towards more realistic cases thanks to the emergence of neural scene representation (Zhang et al., 2021b; Yao et al., 2022; Liu et al., 2023; Verbin et al., 2022: Jin et al., 2023; Attal et al., 2024; Boss et al., 2021; Munkberg et al., 2022). Recently, 3DGS has revolutionized the field of neural scene representation. It is natural to expect inverse rendering could be tackled by 3DGS-based methods to incorporate fast and precise approximation of mate- rials, geometry and lighting (Liang et al., 2024; Jiang et al., 2024; Gao et al., 2023). Despite the rapid success of providing plausible matcrial estimation, the results remain to be improved, mainly because the constraints that model should have for inverse rendering have been overlooked.
3 PRELIMINARIES
In this section, We will cover the basic concepts of 3D Gaussian Splatting, inverse rendering and the Bouguer-Bects Beer-Lambert law as our work is built upon them.
3.1 3D GAUSSIAN SPLATTING
3D Gaussian Splatting (3DGS) (Kerbl et al., 2023) uses a set of points represented by 3D Gaussian primitives to model the 3D scene. Formally, each Gaussian G; is represented by mean μ4, covariance matrix Σ, an opacity o, and a view-dependent outgoing radiance c. Additionally, the covariance matrix is factorized into quaternion q and scale 8; and the outgoing radiance is treated as a set of spherical harmonics coefficients. For the inverse rendering problem, 3DGS-based methods (Liang et al., 2024; Jiang et al., 2024; Gao al., 2023) assign material properties mn and
3

---

Published as a conference paper at ICLR 2025
Material Attributes
Color
Lighting
Image
Gaussian Spheres
U
Alpha Compositing
Albedo
Roughmess
MLP
GT
Shape Attributes
Opecity 0
Coveriance Σ
Alpha
Operation Frow Gredient Flow
Figure 2: Pipeline overview. We add an additional block to 3DGS-based inverse rendering meth- ods (Liang al., 2024; Kerbl al., 2023; Jiang al., 2024; Gao al., 2023). Specifically, instead of modeling opacity as a standalone parameter as done by previous works, we augment against material. By introducing neural network that takes material properties as input and output cross section and multiplying the opacity with it, we are able to incorporate the Bouguer- Bouguer-Beer-Lann Besee---ee-LLabber law into the model. During optimization, material properties not only receive the gradients from color through differentiable PBR, but also the gradients from alpha enforced by the neural network. By doing so, we add an additional constraint to material properties that makes the overall pipeline strictly follow the Bouguer-Beert Lambert law.
normals to each Gaussian to support physically -based rendering. These sets of parameters are used produce PBR color, which serves as & in the original formulation.
After projecting the 3D Gaussians to 2D space using camera parameters 0, 3DGS adopts alpha-blending for rendering the final color of a specific pixel location following
N
((x)
Σ
Go Go,(#) (x),,
(zx) II (1 az(r),,),
where N represents the sorted N Gaussian points, T; is the accumulated transmittance of the first i Gaussians and (2) is computed using
(x(x)) = og exp(- R(u;; 0))" (E.) R(v;;)))
in which R stands for the camera projection operation, Rg the operation of applying the camera projection to the covariance matrix and opacity o; is weighted by the probability determined by distance the pixel location the mean of the projected Gaussian. Afterwards, color loss computed between the rendered image and the ground truth:
L == [=ICC- -113
3.2 THE RENDERING EQUATION
In inverse rendering, the outgoing radiance Colæ, v) at specific surface point from viewing direction determined by the rendering equation (Kajiya, 1986):
Co(I,v) fL. L. 1) x) -n)d.
(4)
where L; 1) is the incoming light from direction I at point x. is the surface normal direction and f is the Bidirectional Reflectance Distribution Function (BRDF). The integral is computed in the upper hemisphere of the surface.
3.3 THE BOUGUER- BERUUURR-BEER-LANEE SEEERRLLAMBER LAW
The Bouguer-Bes Beer-Lambert law states the following (Mayerhöfer et al., 2020; Bouguer, 1729; Lam- bert, 1760; Beer, 1852): Consider radiation of intensity i.e., light of intensity passing through
4

---

Published as a conference paper at ICLR 2025
an absorbing body, the change of its intensity is governed by the following differential equation:
diy = aaplds
(5)
where subscript y means that the term depends on the frequency of light and ap is called the extinction coefficient, attribute that determines how energy is dampened in the absorbing body.
Solving for intensity In, we get
(x() (0)e
here 6 is the transmittance of region 8, meaning that the light intensity is exponentially decreasing when traveling through the absorbing body. the attenuation of light in the body doesn't contain scattering effects, the extinction coefficient can be computed as
10
where n is the number density and oy is cross section. These two terms often appear in chemistry. describes the degree of concentration of countable objects (usually particles) and øg models the the probability that two particles will collide, which determined by the types of the particles. An illustration of the Bouguer- Bouguer-Beer-Lamde Beeb-L law can be found in Fig. 3, the intensity of light passing through a region filled with particles is reduced when "hit" with particles (normally the affecting area bigger than the particle size). Intuitively speaking, the reduction in intensity is determined by the number of particles in the region that the light hits and the affecting area of the specific particle type, which corresponds to Eq. 7 and Eq.
METHOD
We start by introducing the intuition behind our method. It is well-known that different materials re- act differently when hit by a beam of light, e.g., a wooden plank would diffuse the light and let noth- ing pass through, while a glass pane may allow light to travel inside with no dampening. In physics, this phenomenon described by the Bouguer-Becd Beer- Lambert law, suggesting that the opacity of object depends on its material. Motivated by this, we do series of derivation and propose plug-and-play solution suggesting that opacity should be a func- tion of material and the computation of transmit- tance should follow the exponential decay rate as in the Douguer-Beerr-aadd
law.
4.1 GAUSSIAN BLOBS AS ABSORBING BODY
Our approach arises from the Bouguer-Beas Lambert law. Recall Eq. 1, T. stands for the accu- mulated transmittance of the first (i 1) Gaussians, indicating the reduction light intensity of the i-th Gaussian when observed from the camera perspec- tive. For each Gaussian G; the transmittance is cal- culated by 1 (x). According to the Bouguer- Beer-Lambert law, the transmittance of an absorbing body should follow the form aus or -nous as Eq. 6. Therefore we have
Figure 3: An illustration of the Bouguer- Beer-Lambert law. Consider ray passing through an absorbing body consists of parti- cles of some type. The cross section depends on the area that each particle would affect (usually bigger than the size of the particle and equals the size of the particle treated as a hard sphere). When traveling, the inten- sity of the light would be reduced by the ar- eas that each partiele affected. The reduction (8) in intensity is therefore affected by the area around the particles and the number of par- ticles, corresponding to cross section and number density the main paper.
- (xx = -1108
a=(x))
aoye
A natural question then arises: what is n, o, and s in the context of Gaussian Splatting?
5

---

Published as a conference paper at ICLR 2025
To answer this question, recall the definition of (z) in Eq. 2. The term o; indicates how "dense" the Gaussian is, and the exponential term suggests that the Gaussian densest at the center and becomes lighter when away from it shown in Fig. 2. Since each Gaussian blob is not solid, it is natural to think it as a blob of gas with its density following a Gaussian distribution. Notice that in this formulation, the density corresponds the number density in Eq. 8. Formally, have
1 n1 = oz eeppl=250 2(( R(p
(E,)
R()30)))
As for path length 8, since each Gaussian is "splatted" to a 2D plane, i.c. density marginalized along the viewing axis, the distance of the light passing through each Gaussian be set constant 1. In fact, the blending algorithm omits the depth of the Gaussians into the formulation of treat this opacity value
the opacity value at the pixel location. We therefore observed that as the density and since there is no concept of depth on the 2D plane, each Gaussian is then treated equally from this perspective, therefore reasonable treat the path length to constant 1.
4.2 MATERIAL NETWORK FOR CROSS SECTION MODELING
After deciding on the correspondence of existing terms, we found that there is still one term missing from the Bouguer- Seee-Beer--aabse law, namely the cross section ø,- Cross section, by definition, is determined by the type of particle that fills the region as illustrated Fig. 3, namely the material of the region. On macro level, cross section affects the color that one perceive from the region. Since in inverse rendering, material properties, i.e., albedo, metalness and roughness, are explicitly included in the model (for 3DGS-based method they's re set additional parameters for each Gaussian), we can directly use the parameter for representing the cross section. In practice neural network f() that takes input the material properties and outputs cross section oy is introduced, namely
o. = f(m).
(10)
In summary, we have now determined the three terms suggested by the Bouguer- Booguer--Berr-ane Beee--Lmmert law in the context of Gaussian Splatting. Therefore, we can compute as (z(2) in Eq. 2 differently as
(z(z) = 1 -n(())a
(11)
= =1- - (,))(()))(m))
where is the original opacity term for the i-th Gaussian, G, (2) is the normal distribution in Eq. 2, f is the neural network that produces cross section and m, is the material property.
4.3 GRADIENT FLOW
Consider G; in Eq. 1, in inverse rendering: it is computed using the material properties in Eq. 4. Therefore, our formulation of material takes gradients not only from the color term itself, but also from the term. In this case,
ØL dc, ØL T,T + cs ac
OL T
da: 0m,
N
OL cj a
Da:
T,a ac
(12)
0m, dm, ac
1 - 0m:
where L stands for loss. Our formulation augments the gradient for material learning, allowing the gradient to flow from opacity to serve as an additional constraint in a physics-inspired way, as illustrated by Fig. 2.
4.4 ANALYSIS
Apart from the derivation of our formulation from the Bouguer-Beer Lambert law, we found that our formulation could also be interpreted from two other perspectives, indicating the correctness of approach.

---

Published as a conference paper at ICLR 2025
NeRF.
The dipha-blending algorithm in NeRF (Mildenhall et al., 2021) takes the form
N C(r) Σ 4(1
1=1
(13)
T. exp(- Σ ,,,z),
in which stands for the ray direction, G is view-dependent color of the sampled points, 0; is volume density and 6 is the distance between two sampled points. this formulation, 0g = exp(-o;;;), which takes exactly the same form as our derivation from the Nouguer-Beer-Landne law. Neural field-based methods (Bi et al., 2020; Boss et al., 2021) that use ray marching in particle-based manner for inverse rendering handle cross section implicitly by using the same MLP for modeling volume density and material, suggesting the necessity of introducing cross section into the disentangled representation of 3DGS-based methods.
Taylor Expansion. Through Taylor expansion w.r.t. the function att) = 1 exp(-t), we have
att) = =1 e 1 -1- o(t))
(14)
= =t+ o(t),
where o(t) is the little-o notation. This indicates that Eq. is an approximation to the function (x) = 11-expp -xx((=;;G,(x)) which is the exact same form as Eq. 11, omitting the newly introduced cross section (m.). This derivation indicates the mathematical correctness of the newly introduced activation function.
EXPERIMENTS
We discuss the implementation details to show the experimental design and present experimental results on synthetic and real- data to provide a comprehensive understanding of our method.
5.1 IMPLEMENTATION DETAILS
Since our proposed method independent of the underlying model for inverse rendering, we can freely plug it into three state-of-thc-ae baselines, namely GaussianShader (Jiang et al., 2024), GS-IR (Liang et al., 2024) and R3DG (Gao al., 2023) for evaluation. All the experiments are conducted on a single NVIDIA RTX 6000 Ada Generation GPU and we report the reproduced baseline results for fair comparison. We use an MLP that has hidden layers with size 128 for the hidden dimension and ReLU activation function for cross section and the output activation is sigmoid function. The input to the MLP is designed as follows: for GaussianShader, an end-to-end trained material modeling pipeline, as in their original model for MLP input. For GS-IR and R3DG, since they follow a two-stage training pipeline. in which the first stage uses normal rendering for geometry reconstruction and second stage uses anysically-based rendering for material modeling, we input the material properties well the SH coefficients used for color in the first stage to the MLP. The logic behind this design is to make the optimization of the MLP in the first stage meaningful. We follow the experimental settings as provided by the original baselines. GaussianShader evaluated on Shiny Blender (Verbin et al., 2022) and Glossy Synthetic (Liu et al., 2023) as in the orignal work. For R3DG, Synthetic4Relight dataset (Zhang et al., 2022) is used for getting material modeling evaluation results. GS-IR is evaluated on MIP-NeRF 360 dataset (Barron et al., 2022) to provide insights on real-world data. For albedo and relighting evaluation, we follow the standard practice to standardize the albedo prediction against the ground truth.
directly input the albedo, roughness and specular tint terms
5.2 RESULTS ON SYNTHETIC DATA
Material Modeling. We first evaluate our proposed module on top of R3DG (Gao et al., 2023) for material modeling evaluation. As can be seen from Tab. with the help of our modeling.
7

---

Published as a conference paper at ICLR 2025
Rendering
Albedo
Roughness
Relighting
Relighting
R3DG
Ours
GT
Qualitative comparison of albedo, relighting and roughness Synthetic4Relight dataset (Zhang et al., 2022). Our method is able to decouple specular effects from albedo compared to R3DG, the baseline implemented method It is also capable of providing roughness estimation more precisely. We add gamma correction to the relighting results to make the visual differences clearer.
Figure 4:
Table 1: Quantitative results on Synthetic4Relight dataset (Zhang et al., 2022). Our method is able to provide significant improvements universally. The improvements in novel view synthesis and relighting can be attribute to the improved estimation of albedo and roughness thanks to the correct modeling use for constraining material properties.
Novel View Synthesis
Relighting
Albedo
Roughness
PSNRT
SSIM
LPIPS,
PSNR1
SSIM
LPIPS.
PSNR
SSIM1
LPIPS
MSEJ
R3DG R3DG Ours
34.07
0.975
0.047
32.87
0.967
0.052
28.70 70
0.952
0.064
0.011
0.977
0.046
33,21
0.969
0.050
29.31
0.954
0.059
0.007
R3DG is able produce much better results universally. Our method outperforms the baseline on novel view synthesis by 0.4 db PSNR, as well as on two other metrics. The relighting results shares similar performance boosts as novel view synthesis, which could be attributed the significant improvements in albedo (~ 0.6 db PSNR boost) and roughness estimation. Qualitatively speaking, as can seen from the visualization in Fig. 4, our method is capable of producing albedo estimation with less specular effects and roughness estimation that aligns better with the ground truth, resulting pleasing relighting results.
Novel View Synthesis on Synthetic Data Other than the results on Synthetic4Relight, we also implemented our method on top of GaussianShader (Jiang et al., 2024) to get more insights on how our model would perform on synthetic data. Tab. 2 showcases the result on Shiny Blender (Verbin et al., 2022) and Glossy Synthetic (Liu et al., 2023). Our method is able to produce significant improvement in terms of novel view synthesis when built on top GaussianShader (specifically 0.3 db PSNR boost Shiny Blender and 0.4 db boost Glossy Synthetic). Fig. 5 showcases some qualitative comparisons add any constraint to how the model predicts normals, method is able to produce much more satisfactory normal estimation compared to the baseline as well as better modeling of light reflection.
our model with the GaussianShader baseline. While we did not
8

---

Published as a conference paper at ICLR 2025
GT
GaussianShader
Ours
Rendering
Normal
Rendering
Normal
Ball
Angel
Toaster
Figure 5: Qualitative comparison on Shiny Blender (Verbin et al., 2022) and Glossy Syn- thetic (Liu al., 2023) datasets. Our method able to provide satisfactory normal estimation compared to the baseline. Notice that method does not include any additional supervision or constraints for normal estimation. Our method is also able to provide much more accurate view synthesis results when doing physically- based rendering. These results suggest the significance of introducing the physically correct model for inverse rendering.
Table 2: Quantitative results when combined with GaussianShader (Jiang al., 2024). We compare with GaussianShader on Shiny Blender (Verbin et al., 2022) and Glossy Synthetic datasets (Liu al., 2023). Our method is able outperform GaussianShader novel view synthesis in tems of all three standard image error metrics.
Shiny Blender
Glossy Synthetic
PSNRT
SSIM
LPIPS
PSNR
SSIM
LPIPS
GaussianShader
30.77
0.953
0.081
27.25
0.925
0.083
GaussianSSader + Ours
31.07
0.957
0.076
27.68
0.929
0.081
5.3 RESULTS REAL- WORLD DATA
Apart from the results synthetic data, also analyze the effect of our method when applied to real-world data. To verify the plug-and-play attribute of our method, we apply it to GS-IR (Liang et al., 2024) and conduct experiments the MIP-NeRF 360 dataset (Barron al., 2022). The experimental results shown Tab. 3 indicate a significant universal improvement in terms of novel view synthesis thanks the correct modeling and better estimation of materials indicated by the results in Sec. 5.2. Qualitatively speaking, our method able to produce less blurry images compared the GS- baseline as shown by the bicycle scene in the first row Fig. 6.
Analysis. Since our method augments opacity with a material-dependent term, it turns the opacity from a purely geometry-indicato to combined representative of geometry and material. That being said, the optimization of geometry is also affected method. This is obvious if we consider the gradient of a; w.r.t o;:
da: 0o;
G.f((m) ot),
(15)

---

Published as a conference paper at ICLR 2025
Mip-NeRF 360|
Avg.
Bicycle Flowers Garden Stump Treehill Room Counter
Kitchen Bonsai
GS-IR
25.38
23.31
20.43
25.76
25.46
21.75
28.32
26.31
28.66
28.44
PSNRT
Ours
25.85
24.09
20.16
26.24
25.54
21.97
29.64
26.85
28.92
29.27
GS-IR
00.60
0.688
0.542
0.808
0.717
0.583
0.874
0.847
0.877
0.901
SSIM
Ours
0.767
0.720
0,510
0.824
0.720
0.586
0.897
0.859
0.884
0.905
GS-IR
0.2665
0.269
0.368
0.154
0.256
0.367
0.281
0.255
0.179
0.254
LPIPS
Ours
0.254
0.230
0.401
0.140
0.245
0.368
0.247
0.244
0.168
0.243
Table 3: Quantatitive Comparison Mip-NeRF 360 (Barron et al., 2022). The results show that our approach, when combined with GS-IR, can significantly improve the novel view synthesis results of real-world scenes on MIP-NeRF 360 (Barron et al., 2022). Thanks to the constraint we add to opacity, our method is able to produce db PSNR gains along with SSIM and LPIPS improvements.
pixel location x is omitted for simplicity. We hypothesize that since MIP-NeRF 360 supports SIM initialization, it helps our method by easing the burden for geometry reconstruction, therefore lead- ing to more improvements compared to synthetic data which uses random initialization.
GS-IR
Ours
Rendering
Normal
Rendering
Normal
Bicycle
Bonsai
Figure 6: Qualitative Comparison of GS-IR on MIP-NeRF 360 (Barron et al., 2022). Our method is able to provide much more accurate novel view synthesis results compared to the baseline GS-IR (Liang al., 2024) in which we plugged our model. The blurriness in the Bicycle scene in the first row predicted by GS-IR overcome by our approach. The second row showcase that our approach is able provide much more fine-grained and accurate normal estimation.
6 CONCLUSION
In this work, we propose OMG, plug-and-play module for inverse rendering based on Gaussian Splatting. Inspired by the Bouguer-Beer-Lamdee Beer-Lambert law, we derive the exact form of how opacity should be modeled in particle-based model. We analyze the proposed formulation against NeRF-based methods and from a mathematical perspective, providing more insights in the right modeling for inverse rendering. Our method also reveals again the similarity between NeRF-based methods and 3D Gaussian Splatting. To validate the effectiveness and the plug-and-play nature of the proposed modeling, we implement on top of three different ofate-of-the-an baselines and conduct comprehensive experiments on synthetic and real-world data. The significant improve- ments showcased by the experiments indicate the correctness the modeling and point out the significance of the right prior and modeling for visual computing. We believe work could inspire
10

---

Published as a conference paper at ICLR 2025
the community pay more attention to developing the physically correct model and introducing right priors for modeling the 3D world.
Limitation. In OMG, the dependency of cross section on the frequency of light is not included in
the modeling. Future work could introduce the correct dependency of cross section on frequency of light provide more accurate modeling of the physical world. As shown in Sec. and Fig. D6, the learned cross section is still not ideal if applied to the specular region with strong lighting. This motivates us to explore more on how to regularize the learning of materials and cross section more precise way.
ETHICS STATEMENT
Our work focuses incorporating physically correct priors to improve inverse rendering results, aiming to bring new insights to visual computing community. Our research does not involve human subjects, sensitive data, any practices that pose privacy security concerns.
REPRODUCIBILITY STATEMENT
The baselines used in experiments, such Shader and GS-IR, are open-source and publicly available. We have detailed our experimental setup, and other design choices, in Section of the main paper and Section A in the Appendix to ensure reproducibility. Please also be assured that the code implementation of this work will be made publicly accessible.
ACKNOWLEDGEMENT
The author would like to thank Julia Du for the help of the title and the beautiful figures. This work has been funded in part by the Army Research Laboratory (ARL) award W91 NNF-2322-000 and W911QX-24-F-0049, DARPA award FA8750-23-2-1015, and ONR award N00014-23-1-28401
REFERENCES
Benjamin Attal, Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan Barron, Matthew O O'Toole, and Pratul P Srinivasan. Flash cache: Reducing bias radiance cache based inverse rendering. arXiv preprint arXiv:2409.05867 2024. 3
Jonathan T Barron and Jitendra Malik. Intrinsic scene properties from single rgb-d image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 17-24, 2013.
Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo
and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5855-5864, 2021.
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 5470-5479, 2022. 1, 7.9, 10
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19697-19705. 2023.
Beer. Bestimmung der absorption des rothen lichts farbigen flüssigkketen. Annalen der Physik, 162(5):78-88) 88, 1852. 4
Sai Bi, Zexiang Xu, Pratul Srinivasan, Ben Mildenhall, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffe Geoffreofr David Kriegman, and Ravi Ramamoorthi, Neural reflectance fields for appearance acquisition. arXiv preprint 2008 2038422284 2020. 3,

---

Published as a conference paper at ICLR 2025
Mark Boss, Raphael Braun, Varun Jampani, Jonathan T Barron, Ce Liu, and Hendrik Lensch. Nerd: Neural reflectance decomposition from image collections. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision. pp. 12684-12694, 2021. 1.3.7
Pierre Bouguer. Essai d' d'optique sur la gradation de lumière. Claude Jombert, 1729. 4
Samuel Rota Bulò, Lorenzo Porzi, and Peter Kontschieder. Revising densification in gaussian splat- ting. arXiv preprint arXiv:2404.06100, 2024. 3
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European Conference on Computer Vision, pp. 333-350. Springer, 2022. 1,3
Wenzheng Chen, Huan Ling. Jun Gao, Edward Smith, Jaakko Lehtinen, Alec Jacobson, and Sanja Fidler. Learning to predict 3d objects with an interpolation- based differentiable renderer. Ad- vances in neural information processing systems, 32, 2019.
Paul Debevec. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Acm siggruph 2008 classes, pp. 1-10. 2008.
Yue Dong, Guojun Chen, Pieter Peers, Jiawan Zhang, and Xin Tong. aopearance-from-motte Recovering spatially varying surface reflectance under unknown lighting. ACM Transactions on Graphics (TOG), 33(6):1-12, 2014.
Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, and Zhangyang Wang. Lightgaus- sian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps. arXiv preprint arXiv:2311.12255, 2023.
Sara Fridovich-Keil. Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings the Conference on Computer Vision and Pattern Recognition, pp.
the IEEE/CVF 2022.
Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaus- sian: Real-time point cloud relighting with brdf decomposition and ray tracing. arXiv preprint arXiv:2311.16044 2023. 1, 2, 3, 4, 7, 16
Stephan Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, and Julien Valentin. Fast- nerf: High-fidelity neural rendering at 200fps. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2021.
Jun Guo, Xiaojian Ma, Yue Fan, Huaping Liu, and Qing Li. Semantic gaussians: Open-vocabulary scene understanding with 3d gaussian splatting. arXiv preprint arXiv:2403. 156244 2024.
Peter Hedman, Pratul P Srinivasan, Ben Mildenhall, Jonathan T Barron, and Paul Debevec. Baking neural radiance fields for real-time view synthesis. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 5875-5884, 2021.
Lukas Höllein, Aljaž Božič, Michael Zollhöfer, and Matthias Nieβner. 3dgs-Im: Faster gaussian- splatting optimization with levenberg-marquarte, arXiv preprint arXiv:2409.12892. 2024. 3
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In ACM SIGGRAPHI 2024 Conference Papers, pp. 1-11, 2024. 3
Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, and Yuexin Ma. Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces. Pro- ceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 5322- 5332, 2024. 1, 2, 3, 4, 7, 8,9
Haian Jin, Isabella Liu. Peijia Xu, Xiaoshuai Zhang, Songfang Han, Sai Bi, Xiaowei Zhou, Zex- iang Xu, and Hao Su. Tensoir: Tensorial inverse rendering. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 165-174, 2023. 1,3
12

---

Published as a conference paper at ICLR 2025
James T Kajiya. The rendering equation. In Proceedings the graphics and interactive techniques, pp. 143-150, 1986. 3, 4
the 13th annual conference Computer
Yoshihiro Kanamori and Yuki Endo. Relighting humans: occlusion-aware inverse rendering for full-body human images. arXiv preprint / 2019.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Transactions on Graphics (ToG), 42(4))1-14, 2023. 1,
Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Jeff Tseng, Hossam Isack, Ab- hishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 3d gaussian splatting as markov chain monte carlo. arXiv preprint 240::4994.99 2024.
Jean-Henri Lambert. Photometria sive de mensura et gradibus luminis, colorum et umbrae. Sumptibus viduae Eberhardi Klett, typis Christophori Petri Detleffsen, 1760. 4
Zhen Li, Lingli Wang, Mofang Cheng, Cihui Pan, and Jiaqi Yang. Multi-view inverse rendering for large-scale real-world indoor scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pultern Recognition, pp. 12499-12509, 2023. 3
Zhihao Liang. Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. Gs-ir: 3d gaussian splatting for inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21644 21653, 2024. 2, 4, 7,9, 10
Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao. Gaussian-flow: 4d reconstruction with dy- namic 3d gaussian particle. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21136-211444 145, 2024.
Yuan Liu, Peng Wang, Cheng Lin, Xiaoxiao Long, Jiepeng Wang, Lingjie Liu, Taku Komura, and Wenping Wang. Nero: Neural geometry and brdf reconstruction of reflective objects from multi- view images. ACM Transactions on Graphics (TOG), 42(4):1-22 2023. 2, 8, 9, 15
Guanxing Lu, Shiyi Zhang, Ziwei Wang. Changliu Liu, Jiwen Lu, and Yansong Tang. Mani- gaussian: Dynamic gaussian splatting for multi-task robotic manipulation. arXiv preprint arXiv:2403.0832 08321,, 2024. 3
Jonathon Luiten, Georgios Kopamas, Bastian Leibe, and Deva Ramanan. Dynamic gaussians: Tracking by persistent dynamic view synthesis. In 2024 International Conference on 3D Vision (3DV), pp. 800-809. IEEE, 2024.
Saswat Subhajyoti Mallick, Rahul Goel, Bernhard Kerbl, Francisco Vicente Carrasco, Markus Stein- berger, and Fernando De La Torre. Taming 3dgs: High-quality radiance fields with limited sources. arXiv preprint arXiv:2406. 15643, 2024. 3
Thomas G Mayerhöfer, Susanne Pahlow, and Jürgen Popp. The bouguer- beer--amber-rrrr law: Shining light on the obscure. ChemPhysChem, 202292202222 2020.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi. and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1)999-106, 2021. 3,
Bailey Miller, Hanyu Chen, Alice Lai, and Ioannis Gkioulekas. Objects as volumes: stochastic geometry view of opaque solids. In Proceedings of the IEEE/CV Conference on Computer Vision and Pattern Recognition, pp. 87-97, 2024. 3
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with a multiresolution hash encoding. ACM transactions graphics (TOG), (1): 2022. 3,
Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Müller, and Sanja Fidler. Extracting triangular 3d models, materials, and lighting from images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
2022.
13

---

Published as a conference paper at ICLR 2025
KL Navaneet, Kossar Pourahmadi Meibodi, Soroush Abbasi Koohpayegani, and Hamed Pirsiavash. Compactšd: Compressing gaussian splat radiance field models with vector quantization. arXiv preprint arXiv:2311.1815, 2023. 3
Simon Niedermayr, Josef Stumpfegger, and Rüdiger Westermann. Compressed 3d gaussian splatting for accelerated novel view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10349-10358 2024. 3
Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d lan- guage gaussian splatting. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 20051-20060, 2024. 3
Olaolu Shorinwa, Johnathan Tucker, Aliyah Smith, Aiden Swann, Timothy Chen, Roya Firoozi, Monroe David Kennedy, and Mac Schwager. Splat-mover: multi-stage, open-vocabulary robotic manipulation via editable gaussian splatting. In 8th Annual Conference on Robot Learning. 2024.
3
Vincent Sitzmann, Semon Rezchikov, Bill Freeman, Josh Tenenbaum, and Fredo Durand. Light field networks: Neural scene representations with single-evaluation rendering. Advances in Neural Information Processing Systems, 34:19313- 2021. 3
Pratul P Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, and Jonathan T Barron. Nerv: Neural reflectance and visibility fields for relighting and view synthe- sis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7495-7504, 2021. 1,3
Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul P Srini- vasan. Ref-nerf: Structured view-dependent appearance for neural radiance fields. In 2022 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 5481-5490.
IEEE, 2022. 2, 7, 8, 9, 15
Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan T Barron, Todd Zickler, and Pratul P Srini- vasan. Eclipse: Disambiguating illumination and materials using unintended shadows. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 77-86,
2024. 3
Wenqi Yang, Guanying Chen, Chaofeng Chen, Zhenfang Chen, and Kwan- Yee K Wong. Ps-nerf: Neural inverse rendering for multi-view photometric stereo. In European Conference Com- puler Vision, pp. 266-284. Springer, 2022. 3
Yao Yao, Jingyang Zhang, Jingho Liu, Yihang Qu, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. Neilf: Neural incident light field for physically-baser material estimation. In European Conference on Computer Vision, pp. 700-716. Springer, 2022. 3,3
Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. Volume rendering of neural implicit surfaces.
Advances in Neural Information Processing Systems, 34:4805-4815, 2021. 3
Silong Yong, Yaqi Xie, Simon Stepputtis, and Katia Sycara. Gl-nerf: Gauss-laguerr quadrature enables training-free free nerf acceleration. arXiv preprint MxX.::24400198311 2024. 3
Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng. and Angjoo Kanazawa. Plenoctrees for real-time rendering of neural radiance fields. In Proceedings of the IEEE/CVF International
Conference on Computer Vision, pp. 5752-5761, 2021. 3
Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, and Noah Snavely. Physg: Inverse rendering with spherical gaussians for physics-based material editing and relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2021a. 1, 3
Xiuming Zhang, Pratul P Srinivasan, Boyang Deng, Paul Debevec. William Freeman, and Jonathan T Barron. Nerfactor: Neural factorization of shape and reflectance under an unknown illumination. ACM Transactions on Graphics (ToG), 40(6):1-18 2021b. 1, 16
Yuanqing Zhang, Jiaming Sun, Xingyi He, Huan Fu, Rongfei Jia. and Xiaowei Zhou. Modeling indi- rect illumination for inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18643-18651 2022. 2, 7,
14

---

Published as a conference paper at ICLR 2025
OMG:
OPACITY MATTERS IN MATERIAL MODELING WITH GAUSSIAN SPLATTING APPENDIX
A HYPERPARAMETERS
The neural network introduced uses 0.001, 0.0001, 0.0005 and 0.007 learning respectively for Symthetic4Relight, Shiny Blender, Glossy Synthetic and MIP-NeRF 360 datasets. The neural network implemented fully-connected MLP with hidden layers. 128 hidden dimension and ReLU as activation function. We directly feed the material properties into the MLP without any encoding such as positional embedding.
B FULL DERIVATION OF EQUATIONS
Here we give the full derivation of Eq. 12.
0L aL OC
AL 8C
N
0m,
8C
0m;
0m,
aL
9j TTT) T3 + Cj 0m,
daj T3 cjos dm;
0T,
00
m,
0L, dc;
dai
00 Ōm:
a,T. + cTi
0m,
Cjaj
mm,
ax)
(16)
j=+t1
k=1
N
ÖL, dc;
ac dm;
o,T; + ciT;
0m:
Σ
CjOc
exp
ox))
j=1+1
k=1
dc; 0L
ØL. T da: Ca ac
N Σ
aL
Oos
Ti
Tja;
3m, ac
0m;
ac
1
0m:
C ADDITIONAL RESULTS
We present the full results on Synthetic4Relight (Zhang et al., 2022), Shiny Blender (Verbin et al., 2022) and Glossy Synthetic (Liu al., 2023). All the baseline results reproduced by in the main paper. More visualization results can be found in Fig. C1, Fig. C2, Fig. C3, Fig. C4 and Fig. C5.
D VISUALIZATION OF CROSS SECTION
To understand what the cross section network is learning, we visualize some examples in Fig. D6. In the figure, the darker region indicates lower cross section, meaning that it lets more light pass through whereas the brighter regions stands for the regions that have a higher cross section. The example shows that the windows well the plastic bag on the floor actually have a lower cross section assigned to them, which corresponds to human intuition.
15

---

Published as a conference paper at ICLR 2025
Novel View Synthesis
Albedo
Relight
Roughness
Scene
Method
PSNR
SSIM t LPIPS
|SNNR| t SSIM LPIPS 4 PPSNR
SSIM
LPIPS 4
MAE
R3DG
31.90
0.967
0.072
24.82
0,923
0.070
31.38
0.965
0.078
0.0009
Balloons
Ouns
32.11
0.968
0.072
25.84
0.926
0.055
32.50
0.967
0.071
0.0011
R3DG
32.61
0.972
0.050
26.53
0,956
0.089
29.04
0,954
0.067
0.0357
Hotdog
Ours
33.59
0.975
0.047
26.95
0.957
0.085
28.73
0.955
0.066
0.0217
R3DG
34.72
0.978
0.039
30.55
0,963
0.046
33.07
0.966
0.041
0.0036
Chair
Ours
35,00
0.979
0.040
30.82
0.964
0.045
33.46
0.968
0.041
0,0039
R3DG
37.03
0.984
0.027
32.88
0.967
0.064
37.98
0.984
0.023
0.0032
Jugs
Ours
37.17
0.984
0.027
33.64
0.968
0.049
38.16
0.984
0.023
0.0074
Per-scene results on Synthetic4Relight dataset. For albedo reconstruction results, we follow the standard practice (Zhang et al., 2021b; Gao et al., 2023) and scale each RGB channel by a global scalar.
Table CI:
Shiny Blender
Avg.
Ball
Car
Coffee
Helmet
Teapot
Toaster
GaussianShader
30.77
29.05
28.51
31.55
28.91
43.57
23.05
PSNR1
Ours
31.07
29.65
28.14
31.29
28.79
43.48
25.07
GaussianShader
0.953
0.955
0.940
0.970
0.956
0.996
0,902
SSIM
Ours
0.957
0.961
0.939
0.969
0.955
0.996
0.924
GaussianShader
0.081
0.146
0.047
0.085
0.086
0.011
0.110
LPIPS
Ours
0.076
0.137
0.050
0.086
0.085
0.011
0.089
Table C2: Per-scene results Shiny Blender dataset.
Glossy Synthetic
Avg.
|Angel
Bell
Cat
Horse
Luyu Potion Tbell Teapot
GaussianShader
27.25|
26.92
29.73 31.43 26.38 27.31 29.54 23.41 23.25
PSNR
Ours
27.68
28.24
29.91 31.00 26.62 27.32
29.40
24.67 24.31
GaussianShader
0.925
0.923
0.939 0.961 0.934
0.917
0.936
0.897
0.895
SSIM
Ours
0.929
0.929
0.940 0.959 0.934 0.916
0.935
0.907 0.909
GaussianShader
0.083
0.071
0.085 0.058 0.056 0.066 0.096 0.136 0.098
LPIPS
Ours
0.081
0.066 0.084 0.059 0.057 0.067 0.097 0.125 0.091
Table C3: Per-scene results on Glossy Synthetic dataset.
GT
GaussianShader
Ours
Rendering
Normal
Rendering
Normal
Teapot
Horse
Figure C1: More qualitative example in comparison to ShaseianShader on Glossy Synthetic dataset.
16

---

Published as a conference paper at ICLR 2025
GT
GS-IR
Ours
Rendering
Normal
Rendering
Normal
Kitchen
Counter
Garden
Room
Figure C2: More qualitative example in comparison to GS-IR on MIP-NeRF 360 dataset.
Rendering
Roughness
Relighting
Relighting
R3DG
Ours
GT
Figure C3: More qualitative example in comparison to R3DG on Synthetic4Relight dataset.
17

---

Published as a conference paper at ICLR 2025
Rendering
Albedo
Roughness
Relighting
Relighting
R3DG
Ours
GT
Figure C4: More qualitative example in comparison to R3DG on Seelhttic4Relight dataset. We add
a gamma correction to the relighting results to make the visual differences clearer.
GaussianShader
Durs
Rendering
Normal
Rendering
Normal
Coffee
Bell
Figure C5 More qualitative example in comparison to GhuseianShader on Glossy Synthetic and Shiny Blender datasets.
18

---

Published as a conference paper at ICLR 2025
Cross Section
Ground Truth
Rendering
Albada
Roughness
Metalness
Bonsai
Counter
Figure D6: Visulization of the learned cross section Mip-NeRF 360 dataset. Dark indicates the lower cross section (meaning it lets more light pass through) and bright stands for the higher cross section. The example shows that the windows well the plastic bag on the floor actually have lower cross section assigned to them, which corresponds to human intuition.
19


Reviews:
[{"review_id": "fGYxill5jl", "reviewer": "Reviewer_gEKk", "summary": "This paper introduces a plug-and-play approach for enhancing Gaussian Splatting in material modeling by explicitly modeling the relationship between opacity and material properties using the Bouguer-Beer-Lambert law. The authors validate this insight through various analyses, demonstrating its correctness. Qualitative results across different methods and datasets show consistent improvements, as evidenced by performance gains across multiple metrics.", "questions": "Given the simplicity of this formulation and the well-written manuscript, I don't have further questions regarding this submission. In general I enjoy reading this paper and love this simple yet effective solution. Just look at some minor issues in Weaknesses.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The core idea is pretty simple and straightforward, which I personally enjoyed. The authors have demonstrated the validity of their formulation from multiple perspectives, making the approach theoretically sound and insightful.\n2. The proposed formulation leads to improved reconstruction quality across various datasets and methods.\n3. This paper is well written and easy to follow. The entire formulation is physically sound and all the theoretical details are explained in an intuitive way for readers.\n4. Given the simplicity of this method, it should be broadly applicable to future 3DGS-based material modeling approaches, potentially making a significant impact to the community.", "weaknesses": "1. While the reported quantitative metrics indicate improvement, I struggled to notice significant visual differences in the rendered RGB images. The geometric variations are more apparent in some normal renderings. Maybe additional results or visualizations could be provided to better illustrate the quality difference.\n2. As mentioned by the authors, the impact of lighting frequency/spectrum is not considered, which could be an interesting future direction to model more complex materials.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RlMEiekwTA", "reviewer": "Reviewer_G4SH", "summary": "This paper proposes to model opacity, as derived from the existing material properties, to improve Gaussian Splatting for better rendering. The work is inspired by the physical phenomenon that light intensity decays at different rates as it travels through different absorbing media. The authors map existing terms in the  GS formulation to terms in the Bouguer-Beer-Lambert law and add a new term predicted from albedo, metalness, and roughness, to cover the additional cross-section term. \n\nThe authors also provide nice interpretations of their formulation from other perspectives such as Nerf and Taylor expansion, thereby stressing the correctness of their approach.\n\nSince the method is plug-and-play, it was run on synthetic datasets containing glossy objects and also on real-world datasets such as Mip-nerf 360. Both the qualitative and quantitative results show that the method achieves high visual quality.", "questions": "Related to the biggest weakness IMO above, have the authors tried this on translucent objects? If this doesn’t work on those objects, the basis of this paper is fundamentally challenged.\n\nRelated to my final point in Weaknesses, have the authors tried to understand what the cross section network is learning? This can be done in many ways, but the one I have in mind is seeing how a trained network responds to varying inputs of roughness, metalness, and albedo. Again, I’m very skeptical of the network learning anything physically-based. Why would albedo inform cross section? Again, they seem to be from two different worlds of material properties.\n\nNit: I like Figure 1a, but Figure 1b is misleading until I got to the very end of the paper, where the authors said their approach didn’t model the dependency on light frequency. I recommend removing Figure 1b if color of the light doesn’t matter in the entire paper.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "I like how this work incorporates a physically-based factor into the existing GS formulation. The additional term required to cover cross section also makes sense, since that should indeed be material-dependent and disentangled from geometry.\n\nI also like the interpretations the authors provide, to view their proposed approach from other perspectives. That facilitated understanding and also boosted the reader’s confidence to a certain degree.\n\nThe method is plug and play, so if working well, it can have a big impact on how people model Gaussian Splats.", "weaknesses": "It’s surprising that this paper shows no result of transparent/translucent objects. I don’t think it suffices for a method designed to handle transparent/tanslucent appearance to show improvements on opaque objects but not on the original objects of interest. Ideally, there should be videos of view synthesis results on translucent objects as evidence to what this paper claims.\n\n\nAssuming this method does handle translucency well, the big question mark in my head then is how the model deals with shape-material ambiguity. As a specific example: a cup could be due to regular cup geometry paired with regular cup material or cube geometry paired with a translucent material that’s transparent everywhere and opaque around/in the center cup shape. There should be some analysis on why this is not happening if the paper properly models transparency/translucency. A specific experiment the authors can do is setting up a synthetic scene where a shape is enclosed by some translucent material. Does the model reconstruct the translucent part or just ignore it, reconstructing the object directly?\n\nAlso, current improvements reported in the paper on opaque objects may simply be due to a bigger model / gaussian formulation with more parameters we can optimize. If this is actually the case, the whole BBL law inspiration no longer stands.\n\nI am not fully convinced that the cross section network is really learning cross section from albedo, metalness, and roughness. In fact, I don’t think that makes sense.  Consider a translucent material that has some cross section parameter for the network to learn. This material may not have well-defined albedo, metalness, or roughness, which are properties in opaque appearance BRDFs. I hope the authors can provide some clarifications around this (see also my questions below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3JfcNSlZ9Y", "reviewer": "Reviewer_rqsv", "summary": "The paper draws inspiration from the Bouguer-Beer-Lambert law in radiative transfer theory, and applies this theory to 3D Gaussian Splatting. The paper analyzes the alpha blending in 3DGS, considering it as an absorbing body. Based on this analysis, the paper proposes to add a material-affected term into the computation of $\\alpha$ value, namely the cross section modeling, predicted by an MLP from the material parameters. The paper's design can be plugged into exisiting 3DGS-based inverse rendering pipelines, and experiments show that it can achieve superior performance compared with the original inverse rendering method.", "questions": "1. Notation for Guassian covariance matrix. What does the $R_\\theta$ mean in the Gaussian covariance matrix expression $R_\\theta(\\Sigma_i)^{-1}$ in Eq. 2 and 9?\n2. L275-277. The explanation on why the path length $s$ can be simply set to constant 1 is not clear and convincing. The authors should provide a more detailed explanation on it.\n3. Implementation details of the MLP. Does it use positional encoding for the material properties, or just directly feed the properties into the MLP?\n4. Experimental results show improvements on normal estimation. Will the proposed method work better on surface-based Gaussian Splatting methods, e.g. 2D Gaussian Splatting, which has a more well-defined normal and surface geometry representation?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. Good paper writing and analysis. The paper explains its motivation and idea very clearly and intuitively, with mathematical derivations. I like the analysis subsection, which makes the paper's design even more reasonable.\n2. Technical contribution and novelty. The paper points out the limitation of existing methods that disentangle opacity from material, and develops a method to add the dependency between opacity and material. I think this is a very meaningful and valuable point of view.\n3. The paper experiments on 3 baseline methods in 3 different datasets, demonstrating the proposed method's applicability in different scenarios.", "weaknesses": "1. Some designs of the paper requires further explanation. See the \"Questions\" section for my confusion.\n2. Citations. Some suggestions of additional citations:\n    - Huang, Binbin, et al. \"2d gaussian splatting for geometrically accurate radiance fields.\" ACM SIGGRAPH 2024 Conference Papers. 2024\n    - Yariv, Lior, et al. \"Volume rendering of neural implicit surfaces.\" Advances in Neural Information Processing Systems 34 (2021): 4805-4815.\n    - Miller, Bailey, et al. \"Objects as volumes: A stochastic geometry view of opaque solids.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n    - Munkberg, Jacob, et al. \"Extracting triangular 3d models, materials, and lighting from images.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n3. Subtle improvement in Fig. 4. The comparison with R3DG on Synthetic4Relight dataset demonstrates little difference. I wonder if the authors can highlight some areas with more significant differences in the figure.\n4. Needs more geomentry comparison. The proposed cross section modeling provides a more physical constraint on the opacity, which I think can improve the constructed geometry. The experiment section has demonstrated qualitative results on normal maps, and I think the paper should include more quantitative comparisons on the reconstructed geometry, including normal accuaracy, and the extracted mesh if possible.\n5. Lack of interpretability in the cross section modeling. The paper simply uses an MLP to predict $\\sigma_\\nu$, which is an implicit mapping and lacks interpretability. Since the intuition of cross section modeling is based on the law of physics, the design should also be more reasonable.\n6. A minor suggestion: It will be better to use vector graphics (e.g. PDF or SVG) instead of bitmaps (e.g. PNG or JPG) in the figures.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kxh5unLk2u", "reviewer": "Reviewer_RDFN", "summary": "This manuscript introduces a 3D Gaussian Splatting (3D-GS) based inverse rendering task, where opacity is conditioned on material properties as suggested by the Bouguer-Beer-Lambert law. Comparisons within 3D-GS based inverse rendering tasks and ablation studies validate the effectiveness of this approach.", "questions": "1. Maybe experiments on transparent or translucent objects can support the theory.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The Bouguer-Beer-Lambert law is interesting and technically sound.\n2. Experiments validate the effectiveness of the proposed cross-section dependency.", "weaknesses": "1. Minor improvements have been observed compared to the baseline.\n2. A comparison with NeRF-based inverse rendering is missing.\n3. The claim \"The emergence of 3D Gaussian Splatting has boosted it to the next level\" in Line 15 is not convincing. GS-based inverse rendering has proven to be far less effective than SDF-based inverse rendering frameworks like NeRO[1] in terms of performance.\n4. The case presented in Figure 1 considers translucent objects, where opacity is influenced by materials. However, the experiments conducted use opaque objects.\n\n\n[1]Liu Y, Wang P, Lin C, et al. Nero: Neural geometry and brdf reconstruction of reflective objects from multiview images[J]. ACM Transactions on Graphics (TOG), 2023, 42(4): 1-22.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)