DiffTrans: Differentiable Geometry-Materials Decomposition for Reconstructing Transparent Objects
Authors: Changpu Li
Shuang Wu
Songlin Tang
Guangming Lu
Jun Yu
Wenjie Pei
Venue: ICLR 2026 Poster (2026)
Keywords: Differentiable rendering
Transparent object reconstruction
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Reconstructing transparent objects from a set of multi-view images is a challenging task due to the complicated nature and indeterminate behavior of light propagation. Typical methods are primarily tailored to specific scenarios, such as objects following a uniform topology, exhibiting ideal transparency and surface specular reflections, or with only surface materials, which substantially constrains their practical applicability in real-world settings. In this work, we propose a differentiable rendering framework for transparent objects, dubbed \emph{DiffTrans}, which allows for efficient decomposition and reconstruction of the geometry and materials of transparent objects, thereby reconstructing transparent objects accurately in intricate scenes with diverse topology and complex texture. Specifically, we first utilize FlexiCubes with dilation and smoothness regularization as the iso-surface representation to reconstruct an initial geometry efficiently from the multi-view object silhouette. Meanwhile, we employ the environment light radiance field to recover the environment of the scene. Then we devise a recursive differentiable ray tracer to further optimize the geometry, index of refraction and absorption rate simultaneously in a unified and end-to-end manner, leading to high-quality reconstruction of transparent objects in intricate scenes. A prominent advantage of the designed ray tracer is that it can be implemented in CUDA, enabling a significantly reduced computational cost. Extensive experiments on multiple benchmarks demonstrate the superior reconstruction performance of our \emph{DiffTrans} compared with other methods, especially in intricate scenes involving transparent objects with diverse topology and complex texture. Code will be released.

Published as a conference paper at ICLR 2026
DIFFERENTIABLE GEOMETRY- MATERIALS DECOMPOSITION FOR RECONSTRUCTING
DIFFTRANS:
TRANSPARENT OBJECTS
Changpu Li Shuang Wu Songlin Tang", Guangming Lu², Jun Yu', Wenjie Pejf1.2
Harbin Institute of Technology, Shenzhen
Peng Cheng Laboratory
"Nanjing University
(lichangpu29, wushJang9811,
/enfjeccoerreeutloooocco
Langsonglin@stunn hit.
itt.iniiitt..u.......uu cn;
(luguangm, yujun)ehit. wechit.edu.c..
ABSTRACT
Reconstructing transparent objects from a set of multi-view images is chal- lenging task due to the complicated nature and indeterminate behavior of light propagation. Typical methods are primarily tailored to specific scenarios, such as objects following uniform topology, exhibiting ideal transparency and surface specular reflections, or with only surface materials, which substantially constrains their practical upplicability in real- world settings. In this work, we propose dif- ferentiable rendering framework for transparent objects, dubbed Diff/rans, which allows for efficient decomposition and reconstruction of the geometry and materi- als of transparent objects, thereby reconstructing transparent objects accurately intricate scenes with diverse topology and complex texture. Specifically, we first utilize FlexiCubes with dilation and smoothness regularization as the iso-surface representation reconstruct an initial geometry efficiently from the multi-view object silhouette. Meanwhile, we employ the environment light radiance field to recover the environment of the scene. Then we devise a recursive differentiable ray tracer to further optimize the geometry, index of refraction and absorption rate simultaneously in a unified and end-to-end manner, leading to high-quality reconstruction of transparent objects in intricate scenes. A prominent advantage the designed ray tracer is that it can be implemented in CUDA, enabling sig- multiple bench-
nificantly reduced computational cost. Extensive experiments marks demonstrate the superior reconstruction performance of our DiffTrans com- pared with other methods, especially in intricate scenes involving transparent ob- jects with diverse topology and complex texture. The code is available here
INTRODUCTION
The reconstruction of geometry and materials for transparent objects poses a highly complex and ill- posed problem (Kutulakos & Steger 2008). This is primarily due to the intricate interplay between light refraction among an object's ' surfaces and its surrounding environment. As a result, compared to opaque objects, the appearance of transparent objects more intricately entangled with their geometry, such that even a slight change in scene parameters can lead to significant variations in their appearance.
Early approaches employ environment matting (Zongker et al. 1999) Matusik et al. 2002) to recon- struct transparent objects, which could exploit specialized hardware setups (Matusik et al. 2002). Recently, differentiable rendering have demonstrated significant success in novel view synthesis and inverse rendering, unlocking new potential for reconstructing transparent objects. One typical way is to leverage the eikonal field (Bemana et al. 2022 Deng et al. 2024) to model the march-
*Corresponding author. "Equal contribution.

---

Published as a conference paper at ICLR 2026
Multi-view
Geometry and Materials
DiffTrans
Relighting
Images
Overview of the proposed DiffTrans. Our approach reconstructs the geometry and mate- rials of the transparent objects with diverse topology and complex texture from set of multi-view images using differentiable mesh ray tracer, enabling scene editing capabilities such as relighting.
Figure 1:
ing process of light. Despite achieving promising fidelity in novel views and scenes with complex object composition, these methods struggle extract reliable meshes due the lack of constraints on surface geometry, thus unable to deal with objects with intricate meshes (Wang et al., 2023). Another representative way is based on surface reconstruction using representations such as neural SDF field (Yariv et al. 2020 Wang et al., 2021), triangle mesh (Lyu et al., 2020) or 3D gaussian splatting (Huang al. 2025). However, these methods fail reconstruct the geometry of trans- parent objects with complex textures. NEMTO (Wang et al. 2023), NeTO (Li et al., 2023) and NeRRF (Chen et al. 2023) neglect the materials of transparent objects, while Nu-NeRF (Sun et al.) 2024) and TransparentGS (Huang al. 2025) only model the surface materials of transparent ob- jects. None the aforementioned methods can effectively model transparent objects with complex internal absorptive texture, which are commonly observed in real-world scenes, such jewels, glass decorations, and handmade resin.
In this paper, we propose to optimize the geometry and materials of the transparent objects simulta- neously to improve the reconstruction quality. As shown in Figure [ we devise a novel differentiable rendering framework, dubbed DiffTrans, which decouples the geometry and materials of the trans- parent objects with intricate topology and internal texture from set of multi-view images, enabling scene editing capabilities such as relighting. Our DiffTrans consists of three stages that progres- sively facilitate the reconstruction of the geometry, environment and materials. First, we adopt Flex- iCubes (Shen al. (023) as the iso-surface representation to recover the topology of the transparent objects from multi-view object silhouette. By incorporating dilation and smoothness regularization. we can efficiently obtain an initial geometry. Meanwhile, we employ a radiance field (Reiser al., 2023) with voxel grid and triplane to recover the environment of the scene with out-of-mast regions of the input images. Finally, as the most crucial step, we design a recursive diffcrentiable mesh ray tracer to further optimize and refine the geometry, index of refraction and absorption rate of the transparent objects unified and end-to-end manner. In particular, we implement the ray tracer in OptiX and CUDA, significantly enhancing computational efficiency.
In summary, we make following contributions:
We design novel differentiable rendering framework, dubbed DiffTrans, which allows for efficient decomposition and reconstruction of geometry and materials the transparent ob- jects, thereby reconstructing transparent objects accurately in intricate scenes with diverse topology and complex texture.
We propose to employ FlexiCubes with dilation and smoothness regularization as the iso- surface representation, reconstruct the initial geometry of transparent objects using only multi-view object silhouette. Concurrently, we can efficiently recover the environment of the scene using environment light radiance field.
We devise differentiable recursive mesh ray tracer for the joint optimization of geometry, index refraction and absorption rate of the transparent objects. The designed ray tracer is implemented in OptiX and CUDA to enhance optimization efficiency.
2

---

Published as a conference paper at ICLR 2026
Extensive experiments both synthetic and real- wollw datasets demonstrate that Diff- Trans achieves favorable reconstruction quality compared to other state- state-of-the-arr methods for the transparent objects with different topology and intricate texture.
2 RELATED WORKS
Transparent Object Reconstruction. The methods fall into two categories: eikonal-basee recon- struction and surface-based reconstruction. (Bemana et al.) 2022) uses eikonal field in a user- defined area to store IoR and solve the eikonal equation to get differentiable light paths. (Deng et al. 2024) stores the ray marching length and direction and impose straight ray restriction. Lyu
et al. 2020 Wu et al. 2018) leverages patterned hackground for extra ray supervision and optimize on mesh. (Li et al. 2023) uses implicit surface and discard unreliable rays to improved the quality.
(XU et al. (022) further uses a hybrid surface representation of mesh and neural network to stabilize training. (Wang et al. (2023) combines implicit surface and environment matting to recover visually appealing result. (Chen et al. 2023) developed mixture of DM'et (Shen et al., 2021) and neural network to achieve robust mask-only training procedure, (Sun 2024) proposes a ray-traced
iterative strategy to reconstruct nested transparent objects. (Huang et al.) 2025) adopts 3D gaus- sian splatting and light probe to promote the delicacy of reconstruction. (Wu al. 2025) applies multi-level level sets and regularizations pre-trained NeRF to adapt to translucent or thin objects. However, None of the aforementioned methods can handle transparent objects with complex internal absorptive texture
Inverse Rendering. Inverse rendering aims to decompose visual input into geometry, material, and lighting. (Zhang al. 2021) pioneers this direction by factorizing shape and spatially-varying reflectance from a pre-trained NeRF, utilizing surface normals and visibility approximations. To improve optimization efficiency, (Jin et al. 2023) leverages tensorial representation jointly re- construct geometry and material properties under unknown illumination. Recently, explicit repre- sentations have been adapted for relighting tasks; (Gao et al., 2024) integrates BRDF decomposition and ray tracing into 3D Gaussian Splatting to achieve realistic point-based relighting. Addressing the complexities of global illumination, (Dai al. 2025) introduces a multi-bounce path tracing framework combined with reservoir sampling to accurately recover scene parameters.
Mesh-based Optimization. Many differentiable renderers (Laine et al., 2020; Loper Black, 2014 Jakob et al.] (022) support mesh representation. (Nicolet et al. (021) introduces mesh optimizer using the laplace matrix of mesh to avoid seff-intersection and other problems during training. Besides directly optimizing on fixed-topology mesh, some methods (Shen et al.) 2021 2023) take meshes as the proxy of other geometry representations, which are free from common artifacts during mesh optimization and greatly extended its application such as universal inverse rendering (Munkberg et al.) 2022, Hasselgren et al. 2022). To overcome the inefficiency of opti- mizing through visibility gradient, (Mehta et al. (023) derived stronger supervision of meshes which able to dynamically adapt to different topology. (Son et al.] 2024) proposes a differentiable mesh representation that supports topology changes during the inverse process, while [Binninger et al.) 2025) introduces an on-the-fly Delaunay tetrahedral grid strategy to ensure stable isosurface extraction for gradient-based optimization.
mask error
NeRF-based Rendering. NeRF (Mildenhall et al. 2020) achieves photo-realistic rendering by learning implicit scene representation with pure MLPs which incurs slow training speed. To this end, recent works seek to efficient explicit representation to avoid using deep MLPs. Plenox- els (Fridovich-Keil et al. 2022) uses pure explicit sparse voxel grid. DVGO (Sun et al. 2022)
uses dense voxel grid, TensoRF (Chen et al. (222)) uses decomposed tensorial voxel grid. Instant- NGP (Müller et al., 2022) uses multi-rrsoluutonn hash voxel grid and MERF (Reiser al. 2023)
uses low resolution dense voxel grid and high resolution tri-planes, these methods all employ tiny MLPs as voxel fcature decoder to further improve representation capacity. Although these methods achieve great quality of novel view synthesis, they usually suffer from poor extracted geometry due to lack of geometry restriction in volume rendering, especially for transparent objects. The assump- tion of straight ray of volume rendering also ignores the refraction of light which happen frequently in transparent objects.
3

---

Published as a conference paper at ICLR 2026
Initialization Phase
Refine Phase
Differentiable Recursive Ray Tracer
Environment Radiance Field à
Geometry Regularization
Material Field
Material Regularization
Mask Images
Initial Mesh
GT Image
te
InoOtjaei Radinnee En
Out-at-Otyel Radiance Ee
Light Decsy
Decry
NeRF
Fresuel Composing
Out-of-Mask Pixels
Enviromment Radiance Field
Final Radiunce by
Rendered RGB
The framework of our DiffTrans. We commence by reconstructing the initial geometry from multi-view mask images, and recovering the environment light radiance field by employing pixels out of the mask regions (Section 3.2). In the refine phase, we first define the light decay within absorptive medium, and the Fresnel term used for blending in-object radiance and out-of-object radiance (Section 3.3). Then optimize the geometry, loR and absorption rate of the transparent objects simultaneously via our differentiable recursive mesh ray tracer, with geometry and material regularization (Section 3.4).
Figure 2:
3
METHOD
3.1
OVERVIEW
Our DiffTrans aims to reconstruct the geometry and materials of transparent objects from collection of multi-view images, the masks of the transparent objects in the scene and the corresponding camera parameters. Previous works, such as NeRRF (Chen et al. 2202)), are limited to reconstructing trans- parent objects that exhibit ideal transmission and surface specular reflection, while Nu-NeRF (Sun) et al. 2024) models only the surface materials, neglecting the internal absorption rate of transparent objects, which leads subpar reconstruction quality when the textures of transparent objects are highly complex. In contrast, our DiffTrans employs the differentiable decomposition of both ge- ometry and materials, and explicitly models the absorption rate transparent objects, significantly enhancing the reconstruction quality. Figure illustrates the overall framework of our DiffTrans. Similar to previous methods, we adopt a progressive training strategy to achieve more stable results. Specifically, we first utilize the multi-view masks of transparent objects along with the correspond- ing camera parameters to reconstruct the coarse geometry through differentiable rasterizing (Laine) employ the pixels in the out-of-mask regions of the multi-view images
al. 2020). Meanwhile, to recover the 3D environment. Subsequently, we introduce a novel recursive ray tracer to simulta- neously optimize the geometry and materials of the transparent objects, including index of refractive (IoR) and absorption rate, in unified and end-to-end manner.
The inherent ill- ill-posed nature of the reconstruction for transparent objects necessitates simplifications in the scene. To address this, we make three key assumptions. Firstly, we postulate that each point within the transparent object has consistent refractive index, which allows rays to propagate linearly inside the object, thereby eliminating the need for eikonal rendering. Secondly, we posit that the materials of the transparent objects solely consists of the absorption rate and index of refraction. This assumption is essential due to the substantial ambiguity in radiance caused by in en-scattering and background when precise recovery of the geometry and background can not be achieved. Our final assumption is that the surface of the transparent object exhibits specular be- havior, as inverse rendering of rough transparent objects remains highly challenging problem that introduces excessive complexity. Although the scene has been simplified through these assumptions,
Assumptions.
4

---

Published as a conference paper at ICLR 2026
our DiffTrans framework is sufficiently capable of representing majority of transparent objects in the real world.
3.2 GEOMETRY AND ENVIRONMENT INITIALIZATION
Different from reflection-only surfaces (Luu et al. 2023), the appearance of transparent objects is highly sensitive to the environment. In order to ensure the convergence of the optimization, we first initialize the geometry the transparent objects and the environment light radiance field, providing fast and reliable starting point for the subsequent optimization.
Mask-based Geometry Initialization. We leverage diffcrentiable rasterizer (Laine et al., 2020 to recover the initial geometry of transparent objects from multi-view object masks. Specifically, we employ FlexiCubes (Shen et al. 2023) as the isosurface representation to project the 3D mesh to the 2D) image plane, and utilize the ground truth mask M, to supervise the rendered images M, with L.1-loss L-geo-ini-- init- We observe that using only masks for supervision leads to numerous artifacts and cracks in the reconstructed mesh. To mitigate this issue, we apply dilation regularization to penalize the SDF values:
Σ sdf(z;),
Ldilation
N
where sdf(æ;) is the SDF value at point v; randomly drawn from the canonical space. To miti- gate potential noises during training, we apply smoothness regularization smoohh to the depth d and normal n of the objects in the screen space by penalizing the first and second-order gradients. Ad- ditionally, we adopt various other regularization and optimization strategies to enhance the quality of the initialized geometry, with further details provided in Section A. of the appendix.
Environment Initialization. We employ radiance field (Reiser al. 2023) with coarse dense grid and fine triplanes. along with two iNGP-style (Müller et al. 2022) proposal grids to represent the environment of the scene. The environment light radiance field is initialized leveraging out-of- mask pixels in NeRF-like manner:
N-1
Cenv-init
Σ ==0
(1: 011 Ma)l,
where o denotes element wise product.
3.3 LIGHT INTERACTION WITH TRANSPARENT OBJECTS
Based on the initial geometry and environment obtained in the previous stage, we propose a novel differentiable recursive ray tracing to further optimize the geometry and materials of the transparent objects in a unified and end-to-end manner. Unlike previous works (Li et al.) (0200) which utilize feed-forward approach to recover the transparent objects, our method reconstructs these objects in an maalssi-bb--yntheer reconstruction. We will detail our ray tracer in Section 3.4.
manner that adheres to series of physical laws, ensuring the reality of
Light Interaction with Transparent Surfaces. Since we do not consider the roughness of the transparent objects, the light interaction can simplified to a deterministic process that does not require random sampling. We categorize the interaction between rays and the surface of the trans- parent objects into two types: reflection and refraction. The reflected light direction wp and refracted light direction Wi are computed as follows by separating the outgoing direction into the components parallel and perpendicular to the surface normal:
Wr = 2 (wij n) n Wi:
(3)
W. wi way
(4)
w, = -n-
= -v12 172 sin? O..
(5)
w/ 1
(wi (wi: n)n).
5

---

Published as a conference paper at ICLR 2026
where denotes the normal direction, 7) is the ratio of the IoRs between the outgoing medium and
the ingoing medium, which uniform per object, and 8 = w, n. When total internal reflection
occurs, consider only reflection and disregard refraction, i.c. 172 sin 0, < 0. The reflection rate and the refraction rate are calculated based on the Fresnel Equation:
I( IN Cos 0: - ?h COS 02 )² 1 CDS 02 - 1/: cos 0;
R 7. 0, 7) cos 0₁
)
1): 0₁ + 7) cos 8
T
(8)
Optical Transportation in Absorptive Medium. According to our assumption in Section 3.1 simplify the Radiative Transport Equation (Kajiya & Von Herzen 1984) the following form:
we
L(x = =xxx)xxxxx(
Sw
where w denotes the propagation direction of the light, L(x, w/) is the radiance at point x in direction wi, and (x) is the absorption rate at point x. The equation shows the decay of the radiance along the ray direction the medium. The integral form of the equation is:
L(x, w) L(xo, w) exp (-Sx (x(x)dxx)
(10)
where Xy is the starting point of the ray. We calculate L(x, w) in a NeRF-like manner:
L(x, w) L(xo, w) exp (- (-Enixi)
(11)
The absorption rate (x) is represented as a differentiable 3D texture (uüller et al. 2022 and optimized using our proposed differentiable mesh ray tracer.
3.4 DIFFERENTIABLE MESI RAY TRACER
Our differentiable mesh ray tracer simulates the light interactions with the transparent ob- jects in a recursive manner, enabling the uni- fied and end-to-end optimization of geometry, index of refraction and absorption rate. The recursive ray tracer is implemented using Op- tiX and CUDA, significantly reducing compu- tational overhead.
Environment
Transporent Object
1/4 T/ exp
Σ mile)da
x/-1
E/j-UR
Kisz
Σ T(z)a(z)d
Len
RR
Recursive Ray Tracing. We render the color of each camera ray by recursively tracing it un- til it no longer intersects with the transparent objects. Benefiting from our assumptions of co- herent IoR and the absence of roughness, ray tracing can be simplified to deterministic pro- cess without the need estimate rendering in- tegrals. We partition the sampling region into two parts: the interior and exterior of the ob- ject, sampling the absorption rate field and the environment field, respectively. To be specific, process the rays in the following order:
Refraction Ray
Reflection Ray
Normal
Camera
n/ 9omg + funy + Bamy
Outgoing Radiance
Figure 3: The recursive rendering process demon- strated with light path. LiR and LRR are the diance of the reflected ray, which is abbreviated as single variable for simplicity. T, R,, TV and R¹ are calculated with Equation (7) and Equation (8)
1. Terminate the rays that reach the recursive depth Dmax-
2. Evaluate the intersection status of each ray with the object, categorizing it into three sce- narios: non-intersection, intersection from outside the object, and intersection from inside the object.
3. For rays that do not intersect with the object, return the color rendered through the environ- ment field.

---

Published as a conference paper at ICLR 2026
4. For each ray that intersects with the object externally, additional ray is emitted from the intersection point uz in both the reflection and refraction directions, and these rays are recursively traced. The radiance of the reflected and refracted rays then blended using the values of RG and T computed from Equation (7) and Equation (8). Finally, the resulting radiance 7 is mixed with the environmental radiance prior the intersection point. The IoR of the objects is differentiable during this process.
5. For each ray that intersects with the object from the interior, follow the steps in 4), and the radiance is attenuated based on the accumulated absorption rate from the origin to the intersection point, as formulated in Equation [II
Differentiable Mesh Intersection. To further refine the initial mesh obtained in the previous stage, we calculate the intersection between the rays and the mesh in a differentiable manner. Our mesh M is formulated as M (V, E, F) comprising vertices V, edges E, and faces F. The normal n of the intersecting point is barycentrically interpolated from the vertex normals, which are computed as the average of the face normals:
n(x)
Σ v.E((x)
8,
(12)
(V V15,0) x (V5,,2 - V1J.n)
Dop
fYESNEE I[
Σ
- I(vv,,1-V1;,
V1,00) (V5z.2 ~
(13)
where f(x) is the first triangle that the ray intersects, Vy..k is the k-th vertex in triangle fi, and w; is the barycentric coordinate of in f(x). The intersection points are also interpolated from vertex positions to provide the gradient for the mesh. The illustration is shown Figure
Optimization. The mesh vertices, inner absorptive rate and IoR is jointly optimized in this stage. In order to avoid the influence of over-absorbed pixels, we supervise the rendered color è using Le loss, with an additional c;:
Leulor
Σ (c. KEB
c.) call2.
(14)
BB
where B denotes the batch size, and is the ground truth color. Meanwhile, we observed that refracted background may lead to incorrect gradient for the absorption rate due to different darkness. To mitigate the issue, we employ the following regularization to constrain the tone, i.c. the ratio of the three channels in the color space:
(1 |||||c|||
var(c),
(15)
where var(c) denotes the variance of And we adopt AdamUniform (Nicolet et al. 2021 to optimize the mesh to reduce noises during training. Additionally, alleviate the bias caused by incorrect background, we adapt local smoothness for the absorption rate on N randomly sampled
points [v:).Vo- with random perturbation &
Lmat-smooth
Σ Ive(vv)- pe(v: + +.]l
(16)
The total training loss defined as the weighted sum of all individual losses in the form of:
Limal = X1 Ccolor A2 + AnLamat-smooolt smoooth
(17)
where '₁ X2 and X3 are balancing weights for the different terms. In addition, we impose regular- ization terms to mitigate the abrupt variations in the absorption rate, ensuring geometry smoothness and mask consistency. These regularization are applied periodically during training, with further details provided in Section A.2 of the appendix.
EXPERIMENT
We evaluate method on both synthetic and real-world datasets. It is noteworthy that, in trast to previous methods, our DiffTrans can not only reconstruct the geometry of transparent ob- jects but also recover the absorptive material properties. To demonstrate the effectiveness of
7

---

Published as a conference paper at ICLR 2026
approach and the importance of modeling transparency effects, compare the reconstructed ge- ometry quality with other methods, such as NeRO (Liu et al. (023), NU-NeRF (Sun et al., 2024) and NeRRF (Chen et al. 2023). Given that our DiffTrans Is capable of recovering both the geometry and materials of the transparent objects with complex textures, enables scene editing, such as re- lighting. Consequently, also compare the relighting results with those produced by the baseline methods. The implementation details of our approach are included in Section A.3 of the appendix.
4.1 DATASETS
For synthetic data, and cow) from NEMTO (Wang et al.] (003) dataset for reconstruction of objects without
two scenes (bunny
Method
NeRO
NU-NeRF
NeRRF
Ours(S1)
Ours
CDEX10 *)
36,022
7,891
13.341
4.666
3.264
FA010 1y
5.691
8.026
6.916
8.088
8.386
materials, and four scenes (monkey, horse, hand and mouse) from (Lyu et al. 2020; Li et al. 2020) for reconstruction of objects with ma- terials. Due to the absence of released ren- dered images from NEMTO, we follow its con- figuration by utilizing infinite-distance environ- ment maps and absorption-less glass materi- als to render a set of multi-view images. Be- sides, we made series of real-world dataset (flower, etc.) by capturing a video around the object and its environment using iPhone in a continuous one-shot style and extracted the frames using ffmpeg. The camera poses ob- tained via COLMAP (Schönberger et al.] 20166] Schönberger & Frahm (016)) We make utility tool to semi-automatically annotate the masks with SAM (Kirillov et al. [2023), and classify the images into two parts, used to reconstruct the object and the environment respectively, based on the mask proportion in the image. More details and visualization of our dataset are provided in Section A.4 and Section
Average quantitative comparison of geometry reconstruction across all synthetic datasets. Ours(S1) refers to the coarse geometry after the initialization.
Table I:
Dataset
horse
monkey
bunny
Cow
hand
mouse
Ours
1.540
1.577
1.411
1.472
1.484
1.462
1.450
1.600
1,485
1.472
1.485
1.375
Table 2: Comparison of the IoR predicted by DiffTrans with the ground truth across 6 synthetic scenes.
Ah5the the appendix.
RESULTS OF RECONSTRUCTION
NeRRF
NeRO
NU-NeRF
Ours
GT
To validate the quality of the recon- structed geometry produced by our DiffTrans, we conducted experiments in 6 synthetic scenes, with quantita- tive and qualitative comparisons pre- sented in Table D and Figure 4 re- spectively. We adopt the Cham- fer distance (CD) and Fi-score (F1) as the evaluation metrics. To the best of our knowledge, existing meth-
bumny
mouss
monkey
ods struggle to reconstruct transpar- ent objects with intricate textures, sulting in suboptimal geometry qual- ity. For example, the horse recon- structed by NeRRF exhibits notice- able surface roughness, while the void between the hand and head of the monkey is erroneously filled. Furthermore, the bunny and monkey reconstructed by NeRO display structural inaccuracies. Critically, our DiffTrans achieves superior performance, outperforming all baselines in average CD and Fi-score across all the scenes. In the second stage, while optimizing the materials of the transparent objects, Our DiffTrans further refine the geometry obtained from the first stage. As evidenced by the last two rows in Table T the refined geometry demonstrates measurable quality improvements, substantiating the effectiveness of our proposed differentiable recursive mesh ray tracer.
Figure 4: Qualitative comparisons of the reconstructed ge- ometry on bunny, horse an monkey scenes.
In contrast to most approaches that rely on predefined IoR, our DiffTrrans directly optimizes the IoR in the second stage. Table 2 shows the loR predicted by DiffTrans across 6 synthetic scenes,
8

---

Published as a conference paper at ICLR 2026
NeRRF
NeRO
Ours
GT
hoss
monkey
flower
N/A
Figure 5: Qualitative comparison of relighting results on the horse, monkey, and flower scenes.
Damnet
bovse
monkey
flower
hand
mouse
PSNRT
SSIMT
LPPPS]
PSNR1
SSIMI
IPPIS.]
PSNRT
SSIMI
LPIPSS
PSNR
SSIMI
LPIPS]
PSNR
SSIM
LPIPS]
whs Caw-
27.00
0.9708
00514
24.15
0,8942 0.8935
0.0633
24.05
0.7883
0.2731
22.37
0.8719
1481
24.15
0.890 0.8986
0,1225
Full
27.03
0.927%
0.0457
24.62
0.0331
24.14
0.7895
0.2717
23.39
0.8736
0.1420
24.16
0.1219
Table 3: Quantitative results novel view synthesis with/without the tone regularization.
where it can be observed that the discrepancies between the predicted results and the ground truth IoR are minimal. The complete qualitative results of geometry and novel view synthesis provided in Section A.3 of the appendix.
4.3 RESULTS OF RELIGHTING
In addition to the 5 scenes with complex textures (mon- key, horse. flower hand and mouse), we also compare the results of relighting
Method
PSNRT
SSIMI
LPIPS
scenes from NEMTO (Bumny
NeRRF NeRO
19.25
0.8118
0.0812
and cow). To ensure fair comparison, we set a fixed index of refractive (IoR) for the two NEMTO scenes align with the settings used in NeRRF (Chen et al. (2023). For each scene, employ 5 distinct environment maps for illumination and subsequently render images by ran- domly selecting 10 camera viewpoints that collectively cover the upper hemisphere centered on the object, ensur- ing comprehensive validation.
19.64
0.8500
0.0856
Ours
23.17
0.8380
0.0678
Table 4: Quantitative results of relight- ing averaged across all the synthetic scenes.
We compare our DiffTrans with NeRRF (Chen et al. 2023), current state-of-the-art method for transparent ob-
ject reconstruction, and NeRO (Liu et al.] 2023). the re-
cent method for inverse rendering. Since NU-NeRF (Sun)
= 1.4 7= 1.5
= 1.6
et al. 2024) employs the implicit representation, unable to directly obtain its relighting results. Table 4 shows the quantitative results of relighting using the met-
Figure 6: Reconstruction results with
different IoRs on the monkey scene.

---

Published as a conference paper at ICLR 2026
ries including PSNR, SSIM and LPIPS averaged across all scenes. Our DiffTrans outperforms other methods on most metrics, demonstrating the superiority of our approach. We also present the quali- tative comparisons in Figure Due to NeRRF's inability to reconstruct the materials of transparent objects and its relatively low geometry quality, the resulting relighting outcomes are subpar. NeRO focuses solety reflection and fails to simulate the refraction process, leading to unrealistic lighting results. In contrast, our DijfTrans achieves high-quality reconstruction of both geometry and materials, enabling it to achieve plausible relighting.
4.4 ABLATION STUDIES
We conduct ablation experiments to explore the effectiveness of the tone regularization introduced in Equation (15). As shown in Table while the regularization leads to a slight decrease in the SSIM metric for novel view synthesis in the monkey scene, all other metrics have shown improvement. Figure 6 shows that our method is robust under different IoR settings.
Please refer Section the appendix for ablation studies for other components of our method, and the qualitative results the ablation study of the tone regularization.
5 CONCLUSIONS
In this work, we have presented a differentiable rendering framework, DiffTrans, for reconstructing transparent objects with diverse geometry and complex texture. The proposed DiffTrans first constructs the initial geometry and the environment separately. Then it performs joint optimization over the geometry, index of refraction and absorption rate of the transparent objects in unified and end-to-end manner, using the specifically designed recursive differentiable ray tracer. In particular, the ray tracer can be implemented efficiently in CUDA to reduce computational cost. Extensive ex- periments have demonstrate the effectiveness of our method, especially towards transparent objects with intricate topology and texture.
Limitations. We have made three assumptions during the problem formulation, described in Sec- 3.T Despite their correctness in most typical scenes, these assumptions would inevitably intro- duce modeling inaccuracies in some complicated scenes. We will investigate further improvement to relax the assumptions, thereby improving the robustness of our method in future works.
ACKNOWLEDGEMENTS
This work was supported by the National Natural Science Foundation of China (Grant No. 62372133, 62125201 and U24B20174).
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Mojtaba Bemana, Karol Myszkowski, Jeppe Revall Frisvad, Hans-Peter Seidel, and Tobias Ritschel. Eikonal fields for refractive novel-view synthesis. In ACM SIGGRAPH 2022 Conference Proceed- ings, pp. 1-9, 2022.
Alexandre Binninger, Ruben Wiersma, Philipp Herholz, and Olga Sorkine- Tetweave: Isosurface extraction using on-the-fly delaunay tetrahedral grids for gradient-based mesh opti- mization. ACM Transactions on Graphics (TOG), 14(4):1-19, 2025.
Mario Botsch and Leif Kobbelt. A remeshing approach to multiresolution modeling. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, pp. 185-192, 2004.
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European Conference on Computer Vision, pp. 333-350. Springer, 2022.
Xiaoxue Chen, Junchen Liu, Hao Zhao, Guyue Zhou, and Ya-Qin Zhang. Nerrf: 3d reconstruc- tion and view synthesis for transparent and specular objects with neural refractive-reflective fields. ArXiv, awsp/309..1000000 2023. URL
Corpus Se:pusID:26221000
Yuxin Dai, Qi Wang. Jingsen Zhu, Dianbing Xi, Yuchi Huo, Chen Qian, and Ying He. Inverse rendering using multi-bounce path tracing and reservoir sampling. In The Thirteenth International Conference on Learning Representations, 2025.
id=KEXoZxTwbr
Weijian Deng, Dylan Campbell, Chunyi Sun, Shubham Kanitkar, Matthew Shaffer, and Stephen Gould. Ray deformation networks for novel view synthesis of refractive objects. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3118 2024.
Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo the IEEF/CVF
Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings the Conference on Computer Vision and Pattern Recognition, pp. 5501-5510, 2022.
Jian Gao, Chun Gu, Youtian Lin, Zhihao Li, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Re- lightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing. In European Conference on Computer Vision, pp. 73 89. Springer, 2024.
Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. Shape, light, and material decomposi- tion from images using monte carlo rendering and denoising. Advances in Neural Information Processing Systems, 35:22856-22866 2022.
Letian Huang, Dongwei Ye, Jialin Dan, Chengzhi Tao, Huiwen Liu, Kun Zhou, Bo Ren, Yuanqi Li, Yanwen Guo, and Jie Guo. Transparentgs: Fast inverse rendering of transparent objects with gaussians. ACM Transactions on Graphics (TOG), 44(4):1-17, July 2025. URL httpp::// doi meii.org/10.. 1ee.11145333333
Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, and Delio Vieini. Dr.jit: just-in-time compiler for differentiable rendering. ACM Transactions on Graphics (TOG), 41(4):1-19 2022.
Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Songfang Han, Sai Bi, Xiaowei Zhou, Zex- iang Xu, and Hao Su. Tensoir: Tensorial inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165- 2023.
James T Kajiya and Brian P Von Herzen. Ray tracing volume densities. ACM SIGGRAPH computer graphics, 18(3): 165-174, 1984.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. urXiv:2304.02645 2023.
Kiriakos N Kutulakos and Eron Steger. A theory of refractive and specular 3d shape by light-path triangulation. International Journal of Computer Vision, 76: 76:13-29, 2008
11

---

Published as a conference paper at ICLR 2026
Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. Modular primitives for high-perfooe perfoormefrcc differentiable rendering. ACM Transactions on Graphics (ToG), 39(6):1-14, 2020.
Zhengqin Li, Yu- Ying Yeh, and Mammohan Chandraker. Through the looking glass: Neural 3d reconstruction of transparent shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 1262-1271, 2020.
Zongcheng Li, Xiaoxiao Long, Yusen Wang, Tuo Cao, Wenping Wang, Fei Luo, and Chunxia Xiao. Neto: Neural reconstruction of transparent objects with self-occlusion aware refraction- -tratiionntaiigg.... In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18547-18557, 2023.
Yuan Liu, Peng Wang, Cheng Lin, Xiaoxiao Long. Jiepeng Wang. Lingjie Liu, Taku Komura, and Wenping Wang. Nero: Neural geometry and brdf reconstruction of reflective objects from multi- view images. ACM Trunsuctions on Graphics (TOG), 42(4): 2023.
Matthew M Loper and Michael Black. Opendr: An approximate differentiable renderer. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6- 12, 2014, Proceedings, Part VII 13, pp. 154-169. Springer, 2014.
Jiahui Lyu, Bojian Wu, Dani Lischinski, Daniel Cohen-Or, and Hui Huang. Differentiable refraction-tracing for mesh reconstruction transparent objects, ACM Trans. Graph., 39(6), nov 2020. ISSN 0730-0301. doi: 10.1 10.11457414465.3417811 htthttps: //o. org/10.
1145/3414685 3417815
W Matusik, H Pfister, A Ngan, R Ziegler, and L McMillan. Acquisition and rendering of transparent and refractive objects. In Eurographics Workshop on Rendering (EGRW), pp. 267-278, 2002.
Ishit Mehta, Manmohan Chandraker, and Ravi Ramamoorthi. A level set theory for neural im- plicit evolution under explicit flows. In European Conference Computer Vision, pp. 711-729. Springer, 2022.
Ishit Mehta, Manmohan Chandraker, and Ravi Ramamoorthi. A theory of topological derivatives for inverse rendering of geometry. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 419, 2023.
B Mildenhall, PP Srinivasan, M Tancik, JT Barron, R Ramamoorthi, and R Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European conference on computer vision, 2020.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphies prim- itives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1-15, 2022.
Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Müller, and Sanja Fidler. Extracting triangular 3d models, materials, and lighting from images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 8280-8290, 2022.
Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. Large steps inverse rendering of geometry. ACM Transactions on Graphics (TOG), (066))1-13, 2021.
Christian Reiser, Rick Szeliski, Dor Verbin, Pratul Srinivasan, Ben Mildenhall, Andreas Geiger, Jon Barron, and Peter Hedman. Merf: Memory-efficient radiance fields for real-time view synthesis in unbounded scenes. ACM Transactions on Graphics (TOG), 42(4):1-12, 2023.
Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from from-motioon revisited. In Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2016.
Johannes Lutz Schönberger, Enliang Zheng, Mare Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for unstructured multi-vied stereo. In European Conference on Computer Vision (ECCV). 2016.
12

---

Published as a conference paper at ICLR 2026
Tianchang Shen, Jun Gao, Kangxue Yin, Ming Yu Liu, and Sanja Fidler. Deep marching tetrahedra: a hybrid representation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems, 34:6087-6101, 2021.
Tianchang Shen, Jacob Munkberg, Jon Hasselgren, Kangxue Yin, Zian Wang, Wenzheng Chen, Zan Gojcic, Sanja Fidler, Nicholas Sharp, and Jun Gao. Flexible isosurface extraction for gradient- based mesh optimization. ACM Transactions on Graphics (TOG), 42(4):1-16, 2023.
Sanghyun Son, Matheus Gadelha, Yang Zhou, Zexiang Xu, Ming C Lin, and Yi Zhou. Dmesh: A differentiable mesh representation. arXiv preprint 17Xiv:2404.134455 2024.
Oded Stein, Eitan Grinspun, and Keenan Crane. Developability of triangle meshes. ACM Transac- tions on Graphics (TOG), 3714) 2018.
Cheng Sun, Min Sun, and Hwann- Tzong Chen. Direct voxel grid optimization: Super-fast con- vergence for radiance fields reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5459-5469, 2022.
Jia-Mu Sun, Tong Wu, Ling-Qi Yan, and Lin Gao. Nu-nerf: Neural reconstruction of nested trans- parent objects with uncontrolled capture environment. ACM Transactions on Graphics (TOG), 43 (6):1-14, 2024.
Dongqing Wang, Tong Zhang, and Sabine Süsstrunk. Nemto: Neural environment matting for novel view and relighting synthesis of transparent objects. In Proceedings of the TEEF/CVF International Conference Computer Vision (ICCV), pp. 317-327, October 2023.
Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. Advances in Neural Information Processing Systems, 34:27171-27118. 2021.
Bojian Wu, Yang Zhou, Yiming Qian, Minglun Cong, and Hui Huang. Full reconstruction of transparent objects. ACM Transactions Graphics (TOG), 37(4):1-11,200
Tianhao WU, Hanxue Liang, Fangcheng Zhong, Gernot Riegler, Shimon Vainer, Jiankang Deng, and Cengiz Oztireli. asurf: Implicit surface reconstruction for semi-transparent and thin objects with decoupled geometry and opacity. In 2025 International Conference on 3D Vision (3DV), pp. 14-63. IEEE, 2025.
JAMIN XU, ZIHAN ZHU, HUJUN BAO, and WEIWEI XU. A hybrid mesh-neural representation for 3d transparent object reconstruction. ACM Trans. Graph, 1(1), 2022.
Lior Yariv, Yoni Kasten, Dror Moran, Mcirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lip- man. Multiview neural surface reconstruction by disentangling geometry and appearance. Ad- vances Neural Information Processing Systems, 33:2492 2502, 2020.
Xiuming Zhang, Pratul P Srinivasan, Boyang Deng, Paul Debevec. William T Freeman, and Jonathan T Barron. Nerfactor: Neural factorization of shape and reflectance under an unknown illumination. ACM Transactions on Graphics (ToG), 40(6):1-18, 2021.
Douglas E Zongker, Dawn M Werner, Brian Curless, and David H Salesin. Environment matting and compositing. Computer Graphics (Proc. SIGGRAPH99), pp. 205-214, 1999.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 DETAILS ON GEOMETRY INITIALIZATION
Geometry initialization constitutes a critical first step in our proposed method, aiming to derive an initial geometry estimation solely from object masks and camera parameters to enhance training stability. This approach is designed to achieve the following key objectives:
Mask Consistency. The initial geometry must align closely with the input mask, regard- less of its shape. Complex topologies within the mask can sometimes fail to provide ad- equate visibility gradients, particularly for rasterization- MLP to model spatially-varying, data can enhance smoothness and improve visibility gradi- ents (Mehta al., 2022). However, this approach significantly reduces training efficiency and limits the model's 's generalizability for intricate structures. Fully implicit methods that rely on MLP optimization may encounter discretization errors and fractured topologies, while explicit methods often struggle to handle topology changes during training without imposing additional constraints (Mehta et al.] 2022).
methods. Incorporating an
Interpolation Capability. The initial geometry must be sufficiently smooth, particularly in regions within the image mask that lack detailed information. For occluded areas, the image mask does not provide adequate information, necessitating reasonable interpolation to fill these gaps. Methods such as space carving (Mehta al. 2023) can effectively generate meshes that align with the mask, but they are unable to perform interpolation in occluded regions.
Speed. Due the limited information provided by masks, prolonged initialization does not necessarily yield better results. Implicit methods which use MLP for prediction typi- cally exhibit slower training speed and often lead overly smooth surfaces, while explicit methods such as space carving tend to have poorer generalization capabilities. Therefore, it crucial to achieve balance between implicit field optimization and explicit algorithms to ensure both speed and adaptability.
Based on the considerations outlined above, we adopt a simple yet efficient approach for initializa- tion. First, we utilize FlexiCubes as the iso-surface representation, which is optimized by rasteriz- ing the extracted differentiable mesh and minimizing the loss between the rendered results and the ground truth mask. To mitigate potential training instabilities and noise arising from the ill-posed nature of the mask, we impose constraints on smoothness, mesh quality, and SDF during training. This process is both fast and efficient, enabling the geometry to adapt to various topologies while maintaining sufficient smoothness to provide a robust starting point for optimization. However, this dense representation lacks precision and contains significant redundant information. To address these limitations, we extract the mesh from the trained FlexiCubes and remesh it to achieve more detailed and uniform structure. The refined mesh is further optimized using the mask to improve mask consistency. This enhanced mesh serves as the foundation for subsequent optimization steps.
A.1.1 INITIALIZATION- PHASE MESH REPRESENTATION
At the initialization stage, our mesh is defined within Signed Distance Field (SDF) f(x) Specif- ically, the mesh is obtained by extracting an iso-surface S = (xlf(x) = 0} within it. We adopt FlexiCubes (Shen et al.) (023) for geometry initialization at the starting stage, where the SDF values are stored in the uniform cubic grid vertices inside. A mesh can be extracted by Dual Marching Cubes (Shen al. 2023), where primal mesh is firstly extracted by Marching Cubes, and the final mesh is generated such that all its vertices lies on unique faces of the primal mesh. We denote the final mesh as:
M - (V,
(18)
where V (ulv E R2) represents all vertices in the mesh E (fv,, By HyHlu, by E V is the unique edge set, and F = ffu, Uy, 2 Hu,, Vy, 4A E V is the set of unique faces. As vanilla FlexiCubes are limited in bounded region, we provide AABB bounding box (Xmin Xmax) of the estimated object area as the input of the initialization, which is easily obtainable by the MVS progress of creating the dataset AABB boxes several images manually annotated by users. Vertices (v:) of the mesh
14

---

Published as a conference paper at ICLR 2026
transformed as:
o/ U4 (Xmax Xmin) + Xmin
(19)
from the original [-0.5, 0.5] region.
A.1.2 OPTIMIZATION
At each training iteration, we obtain the object's mesh from its grid representation, then we apply nvdiffrast (Laine et al.) 2020) for differentiable rasterization at training views. Concretely, the mask is normally rasterized, and is antialiased with the triangle edges to provide visibility gradient w.r.t the vertex positions, After obtaining the antialiased mask M, we supervise it with the ground truth mask M:
Lgeas init
1 N
Σ IM
(20)
EEB
where B denotes the batch size and N represents the number of pixels.
SDF Dilation. Our geometry is determined by the visibility gradient, which is derived from the differences in masks at the edges of the rendered images. The anti-aliased mask values of the edge pixels are calculated based on the intersection points between the projected triangle edges and the lines connecting the centers of the pixels. These differences in mask values directly contribute to the gradients at the edge positions, thereby influencing the locations of the vertices. Since the gradient is only applied to the edges, achieving proper topology fitting relies on dispersing the edges across a wide area. To achieve this, we adopt the approach proposed in (Munkberg et al. 2022) and initialize the vertex SDF with uniform distribution U|- 0.9), ensuring that our initial geometry is evenly scattered within our area of interest. However, relying solely on the mask loss for geometry recovery leads to poor internal structure. The resulting mesh may have the correct mask but is distorted and noisy internally. In order to address this issue, an alternative approach involves starting with a cleaner initial mesh, such this approach fixes the broken topology within the mask, it also imposes limitations on the range of topologies that can be accommodated.
sphere (Chen al. (023). , instead of random initialization. While
To fill the cracks of the mesh, we dilate its surface with the following loss:
Ldilation
Σ
f(v).
(21)
M
PEV
The regularization tends to decrease the SDF values. expanding the iso-surface and filling the gaps. The utilization of this operation is based on two key observations. Firstly, the areas within the mask do not receive any gradient from the mask loss Lego-init and therefore remain stable after being filled. Secondly, the cracks within the mask are smaller in scale compared to the geometry of the ground truth shape. This implies that the dilation operation will not obscure genuine geometry details. It is important to note that the scalar values stored in the FlexiCubes grid do not represent true SDF, which exhibit a consistent absolute value of spatial gradient. Instead, they serve as indicator functions that are flat in empty and full spaces, respectively. Consequently, Equation (2 not only dilates the mesh within the mask, but also generates new geometry throughout the entire space. We consider this to be beneficial feature that aids in capturing missed geometry.
Mesh Regularization. While the SDF dilation technique successfully recovers the correct object topology, the resulting surface remains rough and exhibits high- freeuerry noise. In order to mitigate these issues, we employ a rendering approach to generate the mesh's normal map n and depth map d. We then introduce smoothness regularization term that operates on the derivatives in screen space:
1 Σ (IIvd:) + [/nll + ||ven:|]))
Lymooth
(22)
N
where V denotes screen space gradient along the direction, and N represents the pixel count over the whole batch. The derivatives in Equation (22) are computed using screen space finite differ- ences. This screen space regularization can be applied to random views. We have noticed that for objects with intricate details, such as holes, that appear small within the view, the mask loss alone may not entirely eliminate all the floating artifacts. To address this issue, we adopt the approach pro- posed in (Munkberg et al. 2022) and employ binary cross entropy function with logistic activation
15

---

Published as a conference paper at ICLR 2026
(C y) log(o(())) (1 y) log(1 o to eliminate sparse floating artifacts among neighboring vertices with different SDF signs:
LBCE
Σ (HE,D))EB 01
BCELogit((fa)))/(t))
(23)
|Eyl
in which
F = ((;;, uy) 1y E A sign(f(r:) + sign(f(v;)) the set of unique and directional edges crossing the extracted mesh face. Due to the dilation process, our recovered mesh may be swelling at unseen faces, so restrict the total surface area by:
(24)
Larea
Σ (f)
(25)
F
SEF'
Our mask loss penalizes out-of-mask triangles, which may result in excessive gradients grazing angles and causes pit the surface. To ensure more consistent mesh structure and avoid the occurrence of pits on the surface, we introduce a series of regularization techniques to mitigate excessive gradients that may arise at grazing angles in our mask loss. The first regularization step involves penalizing the average distance between the mesh vertices and their respective primal faces:
Lavgdis
Σ VEV'
MAD (1llv - ue 12 NN.)),
(26)
VI
where N₂ is the primal face of mesh vertex and MAD means mean absolute deviation of a set of values to their mean. This will add to its ficxibility by leaving places tions. Besides that, we also employ mesh developability loss Laev introduced in (Stein et al. 2018) to encourage flatness and keep hinges and LI loss on grid weights LLI to avoid local minimum. Performing dilation and smoothing too early may cause the initial mesh wrongly filled because dundant floaters are not cleared by the mask, so we mainly use the mask loss at the early stage. We also adapt switching scheme between the dilation and smoothing for training efficiency. The overall loss for the geometry initialization can be written as:
deforming direc-
Lgou
A1 Lyoo--iii + AgLailation + >s + LBCE >5g Ag + >7 + >gL111-
(27)
The regularization requires only little hyper-parameter tuning across datasets and is efficient to train. The scheme details shown in Section A.3.2]
Post-processing. After receiving the mesh from FlexiCubes, to compensate for the over-smooth effect caused by the screen space smoothness regularization, additionally use Botsch-Kobbelt method (Botsch & Kobbelt (2044) to remesh and optimize it using AdamUniform optimizer (Nicolet
al. 2021) with edge normal regularization:
Ledge
Σ
(1- nec nny))²
(28)
E'
A.2 ADDITIONAL DETAILS ON THE RAY TRACER
The recursive ray tracer is leveraged to optimize the geometry, index refraction and absorptive rate of the objects. In addition to the methods described in the main paper, incorporated several regularization strategies during training to further enhance the model's 's generalization ability and robustness.
A.2.1 OPTIMIZATION
Volume Regularization. As discussed in the main paper, our method models the material of trans- parent objects explicitly by directly render under the Lambert-Beer's law, which is highly ill-posed process. To ensure the model's 's robustness and avoid being trapped in local minima, we introduce La penalization term on the volume density br of the object. The regularization term defined as:
Lvol Σ
(x:))1
(29)
where x; is the i-th point sampled randomly from the volumetric material field.
16

---

Published as a conference paper at ICLR 2026
Mesh Regularization. Due the unstable nature of the rendering of transparent objects, apply mask regularization to the mesh to preserve mesh consistency. The regularization term is defined as:
M MMM-M.1, M.ll.
maskk
(30)
where M, is the i-th rendered mask in N-sized batch, and M, is the corresponding ground truth mask.
Smoothness Regularization. We observed that the mesh exhibits noises during the training pro- cess, which may lead to artifacts in the rendered images. To alleviate this issue, we introduce smoothness regularization term on the mesh vertices by smoothing the vertex normals. The regular- ization term is defined as:
eddge
Σ
(1 (1-n,, n., n.;))
(31)
[Ni,D)]EE'
where E' is the set of edges in the mesh, and ny, the normal of vertex U..
Periodical Regularization. To ensure the model's robustness and avoid overfitting, introduce periodical regularization term on the geometry and material parameters. We regularize the mesh with Lmusks Ledge and Laplacian regularization Lupp to improve he quality of the mesh every iterations. The Laplacian regularization detailed in (Nicolet et al., 202T) to control the uniformity of the mesh.
Overall Loss. The overall loss function for the ray tracer is defined in the form of summation of the loss Liout introduced in the main paper and the regularizations:
Lu
Lioaal + hypl + Caak de Ledge-
(32)
A.3 IMPLEMENTATION DETAILS
A.3.1 ENVIRONMENT DETAILS
We conducted our experiments on a machine with an Intel Core 17-10700K CPU and 64GB of RAM, with single RTX 3090 GPU. The software environment includes Python 3.11, Py PyTorch 2.4, CUDA 12.6 and Ubuntu 24.04. We compiled our ray tracer with g++ 13 and nvec 12.6.
A.3.2 TRAINING SCHEME FOR GEOMETRY INITIALIZATION
We employ the Adam optimizer with parameters 31 = 0.9, 32 = 0.999, and a learning rate defined as 0.01 10.0.000 where represents the iteration count. Given the complexity of di- rectly optimizing the explicit geometry, we propose a carefully structured training scheme during the initialization phase, spanning 1000 iterations. This process broadly divided into three distinct stages:
Mask Regression. At the outset, the FlexiCubes representation begins in a random state. To ensure sufficient visibility gradients from the random edges while minimizing redundant geometry, we apply minimal regularization during this stage. By the end of this phase, the geometry concentrates within the mask-indicated region, albeit with significant noise.
Surface Reconstruction. This stage focuses on refining the geometry into complete surface. achieve this by primarily employing dilation loss and smoothness loss, which help form continuous mesh while maintaining geometric smoothness.
Quality Improvement. Finally, the quality of the geometry is enhanced further by incor- porating additional losses, such as developability loss, to refine and improve the surface structure.
Specifically, let denote the current iteration. Throughout all stages, we set As 2, A = 0.5, As 1, and As 0.2 (1 0.00076;). During the initial 100 iterations, define À1 C1
17

---

Published as a conference paper at ICLR 2026
Dataset
horse
monkey
flowers
bunny
cow
C1
7
3
5
C2
0.5
0.1
0.3
0.5
0.1
Table 5: Hyperparameters for geometry initialization across datasets.
Dataset
horse
monkey
flowers
73
0.0001
0.01
0.01
k
300
500
1000
Table Hyperparameters for freze-geeoeete stage.
X2 = 0.2cg, and A = A = 0, where and C2 are hyperparameters. After the first 100 iterations, we employ an alternating strategy by adjusting the settings as follows:
max
0.7-0/1000
-0.05
if mod 50 is even, otherwise,
A
0.6
(33)
0,
0.5,
if mod 50 is odd,
A
(34)
00
otherwise,
27 max {i/100 7,03. if mod 50 is odd, otherwise.
(35)
0,
We adjust ei and 22 across datasets align with the scale of the geometry, as detailed in Table 5 After completing 1000 iterations, we extract the mesh and refine it using the Botsch-Kobbed algo- rithm. Subsequently, we perform an additional 1000 iterations to optimize the mesh. During this phase, we leverage object masks by minimizing the In loss between the rendered mask and the ground truth mask, in conjunction with the smoothness loss Ledge on vertex normals.
A.3.3 TRAINING SCIEMME FOR THE RAY TRACER
To address the potential bias introduced by initializing the material field as fully transparent, given its deviation from the ground truth color of the object, we freeze the object's geometry for the first k iterations, where k is user-defined parameter. During this stage, employ the Adam optimizer with 31 = 0.9, 32 0.999, a weight decay penalty of X = 10 and learning rates of y 3 10-3 for the material network and y 10 for the IOR. The loss weights are set as \ 1, A2 0.001, and \ = 0.0005. For the NEMTO datasets, this initialization stage is omitted, as these datasets lack absorption properties. Additional hyperparameter details are provided Table
Following the freeze-geomety stage, we optimize the geometry and material fields jointly. The same Adam optimizer configuration is used, with the exception that the learning rate for the IoR is updated to 7 10-3, For mesh optimization, jet al., (02T) with a learning rate of 7 = 0.001. To ensure the mesh adheres to the object mask and maintains high geometry quality, we perform 200 iterations of optimization, applying mask constraints and geometry quality regularization as described Section A.3.2 with these constraints enforced every 100 iterations.
employ the AdamUniform optimizer (Nicolet
A.3.4 COMPUTATIONAL AND TRAINING EFFICIENCY
All experiments were conducted on NVIDIA RTX 3090 GPUs with 24 GB of VRAM. The training time for our scenes fell within 1-2 hours, depending on the geometric complexity of the object. Higher geometric complexity requires more ray bounces for fixed batch size, which in turn in- creases the per-iteration training time. In our implementation, rays with more than four bounces discarded during training. For inference, this threshold is increased to eight to achieve better visual quality. The memory consumption is approximately 20 GB during training with a batch size of 5, 5,000 rays, and it never exceeded the available VRAM even in most complex scene.
18

---

Published as a conference paper at ICLR 2026
A.3.5 DYNAMIC RANGE OF INPUT IMAGES
use HDR environments for synthetic scenes. For real-world captures with LDR inputs, we apply gamma correction with 2.2 to mitigate the domain gap between LDR input images and HDR environment maps.
A.4 DETAILS ON DATASETS
As detailed in the main paper, dataset comprises five scenes: monkey, horse, flower, bunny, and cow. The corresponding number of images, material types, and whether the dataset synthetic summarized in Table 7 Objects labeled "N/A" under material do not have specific material
attributes assigned. We provide visualizations of the dataset images in Figure All synthetic
Images
Dataset
Material
Object
Env.
horse
100
100
absorption
N
monkey
100
50
absorption
N
flower
104
56
absorption
Y
flower(2)
absorption
Y
flower(3)
154
69
absorption
Y
flower(4)
119
27
absorption
Y
bunny
100
N/A
N
cow
100
N/A
N
Table 7: Overview of the datasets used in experiments.
datasets are rendered using camera views sampled from a uniform distribution, either on the upper hemisphere for custom datasets or the full sphere for NEMTO datasets, with the object the center. For the real-world dataset (flower), images were captured using an iPhone 11 by recording a video around the object and its environment. Individual frames extracted from the video, and camera poses were estimated using COLMAP (Schönberger & Frahm, 2016). The point cloud generated during the pose estimation process was used to determine the axis-aligned bounding box (AABB) of the transparent object. Masks for the real-world dataset were annotated using a combination of SAM and manual refinements, requiring approximately 30 minutes for the flower dataset.
A.5 MORE VISUALIZATION RESULTS
We present the qualitative comparison of the novel view synthesis results of our method with the ground truth in Figure 9 results available in the nvs directory. As shown in Figure in the horse scene, method success- fully reconstructed the green absorption material and the mesh of the transparent object, demonstrat- ing high fidelity in object recovery. In the monkey scene, method accurately captured the color transition between the upper body and lower part. Additionally, our method achieves high-fidelity reconstruction in the real-world captured flower scene.
All images were rendered using our ray tracing renderer, with additional
Besides the NVS results, we also provide visualization for reconstructed geometry in the geo.vis directory. Especially, results horse, monkey and bunny scenes are shown in Figure T0
We show in Figure g an extra real-world test scene including the input images, recovered normal maps and relighting results.
The rendering result between the meshes from the stage I and the stage are compared in Figure 16 which demonstrates the effectiveness of our stage-2 refinement.
To show the effectiveness of our stage-2 2 reconstruction, we provide a rendered heatmap showing the difference in Hausdorff Distance between the ground truth before and after optimization in Figure and the difference between stage-1 and stage-2 meshes in Figure 22
The figure of recovered scenes shown in Figure 13
19

---

Published as a conference paper at ICLR 2026
howe
CON
bunny
nlwer
Figure 7: Parts of the images from the dataset used in our experiments.
ablational novew synthesis result of different IORs on the monkey and horse scenes are shown in Figure 14
To better visualize the reconstruction result of the geometry, we provide the normal maps of the reconstructed meshes in Figure
A.6 MORE ABLATION STUDIES
A.6.1 ABLATION ON GEOMETRY INITIALIZATION
To evaluate the reliability and effectiveness the proposed method, we conducted ablation studies on the SDF dilation loss and the smoothness loss introduced in Section four meshes: block, frame, horse and monkey. All those meshes are rendered with 100 camera views drawn from the upper semi sphere centered at the object. As shown in Table I0 the
uniform distribution joint usage of our dilation, smoothness and other operations including the original relularizations used in Flexicube has facilitated the reconstruction of high-quality meshes solely from mask super- vision. The absence of floater-removigg regularization leads to unsuccessful recovery on geometry structures containing voids, i.e., frames and blocks. Qualitative results shown in Figure 17
20

---

Published as a conference paper at ICLR 2026
Data
Relighting
Normal
Figure 8: Visualization of an extra real world test scene including the input images, recovered normal maps and relighting results.
A.6.2 ABLATION ON INPUT MASK NOISES
To investigate the impact of mask eπrоr, ablations by adding different levels of noise to the object masks. Specifically, we randomly dilate of erode the masks on their edges by 3 pixels or crop off the masks by 1-3 patches with the width of 10 20 pixels. The quantitative results T The results in Thow that the performance largely unaffected, with metrics varying only within small range. This demonstrates the robustness of our method with respect to mask accuracy.
shown in
A.6.3 ABLATION ON TONE REGULARIZATION
Besides, we conduct ablations to evaluate the effectiveness of tone regularization Equation and the qualitative results are shown in Figure 18
15
21

---

Published as a conference paper at ICLR 2026
GT
horse
rendered
GT
monkey
rendered
GT
flwwe
rendered
Figure 9: Novel view synthesis results on the horse, monkey and flower scenes.
22

---

Published as a conference paper at ICLR 2026
NotOE
Nett
munkey
Dues
NeRRE
NdBO
Ous
Not0u
NeRO
hann
Ous
Figure 10: Reconstructed geometries on the horse, monkey and bunny scenes.
23

---

Published as a conference paper at ICLR 2026
before
after
before
afler
-0.05
before
-0.04
-003
-0002
after
0.01
0.00
before
-0.04
-0.03
-0.02
after
0.00
Figure 11: Per-vertex heatmap illustrating the ground truth difference in Hausdorff Distance before and after optimization.
090012
0.0006
00005
0.0000
Figure 12: Per- Vertex Chamfer Distances betweeen the meshes before and after optimization.
A.6.4 ABLATION ON DIFFERENT IORS
We conducted experiments by using different index refraction (IoR) settings for the same object and performing reconstruction. Table shows the recovered IoR values, which close to the ground truth, demonstrating the stability of our method across different IoR settings. The results of novel view synthesis presented in Figure
24

---

Published as a conference paper at ICLR 2026
Environment Maps
Scene Captures
Figure 13: Visualization of the decomposed scene representations. Top: Recovered 2D environment maps (equirectangular projections) representing the far-field background. Bottom: 3D reconstruc- tions of the scenes. The grey regions visible in the top-right environment map correspond areas (zenith and nadir) that were not observed in the input footage due to the limited vertical field-of-view of the camera trajectory.
1.3
= 1.4
1 1.5
monkey
horse
Figure 14: Rendering of the same object under different refractive indices (IOR).
A.7 THE USE OF LARGE LANGUAGE MODELS
In the process of writing this paper, Large Language Models (LLMs) were not used. Both the research ideation and writing were fully carried out by the authors.
25

---

Published as a conference paper at ICLR 2026
NU-NeRF
Ours(SI)
muskey
Figure 15: Normal maps comparison. The Ours(SI) column shows the normal maps rendered from the first stage of our method.
Method
NeRO
NU-NeRF
NeRRF
Ours(S1)
Ours(S1)*
Ours(S1)**
Ours
Ours*
Ours**
CD(x10-1)L
36.022
7.891
13.341
4.666
4.329
4.468
3.264
3.203
3.341
Fi(x10-1)t
5.691
8.026
6.916
8.088
7.832
7.646
8.386
8.623
8.401
Table 8: The extended result of Table 1 in the paper. * refers to the result obtained using noised masks and refers to the result obtained by cropping patches off the mask.
Object = 1.3
= 1.4
= 1.5
monkey
1.288
1.381
1,473
horse
1.305
1.421
1.512
Table 9: Ablation study of IOR prediction under different ground-truths.
Dataset
CD(x10-31
NC(x10-1)
Method
horse
monkey
frame
block
horse
monkey
frame
block
only Egco-init
3.233
4.915
1.652
6.969
2.403
2,498
2.431
2.794
+ Lailation
0.481
1.039
14.683
31.602
1.298
1.713
0.747
0.933
+ Lymooth
0.594
1.139
14.847
21,456
0.412
0.776
0.747
0.210
full
0.484
0.459
1.072
4.457
0.245
0.166
0.381
0.101
after processing 0.456
0.458
1.084
1.853
0.052
0.053
0.110
0.028
Table Ablation study of initial geometry reconstruction. The second best results are underlined.
26

---

Published as a conference paper at ICLR 2026
mouse
bunny
monkey
Stago
Stage 2
GT
Figure 16: A comparison of relighting results using the coarse mesh from stage I and the refined mesh from stage 2.
27

---

Published as a conference paper at ICLR 2026
only mask
+ dilate
+ smooth
+ other reg.
after proc.
block
frame
horse
monkey
Figure 17: Ablation studies for different regularization.
w/ Prone
w/o Prnne
GT
w/ Prnn
w Lrone
GT
monkey
horse
Figure 18: Reconstruction with/without the tone regularization.
28


Reviews:
[{"review_id": "15d5skYFpm", "reviewer": "Reviewer_jaxD", "summary": "This paper presents DiffTrans, a differentiable rendering framework for reconstructing transparent objects with complex geometry and internal texture from multi-view images. It jointly optimizes geometry and material properties—specifically the index of refraction (IoR) and absorption rate—in an end-to-end manner. The method initializes geometry using FlexiCubes and estimates illumination via an environment radiance field, then refines both using a recursive differentiable ray tracer implemented in CUDA/OptiX. Experiments on synthetic and real data demonstrate strong reconstruction and relighting performance.", "questions": "Please refer to the above section for questions", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The work tackles a highly challenging problem in differentiable rendering, enabling reconstruction and relighting of transparent objects with complex internal geometry and refractions. The decoupling of geometry and materials of transparent objects also presents a reasonable path for modeling light refraction and absorption within an end-to-end differentiable pipeline. The method is efficiently implemented in CUDA/OptiX with strong empirical performance on both synthetic and real-world data.", "weaknesses": "The framework relies on several simplifying assumptions, uniform refractive index, purely specular surfaces, and absorption-only materials, which could limit its applicability to more realistic transparent objects such as frosted, layered, or scattering materials.\n\n\nThe evaluation on real-world data is limited to a single “glass flower” scene, which does not sufficiently demonstrate robustness to real capture conditions such as noise, imperfect masks, or lighting variations. More diverse real-world examples and quantitative comparisons would strengthen the experimental section.\n\nAlthough the recursive differentiable ray tracer is central to the method, its computational and memory efficiency are not analyzed in detail. Providing runtime comparisons or scalability analysis would help clarify its practical feasibility.\n\nThe method’s sensitivity to initialization (e.g., mask accuracy, environment estimation quality) is not thoroughly discussed. An ablation on initialization errors or noisy inputs would clarify robustness in real applications.\n\nThe authors could better articulate their conceptual distinction from prior works such as NeRRF, Nu-NeRF, and TransparentGS, to highlight their unique contributions more clearly. The authors should also discuss the conceptual differences from [1], which also highlights decoupling geometry and radiance for reconstructing semi-transparent surfaces, and may provide quantitative comparisons.\n\n[1] Wu, Tianhao, et al. \"αsurf: Implicit surface reconstruction for semi-transparent and thin objects with decoupled geometry and opacity.\" 2025 International Conference on 3D Vision (3DV). IEEE, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cXkcvLSg7E", "reviewer": "Reviewer_wurg", "summary": "This paper presents a novel multi-view image-based 3D reconstruction framework tailored for transparent objects. By adopting a two-stage design, the method achieves a more accurate geometric representation.\n\nIn Stage 1, the authors model an initial outer surface of the transparent object using object masks. To ensure stable convergence, several operations—including dilation—and regularization terms are introduced to constrain the mesh shape. Additionally, non-mask regions are leveraged to learn an environment field that captures scene appearance and lighting.\n\nIn Stage 2, a custom optical model is employed for joint optimization, enabling simultaneous recovery of the object’s material properties and refinement of the geometry and appearance from Stage 1.\n\nExperimental results demonstrate that the proposed method achieves state-of-the-art (SOTA) performance.\n\nThe main contributions of this work are:\n\n1) A two-stage reconstruction pipeline that produces a high-quality initialization robust to the challenges of transparency.\n\n2) A novel rendering pipeline capable of simulating light transport through transparent materials.\n\n3) A new real-world dataset specifically captured for transparent object reconstruction.", "questions": "1) In the ablation study in Figure 10, does “other reg” refer to the original regularization losses used in FlexibleCube?\n\n2) How does the proposed method perform on the less obvious transparent objects in the NU-NeRF dataset? Would the presence of opaque objects inside transparent ones in the NU-NeRF dataset negatively impact—or even cause failure in—the reconstruction using the proposed method? The approach appears to be designed primarily for solid transparent objects, and may not handle internal occluders or complex layered transparency effectively.\n\n3) Many structural details of transparent objects cannot be captured by silhouette masks alone. Although the paper suggests that the two-stage optimization can recover such missing structures, could the authors provide a direct geometric comparison between the results of Stage 1 and Stage 2 to demonstrate this improvement?\n\n4) Could the authors provide a full 360-degree video rendering of the results?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1) This paper implements a ray tracer for transparent materials based on OptiX and CUDA, which serves as a valuable contribution to the community.\n\n2) To obtain a high-quality initial mesh suitable for ray tracing, the paper proposes a flexible cube-based modeling approach.", "weaknesses": "1) The amount of real-world test data is insufficient—only a single real captured object is used for evaluation. Moreover, the rendered results exhibit inaccurate material appearance, and the paper does not provide any geometric visualization (e.g., mesh or point cloud) of this real object. This raises concerns about the method’s practicality. In contrast, methods like NU-NeRF validate their approach on multiple real transparent objects, offering stronger empirical support.\n\n2) The proposed method relies heavily on an accurate foreground mask for Stage 1 training. In the paper, masks for real data are obtained by combining SAM with manual refinement, which undermines the method’s applicability in fully automatic settings. Notably, prior work such as [1] also leverages vision models for transparent object reconstruction but avoids human intervention entirely, making it more scalable and practical. Also, you may cite this paper.\n\n\n[1] TSGS: Improving Gaussian Splatting for Transparent Surface Reconstruction via Normal and De-lighting Priors", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0uiHWxldhj", "reviewer": "Reviewer_RGa4", "summary": "This paper proposes DiffTrans, a framework for jointly reconstructing geometry and performing inverse rendering from multi-view images of transparent objects. It first initializes the geometry using the silhouette from the object mask with several regularization losses. Then, it refines the geometry and optimizes the refraction parameters using differentiable ray tracing. With the inverse rendering results, it supports applications such as relighting and demonstrates superior quality to baseline SOTA methods.", "questions": "1. In Eq. 1, what are the points $x_i$? Are they mesh vertices, points sampled from the surface of the mesh, or points sampled randomly in the space? Besides, `\\mathcal` is missing for the $\\mathcal L$ symbol.\n2. What are the optimizable refraction material parameters in the last stage (Refine phase)? Are they the IoR $\\eta$ and the absorption rate $\\mu_t(\\mathbf{x})$? Is $\\eta$ a single scalar while $\\mu_t(\\mathbf{x})$ a spatially-varying field?\n3. In Eqs. 5 and 6, what are $\\omega^\\parallel_t$ and $\\omega^\\perp_t$? I guess they refer to the \"parallel and perpendicular part\" of the direction, but I don't know their specific physical meanings, and I think this should be explained.\n4. For the environment initialization, did you handle the LDR-HDR issue? Since the environment NeRF is learned from LDR images, the lighting queried in the ray tracing stage will remain LDR. Using LDR values to compute the rendering equation may lead to overdark estimation, and also be vulnerable to imaging artifacts such as over-/under-exposure.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The proposed framework successfully solved the highly challenging transparent object reconstruction and inverse rendering task, while previous works struggled to handle them simultaneously.\n2. The proposed differentiable rendering algorithm in Sec 3.3 is physically based and technically sound, following the physical laws of light transport.\n3. The experimental results are significantly superior to baseline methods.", "weaknesses": "1. Related work and citations. This paper is an inverse and differentiable rendering paper, but Section 2 does not survey related work in these research directions. I suggest adding a paragraph for a comprehensive review of existing inverse/differentiable rendering and relighting methods.\n2. Paper writing. A lot of necessary details and explanations are missing in the paper, making it hard to understand for people without a background in the related physics laws. Please refer to the \"Questions\" part for some specific confusions when I read the paper, and the authors should include these details/explanations in the revised paper.\n3. Baseline selection. The paper selects NeRO as one of its baselines, but NeRO does not support refraction objects at all, leading to an unfair comparison. It would be better to choose baselines supporting transparent objects.\n4. Geometry initialization. A comparison between the initialized geometry from the first stage and the refined geometry from the last stage should be included, so that we can know how differentiable rendering helps the geometry refinement.\n\nSeveral example citations in differentiable and inverse rendering (both mesh-based and NeRF-based, including but not limited to):\n```\n@article{zhang2021nerfactor,\n  title={Nerfactor: Neural factorization of shape and reflectance under an unknown illumination},\n  author={Zhang, Xiuming and Srinivasan, Pratul P and Deng, Boyang and Debevec, Paul and Freeman, William T and Barron, Jonathan T},\n  journal={ACM Transactions on Graphics (ToG)},\n  volume={40},\n  number={6},\n  pages={1--18},\n  year={2021},\n  publisher={ACM New York, NY, USA}\n}\n@inproceedings{jin2023tensoir,\n  title={Tensoir: Tensorial inverse rendering},\n  author={Jin, Haian and Liu, Isabella and Xu, Peijia and Zhang, Xiaoshuai and Han, Songfang and Bi, Sai and Zhou, Xiaowei and Xu, Zexiang and Su, Hao},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={165--174},\n  year={2023}\n}\n@inproceedings{gao2024relightable,\n  title={Relightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing},\n  author={Gao, Jian and Gu, Chun and Lin, Youtian and Li, Zhihao and Zhu, Hao and Cao, Xun and Zhang, Li and Yao, Yao},\n  booktitle={European Conference on Computer Vision},\n  pages={73--89},\n  year={2024},\n  organization={Springer}\n}\n@inproceedings{dai2025inverse,\n  title={Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling},\n  author={Dai, Yuxin and Wang, Qi and Zhu, Jingsen and Xi, Dianbing and Huo, Yuchi and Qian, Chen and He, Ying},\n  booktitle={The Thirteenth International Conference on Learning Representations},\n  year = {2025},\n  url = {https://openreview.net/forum?id=KEXoZxTwbr}\n}\n@article{son2024dmesh,\n  title={DMesh: A Differentiable Mesh Representation},\n  author={Son, Sanghyun and Gadelha, Matheus and Zhou, Yang and Xu, Zexiang and Lin, Ming C and Zhou, Yi},\n  journal={arXiv preprint arXiv:2404.13445},\n  year={2024}\n}\n@article{binninger2025tetweave,\n  title={TetWeave: Isosurface Extraction using On-The-Fly Delaunay Tetrahedral Grids for Gradient-Based Mesh Optimization},\n  author={Binninger, Alexandre and Wiersma, Ruben and Herholz, Philipp and Sorkine-Hornung, Olga},\n  journal={ACM Transactions on Graphics (TOG)},\n  volume={44},\n  number={4},\n  pages={1--19},\n  year={2025},\n  publisher={ACM New York, NY, USA}\n}\n```\nI generally like the paper's proposed method, but the current writing quality needs significant improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CJ6g2TXa2W", "reviewer": "Reviewer_6MNx", "summary": "This paper proposes DiffTrans, a differentiable rendering framework for reconstructing the 3D geometry and material properties (index of refraction and absorption rate) of transparent objects from multi-view images. The method operates in three stages: 1) Geometry Initialization: Using FlexiCubes and multi-view object masks to reconstruct a coarse mesh, regularized for smoothness and completeness. 2) Environment Initialization: Recovering the surrounding scene lighting using a radiance field from the out-of-mask image regions. 3) Joint Optimization: A novel, efficient, recursive differentiable ray tracer implemented in OptiX/CUDA that jointly refines the geometry, IoR, and absorption rate in an end-to-end manner. The key advantage is the ability to handle transparent objects with complex geometry, enabling high-quality reconstruction and relighting. Experimental results and ablation studies demonstrate nice results compared with the state-of-the-art methods.", "questions": "As discussed above.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Basically, the explicit and joint optimization of absorption rate alongside geometry and IoR is a significant step beyond prior work, which often focused only on surface properties or ideal transparency. This directly addresses a major limitation in reconstructing real-world objects.\n\nOverall, the work combines several advanced techniques (FlexiCubes for mesh initialization, a radiance field for environment, and a custom differentiable ray tracer) into a cohesive, progressive pipeline. The integration of a mesh-based differentiable ray tracer, as opposed to a volume-based or implicit one, is a distinct choice for this problem.\n\nThe design of the recursive ray tracer and the specific regularization terms (like the tone loss L_tone) are novel contributions tailored to the ill-posed nature of reconstructing absorptive transparent objects.", "weaknesses": "First of all, reconstructing transparent objects has always been a very challenging problem in both the vision and graphics communities. The method proposed in this paper takes another step forward in this difficult area. Although the results are quite good, I still have several questions. I am open to hearing the authors’ feedback.\n\n(1) In fact, this method can be categorized as a mesh optimization approach rather than a neural radiance one. Therefore, I am curious why it is not compared with [Lyu et al. 2020]. In other words, the comparison in Table 1 seems quite unfair, as all other methods are NeRF-based, which are inherently inferior in reconstruction quality compared to mesh optimization approaches.\n\n(2) The initialization seems to have a significant impact. As shown in Table 1, the optimization process itself does not lead to a very noticeable improvement. I would like to see a comparative visualization, such as a per-vertex heatmap showing the difference in Chamfer Distance / Hausdorff Distance before and after optimization, which would be more intuitive to check the effectiveness of optimization.\n\n(3) IOR optimization has always been a difficult problem. I suggest that the authors analyze the recovery of the IOR under different refractive indices for the same object. As the IOR increases, the proportion of total internal reflection rises sharply, making IOR estimation more challenging. It is unclear whether the ray tracer can correctly handle such cases.\n\n(4) Regarding the environment map, it would be helpful to include some results showing the recovery of the environment map.\n\n(5) The geometric details could be better illustrated using a normal map, since the current white model rendering makes it hard to perceive geometric enhancement. From Fig. 4, for example, the eyes of the Monkey model are almost not recovered at all. Another useful metric would be the MAE between the predicted and ground-truth outgoing ray directions.\n\n(6) In recursive ray tracing, it is not mentioned how the method handles cases of internal total reflection.\n\n(7) The training time for a single object? Any failure cases? Moreover, given that the optimization involves a large number of variables, I wonder whether the entire optimization process remains stable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)