TetSphere Splatting: Representing High-Quality Geometry with Lagrangian Volumetric Meshes
Authors: Minghao Guo
Bohan Wang
Kaiming He
Wojciech Matusik
Venue: ICLR 2025 Oral (2025)
Keywords: geometry representation
3D modeling
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We introduce TetSphere Splatting, a Lagrangian geometry representation designed for high-quality 3D shape modeling. TetSphere splatting leverages an underused yet powerful geometric primitive -- volumetric tetrahedral meshes. It represents 3D shapes by deforming a collection of tetrahedral spheres, with geometric regularizations and constraints that effectively resolve common mesh issues such as irregular triangles, non-manifoldness, and floating artifacts. Experimental results on multi-view and single-view reconstruction highlight TetSphere splatting's superior mesh quality while maintaining competitive reconstruction accuracy compared to state-of-the-art methods. Additionally, TetSphere splatting demonstrates versatility by seamlessly integrating into generative modeling tasks, such as image-to-3D and text-to-3D generation.

Published as a conference paper at ICLR 2025
TETSPHERE SPLATTING: REPRESENTING HIGH-QUALITY GEOMETRY WITH LAGRANGIAN VOLUMETRIC MESHES
Minghao Guo',, Bohan Wang 12+ Kaiming He', Wojeiech Matusik'
MIT CSAIL, National University of Singapore
(8)
(b)
Ground Truth
Lagrangian
Bulerian
(TetSphere Splatting)
Reconstructed Shape
"A racliner chair"
"A bear dressed in nedieval
Input Image
Reconstructed Shape
(a) Eulerian vs. Lagrangian geometry representations: Compared to Eulerian methods that rely on a fixed grid, TetSphere splatting, Lagrangian method, uses a set of volumetric tetrahedral spheres that deform to represent the geometry. TetSphere splatting supports applications such as reconstruction, image-to-3DD, and text-to-3D) generation (b-d).
Figure 1:
ABSTRACT
We introduce TetSphere Splatting, a Lagrangian geometry representation designed for high-quality 3D shape modeling. TetSphere splatting leverages an under- used yet powerful geometric primitive volumetric tetrahedral meshes. rep- resents 3D shapes by deforming collection of tetrahedral spheres, with geo- metric regularizations and constraints that effectively resolve common mesh is- sues such irregular triangles, non-manifoldness, and floating artifacts. Ex- perimental results on multi-view and single-view reconstruction highlight Tet- Sphere splatting's superior mesh quality while maintaining competitive recon- struction accuracy compared to state-of-the-art methods. Additionally, TetSphere splatting demonstrates versatility by seamlessly integrating into generative mod- eling tasks, such as image-to-3D and text-to-3D generation. Code is available at https://github. com/ httpa//gggh//tttatttta
INTRODUCTION
Accurate 3D shape modeling is critical for many world applications. Recent advancements in reconstruction (Mildenhall et al. 2020 Wang et al. [202 lb; Kerbl et al., (002)), generative model- ing (Poole et al. 2022 Liu et al. 2023b ct Long et al.) 2023), and inverse rendering (Mehta et al.) 2022 Nicolet et al.; 2021 Palfinger 2022) have significantly improved the geometric precision and visual quality of 3D shapes, pushing the boundaries of automatic digital asset generation.
Both authors contributed equally to this research.

---

Published as a conference paper at ICLR 2025
NeRF
FlexiCubes
DMesh
Gaussian Splatting
Ours
Figure 2: Visual comparison of mesh quality across widely used shape representations, including NeRF (Mildenhall et al. 2020), FlexiCubes (Shen et al., 2023b) (Fulerian), DMesh (Son et al. 2024), and Gaussian Splatting (Huang et al.] 2024) (Lagrangiann. These methods exhibit mesh quality issues, such as irregular or degenerated triangles, non-mmniifldness, and floating artifacts. Our method demonstrates uniform surface triangles, improved mesh quality, and structure integrity.
Central to these advancements geometry representations, which can be broadly categorized into two types: Fulerian and Lagrangian representations. Fulerian representations describe a geometry on set of pre-defined, fixed coordinates in 3D world space, where each coordinate position is sociated with properties, such as occupancy within the volume or distance from the surface. Widely used Eulerian representations include neural networks that take continuous spatial coordinates as put to model density fields (Mildenhall et al. 2020) or signed distance functions (Wang et al. [021b] Mehta et al., (022), as well as deformable grids that use discrete coordinates, with signed distance values defined at grid vertices (Shen et al.) 2021; Gao et al. [022]] [hen et al.] (023b). Despite their popularity, Bulerian representations face trade-off between computational complexity and geometry quality: Capturing intricate geometric details of the shape requires either high-capacity neural network or a diihutocoouution grid, both of which are computationally expensive to optimize in terms of time and memory. This trade-off often limits Eulerian representations when modeling thin, slender structures, as their pre-defined resolution is often insufficient to capture fine details.
Recently, there has been a growing shift in the community towards Lagrangian representations, which are typically more computationally efficient than Eulerian methods (Kerbl et al.) [003]]
Guédon & Lepetit 2024, Huang al.] 2024| [on et al., 2024: Chen al.) 2024). Lagrangian representations describe a 3D shape by tracking the movement of set of geometry primitives in 3D
world space. These geometry primitives can be adaptively positioned based on the local geometry of the shape, typically requiring fewer computational resources than Eulerian methods, especially when modeling shapes with fine geometric details. An illustrative comparison between Eulerian and Lagrangian geometry representation is shown in Fig. [ (a). Two of the most commonly used Lagrangian primitives are 3D Gaussians (Kerbl et al. (203)) which represent geometry using point clouds, and surface triangles (Son et al.) [0224] Chen et al. 2024), which are coupled with their connectivity to form surface meshes. While these Lagrangian representations are favored for their computational efficiency, they often struggle with poor mesh quality due their reliance tracking individual points or triangles, which can lack overall structural coherence. For instance, Gaussian point clouds can move freely in space, often resulting in noisy meshes, while surface triangles, when coupled with connectivity, can form non-manifold surfaces or irregular, and sometimes degenerated, triangles. The resulting geometry, exhibiting these geometric issues, is unsuitable for downstream tasks such as rendering and simulation, where high-quality meshes are crucial for high fidelity.
To address these challenges, we propose novel Lagrangian geometry representation, TerSphere Splatting, designed to construct geometry with an emphasis on producing high-quality meshes. Our key insights stem from the fact that existing Lagrangian primitives are too fine-grained high-quality meshes. Mesh quality depends not only on individual primitives but also on their inter- actions: For example, the absence of irregular or degenerated triangles relies on the proper alignment of primitives, while manifoldness depends on how well they are connected. Our representation uses volumetric tetrahedral spheres, termed TeiSphere, as geometric primitives. Unlike existing primi- tives that are individual points or triangles, each TetSphere is a volumetric sphere composed of set of points connected through tetrahedralization. Initialized as a uniform sphere, each TetSphere can be deformed into complex shapes. Together, a collection of these deformed TetSpheres represents 3D shape, which is in line with the Lagrangian approach. This more structured primitive allows ge- ometric regularization and constraints to be imposed among points within each TetSphere, ensuring
ensure
2

---

Published as a conference paper at ICLR 2025
mesh quality is maintained throughout deformation. The volumetric nature of TetSpheres also tablishes a cohesive arrangement of points throughout the volume, ensuring structural integrity and effectively reducing common surface mesh issues such as irregular triangles non non-manifoldness.
We further present a computational framework for TetSphere splatting. Similar to Gaussian splat- ting (Kerbl et al. ) our method "splats" TetSpheres conform to the target shape. We for- mulate the deformation of TetSphere as geometric energy optimization problem, consisting of dif- ferentiable rendering loss, bi-harmonic energy of the deformation gradient field, and non-inversion constraints, all effectively solvable via gradient descent. For evaluation, we conduct quantitative comparisons on two tasks to assess the geometry quality: multi and single-view reconstruction. In addition to the commonly used metrics for evaluating reconstruction accuracy, we introduce three metries to evaluate mesh quality, focusing on key aspects of 3D model usability: surface triangles uniformity, manifoldness, and structural integrity. Compared to state-of-the-arr methods, TetSphere splatting demonstrates superior mesh quality while maintaining competitive performance on other metrics. Furthermore, as demonstrations of its versatility, we showcase TetSphere splatting's 's utility in downstream applications of 3D shape generation from both single images and text.
RELATED WORK
Eulerian and Lagrangian geometry representations. The differentiation between Eulerian and Lagrangian representations originates from computational fluid dynamics (Chung 2002) but ex- tends more broadly into computational geometry and physics. Using fluid simulation as an analogy, an Eulerian view would analyze fluid presence at fixed points in space, whereas a Lagrangian per- spective follows specific fluid particles. Neural implicit representations, such as DeepSDF (Park) jet al., 2019) and NeRF (Mildenhall et al. (02T) are modern adaptations of Fulerian concepts, pro- cessing 3D positions as inputs to neural networks. These methods theoretically allow for infinite resolution through NN-based parameterization but can result in slow optimization speeds due to NN training. Explicit hybrid Eulerian representations, such as DMTet (Shen et al., (222]) and TetGAN (Yang et al. 2019), incorporate explicit irregular grids but can still cause substantial mem- ory usage for high-resolution shapes. Mosaic SDF (Yariv et al., 2024) uses Lagrangian volumetric grids that move in space but are designed for 3D generation tasks only where ground-truth truth shapes are required. Gaussian splatting (Tang et al., (023a) exemplifies Lagrangian approach by moving Gaussian point clouds in space. Surface triangles (Son et al.) 2024 Chen et al., 2024) are another example of Lagrangian approach, where the surface discretized into collection of connected triangular elements tracked individually. Our TetSphere can be viewed as introducing constraints among points due to tetrahedral meshing, with enhanced mesh quality.
3D object reconstruction. 3D reconstruction is an inherently ill- ill-posed problem, and extensive research has been dedicated addressing (Fu al., [22T]] |aahi et al.] (022)). Early approaches utilized combination of 2D image encoders and 3D decoders trained on 3D data with both explicit representations, including voxels (Chen & Zhang 2019 Xie et al. 2019 2020), meshes (Wang et al.,
2018 Gkioxari et al., 2019), and point clouds (Mandikal et al. 2018 Groueix et al.] 2018). and
implicit representations such as NeRF YYu et al.,
2020 [020]Jaag & Agapito 2021 Müller et al. 2022).
SDr et al. [ Mittal et al. 2022 et al. (2019), and occupancy Settorks(MMechedeee et al.
2019 Bian et al. (021). Recently, an active research direction has been leveraging 2D generative
models for 3D reconstruction, including the use of SDS and supplementary losses (Lin et al.
2023
Sun et al. 2023 Liu et al. 2023b Melas- et al. 2023 Shen et al.] 2023a Xu et al.
2022
Gu et 2023 Deng et al.
2022). The recent introduction of large-scale 3D dataset propelled feed-
forward large reconstruction models (Hong et al. 20235 Wang et al. 20236) Xu et al.
2023 Weng
Zhang et al., 2024 Xu et al. 2024b). Their feed-forward inference accelerates the speed of 3D reconstruction,
al. 2024 Tang et al. 2024 He & Wang [023 Tochilkin et al. [024]] Wei et al. 2024
but often at a sacrifice of relatively low resolution and geometry quality. There is also a body of work
Palfinger] 2022), including two leveraging explicit Lagrangian representations with differentiable rendering
from the inverse rendering community focused on 3D reconstruction (Nicolet et al.] [222] P
losses NNcolet et al.) 202 Palfinger 2022 Vicini al. 2022; Worchel et al. 2022)). Nicolet et al. 2022 uses Laplacian preconditioning when computing the gradient. Paalinger (2022) introduces an adaptive remeshing algorithm based estimating the optimal local edge length. Both methods use a single surface sphere, whereas our method uses multiple tetrahedral spheres as primitives. Detailed discussion and experimental comparisons provided in Appendix A
3

---

Published as a conference paper at ICLR 2025
TetSphere Deformation
F
Deformation Gradient
Initial TetSphere
Deformed TetSphere
TetSphere Splatting
Rendering Loss
Rendered Image
g.t.
Geometry Energy
Reg (F)
Initial TeiSpheres
Reconstructed Shape
Figure 3: Overall pipeline: TetSphere splatting represents 3D shape using collection of Tet- Spheres. Each TetSphere is a tetrahedral sphere that can be deformed from its initial uniform state through deformation gradient. The deformation process is optimized by minimizing rendering loss and geometric energy terms.
Our method also relates to text-to-3D content generation as it is one of the applications of TetSphere splatting. We leave detailed discussion on these related works in Appendix
TETSPHERE SPLATTING
We use tetrahedral spheres as our primitive of choice. Unlike point clouds, tetrahedral meshes en- force structured local connectivity between points owing to tetrahedralization. This preserves the geometric integrity of the 3D shape and also enhances the surface quality by imposing geometric regularization across the entire mesh interior. We formulate the reconstruction of shapes through Tet- Sphere splatting as a deformation of tetrahedron spheres. Starting from a set of tetrahedral spheres, adjust the positions of their vertices align the rendered images of these meshes with the cor- responding target multi-view images. Vertex movement is constrained by two geometric regular- izations on the tetrahedral meshes, derived from the field of geometry processing (Bærentzen al. 2012). These regularizations, which penalize the non-smooth deformation (via bi-harmonic energy) and prevent the inversion of mesh elements (via local injectivity), have proven effective in ensuring that the resulting tetrahedral meshes are of superior quality and maintain structural integrity. Fig. illustrates the overall pipeline.
3.1 TETRAIEDRAL SPIERE PRIMITIVE
The primitive of TetSphere splatting is tetrahedralized sphere, called TetSphere, with N vertices and T tetrahedra. By applying principles from the Finite Element Method (FEM) (Sifakis & Barbic 2012). the mesh of each sphere is composed of tetrahedral elements, with each tetrahedron consti- tuting 3D discrete piecewise linear volumetric entity. We denote the position vector of all vertices of the i-th deformed sphere mesh 2; R3N. 'he deformation gradient of the j-th tetrahedron in
the i-th sphere is denoted as F(f-)) R3333 which quantitatively describes how each tetrahedron's
shape transforms Sifakks & Barbic 2012). Essentially, the deformation gradient (t.J) serves as a measure of the spatial changes a tetrahedron undergoes from its original configuration to its de- formed state. Refer to Fig. 3 for visual explanation and Appendix [ for in-depth derivation.
Rather than using a single sphere, our representation employs a collection of spheres to accurately represent arbitrary shapes. Consequently, the complete shape is the union of all spheres. By adopting multiple spheres, this approach ensures that each local region shape is detailed independently, enabling highly accurate representation. Moreover, allows for the representation of shapes with
4

---

Published as a conference paper at ICLR 2025
Ieer-O
Iter-100
Iter-200
Iter-400
Iter-800
Isee=3000
Figure 4: TetSphere splatting with deforming tetrahedral spheres. Color-coded regions represent the bi-harmonic energy values (red: high, blue: low) across tetrahedra, of the geometric regulariza- tions employed in our deformation optimization process.
arbitrary topologies. Such claim is theoretically guaranteed by the paracompactness property of manifold shapes (James) 2000).
Using tetrahedral spheres offers several technical benefits compared with prevalent representations for object reconstruction, as demonstrated Fig. 2
Compared neural representations (e.g., NeRF), tetrahedral representation does rely neural networks, thus inherently accelerating the optimization process.
Compared to Eulerian representations (such as DMTet), our approach entirely avoids the need for iso-surface extraction resolution of the grid space.
operation that often degrades mesh quality owing to the predetermined
Compared other Lagrangian representations, such as Gaussian point clouds and triangle meshes, our method offers a volumetric representation through the use of tetrahedral meshes. Each tetra- hedron imposes constraints among vertices, leading superior mesh quality.
3.2 TETSPHERE SPLATTING AS SHAPE DEFORMATION
To reconstruct the geometry of the target object, deform the initial TetSpheres by changing their vertex positions. Two primary goals govern this process: ensuring the deformed TetSpheres align with the input multi-view images and maintaining high mesh quality that adheres necessary geometry constraints. Fig. illluuttatts the iterative process of TetSphere splatting.
To maintain the mesh quality, we leverage bi-harmonic energy defined in the literature ge- ometry processing [Botsch & Sorkine 2007) as an energy quantifying smoothness throughout a field to the deformation gradient field. This geometric regularization ensures the smoothness of the deformation gradient field across the deformation process, thus preventing irregular mesh or bumpy surfaces. over-smoothness of the final result. This is because the energy targets the deformation gradient field, which measures the relative changes in vertex positions, rather than the absolute positions themselves. Such an approach allows for the preservation of sharp local geometric details, akin to techniques used in physical simulations (Wen et al. 2023). Furthermore, we introduce a geometric constraint to guarantee local injectivity in all deformed elements (Schüller al. 2013). This ensures that the elements maintain their orientation during the deformation, avoiding inversions or inside-out 0. Importantly,
important highlight that this bi-harmonic regularization does not lead to
configurations. This constraint can be mathematically expressed as (et(F these two terms bi-harmonic energy for smoothness and local injectivity for element orientation are universally applicable to any tetrahedral meshes, stemming from their fundamental basis in geometry processing (Bærentzen et al. 2012). More details are discussed in Appendix C
R denote the positions of vertices across all M TetSpheres, and Fx E ROMT = (vec(F((.)) (1.1)) )... vCc(F (().)) denote the flattened deformation gradient fields of all TetSpheres, where N and T denote the number of vertices and tetrahedra within each TetSphere, respectively. In the bi-harmonic energy, the Laplacian matrix is defined based on the connectiv- ity of the tetrahedron faces, denoted as E x9MTXXMT This matrix is block symmetric, where
.... ...,..... ÆM] Let x [.,,...,

---

Published as a conference paper at ICLR 2025
each block Lpg E R -p= # q is set to negative identity matrix -I if the p-th and q-th tetrahedron shares a common triangle; or kI for Lyp. where & is the number of neighbors of the p-th tetrahedron. The deformation of the TetSpheres is formulated as an optimization problem:
min & (R(x)) + ||L x
s.t. Fe ()))) > 0, Vi (L,., J {1....
(1)
where R(.) is the rendering function, ( is the rendering loss matching the union of deformed tetrahedral spheres with the input images. The second term regulates the bi-harmonic energy across the deformation gradient field. The non-inversion constraint ensures that tetrahedra maintain their orientation. To manage this constrained optimization, we reformulate it by incorporating the non- inversion hard constraint soft penalty term into the objective,
)
min ((((x)) w1 |F 11 + W2 Σ dee
det (e
allowing for optimization via standard gradient descent solvers.
In the proposed optimization framework, three considerations have been outlined. The adaptive loss function (-)), designed to be flexible, supports a variety of metrics, including l for color images, MSE for depth images, and cosine embedding loss for normal images. These losses can all be optimized using gradients provided by differentiable renderers, such as mesh rasterizers (Laine al. 2020).. 2) Given that the tetrahedron is linear element, the deformation gradient F is a linear function of x, making the bi-harmonic energy a quadratic term. 3) The weights wi and W2 are dynamically adjusted using a cosine scheduler. We provide details of the scheduler's hyperparameterss in Appendix !I
4 TETSPHERE INITIALIZATION AND TEXTURE OPTIMIZATION
TetSphere initialization. Given multi-view images as inputs, we select feature points to initial- ize the 3D center positions of the TerSpheres. We aim to achieve a uniform distribution of these TetSpheres, ensuring comprehensive coverage of the silhouette depicted in the multi-view images.
We introduce algorithm, silhouette coverage, inspired by Coverage Axis (Dou et al. 2022 to
select initial centers of TetSpheres for an arbitrary shape automatically. This process begins with constructing a coarse voxel grid and initially assigning a zero value each voxel. By projecting these voxels into the image spaces using the same camera poses as the input multi-view images, voxels within the foreground of all images are marked with a value of 1. These voxel positions are identified as candidate positions of TetSphere centers. From these marked positions, the objective to pick a minimal subset of candidates to ensure that all candidates are fully encapsulated by Tet- Spheres and centered on these points. This involves placing uniform spheres with varying radius values at all candidate points and choosing a minimal subset that collectively covers all the candi- date points. We formulate linear programming problem to perform the selection efficiently. The detailed formulation is provided in Appendix G In our implementation, with a voxel grid resolution 300 x 300 and M = 20, the whole TetSphere initialization completes in ~ minute average.
Texture optimization. While the primary purpose of TetSphere splatting is to represent high-quality geometry, its explicit structure allows textures and materials to be directly applied to the surface ver- tices and faces of the TetSpheres. A key advantage of TetSphere splatting is that the deformation of tetrahedral spheres preserves the surface topology, enabling the seamless integration of advanced material models, such as Disney's 's principled BRDF (Burley & Studios 2012). in physically-bast rendering for applications like text-to-3D generation. However, it is important to note that texture optimization is an optional feature of our representation. The primary objective of TetSphere splat- ting remains the accurate representation of geometry, with textures and materials being secondary enhancements. Further details are provided in Appendix
5 EXPERIMENTS AND RESULTS
We provide detailed quantitative evaluations on 3D reconstruction from both multi-view and single- view images demonstrate the effectiveness of TetSphere. Additionally, showcase TetSphere's

---

Published as a conference paper at ICLR 2025
Table I: Multi-View reconstruction results: Evaluating reconstruction accuracy with Chamfer Dis- tance (Cham.) and Volume IoU, alongside mesh quality metrics: Area-Length Ratio (ALR), Mani- foldness Rate (MR), and Connected Component Discrepancy (CC Diff.). For additional results on other metrics, please refer to Table 5 in Appendix E.1
Method
Geo. Rep.
Cham.
Vol. IoU
ALR 1
MR(%) f
CC Diff.
NIE FlexiCuhes
Eulerian
0.0254
0.1863
0.0273
72.3
7.0
Fulerian
0.0247
0,5887
0.0722
45.5
201.3
NeuS
Fulerian
0.0192
0.6182
0.0573
72.3
8.1
VoISDF
Fulerian
0.0185
0.6423
0.0622
81.8
7.3
2DGS
Lagrangian
0.0322
0.4923
0.0209
27.3
25.1
DMesh
Lagrangian
0.0136
0,5616
0.1193
9.09
3.75
Ours
Lagrangian
0.0184
0.6844
0.6602
100
0.0
versatility through case studies on image-to-33 and text-to-3D shape generation. We further analyze the effect of energy coefficients and offer a comparative study on computational costs, particularly focusing GPU requirements and memory usage for image-to-3D generation with SDS loss, where existing methods struggle with high computational demands.
5.1 IMPLEMENTATION DETAILS
Our implementation of the TetSphere initialization algorithm is developed in C++ and uses the Gurobi linear programming solver. The optimization of geometric energies is implemented using CUDA as a PyTorch extension to enhance computational efficiency. 1) For multi-viww reconstruc- tion, we render multi-view RGBA and depth images of the ground truth shape as the input of Tet- Sphere splatting. The optimization objective ((.) includes both rendering loss (MSE on the alpha mask) and depth loss, following (on et al. (2024). 2) For single-view reconstruction,
use Won- der3d (Long et al. 2023) to generate six multi-view images with predefined camera poses. The optimization objective (-) includes rendering loss norm on tone-mapped color and MSE on the alpha mask) and normal loss (cosine loss on normals), following Munkberr et al. (2022). The imple- mentation details for the two applications are described in Appendix ! To enhance robustness and efficiency, we apply a cosine scheduler to scale the coefficients of the geometry loss, formulated as gsin( in where denotes the current iteration and T represents the total number of iterations.
5.2 BASELINES AND EVALUATION PROTOCOL
Baselines. For multi-viww reconstruction, we quantitatively compare our method with state-of- the-art geometry representations: Neural Implicit Evolution (NIE) (Mehta et al. 2022). Flexi- Cubes (Shen et al. 2023b). 2DGS (Huang et al., 2024). and DMesh (Son et al. 2024). The first two Eulerian representations, whereas the latter two are Lagrangian representations. For single-view reconstruction, we quantitatively compare with several state-of-the-art methods: Magic123 (Qian) 2023a), SyncDreamer (Liu al.) 2023c), Wonder3d (Long et al. 2023), Open-LRM (Hong et al. 2023b He & Wang (023), and DreamGaussian (Tang et al. 2023a). DreamGaussian is the only Lagrangian representation, as the field of single-view recon-
let al., 2023), 2-e-2-3-45 (Liu al.,
struction is largely dominated by Eulerian methods. For all baselines, we follow their original implementations and use their publicly available codebase get the results in this paper.
Evaluation datasets. For multi-view reconstruction, we follow Son et al. 2024) and use four closed-surface models from the Thingi32 dataset (Zhou & Jacobson 2016). four open-surface mod- els from the DeepFashion3D dataset (Heming et al.) 2020), and two additional models with both closed and open surfaces from the Objaverse dataset (Deitke al. 2023). We also add the chal- lenging sorter shape from Google Scanned Objects (GSO) dataset (Downs al. (022) it features slender structures. For single-view reconstruction, following prior research (Liu et al. 2023c Long al. 2023), we use the GSO dataset for evaluation, which covers broad range of everyday objects The evaluation dataset aligns with those used by SyncDreamer and Wonder3D, featuring 30 diverse objects ranging from household items to animals.
Metrics. To assess the accuracy of the reconstruction, we use two commonly used metrics: Cham- fer Distance (Cham.) and Volume IoU (Vol. loU), comparing the ground-truth shapes with the

---

Published as a conference paper at ICLR 2025
NIE
FlexiCubes
DMesh
20GS
Ours
Figure 5: Qualitative results on multi-view reconstruction, with surface mesh visualizations and ren- dered normal maps. Our method excels over baseline methods regarding mesh quality, less bumpy surface, correct surface orientation, and accurately capturing slender and thin structures.
reconstructed ones. Following established practices, we use the rigid Iterative Closest Point (ICP) algorithm to align the generated shapes with their ground-truth counterparts before metric calcula- tion. For multi-view reconstruction, we also report F-Score, Normal Consistency. Edge Chamfer Distance, and Edge F-Score following prior research (Shen al., 2023b Son et 2024).
While these metrics effectively measure the reconstruction accuracy, they fail to evaluate mesh qual- ity. We three additional metries assess the geometry quality of the reconstructed shapes: 1) Area-length Ratio (ALR): This metric computes the average ratio of a triangle's 's area to its perime-
Table 2: Single- View reconstruction results on the GSO Dataset: Evaluating reconstruction accuracy with Chamfer Distance (Cham.) and Volume IoU, alongside mesh quality metries: Area-Length Ratio (ALR), Manifoldness Rate (MR), and Connected Component Discrepancy (CC Diff.).
Method
Geo. Rep.
Cham.
Vol. IoU
ALR 1
MR(%) f
CC Diff. 4
Magic123
Fulerian
0.0516
0.4528
0.0383
13.7
One-2-3-45
Fulerian
0.0629
0.4086
0.0574
96
0.83
SyncDreamer
Fulerian
0.0261
0.5421
0.0201
10
0.3
Wonder3d
Eulerian
0.0329
0.5768
0.0281
100
0.0
Open-LRM
Eulerian
0.0285
0.5945
0.0252
0.0
DreanGaussian
Lagrangian
0.0641
0.3476
0.0812
100
237.4
Ours
Lagrangian
0.0351
0.6317
0.3665
100
0.0
8

---

Published as a conference paper at ICLR 2025
Comparison of memory cost and run-time speed image-to-3D generation with SDS loss. We report the maximal batch size of 256 x 256 images that can occupy a 40GB A100 and the run-time speed for training with batch size 4.
Table 3:
Maximal Batch Sizet
Speedf (#iter/s)
Method
Make-it-3D
1.22
Magic123
4
1.03
Eulerian
SyncDreamer
48
1.8
DreamCraft3D
1.23
conmal loos
GeoDream
8
1.30
Lagrangian
DreamGaussian
80
4.43
Figure 6: Analysis geometry energy coef- ficients, Dark regions indicate flipping of the surface triangles.
Lagrangian
Ours
120
6.59
ter (scaled by constant coefficient) within the surface mesh. Values range from 0 1, where meshes with higher ALR values contain mostly equilateral triangles, therehy indicating superior triangle quality; 2) Manifoldness Rate (MR): Manifoldness verifies whether mesh qualifies as closed manifold. Non- Non-manifold meshes can manifest anomalies, such as edges shared by more than two faces, vertices connected by edges but by a surface, isolated vertices and edges, and vertices where more than one distinct surface meets, which can cause problems in downstream applications such as simulation and rendering. We report percentage of manifold shapes within the evalua- tion dataset as MR; and 3) Connected Component Discrepancy (CC Diff.) from the ground-truth shape: This measure identifies the presence of floating artifacts or structural discontinuities within the mesh, highlighting the integrity and cohesion of the reconstructed shape.
5.3 RESULTS
Multi-view reconstruction. Table and Table 5 in Appendix H.1 present the quantitative com- parison results. Fig. 5) shows the qualitative comparison on four different shapes. Our method ex- cels over baseline methods regarding mesh quality while achieving similar reconstruction accuracy. Specifically, our approach features uniform meshing (as indicated by ALR) and eliminates floating artifacts (as shown by CC Diff.). This improvement is attributed to the volumetric representation of TetSphere and its geometric energy regularization. The qualitative results further support these find- ings: our method accurately captures slender and thin structures, such as the fingers of Lucy's statue, the sorter, and the dress, which other methods struggle with. Additionally, our method demonstrates correct surface orientation, whereas bascline methods, particularly FlcxiCubes, and DMesh, often exhibit inverted triangles, approach is less effective when modeling thin shells, such as the leaves of the plant, where DMesh shows better accuracy.
in the noisy rendered normal images. However, our TetSphere
Single-view reconstruction. Table shows the comparison results. Fig. II in Appendix shows qualitative results of our method. Our TetSphere splatting excels baseline methods terms of mesh quality while also achieving competitive levels reconstruction accuracy. This improvement attributed to the regularizations embedded within the TetSphere splatting optimization. ensuring uniform meshing and eliminating common artifacts like floating components. These findings are consistent with the results observed in multi-view reconstruction.
5.4 ANALYSIS
Applications to generative modeling. TetSphere splatting is a versatile approach that can effec- tively applied to generative modeling tasks. We showcase two applications: image-to-3D and text- to-3D generation. Fig. 12 in Appendix E3 illustrate the results on image-to-3D generation. Our approach outperforms Magic123 and DreamCraft3D regarding mesh quality, achieving smoother surfaces for broad regions, such as animal bodies while retaining local sharp details areas like eyes and noses. Magic123 tends to produce overly smooth surfaces that sometimes deviate from the

---

Published as a conference paper at ICLR 2025
"A lush, green dragon camouflaged within dense forest, with leafy wings and wooden horns."
"A DSLR photo of wine bottle and full wine glass on chessboard."
RichDreamer
Ours
Figure 7: Results text-to-3D shape generation. Our TetSphere splatting excels in handling slen- der, thin structures, such as the wine glass stem and the dragon head. More results are available in Appendix E.4
correct geometry. DreamCraft3D suffers from noisy and bumpy surface meshes, indicating lower mesh quality. Fig. 7 and Fig. 13 in Appendix E.4 shows comparison results text-to-3D gener- ation. Our TetSphere splatting can produce slender structures such as the dragon's head and the goblet. Furthermore, the results demonstrate that using TetSphere splatting with SDS enhances the geometric detail the generated shapes, resulting in both diverse and high-quality textures.
Analysis computational cost As shown in Table in Appendix E. our method outperforms the Eulerian method NIE regarding running time while remaining competitive with other geometry representation methods. To further assess the performance of TetSphere splatting, we conduct an extreme test by applying it to the task of image-to-3D generation with SDS loss. We evaluate several baseline methods and report the maximum batch size of 256 256 images that can processed on 40GB A100 GPU, as well as the training run-ting cun-time speed with a batch size of 4. as shown in Table TetSphere splatting stands out for its minimal memory usage and achieves the fastest run-time speed. This efficiency underscores the benefits of TetSphere's explicit and Lagrangian properties.
Effects of energy coefficients. Fig. 6 demonstrates how different energy coefficients influence the reconstruction outcome. Larger coefficients lead to smoother surface, but excessively small coefficients may cause tetrahedron inversion. In our experiments, we choose w x 10 w2 2 x 10 and apply cosine increasing schedule.
CONCLUSION
We introduced TetSphere splatting. Lagrangian geometry representation for high-quality 3D shape modeling. Our computational framework leverages geometric energy optimization and differen- tiable rendering to deform TetSpheres into complex shapes, producing results that demonstrate perior mesh quality compared to state-of-the-an methods.
Limitations. TetSphere splatting has limitations, including the generation of piecewise meshes rather than a globally consistent tetrahedral mesh and the lack of strong topological guarantees. More details are discussed in Appendix K Future work could extend TetSphere splatting to leverage direct 3D supervision with volumetric data, eliminating the need for rendering to images. The current TetSphere splatting cannot guarantee topology preservation due to the union of all tetrahedral spheres. This highlights the need for future development in shape generation that can better adhere topology constraints.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Brian Amberg. Sami Romdhani, and Thomas Vetter. Optimal step nonrigid icp algorithms for sur- face registration. In 2007 IEEE conference on computer vision and pattern recognition, pp. 1-8. IEEE, 2007.
Mohammadreza Armandpour, Ali Sadeghian, Huangjic Zheng, Amir Sadeghian, and Mingyuan uan Zhou. Re-imagine the negative prompt algorithm: Transform 2D diffusion into 3d, alleviate janus problem and beyond. April 2023.
J Andreas Bærentzen, Jens Gravesen, François Anton, and Henrik Aanæs. Guide to computational geometry processing: foundations, algorithms, and methods. Springer Science Business Media, 2012.
Wenjing Bian, Zirui Wang, Kejie Li, and Victor Adrian Prisacariu. Ray-onet: Efficient 3d recon- struction from single rgb image. In BMVC, 2021.
Mario Botsch and Olga Sorkine. On linear variational surface deformation methods. IEEE transac- fions on visualization and computer graphics, 14(1):213-23)) 2007.
Brent Burley and Walt Disney Animation Studios. Physically-based shading at disney. In Acm Siggraph, volume 2012, pp. 1-7. vol. 2012, 2012.
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation. arXiv preprint arXiv:2303.13887, 2023.
Yiwen Chen, Tong He, Di Huang, Weicai Ye, Sijin Chen, Jiaxiang Tang, Xin Chen, Zhongang Cai, Lei Yang, Gang Yu, et al. Meshanything: Artist-created mesh generation with autoregressive transformers. arXiv preprint arXiv:2406. 10163, 2024.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939-5948, 2019.
T Chung. Computational fluid dynamics. Cambridge university press, 2002.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of anno- tated 3d objects. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13142-13153, 2023.
Congyue Deng, Chiyu "max" Jiang, Charles R Qi, Xinchen Yan. Yin Zhou, Leonidas Guibas, and Dragomir Anguelov. NeRDi: Single- View NeRF synthesis with Language- Guided diffusion as general image priors. December 2022.
Zhiyang Dou, Cheng Lin, Rui Xu, Lei Yang, Shiqing Xin, Taku Komura, and Wenping Wang. Coverage axis: Inner point selection for 3d shape skeletonization. Computer Graphics Forum, volume 41, pp. 419-432. Wiley Online Library, 2022.
Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Hickman, Krista Reymann, Thomas B McHugh, and Vincent Vanhoucke. Google scanned objects: A High-Quality dataset of 3D scanned household items. April 2022.
George Fahim, Khalid Amin, and Sameh Zarif. Single-view 3d reconstruction: A survey of deep learning methods. Computers & Graphics, 94:164-190, 2021.
Pascal J Frey and Houman Borouchaki. Surface mesh quality evaluation. International journal for numerical methods in engineering, 45(1):101-118, 1999.
Kui Fu, Jiansheng Peng, Qiwen He, and Hanxiao Zhang. Single image 3d object reconstruction based on deep learning: A review. Multimedia Tools and Applications, 80:463-498, 2021.
William Gao, April Wang. Gal Metzer, Raymond A Yeh, and Rana Hanocka. Tetgan: A convolu- tional neural network for tetrahedral mesh generation. arXiv preprint arXiv:2210.05735, 2022.
11

---

Published as a conference paper at ICLR 2025
Georgia Gkioxari, Jitendra Malik, and Justin Johnson. Mesh r-cmn. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9785-97999 2019.
Álvaro González. Measurement of areas on a sphere using fibonacei and latitude- longitude lattices.
Mathematical Geosciences, 42:49-65 2010.
Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. AtlasNet:
A Papier-Mäché approach to learning 3D surface generation. February 2018.
Jiatao Gu, Alex Trevithick, Kai-En Lin, Josh Susskind, Christian Theobalt, Lingjee Liu, and Ravi Ramamoorthi. Nerfdiff: Single-image view synthesis with nerf-guided distillation from 3d-aware diffusion. In ICML, 2023.
Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 1334,,363, 2024.
Zexin He and Tengfei Wang. Openirm: Open-source large reconstruction models.
https://
github. com httpa////ooooopppneaaaee 2023.
Zhu Heming, Cao Yu, Jin Hang, Chen Weikai, Du Dong, Wang Zhangye, Cui Shuguang, and Han Xiaoguang. Deep fashion3d: A dataset and benchmark for 3d garment reconstruction from single images. In Computer Vision - ECCV 2020, pp. 512-530. Springer International Publishing, 2020. ISBN 978-3-030-58452-8.
Susung Hong, Donghoon Ahn, and Seungryong Kim. Debiasing scores and prompts of 2D diffusion for view-consistent Text-to-3D generation. March 2023a.
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. LRM: Large reconstruction model for single image to 3D. November 2023b.
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In ACM SIGGRAPH 2024 Conference Papers, pp- 2024.
Yukun Huang, Jianan Wang, Yukai Shi, Xianbiao Qi, Zheng-Jun Zha, and Lei Zhang. DreamTime: An improved optimization strategy for Text-to-3D content creation. June 2023.
Alec Jacobson and Daniele Panozzo. Libigl: Prototyping geometry processing research in c++. In SIGGRAPH Asia 2017 courses, pp. 1-172. 2017.
Ajay Jain, Ben Mildenhall, Jonathan T. Barron, Pieter Abbeel, and Ben Poole. Zero-shot text-guided object generation with dream fields. CVPR, 2022.
R Munkres James. Topology. Prentic Hall of India Private Limited, New delhi, 2000.
Wonbong Jang and Lourdes Agapito. CodeNeRF: Disentangled neural radiance fields for object categories. September 2021.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimküüler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023.
Nasir Mohammad Khalid, Tianhao Xie, Eugene Belilovsky, and Popa Tiberiu. Clip-mesh: Gen- erating textured meshes from text using pretrained image-text models. SIGGRAPH Asia 2022 Conference Papers, December 2022.
Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. Modular primitives for nigh-performance differentiable rendering, ACM Transactions on Graphics, 39(6), 2020.
Weiyu Li, Rui Chen, Xuelin Chen, and Ping Tan. SweetDreamer: Aligning geometric priors in 2D diffusion for consistent Text-to-3D. October 2023.
12

---

Published as a conference paper at ICLR 2025
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming- Yu Liu, and Tsung- Yi Lin. Magic3d: High-resolution text-to-33 --3d content creation. In CVPR, 2023.
Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Varma T Mukund, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3D mesh in 45 seconds without Per-Shape optimization. June 2023a.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-I-to-3; Zero-shot one image to 3d object, 2023b.
Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Learning to generate multiview-consister images from single-view image. arXiv preprint arXiv:2339.0344 2023c.
Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Mare Habermann, Christian Theohalt, and Wenping Wang. Wonder3I: Single image 3D using Cross-Domain diffusion. October 2023.
Priyanka Mandikal, K L Navaneet, Mayank Agarwal, and R Venkatesh Babu. 3D-LMNet: Latent embedding matching for accurate and diverse 3D point cloud reconstruction from a single image. July 2018
Ishit Mehta, Manmohan Chandraker, and Ravi Ramamoorthi. A level set theory for neural im- plicit evolution under explicit flows. In European Conference on Computer Vision, pp. 711-729. Springer, 2022.
Luke KeraazKyriazzi, Christian Rupprecht, Iro Laina, and Andrea Vedaldi. RealFusion: 360 recon- struction of any object from a single image. February 2023.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Oc- cupancy networks: Learning reconstruction in function space. In Proceedings ofe IEEE/CVF conference on computer vision and pattern recognition, pp.
2019.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. ECCV, 2020.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1)999-106, 2021.
Paritosh Mittal, Yen-Chi Cheng, Maneesh Singh, and Shubham Tulsiani. Autosdf: Shape priors for 3d completion, reconstruction and generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 306-315, 2022.
Norman Müller, Andrea Simonelli, Lorenzo Porzi, Samuel Rota Bulo, Matthias Nießner, and Peter Kontschieder. Autorf: Learning 3d object radiance fields from single view observations. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3971- 3980, 2022.
Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Müller, and Sanja Fidler. Extracting Triangular 3D Models, Materials, and Lighting From Im- ages. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8280-8290, June 2022.
Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. Large steps in inverse rendering of geometry. ACM Transactions on Graphics (TOG), 40(6):1-13, 2021.
Werner Palfinger. Continuous remeshing for inverse rendering. Computer Animation and Virtual Worlds, 3315), 2022.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 165-174, 2019.
13

---

Published as a conference paper at ICLR 2025
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 2022.
Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin-Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, and Bernard Ghanem. Magic123: One image to -uaalty 3d object generation using both 2d and 3d diffusion priors. arXiv preprint arXiv:2306. 2023.
Lingteng Qiu, Guanying Chen, Xiaodong Gu, Qi Zuo, Mutian Xu, Yushuang Wu, Weihao Yuan, Zilong Dong. Liefeng Bo, and Xiaoguang Han. RichDreamer: A generalizable Normal-Depth diffusion model for detail richness in Text-to-3D. November 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In ICML, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 2022.
Christian Schüller, Ladislav Kavan, Daniele Panozzo, and Olga Horkine-Hrmuug. Locally injective mappings. Computer Graphics Forum, volume 32, pp. 125-135. Wiley Online Library, 2013.
Qiuhong Shen, Xingyi Yang, and Xinchao Wang. Anything-3D. Towards single-view anything reconstruction in the wild. April 2023a.
Tianchang Shen, Jun Gao, Kangxue Yin, Ming- Yu Liu, and Sanja Fidler. Deep marching tetrahedra: hybrid representation for high-resolution 3d shape synthesis. In NeurlPS, 2021.
Tianchang Shen, Jacob Munkberg, Jon Hasselgren, Kangxue Yin, Zian Wang, Wenzheng Chen, Zan Gojeic, Sanja Fidler, Nicholas Sharp, and Jun Gao. Flexible isosurface extraction for gradient- based mesh optimization. ACM Trans. Gruph., 42(4))37-1,
Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. MVDream: Multi-view diffusion for 3D generation. August 2023.
Eftychios Sifakis and Jernej Barbic. Fem simulation of 3d deformable solids: a practitioner's guide to theory, discretization and model reduction. Acm siggraph 2012 courses, pp. 1-50. 2012.
Sanghyun Son, Matheus Gadelha, Yang Zhou, Zexiang Xu, Ming C Lin, and Yi Zhou. Dmesh: A differentiable representation for general meshes. arXiv preprint 2tti:22004134445 2024.
Robert W Sumner and Jovan Popović. Deformation transfer for triangle meshes. ACM Transactions on graphics (TOG), 23(3):399-405, 2004.
Jingxiang Sun, Bo Zhang, Ruizhi Shao, Lizhen Wang. Wen Liu, Zhenda Xie, and Yebin Liu. Dream- Craft3D: Hierarchical 3D generation with bootstrapped diffusion prior. October 2023.
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. arXiv preprint (XXi::2009116533 2023a.
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. LGM: Large Multi-Vien gaussian model for High-Resoluttio 3D content creation. February 2024.
Junshu Tang, Tengfei Wang, Bo Zhang, Ting Zhang, Ran Yi, Lizhuang Ma, and Dong Chen. Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior. arXiv preprint arXiv:2303. /4184, 2023b.
Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from single image. arXiv preprint 24X.v2240.00155 2024. 2024.
14

---

Published as a conference paper at ICLR 2025
Christina Tsalicoglou, Fabian Manhardt, Alessio Tonioni, Michael Niemeyer, and Federico Tombari. TextMesh: Generation of realistic 3D meshes from text prompts. April 2023.
Delio Vicini, Sébastien Speierer, and Wenzel Jakob. Differentiable signed distance function render- ing. ACM Transactions Graphics (TOG), 41(4):1-18, 2022.
Bohan Wang, George Matcuk, and Jemnej Barbič. Modeling of personalized anatomy using plastic strains. ACM Transactions on Graphics (TOG), 40(2):1-11, 2021a.
Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. CVPR, 2023a.
Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European conference computer vision (ECCV), pp. 52-67, 2018.
Peng Wang, Lingje Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. NeuS: Learning neural implicit surfaces by volume rendering for multi- view reconstruction. June 202
Peng Wang, Hao Tan, Sai Bi, Yinghao Xu, Fujun Luan, Kalyan Sunkavalli, Wenping Wang, Zexiang Xu, and Kai Zhang. PF-LRM: Pose-Free large reconstruction model for joint pose and shape prediction. November 2023b.
Zhengyi Wang. Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lifiedreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. NeurIPS, 2023c.
Xinyue Wei, Kai Zhang. Sai Bi, Hao Tan, Fujun Luan, Valentin Deschaintre, Kalyan Sunkavalli, Hao Su, and Zexiang Xu. Meshlmm: Large reconstruction model for high-quality mesh. arXiv preprint arXiv:2404. 12385, 2024.
Jiahao Wen, Bohan Wang, and Jernej Barbic. Large-strain surface modeling using plasticity. IEEE
Transactions Visualization and Computer Graphics, 2023.
Zhenzhen Weng, Jingyuan Liu, Hao Tan, Zhan Xu, Yang Zhou, Serena Yeung-Levy, and Jimei Yang. Single-Vier View 3D human digitalization with large reconstruction models. January 2024.
Markus Worchel, Rodrigo Diaz, Weiwen Hu, Oliver Schreer, Ingo Feldmann, and Peter Eisert. Multi-view mesh reconstruction with neural deferred shading. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6187-6197, 2022.
Haozhe Xie, Hongxun Yao, Xiaoshuai Sun, Shangchen Zhou, and Shengping Zhang. Pix2Vox: Context-aware 3D reconstruction from single and multi-view images. January 2019.
Haozhe Xie, Hongxun Yao, Shengping Zhang, Shangchen Zhou, and Wenxiu Sun. Pix2Vox++:
Multi-scale context-aware 3D object reconstruction from single and multiple images. Int, J. Com- put. Vis., 128(12):2919299) December 2020.
Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Yi Wang. and Zhangyang Wang. NeuralLift-360: Lifting an in-the-wild 2D photo to a 3D object with 360 views. November 2022.
Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomir Mech, and Ulrich Neumann. DISN: Deep implicit surface network for high-quality single-view view 3D reconstruction. May 2019.
Rui Xu, Longdu Liu, Ningna Wang, Shuangmin Chen, Shiqing Xin, Xiaohu Guo, Zichun Zhong, Taku Komura, Wenping Wang, and Changhe Tu. Cwf: Consolidating weak features in high- quality mesh simplification. ACM Transactions Graphics (TOG), 43(4):1-14, 2024a.
Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang. Jiahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, and Kai Zhang. DMV3D: Denoising Multi View diffusion using 3D large reconstruction model. November 2023.
Yinghao Xu, Zifan Shi. Wang Yifan, Hansheng Chen, Ceyuan Yang. Sida Peng, Yujun Shen, and Gordon Wetzstein. Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation, arXiv preprint arXiv:2403. /462/, 2024b.
15

---

Published as a conference paper at ICLR 2025
Shuai Yang, Jiaying Liu, Wenjing Wang, and Zongming Guo. Tet-gan: Text effects transfer via stylization and destylization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1238-1245, 2019.
Lior Yariv, Omri Puny, Oran Gafni, and Yaron Lipman. Mosaic-sdf for 3d generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4630-4639, 2024.
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelNeRF: Neural radiance fields from one or few images. December 2020.
Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-Irm: Large reconstruction model for 3d gaussian splatting. arXiv preprint arXiv:2404. 19702, 2024.
Qingnan Zhou and Alec Jacobson. preprint arXiv:1605.0me 04797, 2016.
Thingi10k: dataset of 10,000 3d-printing models. arXiv
16

---

Published as a conference paper at ICLR 2025
[Palfinger 2022]
[Nicolet et al. 2021]
Ours
Figure 8: Qualitative comparisons with single-primitie approaches in inverse rendering (Nicolet
al. 2021 Palfinger 2022).
A COMPARISON WITH SINGLE-PRIMITIVE APPROACHES IN INVERSE RENDERING
Our method is related to single-primitive inverse rendering methods, such as Nicolet al. (2021) and Palfinger (2022). While these existing methods use single surface sphere for reconstruction, method leverages volumetric representation composed of multiple tetrahedral spheres instead of just one. Both Nicolet et al.) (20II;; Pallinger (2022) and our approach employ regularizations prevent extreme deformations that compromise geometric quality, thereby inherently limiting
17

---

Published as a conference paper at ICLR 2025
Table 4: Quantitative comparisons with single-primitive approaches in inverse rendering (Nicolet
et al. 2021; Pallinger 2022). We omit MR and CC Diff., as all three methods achieve identical values.
Method
Cham. +
Vol. IoU
ALR t
Nicolet (2021)
0.0213
0.6184 0.6583
0,5235
Pallinger 2022
0.0191
0.6332
Ours
0.0184
0.6844
0.6602
each sphere's expressivity to a certain extent. Nicolet et al. (2021); Palfinger (2022) address this limitation by using intermediate remeshing to accommodate topological changes and manage dras- tic deformations when the target shape significantly differs from the sphere. However, remeshing results in the reparameterization of the texture, which poses challenges in the context of 3D shape generation pipelines. Maintaining consistent texture parameterization throughout the optimization process is crucial, as the texture image itself is an optimization variable (see Appendix Hp. Ad- ditionally, remeshing can introduce undesired meshing complications when the surface undergoes topological changes. prevent topological changes during each iteration, single primitive must be initialized with topology that matches the target. By contrast, our method eliminates the need for intermediate remeshing by utilizing multiple TetSpheres alongside stronger regularization terms, in- cluding biharmonic energy and non-inversion constraints. By leveraging multiple TetSpheres, each TetSphere undergo a smaller deformations, ensuring robust shape reconstruction.
Table and Fig. illustrate the quantitative and qualitative comparisons of our method against Nicolet et al. 2021) and and Palfinger (0022). We used their publicly available codebases and performed dataset described in Sec. Our method outperforms these ap- proaches, particularly for shapes with complex topologies. Both Nicolet et al.] (2021) and Palfinger (202)) use single sphere for reconstruction, which limits their ability to handle objects with multi- ple holes, such as the dress and the sorter. For the dress example (the second and the third columns in Fig. B). both baseline methods exhibit unrealistic closure of the open areas. In contrast, our ap- proach accurately represents the thin regions and preserves the open areas due to the use of multiple TetSpheres. For the sorter example (the fourth and the fifth columns), the results from Nicolet al. (222T) exhibit folded, overlapping surfaces with noticeable crumpled regions. This occurs because their deformation regularization is only on the surface and cannot effectively penalize large bending. Our tetrahedral-base method, with its volumetric regularization, mitigates these issues.
multi-view reconstruction
B ADDITIONAL ABLATION STUDIES
B. ABLATION STUDY ON THE NUMBER OF TETSPHERES
The number of TetSpheres in method is determined by the initialization algorithm described in Sec. 4 and Appendix G controlled by the scaling and offset parameters and 3 that define the initial radius of each TetSphere. Intuitively, larger values and 3 result in a sparser distribution of TetSpheres. In this section, we present an ablation study on the reconstruction performance with respect different numbers of TetSpheres.
We select two examples from the multi-view reconstruction experiment, the dress and the sorter, as these represent the most challenging cases due to their complex topology. For each example, we evaluate five different combinations of and 3, resulting in (1,59. 87, 32,3 434) TetSpheres for the dress and (1,31, 31, 69, 127, 239} for the sorter. Notably, 323 and 127 TetSpheres are the numbers used in our primary method.
The reconstruction results are illustrated in Fig. 9 with corresponding metrics for Chamfer dis- tance, volume loU, and ALR shown in Fig. 10(a) a). At one extreme, using only a single TetSphere leads to poor reconstruction, unable to capture the intricate topology of the target shapes. As the number of TetSpheres increases, the reconstruction quality improves significantly. Beyond certain point, however, such as with 434 and 239 TetSpheres, the quality stabilizes, and the metrics indicate minimal further improvement although the tessellation density becomes higher.
18

---

Published as a conference paper at ICLR 2025
N=1 N=59 N-87
N-323
N-434
N-1
N-31
N-69
N-127
N-239
Figure 9: Qualitative reconstruction quality comparison using different numbers of TetSpheres. At one extreme, using only single TetSphere leads to poor reconstruction, unable to capture the in- tricate topology of the target shapes. As the number of TetSpheres increases, the reconstruction quality improves. Beyond certain point, however, the quality stabilizes although the tessellation density becomes higher. Our chosen number of TetSpheres (323 and 127) strikes balance between achieving high reconstruction quality and maintaining reasonable computational efficiency.
305
0.03
04
Miilueee 213
Chanferr Distance
0.045
TTTppeees
30 SpSppis Serter
0.03
0.005
12000
13000
Number Tetrahedra per TetSphere
Nathe 125pbercs
Botiphace
(8)
(b)
Figure 10: (a) Quantitative comparison using different numbers TetSpheres. As the number of TetSpheres increases, both the reconstruction accuracy (Chamfer distance and Vol. IoU) and the surface quality (ALR) improve. Beyond 434 and 239, however, the metrics indicate minimal further improvement. (b) Ablation study on the trade-off between the average number of tetrahedra per TetSphere and the total number of TetSpheres. For each setting of the number of TetSpheres, increasing the number tetrahedra per TetSphere improves reconstruction accuracy. However, the improvements are less pronounced compared to increasing the number of TetSpheres.
excessive number of TetSpheres increases computational costs due to the additional memory required to store the vertex positions. Our chosen number of TetSpheres strikes balance between achieving high reconstruction quality and maintaining reasonable computational efficiency.
B.2 ABLATION STUDY ON THE NUMBER OF TETRAHEDRA PER TETSPHERE
We conducted an ablation study to analyze the trade-off between the average number of tetrahedra TetSphere and the total number of TetSpheres required to represent the surface. We used four sets a and 3 values to obtain different granularities of TetSphere counts, averaging {9, 38, 110, 213} across all examples in the multi-view reconstruction experiment. For each granularity, varied the average number of tetrahedra per TetSphere, testing five configurations: (6000, 9000, 12000, 15000, 18000). The Chamfer Distance results for these tests shown in Fig. 10
19

---

Published as a conference paper at ICLR 2025
For each granularity setting, increasing the number of tetrahedra per TetSphere improved reconstruc- tion accuracy. However, the improvements are less significant compared to increasing the number of TetSpheres. While more tetrahedra per TetSphere can provide finer detail within single TetSphere, the primary determinant of reconstruction accuracy is the overall number of TetSpheres. This is because distributing the surface representation across larger number of TetSpheres allows for bet- ter coverage and adaptability to complex geometric features. Therefore, the number of tetrahedra within each TetSphere plays secondary role in the overall surface reconstruction quality.
C FORMULATIONS OF METRICS
Area-Length Ratio (ALR): The ALR is defined as following the definition of TriangleQ in Frey & Borouchaki [1999); Xu al. (2024a):
N ALR
6 A.
N 1=1
P.h.
where is the total number of triangles in the mesh, A; is the triangle area, is the half-perimeter, and h2 is the longest edge length. This metric ranges from 0 to 1, with higher values indicating that the mesh consists primarily of equilateral triangles, thus reflecting superior triangle quality.
Manifoldness Rate (MR): The MR checks whether the mesh is closed manifold. The MR is reported as the percentage meshes in the evaluation dataset that qualify as closed manifolds:
Number of manifold meshes
MR
x 100%.
Total number of meshes
This metric measures the difference in the num- ber of connected components between the reconstructed mesh and the ground-truth shape. Let Cl and (=(8) denote the number of connected components in the reconstructed and ground-truth meshes, respectively. The CC Diff. defined as:
Connected Component Discrepancy (CC Diff.):
CC Diff.
Σ c
c
N
which helps identify floating artifacts or discontinuities, indicating the structural integrity and cohe- sion of the reconstrueted shape.
D SIMPLE STUDY ON TETRAHEDRON INVERSION
In the inset figure (left), we provide a 2D illus- tration showing how inversion occur, with the inverted region highlighted in red. To fur- ther illustrate the importance of regular- ization in preventing tetrahedron inversion, we conducted simple study using target shape of a sphere that is smaller than the initial Tet- Sphere. As depieted in the inset figure (right), without regularization, the rendering loss alone pushes the surface vertices of the initial Tet- Sphere directly onto the target sphere's 's surface. This results significant number of inverted tetrahedra, highlighted in red. In contrast, with volumetric regularization in place, the recon- structed shape successfully adheres to the target surface without any inverted tetrahedra, ensuring a high-quality reconstruction.
Inial Taterkod
Tirget
invened tomsherr
Hoastaroatone
Karahanzaniies
E
ADDITIONAL RESULTS
E. MORE QUANTITATIVE RESULTS ON MULTI-VIEW RECONSTRUCTION
Table 5 shows comparisons of milti-view reconstruction evaluation metrics.
20

---

Published as a conference paper at ICLR 2025
Figure 11: Qualitative results single-vie view reconstruction. We use Wonder3d (Long et al. 2023 to
generate six multi-view images with predefined camera poses. The optimization objective includes rendering loss norm on tone-mapped color and MSE on the alpha mask) and normal loss (cosine loss on normals), following Munkkerg et al. (2022). More details are described in Sec. 5.]
E.2 QUALITATIVE RESULTS ON SINGLE-VIEW RECONSTRUCTION
Fig. 11 shows the results of single-view reconstruction performed on the GSO dataset. Our Tet- Sphere splatting demonstrates superior mesh quality, effectively capturing sharp geometric features, including the boundaries of the shoes.
E.3 QUALITATIVE RESULTS ON IMAGE-TO-33 GENERATION
Fig. 12 shows qualitative comparisons of image-to-3l generation with surface mesh visualizations and rendered normal maps. Our generated meshes exhibit minimal noise and high quality, featuring more regular triangle meshing.
Table 5: Additional results multi-view reconstruction.
Metric
NIE
FlexiCubes
2DGS
DMesh
Ours
F-Score f
0.486
0.502
0.632
0.605
0.653
Normal Consis.t
0.718
0.734
0.812
0.745
0.839
Edge Cham.
0.039
0,034
0.015
0.049
0.014
Edge F-Score
0.016
0.196
0,219
0.193
0.269
Time (sec.) 1
6436
57.67
691
1434
934
# Triangles
151,223.6
30,285.7
270,235.5
25,767.7
70,616.0
21

---

Published as a conference paper at ICLR 2025
Input
Magic123
DreamCraft3D
Ours
Figure 12: Qualitative comparison on image-to-3D generation with surface mesh visualizations and rendered normal maps. Our generated meshes exhibit less bumpiness and high mesh quality with more regular triangle meshing. We use multi-view images generated using the coarse NeRF fitting stage of DreamCraft3D, sampled on Fibonacci sphere, and optimize 2048 2048 texture image directly using rendering loss. More details are described in Appendix [
E,4
MORE RESULTS ON TEXT-TO-3D SHAPE GENERATION
Fig. 13 shows additional results of text-to-3D shape generation. These results highlight our method's
capability to construct complicated material, such as reflections, by leveraging the explicit geometry representation.
F TETRAHEDRON AND ITS DEFORMATION GRADIENT
Following Sifakis & Barbic, 2012). we treat a tetrahedron as piecewise linear element. The initial (undeformed) positions the four vertices a tetrahedron are denoted by X (X ( (3) X( with each ( e R3, Similarly, the positions of the four vertices of
deformed tetrahedron represented by [x))), x(2) x(3) x()),, where each x R The deformation gradient F E R3x3 which quantifies the local deformation of the tetrahedron, is given by:
m F D.D-1
(3)
D. [ (1) (4)
x (44)
x() (4) ], ( (4) (4) ].
(4)
D., ( (4)
(2) (4)
(5)
The deformation gradient essentially captures how a tetrahedron transforms from its initial state to its deformed state, encompassing both rotation and stretching effects.
22

---

Published as a conference paper at ICLR 2025
"An monster that looks like mushroom oom boss game character.
"A cauldron full of gold coins"
*A humanoid robot playing solitaire"
"A cake in the shape of train"
"A small cherry tomato plant in pot with a few red tomatoes growing on it"
statue of angel"
"A steam engine train"
"An amigurumi motorcycle"
panda rowing a boat a pond"
robot couple fine dining"
Figure 13: More results on text-to-3D shape generation. TetSphere splatting is optimized using rendering and SDS losses. The SDS loss for geometry is based on the Normal-depti diffusion model in Qiu et al. (2023), and loss for texture uses the albedo diffusion model from the same work. More details are described in Appendix !
G FORMULATION OF TETSPHERE INITIALIZATION
Assuming there are total of candidate positions obtained from the coarse voxel grid (as shown in Fig. 5), our goal with TetSphere initialization is to select a subset of these candidate points such that the object's shape is adequately covered by tetrahedral spheres centered at these positions. We
23

---

Published as a conference paper at ICLR 2025
first initialize sphere of fixed radius each candidate position, where the radius is calculated as or + B, where r is the minimum distance from each candidate position to the voxel surface. We use 0.07 in all our examples. The objective is to select a subset of spheres that collectively
= 1.2, 8 cover all voxel positions.
We define coverage matrix D {0, m| where each element dz {0, indicates whether voxel position j is covered by a sphere centered on candidate position i. A binary vector v E {0, 13 identifies selected candidate positions, with each element denoting the selection status of corresponding voxel positions. The selection of feature points is formulated as mixed-integer linear programming problem:
min vl s.t. Dv 11,
where |- is the 1 norm, 1 is a vector of ones. This optimization is efficiently solved using standard linear programming solvers.
H TEXTURE AND PBR MATERIAL OPTIMIZATION
Material optimization is facilitated by using differentiable rasterizers (Laine al. 2020), which adjust the textures or materials to closely match the input multi-view color images. A significant advantage of TetSphere splatting is that the deformation of tetrahedral spheres does not alter the sur- face topology. Unlike methods such as DMTet, which require isosurface extraction and subsequent texture parameterization at each step due potential changes in the underlying shape, our method necessitates only single texture parameterization at the beginning of optimization. This parameter- ization remains consistent throughout the process, significantly enhancing the efficiency of texture optimization.
For scenarios with dense input views, we have found that using textured images optimization variables is straightforward and yields high-quality results. In cases with sparse input views, we adopt two-layer MLP that takes the surface vertex positions as inputs and outputs the material parameters, a practice in line with existing methods (Sun al. 2023 Qiu et al., 2023).
I MORE IMPLEMENTATION DETAILS
When a surface mesh is required, we extract the surface triangles from each TetSphere to generate surface spheres, and then perform a union of all these surface spheres to obtain the final surface mesh. While there is no theoretical guarantee of manifoldness in the resulting mesh from union operation, our TetSphere optimization is highly regularized maintain geometric quality. As result, we did not encounter non-manifold issues in our experiments.
For image-to-3D shape generation, obtain multi-view images from the initial stage of Dream- Craft3D (coarse NeRF fitting only), generating 360 views sampled on a Fibonacci sphere (González) 2010). The texture optimization directly employs a 2048 2048 2D texture image, circumventing the need for additional neural networks. Here, the optimization objective (-)) focuses solely on the rendering loss. For text-to-3D generation, we use the initial stage of RichDreamer (Quu et al.) 2023) to obtain multi-view images. We optimize our TetSphere splatting using both the rendering loss and the SDS loss. The SDS loss for geometry calculated using the Normal-depth diffusion model as described in Qiu et al. 2023). The SDS loss for texture leverages the albedo diffusion model from the same work.
Regarding baselines: For image-to-3D generation, we compare TetSphere splatting with Magic 13 and Dreamcraft3D (Sun et al. 2023). Both are multi-stage methods starting with NeRF optimization followed by DM'eet to optimize mesh and texture. For text-to-3D, our comparison focuses on Rich- Dreamer, a state-of-fe ofate-of-the-art method known for incorporating PBR materials and generating shapes, showcasing notable results in 3D generation from text prompts.
J RELATED WORK TEXT-TO-3D CONTENT GENERATION
The recent success of text-to-2D image generation models has spurred a growing interest in gen- erating 3D output from text input. In light of the limited availablity of text-annotated 3D datasets,
24

---

Published as a conference paper at ICLR 2025
methods have been developed that leverage the pre-trained text-to-image models to reason between 2D renderings of 3D models and text descriptions. Early works (Khalid et al. 2022 Jain et 2022) adopt the pre-trained CLIP model (Radford et al. 202I to supervise the generation by aligning the
clip text and image embeddings. Recently, 2D diffusion-based generative models (Saharia et al,
22022 Rombach et al. (202)) have powered direct supervision in the image/latent space and ac achieved
superior 3D quality (Poole et al. [2022] Tang et al. 2023b [Cen et al. [023) Lin et al. 2023 Wang
al. 2023c). Notably, DreamFusion (Poole et al. 2022) introduces the Score Distillation Sampling
(SDS) for supervising the NeRF (Mildenhall et al., (2220) optimization using diffusion priors
score function (i.e.. by minimizing the added noise and the predicted noise under the text condition). Follow-up works have since been proposed to improve score sampling formula with Perturb-and- Average and variational method (Wang et al. 2223cc a), improved noise sampling schedules (Huang er al 2023), various 3D representations (L.in et al. [ (Tsalico et al.
[023 Chen et al. 2023), text prompts (Armandpour al. 2023). 3D consistency (Li ct al. 2023 Hong et al.) 2023a), and prior quality with dedicated diffusion models (Qiu al. [202]] Shi al. 2023). Our method lever- ages the Normal-Depth diffusion model used in (Qiu et al.] (023) for text-to-3D shape generation, which is trained on the large-scale LAION dataset, but replaces the geometry representation with our TetSphere.
K LIMITATIONS
While our representation demonstrates improvements in reconstruction quality, there are notable limitations that should be acknowledged. For example, the method only produces piecewise meshes that are overlaid, rather than solving the problem of globally fitting a single tet-mesh. Further- more, while our regularization helps prevent issues such tetrahedron inversion and maintains mesh integrity, it does not provide strong topological guarantees. The union operation of multiple TetSpheres can potentially alter the topology and may not fully preserve manifold properties. Ad- dressing these challenges to ensure global mesh consistency and topological guarantees could be valuable directions future work.
L DISCUSSIONS ON BI-HARMONIC ENERGY ON DEFORMATION GRADIENTS
The incorporation of biharmonic energy in our method is inspired by proven efficacy in geom- etry processing, highlighted by prior research (Botsch & Sorkine, 2007). Furthermore, penal- izing the monssmoohhness of transformations (or deformation gradient) across a domain is a well- explored area that has demonstrated its utility in various geometry processing tasks such shape deformation (Wang et al. 202 deformation transfer (Sumner & Popović 2004), and mesh align- ment Amberg al. (2007). For the surface regions where observations available, the surface deformation is primarily influenced by the rendering loss. Conversely, in regions lacking direct observations, our regularization term plays an important role by penalizing high-frequency defor- mations resulting from non-smooth deformation gradients. This approach ensures reliable results.
The biharmonic energy offers stronger regularization compared to first-order harmonic energy, as demonstrated in (Botsch & Sorkine (2007). Although theoretically higher-order smoothness ener- gies can be considered, in practice, they often lead to numerical instability, as discussed Jacobson & Panozzo (2017). Therefore, biharmonic energy strikes balance by providing effective regular- ization while avoiding numerical issues.
25


Reviews:
[{"review_id": "BsXPIobOit", "reviewer": "Reviewer_AA94", "summary": "This paper proposes a method for reconstructing 3D geometry from multiview images. The geometry is represented by Tetrahedral meshes of sphere topology that are optimized in standard fashion w.r.t a rendering loss, along with two geometric regularizes - the biharmonic energy to ensure a smooth deformation, and a barrier term that prevents tetrahedra from inverting.", "questions": "- can you explain why did you choose to not compare to [Nicolet et al. 2021] and methods that adopted it later? \n- you do not cite \"Continuous remeshing for inverse rendering\" - how do you think your method would fair in respect to it?\n- are there other advantages to using a tetrahedral mesh as opposed to triangle, except the two regularization losses?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper revisits the idea of using meshes for 3D reconstruction, and reaffirms that an explicit representation is extremely efficient in producing high quality 3D geometry from images. The paper is written in a clear way and is easy to understand.", "weaknesses": "1. My main concern with the paper is its novelty and contribution: the paper essentially proposes to deform a mesh w.r.t a visual loss which is standard (e.g., [Nicloet et al. 2021]), while using two standard regularizes to guarantee a \"good\" deformation (smooth and without inversions). This is a very standard approach. The only part that seems less explored is the use of tetrahedra instead of triangles, however this of course has been explored extensively outside of the context of differentiable rendering, hence the only part I can deem truly novel is \"using tetrahedra instead of triangles in tandem with multiview reconstruction\". This feels like a marginal contribution. It could be argued to be a very practical approach which may convince me to champion it, however that leads me to my second concern:\n2. Evaluation seems somewhat selective, and the methods compared to do not strike me as the immediate alternatives. Namely, I find [Nicolet et al. 2021] as a main contender, as well as \"Continuous remeshing for inverse rendering\" which is uncited. Many other techniques that cite [Nicolet et al. 2021] can be considered. \n3. The argumentation for the use of a volumetric mesh instead of a triangle mesh needs to be justified further through experiments. It seems the only argument for a full discretization of the *volume* (as opposed to the surface via a triangle mesh), is to regularize the volumetric deformation with the two losses. This seems like a significant overkill, as simpler regularizers could be employed on the triangle mesh (again, e.g., Nicolet et al. 2021) to ensure it behaves well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WlFkY6PGAd", "reviewer": "Reviewer_hHtw", "summary": "The paper proposes a shape reconstruction method that takes a set of images as input and fits a 3D model with reflectance properties to them. In line with a large body of recent work, it combines a rendering loss with shape regularization to directly optimize a 3D scene representation by \"simple\" numerical descent. The problem is particularly challenging as the paper uses a \"Lagrangian\" representation, specifically, a tetrahedral mesh that co-moves with the surface points obtained.\n\nAs recent work, such as [Nicolet 2021] has pointed out, the success of such an approach critically hinges upon suitable regularization, as the rendering loss is ambiguous and it is easy to get stuck in a bad local minimum with nonsensical mesh structures (which has been an issue for a long time in attempting such direct shape optimization against relatively weak data constraints).\n\nThe key contribution of this paper is to use a volumetric representation, a tet-mesh, with a penalty against inversion of elements along with an incrementally applied smoothness regularizer (a bi-harmonic energy applied to deformation gradients of the current mesh) that promotes changes in low-frequency shape first.\n\nThe paper provides very convincing results and shows a number of interesting applications, some of which only made it into the appendix.", "questions": "When does the method break? At which point does the proposed machinery fail to fit a reasonable geometry to data, and how do the problems look like / introduce themselves? Which conceptual or technical challenges would be needed to overcome to fix this?\n\nDoes the method have fundamental advantages over previous representations? If so, can you make a formal or experimental argument to convince the reader? If the differences are more nuanced, what are the key trade-offs one has to make here?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "I think that the main strength of the paper is the idea itself: Regularization is important to avoid bad mesh structure, and a volumetric representation intuitively gives much more leeway to regularization over a pure thin-shell / surface model (the inductive bias of this being a solid is stronger than regularizing details on thin surface). The proposed setup with inversion constraints and smoothness looks plausible and seems to work very well in practice. I should add that I am not actively working in this area, so I might miss some related work; but if this kind of approach has not yet been tried, the approach itself seems worth publishing.\n\nThe results are also convincing in terms of quality and versatility. The paper is very well written, nicely illustrated and enjoyable to read. Comparisons against some related recent method also underline the quality of the results quantitatively.", "weaknesses": "In my opinion, the main downside is that the paper is rather light on analysis: Why does it use the bi-harmonic energy on deformation gradients? What does this actually mean (application to derivatives should create higher-order smoothness, or in terms of a Fourier-perspective, add an additional high-pass filter in front of the regularizer)? What happens, if other regularizers are used? How important is the non-inversion term, and how brittle are parameter choices? It would also be interesting to discuss the relation to other attempts at stricter regularization, such as Nilcolet et al.'s work. I would guess that the volumetric approach is strictly superior, if fully exploited; showing something like this experimentally would make the paper much stronger.\n\nSome of the limitations could also been discussed more clearly. For example, the paper only produces piecewise meshes, which are overlayed; it does not solve the problem of globally fitting a single tet-mesh. Comparisons in mesh quality should take this into account. Further, this also shows implicitly that the method is still to brittle to handle strong deformations during optimization; otherwise, one could just fit a single sphere globally. In this context, as well as in general, it would be instructive to show how to break the method by slowly stepping outside its \"convergence radius\" where good results can be obtained into cases where results are unsatisfactory. My criticism here is not that there are limitations but that the paper could be strengthened by studying them more in depth.\n\nFinally, the paper seems like a better fit to a conference on computer vision or graphics in terms of methods and problems, but I would consider the topic close enough not to exclude it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z2fkNUoMP7", "reviewer": "Reviewer_zZTV", "summary": "The manuscript introduces a framework named TetSphere Splatting, which can be used to reconstruct 3D meshes from multi-view images. TetSphere uses tetrahedron spheres to represent an object. During optimization, deformation field is predicted to relocate vertex points to minimize cost. Experiments in multi-view reconstruction, image/text to 3D shows potential applications of TetSphere Splatting.", "questions": "-\tWhen supervised purely based on depth/mask, the interior TetSphere not visible to any images will not receive gradients, except for regularization. Is this the main reason manifoldness is maintained?\n\n-\tMosaic SDF [a] is one missing work that also follows the Lagrangian framework, where a set of volumes are moved in space. \n\n[a] Yariv, L., Puny, O., Gafni, O., & Lipman, Y. (2024). Mosaic-sdf for 3d generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4630-4639).\n\n-\tClarification needed, below variables are not properly defined:\n - L265: what is N?\n - L266: what is T?\n - L309: what is n?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-\tThe problem is of interest to the research community.\n-\tThe formulation of TetSphere Splatting is novel, and an interesting theoretical perspective from prior work.\n-\tThe experiments setup is diverse, including many applications.", "weaknesses": "Overall, the paper is quite interesting. However, my concern about the paper is its claim, where experimental results cannot back up.\n\n-\tTetSphere is claimed to have superior surface quality due to the fact Marching Cubes is not needed. However, all qualitative results seem to suggest that the TetSphere needs to be sufficiently dense to produce detailed structures. This leads to unnecessarily dense triangles, which can also be achieved by very dense marching cubes resolution. What is the advantage in this case?\n\n-\tMoreover, while the manifoldness of TetSphere is guaranteed, the baselines compared are all Eulerian representations without sufficient regularization. Approaches such as NeuS and VolSDF use volume rendering to achieve manifold meshes. While I understand the faster optimization process aspect, can authors comment on TetSphere’s advantage over NeuS/VolSDF from the quality perspective?\n\n-\tLastly, the claimed superior surface quality is not backed up by smaller Chamfer Distance, such as in Table 2. Can authors provide more explanation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fZQz24WP6W", "reviewer": "Reviewer_u1pH", "summary": "This paper presents a Lagrangian geometry representation based on TetSpheres, volumetric tetrahedral spheres that deform to fit the desired geometry. Key applications of TetSphere splatting are demonstrated in monocular and multiview reconstruction. The deformation of TetSpheres is formulated as an energy optimization problem with geometric constraints that prevent the generation of irregular surfaces. Experiments are proposed to compare TetSpheres with state-of-the-art methods based on both Eulerian and Lagrangian geometry representations.", "questions": "The proposed optimisation does not appear to prevent non-uniform deformation of TetSpheres. However, the resulting reconstruction Aspect Loss Ratio (ALR) is higher than in other methods, suggesting that the tetrahedra within each TetSphere largely preserve their original volume. Is there any factor specifically contributing to the high ALR?\n\n\nWhat are the values of N (number of tetrahedra) and M (number of TetSpheres) used in the experiments? How does the trade-off between these values affect reconstruction quality and computational complexity?\n\nMore details on the image-to-3D and text-to-3D generation pipelines are needed to ensure the experiments can be reproduced.\nFigures 8 to 10 would benefit from captions with additional information about the experimental setup.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper is generally well-written, conveying complex concepts and methods in a concise style. \n\nThe state-of-the-art is clearly explained and appears to be up-to-date.\n\nThe proposed primitive is original and effectively addresses common issues in other Lagrangian geometry representations, such as irregular triangles, non-manifoldness, and floating artifacts.\n\nThe energy optimization process for splatting is well-designed and effective, with a convincing initialization algorithm.\n\nThe accuracy in monocular and multiview reconstruction applications is competitive with current methods while achieving higher surface quality. This approach is also lighter than other Lagrangian representations in terms of memory and computational complexity.", "weaknesses": "The paper proposes a new Lagrangian primitive that is notably more complex than the primitives used in previous methods, such as 3D Gaussians or triangles. Each TetSphere consists of a collection of N tetrahedra, with an apparent trade-off between N and the total number of TetSpheres needed to represent the surface. This trade-off seems to be overlooked in the experimental section, as specific values for the number of TetSpheres and tetrahedra in each experiment are not provided.\n\nThe surface is represented as a union of TetSpheres. Due to the proposed splatting optimization, TetSpheres may intersect, leading to artifacts at the intersection points. However, the paper does not specify how intersections between tetrahedra are managed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a80KIFbJg3", "reviewer": "Reviewer_fnQo", "summary": "This paper uses tetrahedral spheres to represent three-dimensional shapes and constructs its corresponding differentiable inverse rendering process, which improves tasks such as single-view, multi-view reconstruction, and text to 3d shapes.", "questions": "My main doubts lie in the topological and manifold guarantees claimed by the authors, and I hope to see more detailed analyses and experiments to prove their method. I am also curious about the mesh extraction method and the number and quality of triangles. And if the code is attached, it will also increase the credibility of this paper.\n\nOverall, I like this idea, but it seems to have a lot of minor issues. If the authors can address these issues during rebuttal, I will increase my rating.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I think it’s a good idea to use tetrahedrons to represent shapes, which is very common in classical geometry processing. I’m excited to see that the authors were able to combine computer vision and geometry processing and make good progress on multiple tasks.\n\nThis new method surpasses or catches up with Sota methods on multiple data sets, and because it is based on the tetrahedron geometric primitive, it seems to be able to achieve better topology preservation, manifold preservation and better triangulation quality.", "weaknesses": "The implementation details of this paper are not enough for me, and many questions come to my mind when I read this paper.\n\n1. The author does mention the problem of tetrahedron inversion, and cites theories from other papers to show that it is useful. I think it would be easier to understand if there is a simple ablation experiment.\n2. The paper does not seem to mention how to extract the surface mesh from the tetrahedral mesh, which I think is also an important part of the whole pipeline. At the same time, how to ensure that there is no non-manifold when extracting the surface is also a problem. \n3. How to ensure that the topology is correct? If I understand correctly, this seems to be strongly related to \"silhouette coverage\", which must be consistent with the topology of GT. I would like to see an analysis of the Euler characteristic (Genus) in the tables.\n4. This paper introduces many metrics to evaluate the quality of the mesh, but no corresponding references or mathematical definitions are given, like Manifoldness Rate and Connected Component Discrepancy. I hope to see their mathematical definitions in the appendix, which will make it easier for readers to understand. \n5. Regarding mesh quality, first of all, do all compared methods have similar resolution/triangle numbers? If mesh quality is to be compared, giving the triangle numbers is a must. It would be great if the authors could provide a metric for triangle quality, such as the Edge Chamfer Distance in NMC [Chen and Zhang. 2021] and TriangleQ in CWF [Xu et al. 2024].\n6. In addition, since your initialization is multiple tetrahedralized spheres, what is the connection between the spheres? Are they multiple independent connected branches? If so, how to deal with the intersection between them, and how to ensure that the final mesh is an independent and connected one?\n7. Citation problem, I found multiple citations of same paper in several places, such as One-2-3-45 line 648-653, Syncdreamer line 657-662 etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)