DMesh: A Differentiable Mesh Representation
Authors: Sanghyun Son
Matheus Gadelha
Yang Zhou
Zexiang Xu
Ming Lin
Yi Zhou
Venue: NeurIPS 2024 poster (2024)
Keywords: Differentiable Mesh
3D Reconstruction
Primary area: machine_vision
TL;DR: We present a differentiable representation, DMesh, for general 3D triangular meshes. It considers both the geometry and connectivity information of a mesh in a differentiable manner.

Abstract: We present a differentiable representation, DMesh, for general 3D triangular meshes. DMesh considers both the geometry and connectivity information of a mesh. In our design, we first get a set of convex tetrahedra that compactly tessellates the domain based on Weighted Delaunay Triangulation (WDT), and select triangular faces on the tetrahedra to define the final mesh. We formulate probability of faces to exist on the actual surface in a differentiable manner based on the WDT. This enables DMesh to represent meshes of various topology in a differentiable way, and allows us to reconstruct the mesh under various observations, such as point clouds and multi-view images using gradient-based optimization. We publicize the source code and supplementary material at our project page (https://sonsang.github.io/dmesh-project).

DMesh: A Differentiable Mesh Representation
Sanghyun Son'
Matheus Gadelha² Yang Zhou² Zexiang Xu² Ming C. Lin'
Yi Zhou
'University of Maryland, College Park, 4shh1295, 1rllinnnuumm edu Adobe Research, {gadelha, yaziol,,,,aauu,auaaaa y Cadobe. com
Step 30
Step 2000
Step
Step 2
Step
Step
(a) Random initialization
(b) Sample point initialization
Figure 1: (-) Optimization process. We can optimize our mesh starting from either (a) random state (b) initialization based on sample points for faster convergence. Mesh connectivity changes dynamically during the optimization. To make this topology change possible, we compute existence probability for arbitrary of faces in differentiable manner.
(a) Non-convex polyhedra
(b) Non-orientable geometry
(c) Protein
Figure 2: Versatility of DMesh. DMesh can represent diverse geometry in differentiable manner, including (a) non-convex polyhedra different Fuler characteristics, (b) non-orientable geometries (Möbius strip, Klein bottle), and (c) complex protein structure (colored for aesthetic purpose).
Abstract
We present a differentiable representation, DMesh, for general 3D triangular meshes. DMesh considers both the geometry and connectivity information of a mesh. In our design, we first get a set of convex tetrahedra that compactly tessellates the domain based Weighted Delaunay Triangulation (WDT), and select triangular faces on the tetrahedra to define the final mesh. We formulate probability of faces to exist the actual surface in differentiable manner based on the WDT. This enables DMesh to represent meshes of various topology in a differentiable way, and allows such as point clouds and multi-view images using gradient-hee based optimization. We publicize the source code and supplementary material at project page
reconstruct the mesh under various observations,
htttss///oosaagggiiibb.dddeeee htwp:luub..iidmmse--pppppoooo project
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Introduction
Polygonal meshes are widely used in modeling and animation due to their diverse, compact and ex- plicit configuration. Recent AI progress has spurred efforts to integrate mesh generation into machine learning, but challenges like varying topology hinder suitable differentiable mesh representations. This limitation leads to reliance differentiable intermediates like implicit functions, and subsequent iso-surface extraction for mesh creation (Liao u. a. 2018 Guillard u. a. [021] Munkberg u. a. [002]] Liu u. a. 2023b). However, meshes generated by such approaches can be misaligned at sharp regions and unnecessarily dense (Shen u. a. 2023), not suitable for down-stream applications that require light-weight meshes. This limitation necessitates to develop truly differentiable mesh representation, not the intermediate forms.
Shen u. a.
2023 2021
The fundamental challenge in creating differentiable mesh representation lies in formulating both the vertices' geometric features and their connectivity, defined as edges and faces, in a differentiable way. Given vertex set, predicting their connectivity in free-form way using existing machine learning data-structures can cost significant amount of computation and be difficult to avoid irregular and intersecting faces. Consequently, most studies differentiable meshes simplify the task by using a mesh with a pre-determined topology and modifying it through various operations (Zhou 2020 Hanocka
2019 Palfinger 2022; Nicolet a. 2021). This work, on the contrary, ymbitiously aims to establish a general 3D mesh representation, named as DMesh, where both mesh topology and geometrie features (e.g. encoded in vertex location) can be simultaneously optimized through gradient- based techniques.
Our core insight is to use differentiable Weighted Delaunay Triangulation (WDT) divide a convex domain, akin to amber encapsulating a surface mesh into tetrahedra to form a mesh. To create a mesh with arbitrary topology, we select only a subset of triangular faces from the tetrahedra, termed the "real part", as our final mesh. The other faces, the "imaginary part", support the real part but not part of the final mesh (Figure 4.. We introduce method to assess the probability of face being part of the mesh based on weighted points that carry positional and inclusiveness information. Optimization is then focused the points' features generate the triangular mesh. The probability determination allows us to compute geometric losses and rendering losses during gradient-bar based optimization that optimizes connectivity and positioning.
The key contributions of our work can be summarized as follows.
We present novel differentiable mesh representation, DMesh, which is versatile accom- modate various types of mesh (Figure 2 The generated meshes can represent shapes more effectively, with much less number of vertices and faces (Table
We propose a computationally efficient approach to differentiable WDT, which produces robust probability estimations. While exhaustive approach (Rakotosaona u. a. [021 requires quadratic computational cost, our method runs in approximately linear time.
We provide efficient algorithms for reconstructing surfaces from both point clouds and multi-view images using DMesh as an intermediate representation.
We finally propose effective regularization term which can be used for mesh simplification and enhancing triangle quality.
Additionally, further accelerate the algorithm, we implemented main algorithm and differen- tiable renderer in CUDA, which is made available for further research.
2
Related Work
2.1 Shape Representations for Optimization
Recently, using neural implicit functions for shape representation gained popularity in graphics and
vision applications (Mildenhall u. a. 2021 Zhang u. a. 2020 Liu u. a. 20200 Chen u. a. [022a 2023 Wang u. a. 2021 Yariv u. a. 2020). They mainly use volume density, inspired by (Mildenhall u. a.
2021 to represent& shape. However, because of its limited accuracy in 3D surface representation.
neural signed distance functions (SDFs) (Yariv u. a. 2202T] Wang u. a. 202T| 2023) Oechsle u. a.
202 or unsigned distance functions (UDFs) (Liu 2023a Long u. 2023 often preferred.

---

Reconstraction
Loss
Awat Dreat
Mesh
Point Cloud
Malti
3ras Mgy May Mize
Paa Pex P(F)) i Rez P(Fe))
(c)
N Points
Faces
Weight Reg.
Reg.
Quality Reg.
Regularizaion
Peation warge Real
Poleindom
Editence Frobest
(d)
(a)
(b)
Figure 3: Our overall framework optimize mesh according the given observations. (a): Each point is defined by 5-dimensional feature vector: position, weight, and real value. Points with larger real values are rendered in red. (b): Given set of points, we gather possibly existing faces in the mesh and evaluate their probability in differentiable manner. (c): We can compute reconstruction loss based on given observations, such as mesh, point cloud, multi-vier images. (d): To facilitate the optimization process and enhance the mesh quality, we can use additional regularizations.
After optimization, one can recover meshes using iso-surface extraction techniques Lorensen und Cline 1998 Ju u. a. 2002).
Differing from neural representations, another class methods directly produce meshes and optimize
them. However, they assume that the overall mesh topology is fixed (Chen a.
2019
Nicolet
a. 2021] Liu u. a. 2019 Laine u. a. (020), only allowing local connectivity changes through
remeshing (Palfinger 2022) Learning-basee approaches like BSP-Net (Chen
2020) allow
topological variation, but their meshing process is not differentiable. Recently, differentiable iso- surface extraction techniques have been developed, resulting in high-quality geometry reconstruction
of various topologies (Liao u. a. 2018: Shen u. a. 2021 2023;
Wei u. a. 2023 Munkberg u. a.
2022 Liu u. 2023b; Mehta u. a. 2022) Unfortunately, meshes relying on iso-surface extraction
algorithms (Lorensen und Cline 1998] Ju u. a. 2002) often result in unnecessarily dense meshes that
could contain geometric errors. In contrast, our approach addresses these issues: we explicitly define faces and their existence probabilities, and devise regularizations that yield simplified, but accurate meshes based on them (Table See Table for more detailed comparisons to these other methods.
2.2 Delaunay Triangulation for Geometry Processing
2013) has been proven to be useful for recon- structing shapes from unorganized point sets. It's been shown that DT of dense samples on smooth 2D curve includes the curve within its edges (Brandt und Algazi 1992, Amenta u. a. 1998a). This idea of using DT to approximate shape has been successfully extended to 3D, to reconstruct three- dimensional shapes (Amenta a. 1998b) for point sets that satisfy certain constraints. However, these approaches are deterministic. Our method can be considered as a differentiable version of these approaches, which admits gradient-base based optimization.
Delaunay Triangulation (DT) (Aurenhammer
More recently, Rakotosaona r a. (2021) focused on this DT's property connect points and tessellate the domain, and proposed differentiable WDT algorithm to compute smooth inclusion, namely existence score of 2 2-simplexes (triangles) in dimensional space. However, it is not suitable apply this approach to our 3D case, as there are computational challenges (Section 3.2) Other related work, VoroMesh (Maruani u. a. 2023), also used Voronoi diagrams in point cloud reconstruction, but their formulation cannot represent open surfaces and is only confined to handle point clouds.
Preliminary
3.1 Probabilistic Approach Mesh Connectivity
To define traditional, non-differentiable mesh, we specify the vertices and their connectivity. This connectivity discrete, meaning for any given triplet of vertices, we check they form a face in the mesh, returning if they do and 0 otherwise. To overcome this discreteness, propose

---

(a) 2D Font
(b) 3D Dragon
Figure 4: Illustration of our mesh representation for 2D and 3D cases. (a): Our representation in 2D for a letter "A". (b): Our representation in 3D for a dragon model. Blue faces are "real part" and yellow ones "imaginary part".
probabilistic approach to create fully differentiable mesh given a triplet of vertices, we evaluate the probability of face existing. This formulation enables differentiability not only of vertex positions but also of their connectivity.
Note that need procedure that tells the existence probability of any given face realize this probabilistic approach. This procedure must be 1) differentiable, 2) computationally efficient, and 3) maintain desirable mesh properties, such avoiding non-manifoldness and seff-intersections, when
determining the face probabilities.
Among many possible options, we use Weighted Delaunay Triangulation (WDT) (Figure 6 a)) and point-wise feature called the "real" value (17) to define our procedure. Each vertex our framework represented as a 5-dimensiona[ vector including position (3), WDT weight (I), and real value (I) (Figure (a)).. Given the precomputed WDT based on vertices' positions and weights, we check the face existence probability of each possible triplets. Specifically, 1) face F must exist in the WDT, and then 2) satisfy condition on the real values of its vertices to exist on the actual surface. We describe the probability functions for these conditions as Awdt and Areal:
Awas (F) = P(F E WDT), Areal = P(F E Mesh FF WDT).
(1)
Then we get the final existence probability function, which can be used in downstream applications (Figure B), as follows:
A(F) = P(F E Mesh) = Awwde (F)) Areat (F).
This formulation attains nice property in determining the final mesh that is, it prohibits self-intersections between faces. When it comes to the other two criteria about this procedure, Awat function's 's differentiability and efficiency is crucial, differentiable function based on real values (Section (.2).. Thus, we first introduce how we can evaluate Awdt in differentiable and efficient manner, which one of main contributions.
design Ayeal to be very efficient
3.2 Basic Principles
k = d
==d-1
To begin with, we use (d, k) pair to denote a k-simplex (AF) in d-dimensional space. For 3D mesh, we observe that our face F corresponds to (d = 3, k = 2) in Fjg To compute the prob- ability Audt for the face, we use power diagram (PD), which is the dual structure of WDT (Figure ((a).. While previously Rakotosaona al. (2021) proposed differentiable 2D triangulation method for the (d (d= k = 2) case (Figure ()))), it suffers from quadratically increasing computational cost (e.g. it takes 4.3 seconds to process 10K points in 2D, Table and unreliable estimation when (k < d).
(b)
2=
We will discuss later how our formulation conquer these compu- tational challenges. Our setting for 3D meshes are similar to 2D meshes, the (d 2, k 1) case in Figure(5)d), where triangular face reduces to line. Therefore, we will mainly use this setting to describe and visualize basic concepts for simplicity. However, note that it can be generalized to any (d, k) case without much problem.
(0)
(d)
Figure 5: Renderings of A for different pairs of (d, k). Different A are rendered in different colors.
In 2D case, vertex is dimenssionl vector including position (2), WDT weight (1), and real value (1).

---

P
P
D7
P
P1
p2
p
Pa
p
p
D5
Plo
(c) Reduced power cell.
(d) Reduced power cell, A = (PI.Pr)
(a) WDT and PD
(b) Power cell, A = (pI.p2)
A [PL.P2}
Figure 6: Basic concepts to compute existence probability of given 1-simplex (A when d = 2. (a): WDT and PD given set of weighted vertices are rendered solid and dotted lines. The size of vertex represents its weight. (b): Power cell of p1 (Cpz) is rendered in grey. Also, A is rendered in black line, of which dual line (Da:) is rendered in red. (c), (d): For given A reduced power cell of P1 for the A (Rp, (A1) is rendered in blue, with the original power cell (grey). We can evaluate the existence of A in WDT by computing the signed distance from D to RpilA
We generalize the basic principles suggested by Raaotosaona u. formal definitions of the concepts in this section, please refer to Appendix
2021 address cases. For
Let S be a finite set of points in Id== with weights. For a given point E S, we denote its weight as Wp. We call those weighted points in S as "vertices", to distinguish them from general unweighted points in 2 Then, we adopt power distance x (p. q) d(p, q)² Wp - Wg as the distance measure between two vertices S. As depicted in Figure A/=l a I-simplex, which a line connecting R3
two vertices pr and Pj in S. Its dual form D (red line) is the set of unweighted point: in R that are located at the same power distance to p4 and pj- the power diagram, the power cell C, (gray cell) of vertex p is the set of unweighted points that are closer to p than to any other vertices in S. We can use Cp and DA to measure the existence of A From Figure 6. we can see that when A {p: exists in WDT, its dual line DA aligns exactly with Cp. boundary, while when
A Pj. doesn't exist in WDT, DA 1 is outside Cp.
To make this measurement less binary when A exists, we use the expanded version of power cell called "reduced" power cell (Rpla), introduced by |Rakotosaona a. (202T). The reduced power cell of p; E for A = {p computed by excluding Pj from when constructing the power cell For example, when A exists, AA will expand towards Py direction (Figure c)). and D will "go through" At In contrast, when A doesn't t exists, even though we have removed Pj. Rp.14 will not expand (Figure 6 d)), and thus DA stays outside of Rp.
Now
define signed distance field (x, R) for given reduced power cell R, where the signed distance is measured the distance from the point R to the boundary of the reduced power cell (sign is positive when inside). Then, we can induce the signed distance between a dual form D and reduced power cell R:
T TD, R) = max (I. ZED
(3)
As illustrate in Figure and (d), 7 (DAi, Rp, 1) is positive when A exists, while negative when it does not exist in WDT. This observation can be generalized as follows:
Remark 3.1. A exists in WDT if and only if Vp, E A A TT(("= Rp.la)) > 0.
In fact, the sign of every T(DAa, Rp.1a*) average measure the existence probability of Ak. along with sigmoid function
(A)) is same for every P: E A Therefore, we can use its
Awode (A
Σ a(T(Ds.
) Olwddt))
(4)
(E(A*)
where Owdt is a constant value used for the sigmoid function. Awde((A) ) is greater than 0.5 when A exists, aligning with our probabilistic viewpoint and being differentiable.
We treat unweighted points' weight as when computing the power distance.
In 3D case where = 2 and A {pi, Pj, Pp.pp Pj and pa would be ignored for Rp,1
5

---

Computational Challenges. As mentioned before, Rakotosaona et al. (2021) solved the problem for the case where (d = 2) where the dual Da is single point. Naïvely applying their
approach for computing Eq. 3 to our cases poses two computational challenges:
Precision: When (d = 3, k = 2) (d = 2. k 1), Da, becomes line, not single point.
Finding a point on the line that maximizes Eq. 3 is not straightforward.
Efficiency: When naively estimating the reduced power cell in exhaustive manner, the computational cost increases with the number of points at rate of O(N2), where N is the number of points.
See Appendix B.2 for a detailed discussion of these limitations.
4 Formulation
4.1 Practical approach to compute Awdt
We introduce practical approach to resolve the two aforementioned challenges. Specifically, propose constructing the PD first and use it for computing lower bound of Eq. in an efficient way. We also propose handle the two cases separately: whether A cxists in the WDT or not.
First, when the simplex A k does not exist, we choose to use the negative distance between the dual form and the normal power cell, -d(D Cp). This is the lower bound of Eq. 3
-d(Dat, Cp) S T(D Rpja*) < O,
(5)
Cp C. Ryl See Figure d( for this case in (d = 2, 1) case. We can observe that computing this distance is computationally efficient, because the normal PD) only needs to he computed once for all in advance. Moreover, Cp is convex polyhedron, and Da* is (convex) line, which allows to find the distance between line segments the boundary of C and DA,, and choose the minimum.
Second, we analyze the case when the simplex A exists in WDT. In this case, we have to construct the reduced power cell RyJa* * for given A which requires much additional cost. Instead of doing it, we leverage pre-computed PD to approximate the reduced power cell. Then, we pick a point E DA* n RplA*, where p E {A*}. and the following holds:
S (p Rp)a,) S R(D)),Rpa+))
(6)
because v E RyIA* and by the definition at Eq. 3 In our case, since D3. RpIs+ is a line segment, choose its middle point to tighten this lower bound. See Figure[[cc) for the line segment in (d = 2, k = 1) case. We use this bound when A exists. Note that computing this lower bound is also computationally efficient, because we can simply project v to the reduced power cell.
To sum up, we can rewrite Eq. 3 as follows.
T(D Rag.R Ra)a)) = (u RAIA=) -d( C2) else
if A E WDT
By using this relaxation, we can get lower bound of Eq. 3 which is reliable because it always has the same sign. Also, we can reduce the computational cost from ((N2) to nearly O(N), which is prerequisite for representing meshes that have more than 1K vertices general. See Appendix [B.2 for the computational speed and accuracy of our method, compared to the previous one. Finally, implemented our algorithm for computing Eq. 7 in CUDA for further acceleration.
4.2 Definition of Areal
Areat evaluates the existence probability of a k-simplex A in our mesh when it exists in WDT.
To define it, we leverage per-point value t 10. To be specific, we compute the minimum
of the points in A in differentiable way: Areat
Epesr k(p) T (p), where x(p) Egease" 3---((a), and V is function that maps point p to its & value. We set 3 = 100.
e (p)(())
Along with Awet that we discussed before, now we can evaluate the final existence probability of faces in Eq. [ We also note here, that when we extract the final mesh we only select the faces of which Amnt and Areat are larger than 0.5.
Thiis holds when d = 3. When d = 2, we use vertices of of Cp.

---

4.3 Loss Functions
DMesh can be reconstructed from various inputs, such as normal meshes, point clouds, and multi- view images. With its per-vertex features and per-face existence probabilities A(F), we can optimize it with various reconstruction losses and regularization terms, Please see details in Appendix
4.3.1 Reconstruction Loss (Lrecon)
First, if we have normal mesh with vertices I' and faces F, and we want to represent it with DMesh, we should compute the additional two per-vertex attributes, WDT weights and real values. We optimize them by maximizing ((F) since these faces lies on the reference mesh. Conversely, for the remaining set of faces IF that can be defined on P. we should minimize A (F). Together, they define the reconstruction loss for mesh input (Appendix C.I
For reconstruction from point clouds or multi-view images, we need to optimize for all features including positions. For point clouds, we define our loss using Chamfer Distance (CD) and compute the expected CD using our face probabilities (Appendix C.2). For multi-view images, we define the loss as the L1 loss between the given images and the rendering of DMesh, interpreting face probabilities as face opacities. We implemented efficient differentiable renderers allow gradients to flow across face opacities (Appendix C.3
4.3.2 Regularizations
Being fully differentiable for both vertices and faces, DMesh allows us to develop various reg- ularizations to improve the optimization pro- cess and enhance the final mesh quality. The first is weight regularization (Lweeigtt). applied to the dual Power Diagram of the WDT (Ap- pendix C.4) This regularization reduces the structural complexity of the WDT, controlling the final mesh complexity (Figure 7). The next real regularization (Lreat), which enforces nearby points to have similar real values and increases the real values points adjacent to high real value points (Appendix C.3). This helps remove holes or inner structures and makes faces the current surface more likely to considered (Appendix [D)
(6) Aweeghe 10
(b) Aneiate 10
(4) Aweight 10
Figure 7: Results with different Aweight
The final regularization, quality regularization (Iguat), aims to improve the quality of triangle
faces by minimizing the average expected aspect ratio of the faces, thus removing thin triangles (Appendix C.6))
To sum up, our final loss function can be written as follows:
L = Lrecon + Aweight
Lweiihtt + Aveal Lyeat + Aqual Liuall
where A values are hyperparameters. In Appendix we provide values for these hyperparameters for every experiment. Also, in Appendix E.3 we present ablation studies for these regularizations.
5 Experiments and Applications
In this section, we provide experimental results to demonstrate the efficacy of our approach. First, we optimize vertex attributes to restore given ground truth mesh, directly proving the differentiability of our design. Next, we conduct experiments on 3D reconstruction from point clouds and multi-view images, showcasing how differentiable formulation can be used in downstream applications.
Table 1: Mesh reconstruction results.
Bunny
Dragon
Buddha
RE
99.78%
99.72%
99.64%
FP
0.00%
0.55%
0.84%
For the mesh reconstruction problem, we used three models from the Stanford 3D Scanning Reposi- tory (Curless und Levoy 1996). For point cloud and multi-view reconstruction tasks, we used four closed-surface models from the Thingi32 dataset, four open-surface models from the Da dataset, and three additional models with both closed and open surfaces from the Objaverse dataset
7

---

Closed
Open
Mixed
(a) GT
(b) Ours
(c) PSR (d) VoroMesh (e) NDC
( Ours (g) FlexiCube (h) NIE Muet Reconstruction
Point Cloud Reconstruction
Figure 8: Point cloud and multi-view reconstruction results. (a): Ground truth mesh. (b), (f): Our
method restores the original shape without losing much detail. (c), (d), (g), (h): PSR (Kazhdan und Hoppe 2013), VoroMesh (Maruani a. 2023), FlexiCube (Shen 2023). and NIE (Mehta a. 2022 fail for open and mixed surfaces. (e): NDC (Chen u. a. 2022b) exhibits artifacts from grids.
Table 2: Quantitative comparison for point cloud and multi-view reconstruction results. Best results are written bold.
Methods
CD (x 10
Fif
NCT
FCD2
EFI
# Verts.
Faces.
Time (sec))
PSR
0.770
0.931
0.209
0.129
159K
319K
10.6
VoroMesh
>1K
0.671
0.819
>1K
0.263
121K
242K
12.2
20.7K
Ours (Wio normal)
3.7%6
0.866
0.936
0.067
0.342
387K
10.4K
773
Ours (w/ normal)
3,364
0.886
0.952
0.141
0.438
3,56K
7.54K
743
NIE
0.848
0.064
MV
FlexiCube
273
0.591
0.881
0.039
0.152
109K
219K
56.47
0.094
and Adobe Stock. These models are categorized "closed," "open," and "mixed" in this section. Ad- ditionally, we use nonconvex polyhedra of various Euler characteristics and non-orientable geometries to prove our method's s versatility.
We implemented our main algorithm for computing face existence probabilites and differentiable renderer used for multi-view image reconstruction in CUDA (Nickolls u. 2008). Since need to compute WDT before rumning the CUDA algorithm, we used WDT implementation of CGAL (Jamin u. a. 2023). We implemented the rest of logic with Pytorch (Paszke u. a. (017).. All of the experiments were run on system with AMD EPYC 7R32 CPU and Nvidia A10 GPU.
5.1
Mesh to DMesh
In this experiment, demonstrate that can preserve most of the faces in the original normal triangular mesh after converting it to DMesh using the mesh reconstruction loss introduced in 4.3
In Table we show the recovery ratio (RE) and false positive ratio (FP) of faces in reconstructed mesh. Note that we could recover over 99% of faces the original mesh while only having under 1 1% of false faces. Please Appendix E.I more details. This result successfully validates differentiable formulation, but also reveals its limitation in reconstructing some abnormal triangles in the original mesh such long, thin triangles.
5.2 Point Cloud & Multi- View Reconstruction
In this experiment, we aim to reconstruct a mesh from partial geometric data, such as (oriented) point clouds multi-view images. For point cloud reconstruction, we sampled 100K points from the ground truth mesh. We can additionally use point orientations, if they are available. For multi-view reconstruction, rendered diffuse and depth images of the ground truth mesh from 64 view points.

---

8
8
Figure 9: (-) Shape interpolation using DMesh exhibiting topology change. After fitting DMesh to torus (upper left), we optimize again to reconstruct double torus (lower right), which has a different genus. We use multi-view images for the optimization.
In Appendix E. we illustrated the example inputs for these experiments. Also, please see Appendix D to see the initialization and densification strategy we took these experiments.
To validate our approach, we compare our results with various approaches. When it comes to point cloud reconstruction, first compare our result with classical Screened Poisson Surface Reconstruction (PSR) method (Kazhdan und Hoppe, (2033) Then, to compare our method with optimization based approach, we use recent VoroMesh (Maruani u. a. 2023) method. Note that these two methods are not tailored for open surfaces. To compare our method also for the open surfaces, we use Neural Dual Contouring (NDC) (Chen u. a. 2022b), even though
learning-based approach.
Finally, for multi-view reconstruction task, we compare our results with Flexicube (Shen a. 2023) and Neural Implicit Evolution (NIE) (Mehta a. [022], which correspond to volumetric approaches that can directly produce meshes of varying geometric topology for given visual inputs.
In Figure 8 we visualize the reconstruction results along with the ground truth mesh for qualitative evaluation. For closed meshes, in general, volumetric approaches like PSR, VoroMesh, and Flexicube, capture fine details better than methods. This is mainly because currently have limitation in the mesh resolution that we can produce with our method. NIE, which is also based on volumetric principles, generates overly smoothed reconstruction results. However, when it comes to open or mixed mesh models, which are more ubiquitous in real applications, we can observe that these methods fail, usually with false internal structures or self-intersecting faces (Appendix P..2). Since NDC leverages unsigned information, it can handle these cases without much problem as ours. However, we can observe step-like visual artifacts coming from its usage of grid in the final output, which requires post-processing. Additionally, to show the versatility of our representation, also visualize various shapes reconstructed from oriented point clouds Figure
Table 2 presents quantitative comparisons with other methods. We used following metrics from Chen u. a. (002b)) to measure reconstruction accuracy: Chanfer Distance (CD), F-Score (F1), Normal Consistency (NC), Edge Chamfer Distance (ECD). and Edge F-Score (EF1) to the ground truth mesh. Also, we report number of vertices and faces of the reconstructed mesh to compare mesh complexity, along with computational time. All values are average over 11 models that we used. In general, our method generates mesh of comparable, or better accuracy than the other methods. However, when it comes to ECD and EF1, which evaluate the edge quality of the reconstructed mesh, our results showed some weaknesses, because our method cannot prevent non-manifold edges yet. However, our method showed superior results in terms of mesh complexity - this is partially due to the use of weight regularization. Please see Appendix B.3 to see how the regularization works through ablation studies. Likewise, our method shows promising results in producing compact and accurate mesh. However, we also note that our method requires more computational cost than the other methods in the current implementation.
Before moving on, we present an experimental result about shape interpolation using DMesh in Figure We used multi-view images to reconstruct a torus first, and then optimized the DMesh again
We provide point orientations for PSR, which is optional for method.

---

800
700
Prob
500
400
Total
300
200
100
10ms 13
ms
23 t03
44
500
Number of Points
Figure 10: Analysis of computational cost for computing face existence probabilities (A(F).. The computational cost rises sharply beyond 20K points, with most of the time spent on WDT construction ("WDT"", while the probability computation ("Prob") requires significantly less time.
to a double torus. The results show that DMesh effectively reconstructs the double torus, even when initialized from converged single torus, highlighting the method's robustness local minima. However, this also indicates that our representation lacks meaningful shape interpolation, as does not assume any specific shape topology.
Conclusion
6.1 Limitations
As shown above, our method achieves a more effective and complete forms of differentiable meshes of various topology than existing methods, but still has several limitations overcome.
Computational Cost. Currently, the resolution of DMesh limited by computational cost. Although theoretical relaxation and CUDA implementation reduce this burden, processing meshes with over 100K vertices remains challenging due to the computational bottleneck of constructing the WDT at each optimization step. In Figure 10] we analyze computational costs relative to the number of points. As shown, costs rise sharply beyond 20K points, with WDT construction consuming most of the time. This limits method's d ability to handle high resolution mesh.
Non-Manifoldness. As we have claimed so far, DMesh shows much better generalization than the other methods as it does not have any constraints on the shape topology and mesh connectivity. However, due to this relaxation of constraint, we can observe spurious non-manifold emrors in the mesh, even though we adopted measures to minimize them (Appendix (.2277
Specifically, an edge must have most two adjacent faces be "manifold" edge. Similarly, "manifold" vertex should be adjacent to set of faces that form closed or open fan. We refer to edges or vertices that do not satisfy these definitions as "hon-manifold." In results, we found that 5.50% of edges and 0.38% of vertices were non-manifold for point cloud reconstruction. For multi-view reconstruction, 6.62% edges and 0.25% vertices were non-me manifoldf.d. Therefore, we conclude that non non-manifold edges are more prevalent than non manifold vertices in our approach.
6.2 Future Work
To address the computational cost issue, can explore methods that reduce reliance on the WDT algorithm, as its cost increases significantly with the number of points. This is crucial since represent- ing complex shapes with fine details often requires over 100K vertices. To tackle the non-man mannioooo issue, we could integrate approaches based on (un)signed distance fields (Shen u. a. 2023 Liu u. a. 2023b) into our method, ensuring manifold mesh generation. Finally, future research could extend this work to solve other challenging problems, such as 3D reconstruction from real-world images, or applications like generative models for 3D shapes. This could involve encoding color or texture information within framework, opening up exciting new directions for exploration.
10

---

Acknowledgements We thank Zhiqin Chen and Matthew Fisher for helpful advice. This research a joint collaboration between Adobe and University of Maryland College Park. This work has been supported in part by Adobe, IARPA, UMD-ARI Cooperate Agreement, and Dr. Barry Mersky and Capital One Endowed E-Nnovate Professorships.
References
[Amenta u. a. 1998a] AMENTA, Nina; BERN, Marshall; ; EPPSTEIN, David: The crust and the 8-skeleton: Combinatorial curve reconstruction. In: Graphical models and image processing 60 (1998), Nr. 2, S. 125-135
[Amenta u. a. 1998b] AMENTA, Nina BERN, Marshall KAMVYSSELIS, Manolis: A new Voronoi-based surface reconstruction algorithm. In: Proceedings of the 25th annual conference on Computer graphics and interactive techniques, 1998, S. 415-421
[Aurenhammer u. a. 2013] AURENHAMMER, Franz: KLEIN, Rolf LEE, Der-Tsai: Voronoi diagrams and Delaunay triangulations. World Scientific Publishing Company, 2013
[Brandt und Algazi 1992]
BRANDT, Jonathan W. ALGAZI, R.: Continuous skeleton computa- tion by Voronoi diagram. In: CVGIP: Image understanding 55 (1992), Nr. S. 329-338
[Chen a. 2022a]
CHEN, Anpei Xu, Zexiang GEIGER. Andreas Yu, Jingyi Su. Hao: TensoRF: Tensorial Radiance Fields. In: European Conference on Computer Vision (ECCV), 2022
[Chen a. 2023] CHEN, Anpei; XU, Zexiang: WEI, Xinyue TANG, Siyu SU, Hao GEIGER, Andreas: Dictionary Fields: Learning a Neural Basis Decomposition. In: ACM Trans. Graph. (2023)
[Chen u. 2019]
CHEN, Wenzheng LING, Huan: GAO, Jun SMITH, Edward LEHTINEN, Jaakko ; JACOBSON, Alec FIDLER, Sanja: Learning to predict 3d objects with an interpolation- based differentiable renderer. In: Advances neural information processing systems 32 (2019)
[Chen u. 20226]
CHEN, Zhiqin: TAGLIASACCHI, Andrea ; FUNKHOUSER, Thomas ZHANG, Hao: Neural dual contouring. In: ACM Transactions Graphics (TOG) 41 (2022), Nr. 4, 1-13
[Chen u. a. 2020] CHEN, Zhigin: TAGLIASACCHI, Andrea ; ZHANG, Hao: Bsp-net: Generating compact meshes via binary space partitioning. In: Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2020, S. 45-54
[Cheng u. 2013] CHENG, Siu-Wing DEY, Tamal K. SHEWCHUK, Jonathan SAHNII, Sartaj: Delaunay mesh generation. CRC Press Boca Raton, 2013
[Cignoni a. 2008] CIGNONI, Paolo; CALLIERI, Marco CORSINI, Massimiliano: DELLEPIANE.
Matteo GANOVELLI, Fabio ; RANZUGLIA, Guido u.a.: Meshlab: an open-soure, mesh processing tool. In: Eurographics Italian chapter conference Bd. 2008 Salerno, Italy (Veranst.), 2008, S. 129-136
[Curless und Levoy 1996] CURLESS, Brian; ; LEVOY, Mare: A volumetric method for building complex models from range images. In: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, 1996, S. 303-312
[Guillard a. 2021] GUILLARD, Benoit REMELLI, Edoardo LUKOTANOV, Artem ; RICHTER,
Stephan R. ; BAGAUTDINOV, Timur BAQUE, Pierre FUA, Pascal: DeepMesh: Differentiable
iso-surface extraction. In: arXiv preprint arXiv:2/06.1 /795 (2021)
[Hanocka u. a. 2019] HANOCKA, Rana; ; HERTZ, Amir ; FISH, Noa ; GIRYES, Raja; FLEISHMAN, Shachar COHEN-QU -OR, Daniel: Meshenn: network with edge. In: ACM Transactions
Graphics (ToG) 38 (2019), Nr. 4, 1-12
[Jamin 2023] JAMIN, Clément ; PION, Sylvain TEILLAUD, Monique: 3D Triangulations. CGAL. User and Reference Manual. 5.6. CGAL Editorial Board, 2023. URL httpss //dcc. ogal org/5. 6/ "Maaual/packaaa htm htt.::PPghTrringullltt....

---

[Ju u. a. 2002] Ju, Tao LOSASSO, Frank: SCHABFER, Scott; WARREN, Joe: Dual contouring of hermite data. In: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, 2002, S. 33939-33 339-346
[Kazhdan und Hoppe 2013]
KAZHDAN, Michael; HOPPE, Hugues: Screened poisson surface reconstruction. In: ACM Transactions on Graphics (ToG) 32 (2013), Nr. 3, S. 1-13
[Kerbl u. a. 2023] KERBL. Bernhard: ; KOPANAS, Georgios LEIMKÜHLER, Thomas; DRETTAKIS, George: 3D Gaussian Splatting for Real-Time Radiance Field Rendering. In: ACM Transactions on Graphics 42 (2023), Nr.
[Kingma und Ba 2014] KINGMA, Diederik P. BA. Jimmy: Adam: A method for stochastic optimization. In: arXiv preprint arXiv: 1412.6980 (2014)
[Laine u.a. 2020] LAINE, Samuli HELLSTEN, Janne KARRAS, Tero SEOL, Yeongho LEHTINEN, Jaakko AILA, Timo: Modular primitives for high-performance differentiable
rendering. In: ACM Transactions on Graphics (TOG) 39 (2020), Nr. 6, S. 1-14
[Lee 2010] LEE. John: Introduction topological manifolds. Bd. 202. Springer Science & Business Media, 2010
[Liao u. a. 2018] LIAO, Yiyi; ; DONNE, Simon GEIGER. Andreas: Deep marching cubes: Learning explicit surface representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, S. 2916-2925
[Liu u. 2020] LIU, Lingjie Gu, Jiatao LIN, Kyaw Z. CHUA, Tat-Seng; THEOBALT, Christian: Neural Sparse Voxel Fields. NeurlPS (2020)
[Liu u. a. 2019] LIU, Shichen LI, Tianye CHEN, Weikai LI. Hao: Soft rasterizer: A differ-
entiable renderer for image-based 3d reasoning. In: Proceedings the IEEE/CVF International Conference on Computer Vision, 2019, S. 7708-7717
[Liu u. a. 2023a] LIU, Yu-Tao; WANG, Li; YANG, Jie CHEN, Weikai MENG, Xiaoxu YANG,
Bo ; GAo, Lin: Neudf: Leaning neural unsigned distance fields with volume rendering. In:
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023,
S. 237-247
[Liu u. 2023b] LIU, Zhen FENG, Yao XIU, Yuliang LIU, Weiyang PAULL, Liam; BLACK, Michael J. SCHÖLKOPF, Bernhard: Ghost on the Shell: An Expressive Representation of General 3D Shapes. In: arXiv preprint arXiv:2310.15166 (2023)
[Long 2023] LONG, Xiaoxiao LIN, Cheng LIU, Lingjie LIU, Yuan WANG, Peng THEOBALT, Christian KOMURA, Taku WANG, Wenping: Neuraludf: Learning unsigned distance fields for multi-view reconstruction of surfaces with arbitrary topologies. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2023. S. 20834-20843
[Lorensen und Cline 1998] LORENSEN, William E. CLINE, Harvey E.: Marching cubes: A high resolution 3D surface construction algorithm. In: Seminal graphics: pioneering efforts that shaped the field. 1998, S. 347-353
[Maruani u. a. 2023] MARUANI, Nissim KLOKOV, Roman OVSIANIKOV, Maks ALLIEZ, Pierre DESBRUN, Mathieu: VoroMesh: Learning Watertight Surface Meshes with Voronoi Diagrams. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023,
S. 14565-14574
[Mehta u. a. 2022]
MEHТA, Ishit; ; CHANDRAKER, Manmohan RAMAMOORTHI, Ravi: A level set theory for neural implicit evolution under explicit flows. In: European Conference Computer Vision Springer (Veranst.), 2022, S. 711-729
[Mildenhall u. a. 2021] MILDENHALL, Ben SRINIVASAN, Pratul P.; TANCIK, Matthew BARRON, Jonathan : RAMAMOORTHI, Ravi ; NG, Ren: Nerf: Representing scenes as neural radiance fields for view synthesis. In: Communications of the ACM 65 (2021), Nr. 1, S. 99-106
12

---

[Munkberg u. a. 2022]
MUNKBERG. Jacob; HASSELGREN, Jon; SHEN, Tianchang GAO, Jun CHEN, Wenzheng EVANS. Alex ; MÜLLER, Thomas FIDLER, Sanja: Extracting triangular 3d models, materials, and lighting from images. In: Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, 2022, 8280-8290
[Nickolls u. a. 2008] NICKOLLS, John: BUCK, Ian ; GARLAND, Michael SKADRON, Kevin: Scalable parallel programming with cuda: Is cuda the parallel programming model that application developers have been waiting for? In: Queue 6 (2008), Nr. 2, S. 40-53
[Nicolet u. a. 2021] NICOLET, Baptiste: ; JACOBSON, Alec; ; JAKOB, Wenzel: Large steps inverse rendering of geometry. In: ACM Transactions on Graphics (TOG) 40 (2021), Nr. 6, S. 1-13
[Oechsle 2021] OECHSLE, Michael; PENG, Songyou ; GEIGER, Andreas: UNISURF: Unify- ing Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction. In: International Conference Computer Vision (ICCV), 2021
[Palfinger 2022] PALFINGER, Werner: Continuous remeshing for inverse rendering. In: Computer Animation and Virtual Worlds 33 (2022), Nr. 5, S. c2101
[Paszke u. a. 2017]
PASZKE, Adam GROSS, Sam CIINTALA, Soumith CHIANNN, Gregory: YANG, Edward; ; DEVITO, Zachary: LIN, Zeming: DESMAISON, Alban ANTIGA, Luca LERER, Adam: Automatic differentiation in pytorch. (2017)
[Rakotosaona u. a. 2021] RAKOTOSAONA, Marie-Julie ; AIGERMAN, Noam ; MITRA, Niloy J.; OVSJANIKOV, Maks GUERRERO, Paul: Differentiable surface triangulation. In: ACM Transac- tions on Graphics (TOG) 40 (2021), Nr. 6, S. 1-13
[Shen u. a. 2021] SHEN, Tianchang GAo, Jun ; YIN, Kangxue LIU, Ming- Yu FIDLER, Sanja: Deep marching tetrahedra: hybrid representation for high-resolution 3d shape synthesis. In: Advances in Neural Information Processing Systems 34 (2021), S. 6087-6101
[Shen u. a. 2023] SHEN, Tianchang: MUNKBERG, Jacob: ; HASSELGREN, Jon YIN, Kangxue WANG, Zian CHEN, Wenzheng GOJCIC, Zan; FIDLER, Sanja SHARP, Nicholas: ; GAO, Jun: Flexible isosurface extraction for gradient- mesh optimization. In: ACM Transactions on Graphics (TOG) 42 (2023), Nr. 4, S. 1-16
(Wang 2021] WANG, Peng ; LIU, Lingjie: LIU, Yuan ; THEOBALT, Christian KOMURA, Taku; ; WANG, Wenping: Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In: arXiv preprint arXiv:2106. 10689 (2021)
(Wang a. 2023] WANG, Yiming: HAN. Qin HABERMANN, Mare DANIILIIIS, Kostas THEOBALT, Christian LIU, Lingjie: NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-
view Reconstruction. In: Proceedings of the IEEE/CVF International Conference on Computer
Vision (ICCV), 2023
[Wei 2023]
WEI, Xinyue XIANG, Fanbo B1, Sai CHEN, Anpei SUNKAVALLI, Kalyan: XU, Zexiang Su, Hao: NeuManifold: Neural Watertight Manifold Reconstruction with Efficient and Sigh-Quality Rendering Support. In: arXiv preprint arXiv:2305.17114 (2023)
(aariv a. 2021]
YARIV, Lior; GU, Jiatao ; KASTEN, Yoni LIPMAN, Yaron: Volume rendering of neural implicit surfaces. In: Thirty-Fifth Conference on Neural Information Processing Systems, 2021
(Yariv u. 2020]
YARIV, Lior; KASTEN, Yoni ; MORAN, Dror GALUN, Meirav ATZMON, Matan RONEN, Basri ; LIPMAN, Yaron: Multiview Neural Surface Reconstruction by Disen- tangling Geometry and Appearance. In: Advances in Neural Information Processing Systems 33 (2020)
[Zhang u. a. 2020] ZHANG, Kai RIEGLER, Gernot : SNAVELY, Noah : KOLTUN, Vladlen: NeRF++: Analyzing and Improving Neural Radiance Fields. In: arXi::2010.07492 (2020)
[Zhou a. 2020] ZHOU, Yi; WU, Chenglei LI, Zimo CAO, Chen: YE, Yuting SARAGIH, Jason ; LI, Hao SHEIKH, Yaser: Fully convolutional mesh autoencoder using efficient spatially varying kernels. In: Advances in neural information processing systems 33 (2020), S. 9251-9262
13

---

Table Traits of different optimization -baeedddo---ddddddd shape reconstruction methods. We compare methods
based on template mesh (Palinger [2022] Nicolet u. a. 202II, neural SDF (Wang u. a. 2021 2023).
neural UDF (Long u. a. 2023 Liu u. a. 2023a). differentiable isosurface extraction techniques (Shen
a.
2021
Munkberg
2022
Shen u. a.
2023
Liu u. a. 2023b) with ours.
Methods
Closed
Open
Diff. Mesh
Diff. Render.
Geo. Topo.
Mesh Topo.
Manifold
Template Mesh
O
0
X
X
O
Neural SDF
X
0
0
X
0
Neural UDF
X
0
0
X
A
Diff. Isosurface
0
A
0
0
0
X
0
DMesh (Ours)
0
0
0
0
X
A
Comparison to Other Shape Reconstruction Methods
Here we provide conceptual comparisons between our approach and the other optimization-ben bbasodd----dddd--- 3D
reconstruction algorithms, which use different shape representations. To be specific, compared our method with mesh optimization methods starting from template mesh (Palfinger 2022 Nicolet
202T), methods based on neural signed distance fields (SDF) (Wang u. [02 2023), methods
based on neural unsigned distance fields (UDF) (Liu u. a. 2023a Long u. a. (023). and methods
based on differentiable isosurface extraction (Shen u. a. [022] Munkherg a. 2022 Shen u. a. 2023
Liu u. a. 2023b). We used following criteria to compare these methods.
Closed surface: Whether or not the given method can reconstruct, or represent closed surfaces.
Open surface: Whether or not the given method can reconstruct, or represent open surfaces.
Differentiable Meshing: Whether or not the given method can produce gradients from the loss computed on the final mesh.
Differentiable Rendering: Whether or not the given method can produce gradients from the loss computed on the rendering results.
Geometric topology: Whether not the given method can change geometric topology of the shape. Here, geometric topology defines the continuous deformation of Euclidean subspaces (Lee) 2010). For instance, genus of the shape is one of the traits that describe geometric topology.
Mesh topology: Whether or note the given method can produce gradients from the loss com- puted the mesh topology, which denotes the structural configuration, or edge connectivity of a mesh
Manifoldness: Whether or not the given method guarantees manifold mesh.
In Table 3 we present a comparative analysis of different methods. Note that our method meets all criteria, only except manifoldness. It is partially because our method does not assume volume, which is the same for methods based on neural UDF However, because our method does not leverage smoothness prior of neural network like those methods, it could exhibit high frequency noises in the final mesh Because of this reason, we gave A to the neural UDF methods, while giving X to our approach. When it comes to methods based on differentiable isosurface extraction algorithms, we gave A to its ability to handle open surfaces, because of (Liu u. a. 2023b). They can represent open surfaces as subset of closed ones, but cannot handle non-orientable open surfaces. Finally, note that our method is currently the only method that can handle mesh topology.
Likewise, DMesh shows promise in addressing the shortcomings found in previous research. Nonethe- less, has own set limitations (Section(6) Identifying and addressing these limitations is crucial for unlocking the full potential of our method.
14

---

B Details about Section 3.2
B.1 Mathematical Definitions
Here, we provide formal mathematical definitions of the terms used in Scction 3.2 We mainly use notations from Aurenhammer (2013) and [Cheng
(2013).
Generalizing the notations Section 3.2 let S(w) be finite set of weighted points in Rd, where w is a weight assignment that maps each point p E S to its weight wy We denote a weighted point as p. [/yp] and define power distance to measure distance between two weighted points.
Definition B.1 (Power distance). Power distance between two weighted points p(wy] and g(w;, is measured as:
x (y(wpp) g(w,)) d(p, 9)² up Wlgs
where d(p,2) 2) is the Fuclidean distance.
Note that unweighted point is regarded as carrying weight of O. Based this power distance, can define the power cell Cp of a point p[w,] as the set of unweighted points in IRd that are closer to p [p] than to any other weighted point in S(w)]-
Definition B.2 (Power cell). Power cell of a point plupl E S[w] is defined as:
c= = fxr E Idd Vg[we)]
S[w)), x (x., p[w,)) S ((x, a(w,))).
Note that some points may have empty power cells their weights are relatively smaller than neighboring points. We call them "submerged" points. As will see later, weight regularization aims at submerging unnecessary points in mesh definition, which leads to mesh simplification.
To construct the power cell, we can use the concept of half space. A half space Hpcg is the set of unweighted points in Id that are closer to p[up] than g[ua]] Definition B.3 (Half space). Half space Hp<q is defined as:
Hy<q fo E R [x. p(wp)) S x(x g(wg)))-
(10)
Note that can construct power cell by intersecting half spaces, which proves the convexity of the power cell. Now we call H (p, q) as a half plane that divides IR into two half spaces, Hpcc and H9sp. Definition B.4 (Half plane). Half plane Hp.g is defined as:
Hp. for E Rdd n (Ix p|w,]) (2, alw,))).
(11)
Then, for given k-simplex A comprised of weighted points (pi [wp.l.... PE++ [pp41]] C S[wo], the dual structure DA is the intersection of half planes between the points in (A*), which is a convex set.
(4*)
Definition B.5 (Dual form). Dual form DA of A = [pi[[ppl-... .eell.........11 [wep))))))) is defined as:
DAL
N
H(p,,Ps),
(12)
which is equivalent to:
DA fo E Ed Vp, [. Py [,, E S(w], (2, p((wp.)) x (x. Py [u[ 1)}-
(13)
Note than when h = d, D becomes a point, while it becomes line when k = d - 1. As discussed in Section 3.2 we leverage the distance between this dual form D DA * and reduced power cell (Rakotosaona u. a. 2021) of the points in DA to query the existence of A The reduced power cell RpIA= is power cell of p that does not concern the other points * in construction.
Definition B.6 (Reduced power cell), Reduced power cell of a weighted point p[w] e S[w] for given A is defined as: RØJA= {ar E Rdd Vglus) E S[w] {A*), [(z,p(u))) S a (ar, a(w,])).
(14)
Using these concepts, we can measure the existence probability of a k-simplex, as provided in Section 3.2
15

---

B.2 Analysis of previous approach
B.2.1 Theoretical Aspect
As mentioned in Section 3.2, the previous approach Rakotosonna u. a. (202T) has two computational
challenges in computational efficiency and precision. These limitations are mainly rooted in not knowing the power diagram structure before evaluating Eq. 3] We summarize the overall procedure of the previous approach, and point out how our method is different from it.
Collecting simplexes evaluate First, we need to collect simplexes that we want to compute probabilities for. Here we assume the number of simplexes increases linearly (O(N) as the number of points (N) increases. This is plausible assumption, because we often search for k-nearest neighbors for each point, and combine them to generate the query simplexes.
Sampling a point on the dual forms of the simplexes The previous method relies on point projections to evaluate Eq. 3 This did not incur any problem for their case (d = k = 2). because the dual form was a single point. However, when (k < d) as our cases, the dual form contains infinite number of points, which makes unclear how to apply this point-based approach. One possible solution sampling the most "representative" point on the dual form, and leveraging the point to estimate Eq. 3 By definition, this estimation is lower bound of Eq. []] However, the problem arises when this sample point does not give reliable result, For instance, we can consider the case shown in Figure(jc). In the illustration, we can observe that A' exists in WDT, and thus DA goes thorugh Rp. D If we sample a point on DA that is included in Rp a the signed distance from the sample point would have same sign as Eq. 3 However, if the sample point is selected outside Ryi D the sign would be different from the real value. In this case, even if A exists, we can recognize as not existing. Note that this false estimation produces false gradients, which could undermine optimization process.
In contrast, we do not have to concern about this precision issue, because we construct PD, which tells us good sample points that give reliable lower bounds of Eq. when the given simplex exists in WDT. Otherwise, we explicitly compute minimum distance between the dual form and the reduced power cell, as discussed in Section
Projecting sample points to reduced power cells The final step is point projection, where project the sample points from dual forms to the reduced power cells to estimate Eq. B Based on the definitions in Appendix B we can observe that (reduced) power cell's boundaries are comprised of half planes. That is, the boundaries of Cp is comprised of multiple half planes between p and the other weighted points. However, when we do not know which half planes comprise the boundaries, we have to do exhaustive search to find the signed distance from the sample point to the boundaries of the reduced power cell. As the number of half planes that are associated with point p is N, the computational cost to precisely compute the signed distance is O(N).
Note that it does not hold for our case, because by constructing WDT and PD, we know which half planes form the boundaries of each power cell. Also, note that even when the number of points increases, the average number of half planes that comprise the boundaries of power cells remains constant. Therefore, in our case, this step requires only (1) computational cost.
Summary To sum up, the computational cost of the previous approach amounts to O(N²), the number of simplexes to evaluate increases linearly, and the cost for the projection step also increases linearly the number of points increase. However, the cost for ours remains O(N). Moreover, the previous approach does not guarantee satisfactory estimations of Eq. 3
Before moving on, we point out that the original implementation limited the number of half planes to consider in evaluating Eq. 3 to reduce the computational cost to O(N). This relaxation is permissible to the case where the precision is not very important. However, the precision is important in our case, because we aim at representing mesh accurately.
B.2.2 Experimental Results
To prove the aforementioned theoretical claim, we conducted experiments to measure the computa- tional speed and accuracy of the probability estimation for (d = 2, k = 2) and (d = 2, k 1) cases, for varying number of points. We randomly sampled points in unit cube uniformly, and set the
16

---

Table 4: Comparison of computational speed and accuracy between the previous method (Rakotosaonal u. a. 2021) and ours. For each number of points in 2D, we report the triplet of (computational speed (ms) false positive ratio (%) false negative ratio (%)), along with the number of query simplexes
that we give to the algorithm.
Pis.
100
300
IK
3K
10K
30K
Prev.
6.38/070
19.7/070
/39/0/0
739/070.16
4376/070.12
(d=2, k=2)
Ours.
1057070
107/070
108 7070
1117070
116/070.12
1257070.25
Simp.
342
992
2274
3672
4974
6332
Prev.
5.95/0/49.9
16.970749.8
11670/49
609 /9770799.9
3681 79749.9
(d=2, k=1)
Ours.
99.47070
1007070
1037 1037070
105 700070
111/070.08
123/070.20
Simp.
526
1504
3428
5522
7482
9524
weights by random sampling from a normal distribution: (0, 10- For fair comparison, we did not use CUDA implementation that we mainly use in the paper. We implemented both algorithms in PyTorch, and ran 10 times for each setting to get fair values. In each experiment, we fed the query simplexes into the algorithm, and computed their existence probabilities. If the computed probability for a simplex was over 0.5, but did not exist, it is counted to false positive ratio. In contrast, if the computed probability for simplex was below 0.5, but did exist, it is counted to false negative ratio. We measured the computational accuracy with these metrics.
In Table we can see that as the number of points increases, the computational cost increases exponentially for the previous approach, while ours remain fairly stable up to 30K points. When the number of points reached 100K, the previous method failed because of excessive memory consumption. However, when the number of points is smaller than IK, the previous method ran faster than ours, because need to construct PD even for the small number of points, which consumes most of the time for all these cases.
In terms of accuracy, we can observe that the previous method and ours both give accurate estimations when (d. = 2, k 1). However, when (d 2, k = 1), we can see that the false negative ratio is almost 50% as we set the number of existing simplexes and non-existing simplexes in the query set as the same, it means that the previous method chose wrong sample points for most of the query simplexes, so that it predicted most of them as not existing. To select the most representative point, we found the intersection point between the dual form and the affine space created by the weighted points in the simplex. This could be another interesting direction to explore, but this approach found out to be not very accurate in this experiment. In contrast, our approach gave stable estimations for all the cases. This result demonstrates that our method is more scalable, and gives reliable probability estimations than the previous approach, which is necessary for accurate 3D mesh representation.
C Loss Functions
Here we provide formal definitions for the loss functions that we use in the paper.
C.1 Mesh to DMesh
In this section, we explore the loss function used to transform the ground truth mesh into our DMesh representation. As previously mentioned in Section 4.3 the explicit definition of ground truth connectivity in the provided mesh allows us to establish a loss function based on it.
Building on the explanation in Section 4.3 the ground truth mesh consists of vertices and faces F, we can construct an additional set of faces F. These faces are formed from vertices in IP but do not intersect with faces in F.
F F F, where F every possible face combination on P.
Then, we notice that we should maximize the existence probabilities of faces in F, but minimize those of faces in F. Therefore, we can define our reconstruction loss function as
Lrecon
ΣA(F) HU0+>AAE) A(P).
(15)
FER
FEB
If the first term of the loss function mentioned above not fully optimized, it could lead to the omission of ground truth faces, resulting in poorer recovery ratio (Section 5.1). Conversely, if the
17

---

second term is not fully optimized, the resulting DMesh might include faces absent in the ground truth mesh leading to higher false positive ratio (Section Refer to Appendix for details on how this reconstruction loss is integrated into the overall optimization process.
C.2 Point Cloud Reconstruction
In the task of point cloud reconstruction, we reconstruct the mesh by minimizing the (Li-norm based) expected Chamfer Distance (CD) between the given point cloud (P. and the sample points (P ourr) from our reconstructed mesh. We denote the CD from Fgt to I Pours CDgt. and the CD from F ours to P gt as CD The final reconstruction loss is obtained by combining these two distances.
Lieeon CDtt + CDourss
(16)
C.2.1 Sampling P Porrs
To compute these terms, we start by sampling Fours from our current mesh First, sample set of faces that we will sample points from. We consider the areas of the triangular faces and their existence probabilities. To be specific, define n(() for face F as
T(F) = A(F), m(F)) Farea (FF),
and define probability to sample F from the entires faces IF as
(F(F)
P samptel (F) EFeez m(F'))
We sample N faces from FF with replacement and then uniformly sample single point from each selected face to define IP Pours. In our experiments, we set N to 100K.
In this formulation, we sample more points from faces with larger area and higher existence probability to improve sampling efficiency. However, we observed that despite these measures, the sampling efficiency remains low, leading to slow convergence. This issue arises because, during optimization, there an excessive number of faces with very low existence probability.
To overcome this limitation, we decided to do stratified sampling based on point-wise real values and cull out faces with very low existence probabilities. To be specific, we define two different functions:
T (F) = Amat(F) min(miii (b, i (F) = Farea 171 (F) V/a = Awde (F) max 1hy, vh., (F) = Farea 172 (F)
where (1/s, thy, v/) are the real values of the points that comprise F. Note that ? is the same as 17
For the faces in F, we first calculate the 7/ and 172 values and eliminate faces with values lower than predefined threshold en- We denote the set of remaining faces as FF and F2, Subsequently, we sample N. faces from Fz, using the following two sampling probabilities:
N faces from F1 and the other π 2
(F)
(2(F))
Psample.1 (F) EFES "F'F)' Peampte, ((P) Erer, 1/2 (22)F")
The rationale behind this sampling strategy is to prioritize (non-existing) faces closer to the current mesh over those further away. In the original 7 = 7) function, we focus solely on the minimum real value, leading to higher sampling rate for existing faces. However, to remove holes in the current mesh it's beneficial to sample more points from potential faces tnces-those not yet existing but connected to existing ones. This approach, using 72, enhances reconstruction results by removing holes more effectively. Yet, there's substantial potential to refine this importance sampling technique, as we haven't conducted theoretical analysis in this study.
Moreover, when sampling a point from face, we record the face's existence probability alongside the point. Additionally, if necessary, we obtain and store the face's normal. For point IP Pours:
We do use differentiable min operator, we do not require differentiability in the sampling process.
18

---

we introduce functions Apc(.) and Normall-) to retrieve the face existence probability and normal, respectively:
(p) ((F()),, Normal(p) = normal:
F()) the face where p was sampled from.
C.2.2 CD
Now we introduce how we compute the CDgr gt which is CD from Egt to IP ourrs. For each point P Pot, we first find k-nearest neighbors of p in IP Pours, which we denote as (p1. Po..P...... Pp). Then, we define distance function between the point and the k-nearest neighbors as follows, to accommodate the orientation information:
D(p. p.) = ||p -illl + Anormal
(17)
where Dr (p. = -1 Pnormat, Normal
>|
where Anoema! is a parameter than determines the importance of point orientation in reconstruction. Anormal = we only consider the positional information of the sampled points.
After we evaluate the above distance function values for the k-nearest points, we reorder them in ascending order. Then, we compute the following expected minimum distance from p to IP Pours.
D(p,
Σ D(p. P(p.) P(p.).
i=....k
P(p:) Apr(pi) Ipreo (F(pi);
P(p.) II 11=1,..,-1(11 P(pi)),
where Uprey an indicator function that returns I only when the given face has not appeared before in computing the above expected distance. For instance, if the face ids for the reordered points were (1, 2, 3, 2, 3, 4), the Uprev function evaluates to (1, 1, 1, O, O, 1). This indicator function needed, because if select p; the nearest point p with the probability (p), it means that we interpret that the face corresponding to P. already exists, and then we would select pæ on the face as the nearest point to p rather than the other points that than ps and thus come after in the ordered points.
sampled from the same face, but have larger distance
Note that we dynamically change k during runtime to get a reliable estimation of D(p. IP Poure). That is, for current k, if most of ((yk)s )s for the points Fot are still large, it means that there a chance that the estimation could change lot if we find and consider more neighboring points. Therefore, in our experiments, any point For has P(ps) larger than 10-4, we increase by for the next iteration. However, if there is no such point, decrease & by 1 to accelerate the optimization process.
Finally, we can compute CDgt gt by summing up the point-wise expected minimum distances.
CD0t = Σ
D(p. P Poure).
PE
C.2.3 CDours
In computing CD which is CD from Pours to IP we also find k-nearest neighbors for each point p E F ours. which we denote as (P1. P......,,.) Then, for point p. we use the same distance function D in Eq. 17 to find the distance between p and (Pi,P2,.... Pk). After that, we select the minimum one for each point, multiply the existence probability of each point, and then sum them up to compute CD CDours
D(p. Pyt) = min D(p,Pt); i=l....
CDours
Σ
Ap(p)) D(p, Pt).
PEPaur
Finally, we compute the final reconstruction loss for point clouds shown in Eq. 16
19

---

C.3 Multi-Vies Reconstruction
When we are given multi-view images, we reconstruct the mesh by minimizing the L1 difference between our rendered images and the given images. In this work, we mainly use both diffuse and depth renderings to reconstruct the mesh.
If denote the (Nimg)) ground truth images of N Npizel number of pixels Ii" 1,... Nang). and our rendered images as Iqurs, we can write the reconstruction loss function as
Lipecon
E
-
Ning
piizel iinn..,Nim
N
Then, we can define our rendered image as follows:
Tours ((F), MV,, Pi).
where F is a differentiable renderer that renders the scene for the given points P, faces E, face existence probabilities A(F), i-th modelview matrix MV; R4x4 and i-th projection matrix E The differentiable renderer F has to backpropagate gradients along IP, F, and A(F) to update point attributes. Specifically, here interpret (F)) opacity for faces rendering process. This is because opacity means the probability that ray stops when it hits the face, which aligns with face existence probability well. For this reason, ignore faces with very low existence probability under some threshold to accelerate the reconstruction, as they are almost transparent and do not contribute to the rendering lot.
in the
To implement F, we looked through previous works dedicated for differentiable rendering (Laine u. a. 2020 Liu u. a. 2019). However, discovered that these methods incur substantial computational costs when rendering a large number of (potentially) semi-trang transparent triangles, as is the case in scenario. Consequently, developed two efficient, partially differentiable renderers that meet our specific requirements. These renderers fulfill distinct roles within our pipeline- as detailed in Appendix D optimization process encompasses two phases within single epoch. The first renderer is employed during the initial phase, while the second renderer is utilized in the subsequent phase.
C.3.1 FA
there are multiple semi- maiiitranspprent faces in the scene, have sort the faces that covers target pixel with their (view-space) depth values, and iterate through them until the accumulated transmittance is saturated to determine the color for the pixel. Conducting this process for each individual pixel is not only costly, but also requires lot of memory to store information for backward pass.
T₃
T2
Recently, 3D Gaussian Splatting (Kerbl 2023) over- came this issue with tile-based rasterizer. We adopted this approach, and modified their implementation to render triangular faces, instead of gaussian splats. To briefly in- troduce its pipeline, it first assigns face-wise depth value by computing the view-space depth of its center point. Then, after subdividing the entire screen into 16 16 tiles, we assign faces to each tiles if they overlap. After that, by using the combination of tile ID and the face-wise depth as a key, we get the face list sorted by depth value in each tile. Finally, for each tile, we iterate through the sorted faces and determine color and depth for each pixel as follows.
Figure 11: Fg uses tessellation structure efficiently render overlapped faces in the correct order.
C= Σ T. a; C.. (T, =
-a,))
l.....
where is the accumulated transmittance, a; is the opacity of the i-th face, and C2 the color (or depth) of the i-th face. Note that a; A(F)), as mentioned above.
20

---

(a) Rendered Images from FA
(b) Rendered Images from FA
Figure 12: Rendered images from two differentiable renderers, FA and Fx². .. Left and right image corresponds to diffuse and depth rendering, respectively. (a) FA is (partially) differentiable renderer based on tile-based approach. (b) Since FA does not produce visibility- rellttdd gradients, we additionally use Fx (Laine a. 2020) to render images and integrate with ours.
Even though this renderer admits an efficient rendering of large number semi-transparent faces, there still two large limitations in the current implementation. First, the current implementation does not produce disibliity-relttee gradients (near face edges) to update point attributes. Therefore, argue that this renderer is partially differentiable, rather than fully differentiable. Next, since it does not compute precise view-point depth for each pixel, its rendering result can misleading for some cases, pointed out in (Kerbl u. a. 2023).
To amend the first issue, we opt to use another differentiable renderer of Laine u. a. (020). which produces the visibility-related gradients that we lack. Since this renderer cannot render (large number of) transparent faces as ours does, we only render the faces with opacity larger than 0.5. Also, we set the faces to be fully opaque. If we call this renderer as FA our final rendered image can be written as follows.
Ieurs } (FA(C MV, P.) (P.A, P.)).
In Figure 12 we illustrate rendered images from Fa and FAT.
Acknowledging that this formulation is not theoretically correct, we believe that it is an intriguing future work implement a fully differentiable renderer that works for our case. However, we empirically found out that we can reconstruct wide variety of meshes with current formulation without much difficulty.
As mentioned before, this renderer is used at the first phase of the optimization process, where all of the point attributes updated. However, in the second phase, we fix the point positions and weights, and only update point-wise real values (Appendix D.2). D In this case, we can leverage the tessellation structure to implement an efficient differentiable renderer. As the second renderer does precise depth testing unlike the first one, can be used to modify the error incurred by the second limitation of the first renderer (Figure 13
C.3.2 FB
The second renderer performs precise depth ordering in an efficient way, based on the fixed tessellation structure that have. In Figure [T] we illustrate 2D diagram that explains our approach. When the green ray, which corresponds to a single ray to determine the color of a single pixel, goes through the tessellation, we can observe that it goes through a sequence of triangles (tetrahedron in 3D), which are denoted as Ti, T2, and T3. When the ray enters a triangle T; through one of its three edges, we can see that it moves onto the other adjacent triangle T:+1 only through one of the other edges of T; because of compact tessellation. Therefore, when the ray hits one edge of T, it can only examine the other two edges of T; to find the next edge hits. Note that do not have do depth testing explicitly in this approach. Also, unlike the first approach, this renderer does not have to store all the possible faces that a ray collides for the backward pass, because can iterate the same process opposite way in the backward pass to find the edge that it hit before the last edge. If we only store the edge that each hits at the forward pass, we can start from the edge and find the previous edges that it hit to compute gradients. Therefore, this second renderer requires much less memory
21

---

(a) Extracted Mesh after phase 1
(b) Extracted Mesh after phase 2
Figure 13: Reconstructed mesh from multi-view images, rendered in MeshLab's (Cignoni a. 2008
X-ray mode to see inner structure. In multi-view reconstruction, we divide each epoch in two phases. (a) After the first phase ends, where do inaccurate depth testing, lots of false inner faces are created. (b) To remove these inner faces, we require renderer that does the exact depth testing, which we use in the second phase. Also see Appendix D.2 for details about post- post-processing step to remove the inner structure.
than the first one, and also performs precise depth testing naturally. However, note that this renderer also partilly differentiable, because it cannot update point positions and weights.
To sum up, we implemented two partially differentiable renderers to solve multi-view reconstruc- tion problem with DMesh. They serve different objectives in reconstruction process, and empirically found out that they are powerful enough to reconstruct target meshes our experiments. However, we expect that we can simplify the process and improve its stability, if we can implement fully differentiable renderer that satisfy our needs. We leave it as future work.
C.4 Weight Regularization
Weight regularization aims at reducing the complexity of WDT, which supports our mesh. By using this regularization,
discard unnecessary points that do not contribute representing mesh. Moreover, we can reduce the number of points on the mesh, if they are redundant, which ends up in the mesh simplification effect (Appendix (E3)..
formulate the complexity of WDT as the sum of edge lengths in its dual power diagram. Formally, we can write the regularization as follows,
Thoright
Length( (Fa),
where E, are the edges in the dual power diagram, and N the number of edges.
C.5 Real Regularization
Real regularization is a regularization that is used for maintaining the real values of the connected points in WDT as similar as possible. Also, we leverage this regularization to make real values of points that are connected considered in reconstruction more often than the points that are not connected to those points. To be specific, note that we ignore faces with very low existence probability the reconstruction process. By using this regularization, it can remove holes more effectively.
points with high real values to become higher, so that they can be
22

---

This real regularzation can be described as
Lreat = E Ei A(Fi)
E A(F)) (01(F)) + (z()))
i=
(1) 1/2 + v3)
0i (F()
j 1,2,3
Σ
1
[I max (155) dnigh). /-1,2,3
j=1.2,3
Here 2,,, represent the real values points that comprise F,, and &high is threshold to determine "high" real value, which is 0.8 in our experiments. Note that the faces with higher existence probabilities are prioritized over the others.
C.6 Quality Regularization
After reconstruction, we usually want to have a mesh that is comprised of triangles of good quality, rather than ill-formed triangles. We adopt the aspect ratio quality measure for the triangular faces, and minimize the sum of aspect ratios for all faces during optimization to get a mesh of good quality. Therefore, we can write the regularization follows.
Lquat
1
Σ i=
AR(F:) Eman (F. A(F)),
Ei=1....N
A(Fi)
Exnax F2)
/3
AR(F:)
H (Fi)
2
Emaa (F:) Maximum edge length of F;,
Hmin (F)) Minimum height of F;.
Note that we prioritize faces with larger maximum edge length and higher existence probability than the others in this formulation. In Appendix E.3 provide ablation studies for this regularization.
D Optimization Process
In this section, explain the optimization processes, exact reconstruction algorithms, in detail. First, we discuss the optimization process for the experiment Section 5.1] where we represent the ground truth mesh with DMesh. Then, discuss the overall optimization process for point cloud multi-view reconstruction tasks in Section 5.2 from initialization to post processing.
D.1
Mesh to DMesh
Our overall algorithm to convert the ground truth mesh into DMesh is outlined in Algorithm I We explain each step in detail below.
D.1.1 Point Initialization
At the start of optimization, initialize the point positions (P), weights (W), and real values (1) using the given ground truth information ge FFt). To be specific, we initialize the point attributes as follows.
P
P gt:
W
[1, ....... 1].
[1,....1 1).
The length of vector W and s is equal to the number of points. In Figure 14 we illustrate the initialized DMesh using these point attributes, which becomes the convex hull of the ground truth mesh.
23

---

Algorithm Mesh to DMesh
P. got IF gt <- Ground truth mesh vertices and faces P. W, db t Initialize point attributes for DMesh
F < Empty set of faces
while Optimization not ended do
P. w. t- Do point insertion, with P, W DT, PD t- Run WDT algorithm, with IP, W f- Update faces to exclude, with WDY WT A(Fgt), (IF) <- Compute existence probability for faces, with P. db WDT, PD A(F)
Lrecon f- Compute reconstruction loss, with A(F A(F))
Update F, W, & to minimize Lyecon
Bound IP
end
M Get final mesh from DMesh
(a) Ground Truth
(b) Initialization
(c) Point Insertion
(d) 5000 Steps
Figure 14: Intermediate results in converting bunny model to DMesh. For given ground truth mesh in (a), we initialize our point attributes using the mesh vertices. (b) Then, the initial mesh becomes convex hull of the original mesh.
(c) To remove undesirable faces that were not in the
original mesh we insert additional points on the undesirable faces. Then, some of them disappear because of the inserted points.
(d) After optimizing 5000 steps, just before another point insertion, DMesh recovers most of the ground truth connectivity.
Note that during optimization, we allow only small perturbations to the positions of initial points, and fix weights and real values of them 1. This is because we already know that these points correspond to the ground truth mesh vertices, and thus should be included in the final mesh without much positional difference. In our experiments, set the perturbation bound 1% of the model size.
However, we notice that we cannot restore the mesh connectivity with only small perturbations to the initial point positions, if there are no additional points that can aid the process. Therefore, we periodically perform point insertion add additional points, which is described below.
D.1.2 Point Insertion
The point insertion is subroutine to add additional points the current point configurations. It is performed periodically, at every fixed step. The additional points are placed at the random place on the faces in IF, which correspond to the faces that should not exist in the final mesh. Therefore, these additional points can aid removing these undesirable faces.
However, we found out that inserting point for every face in I can be quite expensive. Therefore, use k-means clustering algorithm to aggregate them into 0.1 NE clusters, where Np is the number of faces in F,, to add the centroids of the clusters
running point set. On top of that, select to put additional points directly on them. This is because there are cases placed on the good positions where they can remove the undesirable faces.
1000 random faces where centroids
In Figure 14
we render DMesh after point insertion the initialized mesh. Note that some of the undesirable faces disappear because of the added points.

---

Algorithm 2 Point cloud & Multi-view Reconstruction
T' f- Observation (Point cloud, Multi-view images)
P. W, s t Initialize point attributes for DMesh (using possible) F < Empty set of faces
while epoch not ended do
P. W, 2b t (If not first epoch) Initialize point attributes with sample points from current DMesh,
for mesh refinement
// Phase 1
while step not ended do
W DT, PD +- Run WDT algorithm with P, W
FF &- Update faces to evaluate existence probability for, with W DT
(F)) + Compute existence probability for faces in F. with P, v, W DT, PD Lreeon Compute reconstruction loss, with F, AFF),
Leweight <- Compute weight regularization, with PD
Lreal f- Compute real regularization, with P, v. W DT
Lqual t Compute quality regularization, with P, F. ((F)
Lreeon Aweriht Leeight + Xreal Lreat + Aqual Lquat
Update P, W, v to minimize L
end
// Phase 2
WDT, PD < Run WDT algorithm with P, W
F. + Faces in WDT
Awdr (IF) +-1
while step not ended do
AFF) + Compute existence probability for IF, with P, tb, Awdd (F)
Lirecon < Compute reconstruction loss, with P, F, ((F),
Lreal +- Compute real regularization, with P, v. W DT
L Lrecon Xreal Lreal
Update &i to minimize L
end
end
M +- Get final mesh from DMesh, after post-processing
D.1.3 Maintaining
In this problem, we minimize the reconstruction loss specified in Eq. 15 to restore the connectivity in the ground truth mesh, and remove faces that do exist in it. In the formulation, denoted the faces that are comprised of mesh vertices I,, but are not included in the original mesh as F. Even though we can enumerate all of them, the total number faces in IF mounts O(N"), where N is the number of mesh vertices. Therefore, rather than evaluating all of those cases, we maintain a set of faces IF that we should exclude in our mesh during optimization.
To be specific, at each iteration, we find faces in the current WDT that are comprised of points in P, but do exist in and add them to the running of faces F. On top of that, at every pre-defined number of iterations, in our case 10 steps, we compute k-nearest neighboring points for each point in P. Then, find faces that can be generated by combining each point with of its k-nearest points, following Rakottsaonaa u. a. 2021). Then, we add the face combinations that do not belong to F to In our experiments, we set
D.2 Point cloud & Multi-view Reconstruction
we describe the overall algorithm that is used for point cloud and multi-view reconstruction tasks. We explain each step in detail below.
In Algorithm 2
D.2.1 Two Phase Optimization
We divide each optimization epoch in two phases. In the first phase (phase 1), we optimize all of the point attributes positions, weights, and real values. However, the second phase (phase 2), we fix the point positions and weights, and only optimize the real values.
25

---

(a) Ground Truth
(b) Initialized DMesh (Points, Extracted Mesh)
Figure 15: Initialized DMesh using sample points from ground truth mesh. (a) From ground truth mesh, we uniformly sample 10K points to initialize DMesh. (b) In the left figure, sample points from the ground truth mesh (IP sample) are rendered in red. The points that correspond to voronot rendered blue. In the right figure, we render the initial mesh we can get from the points, which has lot of holes.
This design aims removing ambiguity in our differentiable formulation. That is, even though we desire face existence probabilities to converge to either 0 and 1, those probabilities can converge to values in between. To alleviate this ambiguity, after first phase ends, we fix the tessellation to make watt for each face in F to either 0 or 1. Therefore, in the second phase, we only care about the faces that exist in current DT, which have Awde value of 1. Then, we can only care about real values.
Note that the two differentiable renderers that we introduced in Appendix are designed serve for these two phases, respectively.
D.2.2 Point Initialization with Sample Points
In this work, we propose two point initialization methods. The first initialization method can be used when we have sample points near the target geometry in hand.
This initialization method based on an observation that the vertices Voronoi diagram a point set tend to lie the medial axis of the target geometry (Amenta a. 1998a b). Therefore, for the given sample point set P. sample, we first build Voronoi diagram of it, and find Voronoi vertices Pvoronoi. Then, merge them initialize our point set F:
sample U PP poronoii
all of which weights are initialized to 1. Then, we set the real values (v) of points in P samppe 1, while setting those of points IP Proroooi as O.
In Figure [5] we render the mesh that we can get from this initialization method, when we use 10K sample points. Note that the initial mesh has lot of holes. because there could be Voronoi vertices that are located near the mesh surface, as pointed out by (Amenta u. a. 1998b). However, we can converge to the target mesh faster than the initialization method that we discuss below, because most of the points that we need are already located near the target geometry.
D.2.3 Point Initialization without Sample Points
If there is no sample point that we can use to initialize our points, we initialize our points with 3 points regularly distributed on grid structure that encompasses the domain, all of which has weight I and v value of 1. We set N 20 for every experiment (Figure 16a). Then, we optimize the mesh to retrieve coarse form of the target geometry (Figure 16b). Note that we need to refine this mesh in the subsequent epochs, explained below.
26

---

(a) Epoch Initial State
(b) Epoch Last State
(c) Epoch 2, Initial State
(d) Epoch Last State
(e) Epoch 3, Initial State
(I) Epoch 3, Last State
(g) Epoch 4, Initial State
(h) Epoch 4, Last State
Figure 16: Optimization process for multi-view reconstruction for Plant model. At each row, we present the initial state (left) and the last state (right) of each epoch. For each figure, the left rendering shows the point attributes color coded based on real values, while the right one shows the extracted mesh. (a), (b) In the first epoch, we initialize DMesh without sample points. At the end of each epoch, we sample points from the current mesh, and use them for initialization in the next epoch.
D.2.4 Point Initialization for Different Inputs
Until now, we introduced two point initialization techniques. When the input is point cloud, sample subset the point cloud to initialize our mesh (Figure [5). However, when the input multi-view images, we start from initialization without sample points (Figure 16). because there is no sample point cloud that we can make use of.
D.2.5 Maintaining FF
We maintain the rumning set of faces to evaluate probability existence for in F. At each iteration, after we get DT, we insert every face in W DT to F, as it has high possibility to persist in the subsequent optimization steps. Also, as we did int mesh to DMesh conversion (Appendix D.I). at every 10 optimization step, we find k-nearest neighbors for each point, and form face combinations based on them. Then, we add them to F.
27

---

D.2.6 Mesh Refinement
At start of each epoch, if it is not the first epoch, refine mesh by increasing the number of points. To elaborate, we refine our mesh by sampling N number of points on the current DMesh, and then initialize point attributes using those sample points as explained above. We increase N as number of epoch increases. For instance, in our multi-view reconstruction experiments, we the number of epochs 4, and N (1K, 3K, 10K) for the epochs excluding the first one. In Figure [6] we render the initial and the last state of DMesh of each epoch. Note that the mesh complexity increases and becomes more accurate epoch proceeds, because we more points. Therefore, this approach can be regarded as a coarse-to-fine approach.
D.2.7 Post-Processing
When comes to multi-view reconstruction, we found out that it is helpful to add one more constraint in defining the face existence, In our formulation, in general, face F has two tetrahedra (T₁, T2) that are adjacent to each other over the face. Then, we call the remaining point of T2 and T2 that is not included in F as P1 and Pz. Our new constraint requires at least one of P1 and P2 to have vi value of to let F exist.
This additional constraint was inspired by the fact that F is not visible from outside if F exists in our original formulation, and both Pi and P2 have 1 value of 1. That is, if not visible from outside, we do not recognize its existence. This constraint was also adopted to accommodate our real regularization, which increases the real value points near surface. If this regularization makes the real value of points inside the closed surface, they would end up in internal faces that are invisible from outside. Because this invisibility, our loss function cannot generate a signal to remove them. In the end, signal to remove them. Therefore, we choose to remove those internal faces by applying this new constraint in the pest-processing step.
expect all of the faces inside closed surface will exist, because of the absence of
Note that this discussion based on the assumption that our renderer does a precise depth testing. it does not do the accurate depth testing. internal faces can be regarded as visible from outside, and thus get false gradient signal. In Figure 13a the final mesh after phase rendered, and we can see therer are lots of internal faces the renderer used in phase does not support precise depth testing. However, we can remove them with the other renderer in phase 2, as shown in Figure [3b which justifies our implementation of two different renderers.
Finally, we note that this constraint is not necessary for point cloud reconstruction, because if we minimize CDours in Appendix C.2 the internal faces will be removed automatically.
E Experimental Details
In this section, we provide experimental details for the results in Section and visual renderings of the our reconstructed mesh. Additionally, we provide the results of ablation studies about regularizations that we suggested in Section 4.3
E.1 Mesh to DMMsh
As shown in Table I reconstruct the ground truth connectivity of Bunny, Dragon, and Buddha model from Stanford dataset (Curless und Levoy 1996). For all these experiments, we optimized for 20K steps, and used ADAM optimizer (Kingma und Ba 2014) with learning rate of 10- For Bunny model, we inserted additional points at every 5000 step. For the other models, we inserted them at every 2000 step.
In Figure 17 we provide the ground truth mesh and our reconstructed mesh. We can observe that most of the connectivity is preserved in our reconstruction, suggested numerically in Table [ However, note that the appearance of the reconstructed mesh can be slightly different from the ground truth mesh because allow 1% of positional perturbations to the mesh vertices.
28

---

Sn
(a) Ground Truth Mesh
(b) Reconstructed DMesh
Figure 17: Reconstruction results for mesh to DMesh experiment. From Left: Bunny, Dragon, and Buddha. We can observe that most the edge connectivity is perserved in the reconstruction, though the appearance is slightly different from the ground truth mesh because of small perturbations of vertex positions.
F,2 Point Cloud & Multi-view Reconstruction
E.2.1
Hyperparameters for Point Cloud Reconstruction
Optimizer: ADAM Optimizer, Learning rate 10-1 for open surface meshes and mixed surface meshes (Bigvegas, Raspberry) 3 3 10 -4 for closed surface meshes, and one mixed surface mesh (Plant).
Regularization:
Aweight 10-8 Areal 10-3 Aquat 10-3 for every mesh.
Number of epochs: Single epoch for every mesh.
Number of steps per epoch: 1000 steps for phase 1, 500 steps for phase 2 for every mesh.
E.2.2
Hyperparameters for Multi-view Reconstruction
Optimizer: ADAM Optimizer, Learning rate 10-3 in the first epoch, and 3 10-4 in the other epochs for every mesh.
Weight Regularization: Aweight 10 for every mesh.
Real Regularization: Areat 10 for the first 100 steps in every epoch for open surface meshes and one mixed surface mesh (Plant) / 10 for the first 100 steps in every epoch for closed surface meshes and two mixed surface meshes (Bigvegas, Raspberry).
Quality Regularization: Ayuat 10-3 for every mesh.
Normal Coefficient: Anormal for every mesh (Eq. 17
Number of epochs: 4 epochs for every mesh. In the first epoch, use 20 regularly distributed points for initialization. In the subsequent epochs, sample K, 3K, and 10K points from the current mesh for initialization.
29

---

(a) Ground Truth Mesh
(b) Flexicube
(c) Ours
Figure 18: Reconstruction results for closed surface model in Thingi32 dataset. Flexi-
cube (Sen u. a. 2023) can generate internal structures, while our approach removes them through post-processing.
(a) Ground Truth
(b) Flexicube
(c) Flexicube, self-intersecting faces removed
Figure 19: Reconstruction results for the Plant model. Flexicube (Shen u. a. 2023 can generate redundant, self-intersecting faces for open surfaces, in this case, leaves. To better capture the redundant faces, we rendered the models from upper side, which is shown in the bottom right figures.
Number of steps per epoch: 500 steps for phase 500 steps for phase for every mesh.
Batch size: 64 for open surface meshes, 16 for the other meshes.
E.2.3 Visual Renderings
In Figure 22 and 24 we provide visual renderings of our point cloud and multi-view reconstruction results with ground truth mesh. We also provide illustration input point cloud and diffuse map. Note that we also used depth renderings for multi-view reconstruction experiments.
E,2.4 Additional Discussion
Generally, we can observe that reconstruction results from both point cloud and multi-view images capture the overall topology well. However, we noticed that the multi-view reconstruction results not as good as point cloud reconstruction results. In particular, we can observe small holes in the multi-view reconstruction results. We assume that these artifacts are coming from relatively weaker supervision of multi-view images than dense point clouds. Also, we believe that we can improve these multi-view reconstruction results with more advanced differentiable renderer, and better mesh refinement strategy. In the current implementation, we lose connectivity information the start of each epoch, which is undesirable. We believe that we can improve this approach by inserting points near the regions of interest, rather than resampling over entire mesh.
30

---

(a) Bigvegas
(b) Plant
Figure 20: Point cloud reconstruction results with different Aweight- From Left: 10-6 10-6, and 10-4
Aweight
Also, regarding comparison to Flexicube (Shen u. 2023 in Table 2 we tried to found out the reason why ours give better results than Flexicube in terms of CD the ground truth mesh for closed surfaces in thingi32 dataset. We could observe that Flexicube's reconstruction results capture fine geometric details on the surface mesh, but also observed that they have lots of false internal structure (Figure 18). Note that this observation not only applies to closed surfaces, but also to open surfaces, where it generates lots of false, self-intersecting faces (Figure 19p. Our results do not suffer from these problems, as we do post-processing (Appendix D.2) to remove inner structure, and also our method can represent open surfaces better than the volumetric approaches without self-intersecting faces.
E.3
Ablation studies
In this section, we provide ablation studies for the regularizations that we proposed in Section 4.3 We tested the effect of the regularizations on the point cloud reconstruction task.
E.3.1 Weight Regularization
We tested the influence of weight regularzation in the final mesh by choosing Aweight in (10-6, 10-5 10-4). Note that set the other experimental settings same as described in Sec-
tion E.2 except Aquality. which is set as O, to exclude it from optimization.
In Table we provide the quantitative results for the experiments. For different Aweight, we reconstructed mesh from point clouds, and computed average Chamfer Distance (CD) and average number of faces across every test data. We can observe that there exists a clear tradeoff between CD and mesh complexity. To be specific, when Aweight 10-6 the CD not very different from the results in Table 2 where we use Aweight 10-8 However, when it increases 10-5 and 10-4 we can observe that the mesh complexity (in terms of number of faces) decreases, but CD increases quickly.
31

---

Bigvegas
(b) Plant
Figure 21: Point cloud reconstruction results with different Aquality 10-4, 10-3, and 10-2
From Left:
Areal
The renderings in Figure 20 support these quantitative re- sults. When Aweight 10 we observe good recon- struction quality. When Aweight 10-5, there are small artifacts in the reconstruction, but we can get meshes of generally good quality with fewer number of faces. How-
Table 5: Ablation study for weight regu-
larization, quantitative results.
Ameight
10-6
10-5
10-1
CD
7.48
8.08
10.82
ever, when it becomes 10-4, reconstruction results
Num. Face
4753
2809
1786
deteriorate, making holes and bumpy faces on the smooth surface. Therefore, we can conclude that weight regularization contributes to reducing the mesh complexity. However, we need to choose Aweight carefully, that it does not harm the reconstruction quality. The experimental results tell us setting Awiight to 10-6 could be a good choice to balance between these two contradictory objectives.
E.3.2 Quality Regularization
As we did in the previous section, we test the influence of quality regularization the final mesh by selecting Ayeat among (10-4,10 3,102,10-) We also set the other experimental settings same as before, except Ameight 0.
Table 6: Ablation study for quality regu- larization, quantitative results.
In Table 6 and Figure 21 we present quantitative and qualitative comparisons between the reconstruction results. We provide statistics about average CD, average number of faces, and average aspect ratio faces. Interestingly, unlike weight regularization, we could not observe tradeoff between CD and aspect ratio. Rather than that, we could find that CD decreases aspect ratio gets smaller, and Aspect Ratio
Apual CD
10-4
10-3
10-2 2
7.60
7.42
7.28
Num. Face
8266
8349
10806
2.33
2.06
1.55
thus the triangle quality gets better.
We find the reason for this phenomenon in the increase of smaller, good quality triangle faces. Note that there no significant difference between the number faces between Agual 10-4 and 10-3 Also, cannot find big difference between visual renderings between them, even though the aspect
32

---

ratio was clearly improved. However, when Aqual becomes 10 10-2, the number of faces increase fast, which can be observed in the renderings, too. We believe that this increase stems from our quality constraint, because it has to generate more triangles to represent the same area, if there is less degree of freedom to change the triangle shape. Since has more triangle faces, we assume that they contribute to capturing fine details better, leading to the improved CD.
However, the same time, note that the number of holes increase to visual artifacts. We assume that there are not enough points to remove these holes, by generating quality triangle faces that meet our needs. Therefore, as discussed before, we can find systematic way to prevent holes, or come up with a better optimization scheme to remove them, we expect that we would be able to get accurate mesh comprised of better quality triangles.
increase Aquel, which lead
33

---

(a) Mesh 164
(b) Mesh 30
(c) Mesh 320
(d) Mesh 448
Figure 22: Point cloud and Multi-view Reconstruction results for open surface models. From Left: Ground truth mesh sample point cloud, point cloud reconstruction results, diffuse rendering, multi-view reconstruction results.
34

---

(a) Mesh 64444
(b) Mesh 292119
(c) Mesh 313444
(d) Mesh 527631
Figure 23: Point cloud and Multi-view Reconstruction results for closed surface models. From Left: Ground truth mesh sample point cloud, point cloud reconstruction results, diffuse rendering, multi-view reconstruction results.
35

---

(a) Bigvegas
(b) Plant
(c) Mesh 313444
Figure 24: Point cloud and Multi-view Reconstruction results for mixed surface models. From Left: Ground truth mesh sample point cloud, point cloud reconstruction results, diffuse rendering. multi-view reconstruction results.
36

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In abstract and introduction, we described that we are presenting a differen- going explore various aspects of (e.g. computational cost,
tiable mesh, and reconstruction task, regularization), They are accurately discussed across the entire paper, including Appendix.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discussed the two limitations of our current approach in Section 0 which are about computational efficiency and manifoldness of the generated mesh.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
37

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [No]
Justification: Even though we tried our best to describe every theoretical detail in main paper, especially in Section 3.2 and Appendix [B] our method assumes that the reader has some background knowledge about the geoemtrical concepts. However, we specified the sources that the readers can refer to learn details about the theoretical claims we made the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: In Appendix D and E| we provided the pseudocode of our optimization process, and detailed hyperparameters reproduce the experimental results. Also, we included code in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
38

---

(d)
reproducible, they We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Excluding 2 models that we obtained from Adobe Stock, every model that used is publicly available. Also, we submitted our code. However, due to the size limitation of the supplementary material, we could submit the entire version of the code, because depends on many external libraries. But we believe readers can compare the code and the paper to learn details of our paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
Sttpaa////guuieee//eeuuubbbii wwwsssssiiiccoooeeoo for more details. / //ips.cccpublic/guicce httpp/ullligguuiee//eeeeeeeeee/e
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide experimental details in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
39

---

Justification: For the nature of our optimization based experiments, random initialization has little affect in the results. In addition, our experiments exam the effect of resolution and topology varieties, it does not use large amount of testing data. Thus, error hars for showing statistical significance of the experiments are not suitable for our paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provided the machine that we used for running experiments in Section 5 and reported execution time in the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
Our research conforms to the every guideline in the Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
40

---

10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our research mainly about geometry, especially about triangle mesh formu- lation. So we believe it does not possess societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our research does not deliver any trained model, but suggest a method for formulating differentiable mesh.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [No]
41

---

Justification: We cited the source of assets that we used in experiments in Section 5 but did not include license information about them.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, pappesswi thcode danasatss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's ss creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We used only existing assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our research does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
42

---

Answer:
[NA]
Justification:
Our research does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
43


Reviews:
[{"review_id": "NHvOOcMp6K", "reviewer": "Reviewer_cnRb", "summary": "The authors first propose a differentiable mesh representation, DMesh, which can represent a wider variety of mesh types. They then introduce a computationally efficient approach to differentiable weighted Delaunay triangulation that can run in approximately linear time. Following this, an efficient algorithm is provided for reconstructing surfaces from both point clouds and multi-view images, with DMesh used as an intermediate representation. Finally, an effective regularization term is proposed for mesh simplification and enhancing triangle quality.", "questions": "1. During the post-processing step, the inner structures are removed via depth testing. How does the method handle the boundaries caused by the previous operation?\n\n2. In line 91, \"... and it is not.\" should be completed for clarity. Maybe the authors missed some words at the end.\n\n3. The DMesh is constructed from the normal mesh. However, in real applications we often do not have access to the mesh. How does the DMesh help in reconstructing the result?", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The method proposed by this paper aims to optimize the geometry and topology of the reconstructed mesh. It presents a computationally efficient approach to differentiable weighted Delaunay triangulation, which can achieve this goal. The methods in this work are partly valuable to the NeurIPS community.", "weaknesses": "In this work, the inner structure is removed via a depth test, meaning that any reconstruction method producing an inner structure can be improved by using the depth test to remove the unseen parts of the mesh. While this post-processing step does help to improve the quality of the result, it is not a feature of the proposed method itself.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your comments, but we believe there may have been some major misunderstandings about our work. We have tried to address your comments as thoroughly as possible. If we have not correctly addressed your concerns, please let us know.\n\n---------------------------\n\n**Q1. Weakness about “depth testing”**\n\nA1. We believe there was a misunderstanding regarding the term \"depth test\" in our manuscript. We used this term to *describe the differences between the two renderers we employed in the multi-view reconstruction task* (Appendix C.3.1, C.3.2).\n\nIn estimating the relative ordering between faces, the first renderer $F_A$ (Appendix C.3.1) can produce incorrect orderings because we compute the depth value of each face globally, which might not align with local orderings. This could result in inner faces being perceived as “seen” from the outside, leading to false inner structures (Figure 11(a)). This renderer is used in the first phase of optimization.\n\nIn contrast, the second renderer $F_B$ (Appendix C.3.2) produces precise orderings between faces, thereby removing these false inner structures (Figure 11(b)). This renderer is used in the second phase of optimization, not during post-processing.\n\nIn the post-processing step, we remove unseen inner faces based on the observation inherent to our approach (Appendix D.2.7), which tessellates the entire domain. Since other methods do not use this kind of tessellation, this approach is not applicable to them.\n\nWe acknowledge that our use of the term could have been misleading. However, as described above, *it differs from conventional \"depth test\" or \"visibility test\" techniques that can be applied to any reconstruction method*. We used the term to *describe a situation inherent to our approach, not a general case*. *We did not perform any conventional \"depth test\" or \"visibility test\" to remove inner structures*. We will clarify this point in the revised version.\n\n-------------------------\n**Q2. During the post-processing step, the inner structures are removed via depth testing. How does the method handle the boundaries caused by the previous operation?**\n\nA2. As described above, the post-processing step in our method is based on principles inherent to our approach, not a general depth testing. Therefore, it does not produce additional boundaries on its own or get affected by boundaries produced in previous optimization phases.\n\n-------------------------\n**Q3. In line 91, \"... and it is not.\" should be completed for clarity. Maybe the authors missed some words at the end.**\n\nA3. Thank you for pointing this out. We intended to write: \"... and is only confined to handle point clouds.” We will correct this in the revised version of the paper.\n\n\n-------------------------\n**Q4. The DMesh is constructed from the normal mesh. However, in real applications, we often do not have access to the mesh. How does the DMesh help in reconstructing the result?**\n\nA4. When we reconstruct a mesh from point clouds or multi-view images, **we do not use the normal (ground truth) mesh as input**. In our experiments, we sample point clouds or capture multi-view images of the ground truth mesh, which are then fed into the reconstruction algorithm. *The ground truth mesh is used only to evaluate the reconstruction quality*. Thus, *our reconstruction algorithm can be used even when there is no normal mesh available*. Please also refer to the videos about point cloud and multi-view reconstructions in the index.html file in our supplementary material. These videos demonstrate that our method does not require the normal mesh as input.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aTba58Zw6V", "reviewer": "Reviewer_yemP", "summary": "The work introduces a differentiable mesh representation where both the topology of the surface and the connectivity are differentiable. The paper builds on previous work titled 'Differentiable Surface Triangulation,' which suggests using a soft relaxation of Weighted Delaunay Triangulation (WDT) to differentiate through meshing and vertex positions. DMesh highlights that the original work cannot be used as a differentiable mesh representation in a straightforward way and would be very inefficient due to the quadratic scaling of the time complexity with the number of vertices. In light of this, the paper suggests approximating the weights in Delaunay Triangulation using a tight lower bound, which significantly accelerates the algorithm, making it practically linear. Additionally, the work introduces another per-vertex parameter that defines the probability of existence of each face that includes this vertex (given that it exists in WDT). Extensive validation shows that DMesh is capable of reconstructing complicated meshes, including open surfaces and non-orientable surfaces. Furthermore, DMesh outperforms the baselines on the tasks of surface reconstruction from a point cloud and multi-view rendering, while also being a more flexible and general representation.", "questions": "Here I have a few questions that don't affect the rating, but are rather out of curiosity. Including them in the final manuscript is up to the authors:\n1. It looks like the representation allows the user to freeze the connectivity of the mesh if they want to. Is it also possible to \"freeze\" surface topology ?\n2. Did you have to prevent the optimization process from converging to binary values of the probability ? (which I believe will make the mesh effectively non-differentiable ?)\n3. is there anything we can say about the existence and uniqueness of the additional two parameters ($\\Lambda_{\\text{wdt}}$ and $\\Lambda_{\\text{real}}$) when reconstructing a mesh with the reconstruction loss from section 4.3.1. ?\n\nAfter the authors addressed the questions in the rebuttal, increasing my rating to strong accept.", "limitations": "The limitations are discussed in great detail, apart from the lack of examples of non-manifold meshes as mentioned in Weaknesses. The broader impact discussion does not seem to be necessary here.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is easy to follow, and despite the complicated nature of the topic, the narration remains clear and formal; main concepts are well illustrated. The proposed modifications to Differentiable Surface Triangulation are significant and not straightforward. Extensive evaluation of the proposed method proves its effectiveness and significance for the field.\n\n*Additional notes*:\n+ Many regularization terms can be easily expressed natively for this representation, as demonstrated in the paper.\n+ To the best of my knowledge, the paper compares against all the required baselines.\n+ The supplementary material provides seemingly all the details needed to reproduce the algorithm.\n+ The paper provides an efficient CUDA implementation of the algorithm (though I did not test it).\n+ The demonstrated results are very impressive, and the resulting meshes look very clean.\n+ The limitations of the paper are discussed in great detail.\n\nGiven all of the above, I believe this paper is well-positioned to significantly contribute to the community and potentially return attention to explicit representations in many fields, such as differentiable rendering, physics-based optimization, and so on.", "weaknesses": "I am a bit confused why the paper states that the running time of “Differentiable Surface Triangulation” growth exponentially with the number of vertices, while in fact it is quadratic? I would like to see either an explanation or a corrected version by the rebuttal.\n\nWhile the method is very well evaluated and ablated, I think a few additional results will be interesting to see. Here they are in the order of decreasing significance:\n\n1.\tThe biggest question to me so far is if the interpolations between two DMeshes are possible, especially if there is a topology/connectivity change. All the presented experiments showcase an optimization process that can start in a good initialization of the DMesh. My concern is that, after convergence the DMesh should get somehow more binary in $\\Lambda_{\\text{wdt}}$ and $\\Lambda_{\\text{real}}$, what will make it hard to take it out of this local minima in the further optimization process. By the rebuttal I would like to see a simple demonstration of a mesh-to-mesh optimization with a surface topology change, or a discussion of this in the limitation section if this is not trivial.\n\n2.\tIn the limitations it is mentioned that sometimes the resulting mesh is non-manifold. It would be interesting to see some examples of failures and an estimation how often it happens and why? (at least I did not find this in the paper).\n\n3.\t(Minor) The point cloud reconstruction results are demonstrated for 100k sampled points, which is very dense. It would be interesting to see how the quality of mesh reconstruction depends on the number of sampled points.\n\n4.\t(Minor) Table 2 is evaluated only on 11 meshes, which is not much. Would be great to see a bigger test set by the camera-ready deadline.\n\n*I would be ready to even further increase my rating if these questions are addressed.*\n\nMinor Notes and Typos:\n\n- Line 91: “it is not ..”  seems to miss something\n- Would be great to increase the resolution of illustrations. In particular, the pixelation is very visible on Figure 6., at least in my PDF viewer\n- Footnote 3 on page 5 – seems to be a typo around “and p_j and p_k”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your detailed comments and positive evaluation about our work. Please let us know if we fully address your questions.\n\n------------------------\n**Q1. Description about running time of “Differentiable Surface Triangulation”**\n\nA1. Thank you for pointing out the error. We acknowledge that our description was incorrect. We will change the description to \"quadratic\" instead of \"exponential\" in the revised version.\n\n-------------------------\n**Q2. Interpolations between two DMeshes, especially with topology/connectivity changes**\n\nA2. Please see Global Response A3. Additionally, please refer to Q8 below to understand how we prevent our formulation from becoming too binary or non-differentiable.\n\n-------------------------\n**Q3. Examples of non-manifold meshes, estimation of frequency, and reasons for occurrence**\n\nA3. Please see Global Response A2.\n\n-------------------------\n**Q4. Point cloud reconstruction results with varying densities**\n\nA4. Thank you for the suggestion. We will include experimental results in the final version of the paper to demonstrate how the quality of mesh reconstruction depends on the number of sampled points. We expect the results to degrade as the number of sampled points decreases since we are optimizing only Chamfer distance loss without any topological assumptions. Developing a more robust reconstruction algorithm based on our method would be an exciting future research direction.\n\n-------------------------\n**Q5. Evaluation on a larger test set**\n\nA5. Thank you for the suggestion. We will include more examples from the Objaverse dataset in the final version of our paper.\n\n-------------------------\n**Q6. Minor typos**\n\nA6. Thank you for identifying these. We will fix them in the revised version.\n\n-------------------------\n**Q7. Freezing surface topology**\n\nA7. As we understand, surface topology corresponds to geometric topology (e.g., genus of the shape) rather than mesh topology (individual edge connectivity) (Appendix A). If this is the case, we can freeze the surface topology by freezing all mesh connectivity, as you suggested. However, if you refer to *changing mesh connectivity while freezing surface topology, our method does not support this functionality yet*, because our method does not make any specific assumptions about surface topology during optimization. This would be an interesting research direction to explore. Please let us know if we understood this question correctly.\n\n-------------------------\n**Q8. Preventing optimization from converging to binary values of probability**\n\nA8. To answer this question, let us consider two cases where the existence probability of a face converges to either 1 or 0.\n\nFirst, if the probability has converged to 1, but we want to decrease it, we can simply adjust the real values stored in the 3 vertices of the face to reduce its probability. Since $\\Lambda_{real}$ does not include any nonlinear function like sigmoid, we do not worry about falling into a local minimum due to the vanishing gradient problem.\n\nNext, let us assume the probability has converged to 0, but we want to increase it. There are two possibilities for the probability converging to 0: $\\Lambda_{wdt}$ becoming 0 or $\\Lambda_{real}$ becoming 0. If $\\Lambda_{wdt}$ is almost 0, we did not use any measures because there are many possible combinations with very small $\\Lambda_{wdt}$. Enumerating all possible face combinations (up to $n \\choose 3$) and computing $\\Lambda_{wdt}$ for them requires significant computational cost, making it impractical to amend this problem efficiently. However, using WDT is beneficial because the face existence probability usually increases as its vertices become closer. Thus, we observed that reconstruction loss moves points to minimize the loss, usually increasing (formerly very small) $\\Lambda_{wdt}$ of the desirable face.\n\nIf the probability of a face converges to 0 because $\\Lambda_{real}$ is 0, we need a measure to prevent it because we ignore faces with very small probability to reduce the computational burden (Appendix C.3, C.5). These faces cannot get gradients to escape zero probability without any measures. Hence, we introduced the real regularizer in Appendix C.5. The regularizer increases the real value of points connected to points with high real values, helping escape local minima effectively.\n\nWe will include this detailed discussion about local minima in the revised version of the paper.\n\n-------------------------\n**Q9. Existence and uniqueness of $\\Lambda_{wdt}$ and $\\Lambda_{real}$.**\n\nA9. Thank you for the question. We believe $\\Lambda_{wdt}$ is related to the tessellation of the spatial domain, and $\\Lambda_{real}$ is about selecting desired faces from the tessellation. Thus, $\\Lambda_{real}$  depends on $\\Lambda_{wdt}$: if tessellation changes, selected faces may change accordingly. When $\\Lambda_{wdt}$ is fixed (and thus tessellation is fixed), we could argue that $\\Lambda_{real}$ that globally minimizes the reconstruction loss would exist and be unique.\n\nRegarding $\\Lambda_{wdt}$, we refer to literature on WDT [1]. Certain constraints, like those related to Local Feature Size (LFS), guarantee we can recover a topologically equivalent object to the ground truth mesh by selecting faces from (W)DT. We believe this applies to our case and guarantees the existence of a solution theoretically. However, for uniqueness, we do not think only one $\\Lambda_{wdt}$ reconstructs the given geometry. For instance, we can subdivide faces on the current DMesh by inserting additional points or end up with different $\\Lambda_{wdt}$ due to different initialization. Thus, we believe we can theoretically guarantee the existence of $\\Lambda_{wdt}$ but not its uniqueness.\n\n[1] Cheng, Siu-Wing, et al. Delaunay mesh generation. Boca Raton: CRC Press, 2013.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z4iUhSOTZQ", "reviewer": "Reviewer_8dk4", "summary": "The paper presents a differentiable mesh representation for mesh reconstruction from input meshes, point clouds, and for multi-view reconstruction. It builds off ideas of Rakotosaona 21, and uses a weighted Delaunay triangulation framework to predict simplex probabilities. They borrow the notion of distance to reduced power cell, but consider lower bound estimations and precomputation to speed it up. Their method is the first to give a differentiable mesh representation that is flexible enough to handle open/closed meshes, and accommodate changes in mesh topology and connectivity. The method is tested against several competing SOTA works and performs better or comparably in all scenarios (save for producing exactly manifold output).", "questions": "1. As noted I had some technical questions about S4.1:\n    * For Eq. (5), can you give an instance when this lower bound is not sharp? I thought that if the simplex does not exist in the WDT, then the power cell would remained unchanged under deletion of the other two points?\n    * How is it that one approximates the reduced power cell? This is not explained at all, unless I missed something? \n2. Many of the mins,maxes, absolute values, lengths, etc. that appear in the regularization terms appear to be non-differentiable. Are these simply smoothed, or am I missing something? \n3. What does the optimization do when there is not a shot showing a particular side of the model? E.g. from the bottom of a model? \n4. Why were \"reality\" weights vertex-based? Was there any consideration for edge-based weights? \n5. How often does non-manifoldness occur?", "limitations": "The authors appropriately acknowledged their limitations, some of which I listed in Weaknesses above.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. As noted, the framework is the first to directly represent meshes in a differentiable way that can handle topology change.\n2. They extend (in a straightforward way) the framework of Rakotosaona et al, to the setting of surfaces in 3D, and note the possibility of its use for other k,d (though do not demonstrate use in these scenarios).\n3. They provide computational speedup methods that help to alleviate the computational burden of power diagram computation.\n4. The authors provide extensive validation and implementation details in the supplementary, including some ablation studies.", "weaknesses": "1. The method does not guarantee manifoldness of the end mesh result, which is acknowledged by the authors.\n2. The matter of computational cost is swept under the rug a bit. It would be nice to present detailed statistics of some sort (over meshes/point clouds of various sizes, etc.) in the main paper. \n3. It is unclear to me how much technical novelty is contained in the estimation step of 4.1 over the prior framework. The steps seem relatively natural and easy to consider once you realize that the dual simplex at hand is no longer a simple point.\n4. The earlier parts of the article could be compacted to include at least an overview of the technical pipeline used in the experiments. There are a lot of details that are omitted entirely in each use scenario, especially for the multi-view reconstruction. \n5. There is a small, confusing technical mistake above equation (3). The signed distance function should be positive inside, and negative when outside for this to make sense. This is as done in Rakotosaona.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your detailed comments and positive evaluation about our work. Please let us know if we addressed your questions correctly.\n\n------------------------------\n**Q1. Non-manifoldness**\n\nA1. Please see Global Response A2.\n\n------------------------------\n**Q2. Computational cost**\n\nA2. Thank you for the suggestion. Please see Global Response A1 for detailed statistics about the computational cost. We will include this information in the final version of the main paper.\n\n------------------------------\n**Q3. Technical novelty in the estimation step of 4.1**\n\nA3. There were two main motivations for developing the lower bound estimation step in 4.1 instead of using the prior framework: precision and efficiency, as suggested at the end of 3.2. As shown in the paper, our estimations were much more precise than the previous method. Regarding efficiency, our approach significantly reduces the computational burden, *even when the dual simplex is a single point* (Table 4, d=2, k=2 case). *We argue that our approach effectively relaxes the computational challenges of the previous method, despite being based on simple principles*. Additionally, *its implementation was not as straightforward as the prior framework*, which is why we used CUDA instead of PyTorch. For instance, while the prior framework only requires a point projection operation, we needed other operations, such as line-line distance computation. Therefore, *we believe the technical novelty is not trivial*.\n\n------------------------------\n**Q4. Including an overview of the technical pipeline in the main body**\n\nA4. Thank you for the suggestion. We will modify the paper to incorporate more details about the experiments in the main body if we are granted an additional page in the camera-ready version.\n\n------------------------------\n**Q5. Technical mistake about description above equation (3)**\n\nA5. Thank you for pointing this out. You are correct that the description was wrong. Specifically, Eq. 3 is correct, but in line 154, we should have written, \"sign is positive when inside.\" We will correct this in the revised version of the paper.\n\n------------------------------\n**Q6. Sharpness of the lower bound in Eq. (5)**\n\nA6. Thank you for the insightful question. Let's consider an example under the same setting as Figure 6. In Figure 6(d), we rendered the case when $\\Delta^{1} = \\{ p_1, p_7 \\}$. As you pointed out, the reduced power cell of $p_1$ remains unchanged when we delete $p_7$. However, consider the case where $\\Delta^{2} = \\{ p_1, p_3, p_7 \\}$, which also does not exist in the WDT. In this scenario, the reduced power cell of $p_1$ changes when we delete $p_3$ and $p_7$ because $p_1$ and $p_3$ are connected in the power diagram. This observation also applies to our (d=3, k=2) case.\n\n------------------------------\n**Q7. Approximating the reduced power cell**\n\nA7. We briefly mentioned approximating the reduced power cell with pre-computed PD but did not provide details. To elaborate, as we compute PD at every optimization step, we collect the points that comprise the power cell of a point $P$ at each step. By keeping and accumulating these points, we can effectively find possible half-planes that could comprise the reduced power cell of $P$. Therefore, we use the half-planes between $P$ and the (accumulated list of) points to approximate the (boundary of) reduced power cell of $P$. Although this approximation is not very accurate in the initial optimization steps, it does not undermine the optimization process. This is because, as optimization progresses, most possible half-planes to define the reduced power cell are collected for each point. We will include this detail in the revised version of the paper.\n\n------------------------------\n**Q8. Non-differentiability of regularization terms**\n\nA8. For min/max functions, we used a smoothed version because they are used to compute face probabilities, a main component of our approach. We found that differentiable min/max functions stabilize the optimization process. However, we used absolute values and lengths without modification because they are almost differentiable and used mainly for regularizers, which did not harm the optimization process.\n\n------------------------------\n**Q9. Handling unseen parts of the model in multi-view reconstruction**\n\nA9. In multi-view reconstruction tasks, we use real regularization as described in Appendix C.5. This regularization automatically fills unseen parts of the object (e.g., inner parts). In the post-processing step (Appendix D.2.7), we discard most of these automatically generated, invisible faces. Therefore, when a particular side of the model is not depicted, that part would be reconstructed as the maximum volume that does not violate observations from other views. We will include an example of this case in the revised version of the paper.\n\n------------------------------\n**Q10. Vertex-based reality values**\n\nA10. We did not consider edge or face-based reality values because we aimed to store all mesh information in the points. This design will simplify its adoption in other ML frameworks, such as generative models. It is much easier for neural networks to generate point-wise features than generate unfixed edge or face combinations. By storing all connectivity information in points, we avoid generating face combinations. It is also more efficient since the number of edges or faces becomes excessive as the number of points increases. Storing real values for such edges or faces is overwhelming. However, edge or face-based reality values could be useful in the final optimization step to fine-tune geometry and remove non-manifoldness, as discussed in Global Response A2.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7baSwFj5ae", "reviewer": "Reviewer_jRqE", "summary": "This paper proposes a novel differentiable mesh representation, which focuses on both the vertex position and connectivity. To achieve this, authors come from the probability of face existence, and decompose the probability into two parts: 1) the WDT probability; 2) a probability of vertices' existence on surface. Further, to accelerate the WDT process, authors adopt a novel relaxation operation, making the whole design applicable.", "questions": "I think this is a good work and I believe further improvements as mentioned in the weakness are not easy to complete during the rebuttal. I suggest the author to add experiment on ablation of the influence of \\lambda_WDT and \\lambda_real to make the core contribution more convincing.", "limitations": "Yes, authors discussed limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+: The probabilistic approach to determining the existence of faces in the mesh is innovative and allows for gradient-based optimization of both vertex positions and their connectivity, which reveals the potential application in many optimization or learning-based reconstruction tasks. \n\n+: Both the efficient CUDA implementation and the relaxation design further enhance the usability.\n\n+: Sufficient evaluations on settings: mesh reconstruction from ground truth, point clouds and multi-view images, verifies the effectiveness of the paper. And the quantitative comparisons with other methods are well-presented and solid.", "weaknesses": "-: Although the method is efficient compared to traditional approaches, the computational cost is still significant, especially when dealing with large datasets. This prevents the application beyond small object level task.\n\n-: While the method is tested on synthetic datasets and controlled environments, there is a lack of demonstration on real-world datasets, or texture-rich dataset. Applying DMesh to more complex scenarios would strengthen the paper's contributions.\n\n-: No ablation on the influence of \\lambda_WDT and \\lambda_real.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your detailed comments and positive evaluation about our work. Please let us know if we addressed your questions correctly.\n\n------------------------------------\n**Q1. Significant computational cost**\n\nA1. Yes, in the current implementation, we can handle up to 20K points efficiently, but going beyond that is limited. Please see the detailed analysis about the computational cost in Global Response A1. To overcome this limitation, we need to take a completely different approach that does not require the WDT algorithm, as it is the main computational bottleneck. This is one of our future research directions.\n\n------------------------------------\n**Q2. Testing on real-world and texture-rich datasets**\n\nA2. As pointed out, our current implementation does not support texture reconstruction, which is why our method is not yet applicable to real-world datasets. This is also a future research direction we intend to explore.\n\n------------------------------------\n**Q3. Ablation on the influence of $\\Lambda_{wdt}$ and $\\Lambda_{real}$**\n\nA3. We believe there might be a misunderstanding or typo in this question, as $\\Lambda_{wdt}$ and $\\Lambda_{real}$ are multiplied to get the final face existence probability (Eq. 2). They are not hyperparameters that we can arbitrarily choose. Therefore, there is no specific ablation study we can conduct on them. If you meant $\\lambda_{weight}$, $\\lambda_{real}$, and $\\lambda_{qual}$, which are the coefficients for regularization (Section 4.3.2), please see Figure 7 and Appendix E.3 for the ablation study of these coefficients. Please let us know if we misunderstood your question.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DlEgSOgAiJ", "reviewer": "Reviewer_7cDg", "summary": "This paper proposed a novel differentiable representation of a mesh. It takes advantage of two probabilities, $\\Lambda_{wdt}$ and $\\Lambda_{real}$, to generate the mesh, such that the mesh is differentiable.", "questions": "As there are spurious non-manifold errors in the mesh, what are the ratio of the non-manifold errors?\n\nIs the method robust against noises and holes? Current experiments did not test point clouds with noises or holes.", "limitations": "The limitations should discuss the potential limits to process indoor/outdoor scenes. These data are more challenge due to noises, holes and complicated topology.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Based on weighted Delaunay triangulation and its dual power diagram, the paper proposed a novel approach to compute $\\Lambda_{wdt}$, which makes the WDT generation differentiable.\n\nIt seems that the accuracy of the conversion from mesh to DMesh is high enough. The paper applied the novel representation for various applications, such as, point cloud and multi-view reconstruction.", "weaknesses": "The efficiency of the method seems some what slow due to the WDT construction.\n\nIn the experiments, it is better to show the deformation process of the a mesh with different topologies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your detailed comments and positive evaluation about our work. Please let us know if we addressed your questions correctly.\n\n-----------------------------------------------\n**Q1. Degraded efficiency of the method due to WDT**\n\nA1. As pointed out, the main computational bottleneck in the current implementation is the WDT construction. Please see Global Response A1 for detailed information. Removing the necessity for WDT is one of our future research directions.\n\n-----------------------------------------------\n**Q2. Showing the deformation process of a mesh with different topologies**\n\nA2. Thank you for the suggestion. Please see Global Response A3. We will include this in the revised version of the paper.\n\n-----------------------------------------------\n**Q3. Ratio of non-manifold errors**\n\nA3. Please see Global Response A2 for details on the ratio of non-manifold errors.\n\n-----------------------------------------------\n**Q4. Robustness against noises and holes in point cloud reconstruction**\n\nA4. As pointed out, we did not test our method on point clouds with noises or holes. Currently, our method may not be suitable for such challenging cases because our reconstruction is driven solely by Chamfer Distance loss. Consequently, holes or noises in the input point clouds may persist in the reconstructed mesh. Other point cloud reconstruction methods often impose assumptions about the mesh topology to address these issues. However, our approach does not impose any topological constraints, which is contradictory to these methods.\n\nAdditionally, developing a robust point cloud reconstruction method is beyond the scope of this paper, as the main contribution lies in introducing the differentiable mesh formulation, not the reconstruction itself. Nevertheless, we acknowledge the importance of this aspect and consider it an interesting direction for future research.\n\n-----------------------------------------------\n**Q5. Limitations in processing indoor/outdoor scenes**\n\nA5. Yes, we will include a discussion of the limitations related to processing indoor and outdoor scenes in the paper, as these data are more challenging due to noises, holes, and complicated topology. Thank you for the suggestion.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thank you for all of the reviews. We can further improve this paper based on your feedback.

Before addressing the specific points, *we encourage the reviewers to refer to the index.html file in the supplementary material, which contains videos visualizing the meshes during optimization in our experiments*.

---

**Q1. Details about computational cost (Reviewer 8dk4, jRqE, 7cDg)**

A1. We conducted additional experiments to analyze computational costs, particularly time. **Figure 1** in the attached PDF shows how computational time changes with the number of points.

We classified computational time into WDT (computing WDT and PD of given points) and Prob (computing face probabilities based on WDT and PD). Refer to Algorithm 2 in Appendix D for details. For each point count, we generated 3D point clouds in a unit cube with uniform weight 0 and ran each setting 5 times to get average running times.

As shown in the graph, computational cost increases with the number of points due to an increase in possible faces, thus increasing Prob's cost. However, *WDT takes most of the time, validating our claim that WDT is the main computational bottleneck in the current implementation*. Although the cost rises rapidly beyond 20K points, the algorithm is affordable below 20k and efficient below 10K points. In fact, most single objects can be represented with ~4K points, as shown in Table 2 in paper. *Therefore, our approach is efficient for most single objects*. However, larger scenes require more points, potentially up to millions. We are exploring an accelerated implementation of DMesh that efficiently handle millions of points.

---

**Q2. Details about non-manifoldness (Reviewer yemP, 7cDg)**

A2. To explain cases where the resulting mesh is non-manifold, let's revisit the combinatorial definition of manifoldness. First, an edge should be incident to at most 2 faces. An edge incident to more than 2 faces is a non-manifold edge. Second, faces incident to a vertex should form a closed or open fan. A vertex not meeting this condition is a non-manifold vertex. Finally, there should be no self-intersections between faces. Our method guarantees no self-intersections, so our measurement focuses on the first two conditions. Testing on 11 manifold models, we measure the average ratio of non-manifold edges and vertices.

For point cloud reconstruction results, 5.50% of edges were non-manifold, and 0.38% of vertices were non-manifold. For multi-view reconstruction results, 6.62% of edges were non-manifold, and 0.25% of vertices were non-manifold. Therefore, *non-manifold edges are more significant than non-manifold vertices in our approach*. The main reason for non-manifold edges is a corner case in our formulation. We select faces using point-wise real values from each tetrahedron in the tessellation, but *if we need to select 2 or 3 faces, all points should have a real value of 1, resulting in selecting all faces that could produce non-manifold edges*. Please refer to **Figure 2** in the attached PDF for illustrations. Because of this ambiguity, *redundant faces are created and non-manifold edges arise from them*. We believe *optimizing face-wise real values in post-processing can considerably reduce non-manifold edges*, thereby avoiding this corner case.

Despite not preventing non-manifold cases, we see this as a *double-edged sword regarding representation capacity*.  Enforcing manifold constraints is beneficial for representing real-world objects, BUT less so for abstract geometries, where non-manifold cases often arise, e.g., the plant and great stellated dodecahedron in the teaser and Bigvegas in Appendix E. In fact, many popular 3D assets, such as Mixamo, contain non-manifold meshes for cloth and surfaces. *Therefore, we purposely design our method to be a versatile geometric representation that encompasses all these cases*. Exploring constraints to guarantee *user-desired level of manifoldness* using our proposed representation is a promising future research direction for application-specific use and we would be happy to highlight it in the revision.

---

**Q3. Interpolations between DMesh exhibiting topology / connectivity changes during deformation (Reviewer yemP, 7cDg)**

A3. Firstly, we would like to emphasize that the videos embedded in the index.html file already demonstrate how topology / connectivity changes during optimization. We would appreciate it if reviewers could please review these supplementary videos.

We agree that it would be beneficial to see interpolations between DMesh of different (surface) topologies, especially starting from a converged DMesh due to local minima issues. To illustrate this, we first fitted our DMesh to a **single torus** (torus with 1 hole), and then optimized it to 2 other shapes, which include a **similar shape with the same genus**, and **double torus** (torus with 2 holes), assuming the multi-view images are provided as input. Please refer to **Figure 3** in the attached PDF, which shows the optimization process.

As demonstrated in Figure 3, *our method successfully recovers the target shape, even when starting from an already converged single torus shape*. Therefore, we argue that *concerns about local minima may not be as significant*. This also highlights *our approach's capability in optimizing topology explicitly*. However, we acknowledge that more complex cases might exist. For such cases, we propose using our *coarse-to-fine strategy* (Appendix D.2), to *effectively escape (possible) local minima*.  Therefore, we claim that *our method can robustly address local minima issues*.

However, in Figure 3, we observe that the intermediate states between the two shapes are not semantically meaningful. Even when we optimize for a shape of same surface topology (blue), it does not go through a smooth interpolation. This is because *we do not have any topological assumption*, which is double-edged sword of our formulation, as discussed in above A2.

Meta-review: This paper introduces a differentiable representation for meshes and efficient algorithms supporting it, allowing surface reconstruction. The paper has received favorable reviews from five expert reviewers, who generally appreciated the idea, efficiency (including a CUDA implementation), good presentation and writing, extensive evaluations (including appendix the paper is 42 pages long ...), extensive comparisons, discussed limitations, etc.

Some weaknesses were raised (manifoldness, remaining computational cost), which did not impact the suitability for publication.
The AC concurs and judges the paper to be strong and ready for publication.
Decision: Accept (poster)