Neural Volumetric Mesh Generator
Authors: Yan Zheng
Lemeng Wu
Xingchao Liu
Zhen Chen
qiang liu
Qixing Huang
Venue: Submitted to ICLR 2023 (2023)

Abstract: Deep generative models have shown success in generating 3D shapes with different representations. In this work, we propose Neural Volumetric Mesh Generator (NVMG), which can generate novel and high-quality volumetric meshes. Unlike the previous 3D generative model for point cloud, voxel, and implicit surface, the volumetric mesh representation is a ready-to-use representation in industry with details on both the surface and interior. Generating this such highly-structured data thus brings a significant challenge. We first propose a diffusion-based generative model to tackle this problem by generating voxelized shapes with close-to-reality outlines and structures. We can simply obtain a tetrahedral mesh as a template with the voxelized shape. Further, we use a voxel-conditional neural network to predict the smooth implicit surface conditioned on the voxels, and progressively project the tetrahedral mesh to the predicted surface under regularization. The regularization terms are carefully designed so that they can (1) get rid of the defects like flipping and high distortion; (2) force the regularity of the interior and surface structure during the deformation procedure for a high-quality final mesh. As shown in the experiments, our pipeline can generate high-quality artifact-free volumetric and surface meshes from random noise or a reference image without any post-processing. Compared with the state-of-the-art voxel-to-mesh deformation method, we show more robustness and better performance when taking generated voxels as input.

Under review as a conference paper at ICLR 2023
NEURAL VOLUMETRIC MESH GENERATOR
Anonymous authors Paper under double-blind review
ABSTRACT
Deep generative models have shown success in generating 3D shapes with different representations. In this work, we propose Neural Volumetric Mesh Generator (NVMG), which can generate novel and high-quality volumetric meshes. Unlike the previous 3D generative model for point cloud, voxel, and implicit surface, the volumetric mesh representation is a ready-to-use representation in industry with details on both the surface and interior. Generating this such highly-structured data thus brings a significant challenge. We first propose a diffusion-based generative model to tackle this problem by generating voxelized shapes with close-to-realigy outlines and structures. We can simply obtain a tetrahedral mesh as template with the voxelized shape. Further, we use a voxel-conditional neural network predict the smooth implicit surface conditioned on the voxels, and progressively project the tetrahedral mesh to the predicted surface under regularizations. The regularization terms are carefully designed so that they can (1) get rid of the defects like flipping and high distortion; (2) force the regularity of the interior and surface structure during the deformation procedure for a high-quality final mesh. As shown in the experiments, our pipeline can generate high-quality artifact-free volumetric and surface meshes from random noise or a reference image without any post- processing. Compared with the state-of-the-art voxel-to- mossthh-ess deformation method, we show more robustness and better performance when taking generated voxels as input.
INTRODUCTION
How to automatically create high-quality new 3D contents that are accessible and editable is key problem in visual computing. Although generative models have revealed their power on audio and image synthesis (Goodfellow et al.
& Welling 2013 Higgins et al. 2016 Goodfellow
2014 Brock al. 2018 Ho et al. 2019; [Song & Ermon 2019 Ho et al. 2020 Song
et al. 2020). their pertormance remains limited. A major challenge of the current methods is the
representation of the 3D shapes. Many generative models focus on point clouds (Fan et al. 2017 Yang et al.) 2019 Cai et al., 2020; Luo & Hu 2021a|b). However,
non- con-trivial to convert point
clouds to other shape representations. Another line of work (Park et al. 2019: Niemeyer & Geiger
2021 Schwwrr al. 2020 Jain al. (2021) directly learns generate implicit representations, e.g., neural radiance field (NeRF) (Mildenhall et al. 2020), of shapes. However, for applications in physical simulation, the implicit representation needs to be converted into explicit representations such as meshes, which by itself is not completely solved problem.
In this work, we consider the problem of directly generating ready-to-use volumetric meshes. Vol- umetric mesh is one of the most important representations of 3D shapes, which is widely adopted
in computer graphics and engineering (Nieser et al., 2011 Hang 2013 Hu et al. 2018). However,
it is difficult to be generated with off-the-sheli generative models due to a number of geometric constraints (Aigerman & Lipman, 2013) Li et al.) 2007: 2020 2021 Ni et al.) 102T). Without carefully handling these constraints, the generated meshes have various defects, including flipping. self-intersection. large distortion, etc. To overcome the constraints, existing methods (Wang et al.
2018 Wen et al. 2019 Gupta & Chandraker 2020 Shi et al. 2021) for deep mesh generation usually learn deformation on a template mesh e.g., an ellipsoid mesh, to obtain a new mesh. Unfortunately, the usage of a template mesh limits the topology (number of holes) and large deformation of the generated meshes.
Thus, we present novel pipeline, termed Neural Volumetric Mesh Generator (NVMG), for learning generative models of volumetric meshes. Unlike focusing on designing the neural network on the mesh representation, NVMG takes two-level hybrid approach. First, we utilize the generalization

---

Under review as a conference paper at ICLR 2023
behavior of diffusion models (Ho et al., 2020) synthesis result. We then use another neural network to predict how to perturb the initial synthesis. adding geometric details. At the second level, NVMG employs an optimization procedure to control the quality of the final mesh, addressing issues in flipped faces and distorted faces. The optimization procedure seamlessly combines with the output of neural predictions, adding the strength of neural predictions and optimization-ban -base formulations for mesh optimization.
voxel- representation to obtain initial
We empirically evaluate NVMG on unconditional volumetric mesh generation and conditional image-to- to-e-stteehhhheeeee generation tasks. We show that NVMG outperforms ofat-oo--the-n point cloud generator (Zhou et al. 2021) and image-to-mesh generator (Shi et al. (021). When we compare our mesh deformation module with the state-of-the-art voxel-to-mesh methods and Neural Dual Contouring, we obtain better results on converting generated voxel to mesh.
2 RELATED WORK
3D Shape Generation Generating 3D
shapes is a key challenge in computer vi- sion and computer graphics (Hartley Zis
serman 2003 Moons et al. 2010). Tradi-
tional methods focus reconstructing 3D
shapes from multiple views (Hartley Zis-
serman 2003; Furukawa & Ponce] (2009)
using geometric cues. With the power of deep learning, researchers have advanced the field significantly with data-driven meth- ods. Through the prior knowledge captured by the deep neural networks, current algo- rithms can generate point clouds from sin-
gle image (Fan et al. 2017| Yang et al. 20199 Achlioptas et al. (018) Luo & Hu 2021b a
Figure I: A gallery of generated volumetric meshes. Both the surface and the interior of volumetric meshes generated by NVMG are artifact-free.
Zhou et al. 2021 improve multi-vies -view re-
construction Yao et al.. 2018. Chen al.]
2019). and create high- quality implicit sur-
faces (Mildenhal et al. 2020 Niemeyer & Geiger 2 Genova al. 2020 Different from
this work, our work focuses on an even harder 3D representation, the volumetric mesh. Instead of generating the point position or signed distribution in the 3D space, we need to provide the vertices position and the connection structure for face relations for both the surface and interior. The complex relationship makes the generative model extremely hard to learn.
Mesh Generation with Deep Neural Networks Despite the generality of deep learning, how to incorporate deep neural networks efficiently in mesh generation remains an open problem. The main difficulty is that a valid mesh must satisfy a series of physical constraints. To deal with these constraints, the majority of previous works use mesh template and train neural network to deform the template to obtain the mesh of interest (Wang et al. 2018 Wen et al. 2019 Gupta & Chandraker 2020; Shi et al., 2021 Kanazawa et al.] 2018 Pan et al. 2019 Uy et al., 2020). However, the several aspects, e.g., topology and the magnitude of deformation. Another branch of work is to learn implicit field to reconstruct the mesh
generated meshes limited by the dre-eed template
from the point clouds or voxel prior (Venkatesh et al., 2021 Chen et al.) 2022 Shen et al. [0211b]
Chen & Zhang] 2019 2021; Gao al. [220]] Remelli al. 2020 Chen et al. 2020 2021 2 Chibane ct al. 2020b] Sitzmann et al. 2020; Williams et al. (019). These works rely on the Marching Cube or variant of the marching cube algorithm to convert the learned implicit representation to the surface. However, these methods mainly have three issues. First, most methods need to know the information of the point cloud on the surface for the reconstruction, which indicates that they are not suitable for generating novel shapes. Second, converting implicit surface to mesh is not robust on the shape topology and may cause part and hole losses when converting to mesh. Third, the implicit surface representation can not model the shape interior. PolyGen (Nash et al. 2020) uses sequential model to progressively generate vertex and faces one by one, however, this kind of auto regressive model using transformer structures is costly and exhibits only limited generalization behavior. GET3D (Gao) et al., (222) generates high-quality 3D textured meshes bridging recent advances in the differentiable surface modeling, differentiable rendering as well as 2D Generative Adversarial Networks.
2

---

Under review as a conference paper at ICLR 2023
DDPM
Division
Voxel-Conditional Neural Closest Point Predictor
Voxel Intent
Eapess (t"s Ao(K.))
VoVV
Query Point
Prodict
Resast (M) (
p
Mesh Deformation with Regularized Projection
(a)
(b)
(c)
Figure 2: NVMG combines three modules. (a) A generative model for generating voxelized shapes from noise distribution; (b) Voxel division for the initial volumetric mesh; (c) Physically correct mesh deformation with voxel-conditional neural closest point predictor and regularized projection.
Hybrid representations for shape synthesis. Choosing suitable representation for shape synthesis fundamentally challenging. Several recent works have studied combing hybrid 3D representations for geometry synthesis. Zhang et al. (2020) combine a point-based representation and an image- based representation for scene synthesis. Yang et al. (2021) combine neural networks and parametric priors of object relations for scene synthesis, [Shen et al. (2021a) la) combine implicit representation and an explicit tetrahedral representation for synthesizing shape details from a coarse volumetric grid. In contrast, our approach combine both hybrid representations and procedures for mesh synthesis which prioritize high-quality shape details and mesh quality.
3 NEURAL VOLUMETRIC MESH GENERATOR
This section presents the technical details of our neural volumetric mesh generator (or NVMG). As illustrated in Figure 2 NVMG combines three newly proposed modules: voxel generation with diffusion models, voxel-conditional neural closest point predictor, and physically robust mesh deformation with regularized projection. These modules nicely combine the strength of different approaches. Specifically, diffusion- based methods show great generalization behavior on the global shape, and it provides an initial tetrahedral mesh. The neural closet point predictor adds surface details. while the mesh deformation module promotes the mesh quality.
3.1 VOXEL GENERATION WITH DIFFUSION MODELS
The first module of NVMG is voxel generation. We exploit denoising diffusion probabilistic model (DDPM) (Ho et al. 2020) for this task due to its simplicity and generalization behavior.
The goal of DDPM is learn a parameterized Markov chain that can generate novel samples after finite time. In the forward direction (( ) of the chain, a real data sample æo is gradually diffused step. DDPM then trains a conditional denoising model po (2r (x.) that runs the reverse direction, which reduces the noise
DDPM
sample ør from the standard Gaussian noise by adding noise in each time
in each time step t to obtain synthetic sample.
Formally, have two Markov chains in opposite directions, the forward process q( and the reverse process Pa (.O.T))
q( ((Tar) = ((x)) Here, the transition probabilities are defined as Gaussian distributions,
po( = p(2.)( 12o (x
(1)
g( N( /I 3, B,I), Pe( (( H (o B,I)
(2)
In
practice,
DDPM choose the following parameterization, (s(
B. Rt VI-a, 6s (Z(())))) where ar = - 3e and ār = 16 Br. which controls
vae
3

---

Under review as a conference paper at ICLR 2023
the magnitude of noise, gradually decreases to 0 approaches 0. The training objective is to maximize the evidence lower bound (ELBO). The derivation leads to a simple loss function,
dddpm
||c css (vVVv, + vI- t,
where wo Dirain is a data point sampled from the training set, é uniformly sampled from 1 T. and e ~ NO, I). The neural network c t) learns to estimate the noise from current observation Tt, then subtract it to obtain the new estimation of mean in time step --1 1. After the training process, one can generate new samples through simulating the reverse process. Starting from æT No I).
the reverse process recursively draws _;-1 from ( (.,) until a generated sample do is obtained.
Readers can refer to (Ho et al. 2020) for more details of DDPM.
Figure 4: visualization of transforming voxel tetrahedral mesh. Each voxel can split into tetrahedra.
Figure 3: Generating voxelized shapes with DDPM.
Voxel Generation with DDPM In our scenario, shapes are normalized to a unit cube with x r voxels. voxel representation of a shape is encoded by a binary tensor V E {0, 1 where (xx, y,
0/1 means that the (x, y, 2)tth voxel unfilled/filled. The training set for voxel generation is created by discretizing the original meshes in the training set to their voxel representations with specific resolution. We adopt a similar 3D convolutional neural network as in (Zhou et al. 202T) to parameterize 6g. with several small modifications to accommodate to the voxel generation task. Details can be found in the supp. material.
After the coarse voxel representation of the shape is generated, we then divide it into tetrahedral mesh for later modules of NVMG. A tetrahedron is pyramid-like polyhedron, served a basic volumetric element, with four vertices and four triangular faces. A hexahedron is polyhedron, with eight vertices and six quadrilateral faces. In our method, split each hexahedron into 6 tetrahedra as in Figure to achieve the face-to fo-ce consistency across the tetrahedral mesh.
Tetrahedral Mesh from Voxel Division
(5)
Specifically, we first represent the voxel-based repre- sentation as a hexahedral mesh H = (V,F qqaa) (U (u))
where V is the vertex set, F Fquod
the quadrilateral face set, vi is the i-th vertex of the e-th cube, and ft is the j-th quadrilateral face of the e-th cube. By splitting each cube mesh into 6 tetrahedra, we can further extract tetrahedral mesh T ((r)
Figure 5: (a) Voxelization of torus. (b) Blue query point æ and its yellow closest point CP(x). (c) Query from V to torus. (d) Training samples.
(U I Met.o O vertex set the hexahedral mesh H. Feri is the triangular face set, vi is the i-th vertex of the k-thh tetrahedron in
(E)) where V is the same k=1
k=1
the e-th cube, and fúr is the j-th triangular face of the k-th tetrahedron in the c-th cube. For volumet- ric meshes, as there interior meshes, use y to denote the set of the boundary vertices on the surface.
3.2 VOXEL-CONDITIONAL NEURAL CLOSEST POINT PREDICTOR
The voxelized shape and its corresponding tetrahedral mesh is coarse representation the shape of interest. The second module NVMG employs neural network synthesize surface details from the voxelized shape. This done by fitting a neural network (V,Va)) .) (0. 1)r R + R3 to mimic the closest point function of each training surface S which maps any point E TR3 to its closest point
4

---

Under review as a conference paper at ICLR 2023
on S:
CP(x) = arg min lpp 2//2-
PES
Note that during generation, the surface S is implicitly defined by 5 fa E TR' (V The closest point function becomes discontinuous points the medial axis. Therefore, the closest point prediction network only applies for points that are close to the final surface. This another motivation of using the voxelized shape as the initial mesh reconstruction.
0}.
Data preparation To train the prediction network he (V, x), we first normalize each mesh in the training dataset to a unit bounding box, voxelize and extract the mesh T as described before. Then, we track the boundary 0T, which is union of triangular faces Fire that forms a watertight surface mesh T (V²s Fr Next, we calculate the closest point p* on the ground truth object surface to each vertex v E V2 Instead of sampling spatial points near the surface aggressively as deep implicit function (Park et al. 2019), we sample points around line segments {(1 - az) p + are E [0, 1]] as depicted Figure Therefore, our sampling area for the closest function is tied with the output of the first module voxel shape, i.e. a band within the voxel cube that wraps up the target surface.
Discussion A recent work, CSPNet (Venkatesh et al., (021) also predicts the closest point for surface reconstruction. However, CSPNet need to take the ground truth point cloud and its normal as input to calculate the closest point, which are not available for the generation task.
3.3 PHYSICALLY ROBUST MESH DEFORMATION WITH REGULARIZED PROJECTION
We proceed to introduce the third module of NVMG, which takes the output of the first two modules input and output a deformed mesh. A straightforward approach is directly project each vertex of the volumetric mesh to its closest point inferred by the second module hg:
8t f ho v"), Vo E V*.
(5)
However, this naive projection does work well in practice due three issues. First, the prediction from the neural network can be imprecise, creating bumps on the resulting mesh. Second, the projections can be highly non-uniform the surface, causing highly distorted mesh. Third, the projections are independent with each other, leading to artifacts like flipping.
Therefore, we introduce an optimization formulation to jointly optimize the locations of projected vertices. The objective terms are designed to promote several generic conditions about high-quality volumetric mesh:
C(V)
Σ foros (v"; (V,))) R(V), V'EV"
(6)
where Cproj (v": (V, | - (V, v²)]l2 is the data term and R(V) is the regulatization term. The data term prioritizes that the overall shape of the projection stay close to the output of the second module. The regularization term R(V) promotes the mesh quality, which is designed to be differentiable so that it can be easily optimized with any gradient-based optimizer. In the following, we introduce the regularization term R and the data term Cproj in details.
1) Smooth term for uniform structure The first regularization term Rymooth promotes the smoothness of the resulting tetrahedral mesh T:
Remooth (V) >a Redge (V, Firi) + >o Riaaaplacia (V, + Ruormall (V² FFi))
Here Redge penalizes extremely long edges; Risplscian is the graph-based Laplacian term; Rnormal enforces normal consistency among adjacent faces on the surface. Note that, Radge and Riaplacian are defined over the whole tetrahedral mesh, which differs from (Wang et al. 2018). Xa, As and X are hyper-paac parameters, and we determine them via cross-validation.
2) Orientation term to prevent defects Another important property of tetrahedral is to avoid vanishing face volumes and flips. Flips break local (rrectivit((Abblaae 2018) and cause salient artifacts. Therefore, introduce orientation term which prevent each tetrahedral face from approaching zero volume and thus enforces local injectivity:
Rorientation (V) = Σ ((M*))
(8)
c,k
1(Ma)) =
( (del(MMcc) -v0)2 log det a
dde((Ma.)) tn .et((Mc. Up
20
(9)
5

---

Under review as a conference paper at ICLR 2023
where M R4x4 is the matrix stacked by the homogeneous coordinates of four vertices of the k-th tetrahedron in the e-th cube. The vertices order in Mr. satisfy the right hand rule convention such that every tetrahedron in the initial mesh has positive volume. Note that !(.) is robust -C-function, which is widely used in physical simulation (Li et al.) 2020; 2021 Ni et al. 2021). to a constant thresholding set to activate the penalty, ensuring a minimum volume of tetrahedra. By this prreerrvng term, each tetrahedron keeps a positive volume through the optimization procedure.
3) Robust Projection for distortion suppression We empirically observed that, merely pulling the mesh vertices to the neural prediction of the closest point may result in colliding vertices and mesh faces with large distortion. To address this issue, we modify Lyroj using robust version:
Epros (v": (V. -| - (V, 0²) k22122
(10)
where N (0, 1) is random Gaussian noise, & is coefficient that gradually decays to 0 as optimization proceeds. During the optimization procedure, the 'lucky' vertices arrive at the surface earlier, while the 'unlucky' vertices arrive later. Thus the collisions are avoided, and distortions of the resulting mesh are significantly reduced.
Putting them together problem is,
By combining Eyroj. Romonth and Rorientation, our final optimization
((V)
Σ Eyro (v"; (o0;:hh ho (V, + Remooth (V) + Rorientation (V).
(11)
D'EV"
After obtaining the voxelized shape and the tetrahedral mesh from Sec. 3.1 we optimize Eq. equa- tion 11 to obtain the final volumetric mesh.
EXPERIMENTAL EVALUATION
We begin with the unconditional mesh generation setting in Section mesh generation from image inputs in Section 4.2 Section 4.33 and Section 4.4 present an analysis of the mesh quality of NVMG and its application in shape editing, respectively.
4.1 We then evaluate NVMG for
323 grid. The denoising procedure uses a linear noise schedule from 0.02 to 0.0001, where number of steps is 1000. We train this module 500 epochs with the learning rate 2e 33 The neural closest point module employs multi-scale voxel network inspired by IF-Net (Chibane al. 2020a). This network has 6-layer 3D convolutions that extract the features from layers at resolutions 32 , 163 _and 83, The extracted features are fed into a fully connected layer to predict the closest points. The mesh deformation module uses 60 iterations. We use 4 x RTX2080Ti GPUs for training the neural modules. Please refer to the supp. material for more details.
Implementation details The voxel generation module uses 8-layer network
4.1 UNCONDITIONAL MESI GENERATION
We first evaluate unconditional mesh generation which takes latent code from a random dis- tribution. Baseline approaches include state-of-the-art point cloud generation methods including PointFlow (Yang et al. 2019). ShapeGF (Cai et al. 2020). Point cloud diffusion model (Luo & Hu 2021a), and PVD (hou et al.) 22221), and Neural Dual Contouring (or NDC) (Chen et al. 2022) state-of-the-ar voxel-to mesh method.
Dataset We train this pipeline using ShapeNet Chair and Airplane categories, which are the two
most representative subsets among baseline approaches (Achlioptas et al. 2018 Yang et al. 2019
Cai et al. 2020 Luo & Hu 2021a Zhou et al. 2021 Chen 2022).
Evaluate Metric The same as (Zhou et al. 2021), we evaluate shape generation quality using 1-NN under both Chamfer distance and Earth Mover's 's Distance. Supp. material reports more evaluations. Supp. material reports additional evaluations under Minimum Matching Distance (MMD) and Coverage (COV), which are two other commonly used metrics.
We from Table NVMG gets a comparable and even better performance compared with the state-of-the-ar baseline approaches. Note that this encouraging as NVMG does not optimize the point distribution directly. Figure further shows that one of the advantages of NVMG over prior works in the sense that the generated shape is smooth without outliers.
Result

---

Under review as a conference paper at ICLR 2023
Easy case
Hard
rop.
Airplane
CD
EMD
CD
EMD
Vanel-VA VAB
Brock
Moxel
Vin-diffusion
Vosel-GIAN
Vu nl.
Zhou et dl. 2021
X016
99.75
86.22
98.13
79:59
97.12
74.42
81.29
96.74
Genrated
voxcl
(Voxel generator)
1-GAN Achlicpeas el 2018
87.30
93.95
68.58
83.84
Yang al. 10
Der-Net
Klokow
M000
15.18
66.55
62.00
58.53
NDC
Poinr Cloud
SoftFlow
Kim al. 2020
76.05
65.80
59.21
60.05
StupeGF
et
10200
80.00
76.17
68.96
65.48
DifTusion
202
PVD (7hom al. 2021
73,82
6481
56.26
53.32
PolyCken Nash ui
2020
81.51
72.32
69.12
63.90
Mesh
(w7 Chon oral. 2022
76.41
58.75
NVMG
NVMG
73441
64.29
56,12
53.30
Table Result compared with various point cloud gen- eration baselines. CD: Chamfer distance. EMD: Earth Mover's ' Distance. Lower score means betler generation quality and diversity.
Figure 6: Mesh deformation uses our method and NDC. We notice the heavy artificial and even a missing leg (column in red circle) on the NDC mesh for the hard cases.
Generate vaxel
Mosh deformation
Sample points
Generared samples from our model
Samples with outlier points from PVD
Figure 7: Comparison between NVMG and PVD (Zhou et al.
2021), state-of-ther af-thhcrrt point- based generator. For fair comparisons, we convert the output mesh of NVMG into a point cloud.
4.1.1 ABLATION STUDY
To show our voxel generator and voxel mesh deformation carefully designed for this pipeline and not easily
replaced, we compare each parts with the existing literature and introduce the ablation study results in Table
Effectiveness of the voxel generator First, validate our voxel diffusion model is powerful enough to generate good-emo goough intermediate voxel representation. As we show in the first voxel representation section in the [ we compare several voxel generator and calculate the metric of the generation quality. We reimplement Voxel- (Brock et al., 2016) and Voxel-GAM
(Wul
ct al. 2016) due to the official code base too old and use the Vox-Diffusion result from PVD Zhou et al. 2021 We see that NVMG voxel generator part greatly exceed other voxel generators and thus provide good-quality of intermediate voxels for the following deformation parts.
Robustness of the mesh deformeration There are some brilliant voxel to mesh designs recent work including NDC (Chen et al., (2022) and DMtet (Shen et al. (2021b). Because DMtet requires additional surface points information for the voxel to mesh reconstruction, which is not suitable for our generated voxels, we choose NDC and train it in the same training set for relative fair comparison. We compare our mesh deformation with NDC by taking our generated voxel samples as input. From Table llst two lines, we observed a higher performance of our method compared with NDC, indicating our method are more robust in converting the generated voxel samples into mesh surface. In Figure we visualize the generated mesh samples. In easy case, NDC performs similarly to our method. However, generated voxels may contain irregular noise. so in the hard cases, NDC may shows more artificial on the surface. In addition, we also observed a missing leg on the second column from the NDC's result, showing the insatiability of the marching cube variant algorithm with noisy voxel samples as input. The above result shows our design the deformation-hase method and robustness regularization loss works better in our proposed pipeline comparing with the off-the shell marching cube based voxel to mesh methods.
7

---

Under review as a conference paper at ICLR 2023
Input image
Pixel2mesh
Deepmesh
NVMG
Input image
Pixel2mesh
Deepmesh
NVMG
Figure 8: Qualitative comparisons between NVMG and two baseline approaches Pixel2mesh (Wagg et al. 2018 and Deepmesh (Pan et al. 2019) for the task of single-view shape reconstruction.
4.2 IMAGE CONDITIONAL SHAPE GENERATION.
We proceed to evaluate NVMG for the task of image to mesh generation. To this end, we use ResNet- (e et al. 2016 extract image features and fit the extracted features into NVMG for reconstruction. We compare this approach with state-of-the-ar image to mesh based approaches, including Pixel2mesh (Wang et al. 2018), GEOMetric (Smith et al. 2019). and DeepMesh (Pan et al. 2019).
Dataset and Metric. We use the ShapeNetRender dataset (Choy et al. 2016). We select three categories, i.e., chair, bench and table, which are the most common yet challenging categories among prior works. Same as the previous works, we uniformly sample 10,000 points on the reconstructed mesh and compare them with the ground-truth surface using the Chamfer distance.
Result. Table 2aat NVMG outperforms state-of-the-an approaches consistently. Figure 8 presents qualitative results of NVMG and two top performing baselines Pixel2Mesh and Deepmesh. We can see that NVMG offers reconstructions that preserve more shape details and thin geometric features than baseline approches.
P2M
GEOMetrics
DeepMesh
NVMG
chair
0.610
0.823
0.514
0.471
table 0.498
0.797
0.404
0.342
bench
0.624
0.690
0.516
0.439
Table 2: Comparisons between NVMG and baseline approaches under the Chamfer distance.
Original
Add
Delete
Figure 9: Robust projection regularization improves the quality of the reconstructed mesh. (Top) Without robust projection (Bottom) With robust projection.
Figure 10: An application in shape editing where the user edits the coarse voxel shape and the final shape is updated accordingly
4.3 ABLATION STUDY ON THE LOSS TERMS
A desired property of surface volumetric mesh reconstruction shall be free of flips and distorted cells. Figure shows the effects different combinations of regularization terms. We detect flips by calculating signs of their oriented volumes and color them in red. The quality of a cell is assessed by the aspect ratio (Parthasarathy et al. 1994). which is defined as the ratio between its eircumradius
8

---

Under review as a conference paper at ICLR 2023
(a)
(b)
(c)
(d)
(c)
(f)
Figure 11: We show the ablation study for our method. Red color marks flipped tets; black color marks distorted triangle with an aspect ratio larger than 2.6. (a) One-step mapping by neural closest point function. (b) Only projection term. (c) Robust projection term. (d) Projection smooth terms. (e) projection, smooth and orientation terms. (f) Robust projection, smooth and orientation terms.
and two times its inradius. We color the surface triangles whose aspect ratios are bigger than 2.6 in black.
As shown Figure II (b),), only using the surface term yields good approximations of the underlying surface but can have flipped tests and highly distorted faces. Figure IT e)(() shows that enforcing the orientation term significantly reduce the number of flipped tets. Moreover, Figure IT shows that the robust projection term dramatically reduces the number of distorted triangle faces. Furthermore, as the orientation term also penalizes volume lower than an allowed minimal threshold, it prevents the
thin structure from vanishing and thus promotes physical feasibility. This be seen by comparing the skinny chair legs in Figure (f) and those Figure dT).). Furthermore, many self-intersecting triangles and tets occur without the regularization terms (See Figure IT a)). Please refer to the supp. material for more ablation study results.
4.4 SHAPE EDITING
Voxel, as an easy- easy-to-edit 3D representation, has already been used in 3D creation in art and games. We manually apply editing on the generated mesh create user-preferred shape casily. As we show Figure TO we can remove or add details on the chair back under the voxel-based representation, and our deformation algorithm can provide the desired mesh based on the generative result with editing. The final mesh automatically synthesizes geometric details. Due to the space limitation, show more editing cases including shape assemble and mixture in Appendix.
CONCLUSION AND LIMITATIONS
In this paper, we propose Neural Volumetric Mesh Generator (NVMG), a novel baseline for learning a generative model for generating volumetric mesh. Empirically, we show our pipeline can generate volumetric meshes with high-fidelity and physical robustness. One limitation of our approach is on the limited resolution of the voxel-based representation. We plan address this issue by using Octrees the future. Another limitation is that the three modules are not trained end-to-end. As future direction, we plan to address this issue by developing losses on the local minimums of the deformation energy, which enable end-to-end learning.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Sayed Mazdak Abulnaga. Volumetric mesh parameterization to a canonical template. PhD thesis, Massachusetts Institute of Technology, 2018.
Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3d point clouds. In International conference on machine learning, pp. 40-49. PMLR. 2018.
Noam Aigerman and Yaron Lipman. Injective and bounded distortion mappings in 3d. ACM Transactions on Graphics (TOG), 12,, 2013. 2013.
Andrew Brock, Theodore Lim, James M Ritchie, and Nick Weston. Generative and discriminative voxel modeling with convolutional neural networks. arXiv preprint arXiv:1608 042366
04236, 2016.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In International Conference Learning Representations, 2018.
Ruojin Cai, Guandao Yang. Hadar Averbuch-Elor, Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariharan. Learning gradient fields for shape generation. In European Conference on Computer Vision. pp. 364-381. Springer. 2020.
Rui Chen, Songfang Han, Jing Xu, and Hao Su. Point-based multi-view stereo network. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1538-1547, 2019.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939-5948, 2019.
Zhiqin Chen and Hao Zhang. Neural marching cubes. ACM Transactions on Graphics (TOG), 40(6): 1-15, 2021.
Zhiqin Chen, Andrea Tagliasacchi. and Hao Zhang, Bsp-net: Generating compact meshes via binary space partitioning. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 45-54, 2020.
Zhiqin Chen, Vladimir Kim, Matthew Fisher, Noam Aigerman, Hao Zhang, and Siddhartha Chaudhuri. Decor-gan: 3d shape detailization by conditional refinement. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15740-15749, 2021.
Zhiqin Chen, Andrea Tagliasacchi, Thomas Funkhouser, and Hao Zhang. Neural dual contouring. ACM Transactions on Graphics (Special Issue of SIGGRAPH), 41(4), 2022.
Julian Chibane, Thiemo Alldieck, and Gerard Moll. Implicit functions in feature space for 3d shape reconstruction and completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6970-6981, 2020a.
Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll. Implicit functions in feature space for 3d shape reconstruction and completion. In IEEE Conference Computer Vision and Patlern Recognition (CVPR). IEEE, jun 2020b.
Christopher B Choy, Danfei Xu, Jun Young Gwak, Kevin Chen, and Silvio Savarese. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In European conference on computer vision, pp. 628 644. Springer, 2016.
Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 605 605, 2017.
Yasutaka Furukawa and Jean Ponce. Accurate, dense, and robust multiview stercopsis. IEEE transactions on pattern analysis and machine intelligence, 1362) 1376, 2009.
Jun Gao, Wenzheng Chen, Tommy Xiang, Alec Jacobson, Morgan McGuire, and Sanja Fidler. Learning deformable tetrahedral meshes for 3d reconstruction. Advances In Neural Information Processing Systems, 339996--9947, 2020.
10

---

Under review as a conference paper at ICLR 2023
Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojeic, and Sanja Fidler. Get3d: generative model of high quality 3d textured shapes learned from images. In Advances In Neural Information Processing Systems, 2022.
Kyle Genova, Forrester Cole, Avneesh Sud, Aaron Sarna, and Thomas Funkhouser. Local deep implicit functions for 3d shape. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 4857-4866, 2020.
lan Goodfellow, Jean Pouget-Abadie. Mehdi Mirza, Bing Xu, David Warde-Farles Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
Kunal Gupta and Manmohan Chandraker. Neural mesh flow: 3d manifold mesh generation via diffeomorphic flows. Advances in Neural Information Processing Systems, 33:1747-1758, 2020.
Si Hang. Tetgen, delaunay- quality tetrahedral mesh generator. ACM Trans. Math. Softw, 41 (2):11, 2015.
Richard Hartley and Andrew Zisserman. Multiple view geomelry in computer vision. Cambridge university press, 2003.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, In Proceedings of the IEEE conference on computer vision and patlern recognition, pp. 770-778, 2016.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. 2016.
Jonathan Ho, Xi Chen, Aravind Srinivas, Yan Duan, and Pieter Abbeel. Flow++: Improving flow- based generative models with variational dequantization and architecture design. In International Conference on Machine Learning, pp.
PMLR, 2019.
Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6855 2020.
Yixin Hu, Qingnan Zhou, Xifeng Gao, Alec Jacobson, Denis Zorin, and Daniele Panozzo. Tetrahedral meshing in the wild. ACM Trans. Graph., 37 2018.
Ajay Jain, Ben Mildenhall, Jonathan T Barron, Pieter Abbeel, and Ben Poole. Zero-shot text-guided object generation with dream fields. arXiv preprint arXiv:2112.01455, 2021.
Angjoo Kanazawa, Shubham Tulsiani, Alexei A Efros, and Jitendra Malik. Learning category- specific mesh reconstruction from image collections. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 371-386, 2018
Hyeongju Kim, Hyeonseung Lee, Woo Hyun Kang, Joun Yeop Lee, and Nam Soo Kim. Softflow: Probabilistic framework for normalizing flow on manifolds. Advances in Neural Information Processing Systems, 11116688-16999 2020.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114 14, 2013.
arXiv preprint
Roman Klokov, Edmond Boyer, and Jakob Verbeck. Discrete point flow networks for efficient point cloud generation. European Conference on Computer Vision, pp. 694-710. Springer, 2020.
Minchen Li, Zachary Ferguson, Teseo Schneider, Timothy Langlois, Denis Zorin, Daniele Panozzo, Chenfanfu Jiang, and Danny M. Kaufman. Incremental potential contact: Intersection-and inversion-free, large-deformation dynamics. ACM Trans. Graph. 39(4), jul 2020. ISSN 0730-0301. doi: 10. 41444/3336655933922555
/tttttpp:://iii.... org 10. 1145 3055/33866666 3392 425
Minchen Li, Danny M. Kaufman, and Chenfanfu Jiang. Codimensional incremental potential contact. ACM Trams. Graph., 40(4), jul 2021. ISSN 0730-0301, doi: 10.11457345002264445777. URL
httts:://ooi www./oo..... 1ee...111131000. 3459767

---

Under review as a conference paper at ICLR 2023
Xin Li, Xiaohu Guo, Hongyu Wang. Ying He, Xianfeng Gu, and Hong Qin. Harmonic volumetric mapping for solid modeling applications. In Proceedings of the 2007 ACM symposium on Solid and physical modeling, pp. 109-120, 2007.
Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2837-2845, 2021a
Shitong Luo and Wei Hu. Score-based point cloud denoisinn. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4583-4592, 2021b.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European conference on computer vision, pp. 405-421. Springer, 2020.
Theo Moons, Luc Van Gool, Maarten Vergauwen, et al. 3d reconstruction from multiple images part I: Principles. Foundations and Trends@ in Computer Graphics and Vision, 4(4):287-404, 2010.
Charlie Nash, Yaroslav Ganin, SM Ali Eslami, and Peter Battaglia. Polygen: An autoregressive generative model of 3d meshes. In International conference muchine learning, pp. 7220-7229. PMLR, 2020.
Ruiqi Ni, Teseo Schneider, Daniele Panozzo, Zherong Pan, and Xifeng Gao. Robust amp: asymptot- ically locally optimal uav-trajectory generation based on spline subdivision. In 202/ IEEE International Conference Robotics and Automation (ICRA), pp. 7715-7721, 2021. doi: 10.1 200.CRRAA44566.222.955055
Michael Niemeyer and Andreas Geiger. Giraffe: Representing scenes as compositional generative neural feature fields. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pp. 1155-11464, 2021.
Matthias Nieser, Ulrich Reitebuch, and Konrad Polthier. Cubecover parameterization of 3d volumes. In Computer graphics forum, volume 30, pp. 1397-1406. Wiley Online Library, 2011.
Junyi Pan, Xiaoguang Han, Weikai Chen, Jiapeng Tang, and Kui Jia. Deep mesh reconstruction from single rgb images via topology modification networks. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 9964-9973, 2019.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165-174, 2019.
VN Parthasarathy, CM Graichen, and AF Hathaway. A comparison of tetrahedron quality measures. Finite Elements in Analysis and Design, 15(3):255-261, 1994.
Edoardo Remelli, Artem Lukoianov, Stephan Richter, Benoît Guillard, Timur Bagautdinov, Pierre Baque, and Pascal Fua. Meshsdf: Differentiable iso-surface extraction. Advances in Neural Information Processing Systems, 13:22468-22477 2020.
Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas Geiger. Graf: Generative radiance fields for 3d-aware image synthesis. Advances in Neural Information Processing Systems, 33: 20154-20166 2020.
Tianchang Shen, Jun Gao, Kangxue Yin, Ming- Yu Liu, and Sanja Fidler. Deep marching tetrahedra: hybrid representation for high-resolution 3d shape synthesis. In NewrIPS, pp. 6087-6101, 2021a.
Tianchang Shen, Jun Gao, Kangxue Yin, Ming- Yu Liu, and Sanja Fidler. Deep marching tetrahedra: hybrid representation for high-resolution 3d shape synthesis. In Advances in Neural Information Processing Systems (NeurIPS), 2021b.
Yue Shi, Bingbing Ni, Jinxian Liu, Dingyi Rong, Ye Qian, and Wenjun Zhang, Geometric granularity aware toxelh In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13097-13106, 2021.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems, 33:7462-7473. 2020.
12

---

Under review as a conference paper at ICLR 2023
Edward J Smith, Scott Fujimoto, Adriana Romero, and David Meger. Geometrics: Exploiting geometric structure for graph-encoded objects. arXiv preprint arXiv: 1901.11111,,,, 2019.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
Mikaela Angelina Uy, Jingwei Huang, Minhyuk Sung, Tolga Birdal, and Leonidas Guibas. Deformation-aware 3d model embedding and retrieval. In European Conference on Computer Vision, pp. 397-413. Springer, 2020.
Rahul Venkatesh, Tejan Karmali, Sarthak Sharma, Aurobrata Ghosh, R Venkatesh Babu, Lászłó A Jeni, and Maneesh Singh. Deep implicit surface point prediction networks. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 12653-12662, 2021.
Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu- Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European conference on computer vision (ECCV), pp. 52 67, 2018.
Chao Wen, Yinda Zhang, Zhuwen Li, and Yanwei Fu. Pixel2mesh++: Multi-view 3d mesh generation via deformation. In Proceedings of the IEEE/CVE international conference on computer vision, pp. 1042-1051, 2019.
Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. Deep geometric prior for surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Putlern Recognition, pp. 10130-10139, 2019.
Jiajun Wu, Chengkai Zhang, Tianfan Xue, William Freeman, and Joshua B Tenenbaum. Learning probabilistic latent space of object shapes via 3d generative-adversariall modeling. In Advances in Neural Information Processing Systems, pp. 82-90, 2016.
Guandao Yang, Xun Huang. Zekun Hao, Ming- Yu Liu, Serge Belongie, and Bharath Hariharan. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 4541-4444 145000 2019.
Haitao Yang, Zaiwei Zhang, Siming Yan, Haibin Huang, Chongyang Ma. Yi Zheng, Chandrajit Bajaj, and Qixing Huang. Scene synthesis via uncertainty-driven attribute synchronization. CoRR, abs/2108. 13499, 2021.
Yao Yao, Zixin Luo, Shiwei I., Tian Fang, and Long Quan. Mysnet: Depth inference for unstructured multi-view stereo. In Proceedings of the European Conference Computer Vision (ECCV), pp. 767-783, 2018
Zaiwei Zhang, Zhenpei Yang, Chongyang Ma, Linjie Luo, Alexander Huth, Etienne Vouga, and Qixing Huang. Deep generative modeling for scene synthesis via hybrid representations. ACM Trans. Graph., 1911::7:111111 2020.
Linqi Zhou, Yilun Du, and Jiajun Wu. 3d shape generation and completion through point-voxel diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5826-5835, 2021.
13

---

we compare the trajectories of mesh deformaton using Cpros and Under review as a conference paper at ICLR 2023
A MORE DETAILS ON METHODOLOGY
A.1 OJECTIVE FUNCTION
Smooth term
smoth (V)
>a Redge Ftrs) + Aapllacca (V. + de Roormall (V= Firr)) (12)
Let be the of unordered edges in the volumetric mesh.
Redge (V,
10: 001l (1.))~E
Σ
Recall our definition that V and F are vertices and faces set of the volumetric mesh, while y and J are vertices and faces set on the surface mesh. F and F can be viewed as the graph that records the connectivity of volumetric mesh(surface mesh included) and surface mesh(surface mesh only) respectively. Hence, the volume neighborhood and the surface neighborhood of i-th vertex can be drawn from graph and graph F³, denoted as N₁ and N respectively. Then we define
5,
2y. and give
NT
Rappaciaa
(vi)) = a Σ v.EV
|1
0 +
Σ VVEV
16
oz 112
where a = 1 and 3 = 0.5.
The normal consistency loss is constructed under the assumption that only two faces share edge, which is always satisfied by our meshes. Let n2 denotes the normal of i-th face. We have
Roormal (V*, Fim)
Σ 1- cos(n, n2))
17113
ff
represents the i-th face and the j-th face on the surface sharing the same edge.
Orientation term
R Rorientation (V) Σ ((Mc))
(13)
O,
((Ma)) =
(Mc))) - u0)² log( det(
det(Maa))
S v0
(14)
0
det(MMc.) Un
yo
det (M
21
V1
(15)
22
W/2
22
c.k
133
V/
23
c.k. (o 11 2, 22th²) is the coordinate of the i-th vertex of the k-th tetrahedron in the e-th cube. We set No = 0.01 consistently all of our experiments.
Robust projection In Figure Eyroj-
A.2 MESH QUALITY
In our work, we aim to eliminate the defects of generated meshes, i.e. flipping, distortion and self-intersection. Here, we illustrate them as the preliminary. We refer to B.3 for quantitative reports on these quality statisties.
14

---

forlew Untde
(CLL di paper contstavee a as
2023
フレメつー フユ
Figue 12:
with Firs nowe
Seoon Capoi
with Tow-
Eayut
0. Columirs, step
20. 30, 40. 60.
80.
D
Lett 13: nomnal the of
Figure
intial vesoor
reglarr apensite
inndaaalaion
the Righit
trangge
dde is
sippecd
Note
the that
direvtion
Figure 14, red. to Aeforminge
and Fürst This
sippedi rowe
Seomni cartaasitaa
treadccion rowe vouumecite
from is fippedi rowe A dteraaatinn dehominng is
and
blue
preserrige makess
corsopoontts the
mesh
Thir lieeel,
the eotoaillon
15

---

Under review as a conference paper at ICLR 2023
Figure 15: Green circle: inradius. Blue circle: circumradius. The left black triangle a regular one, while the right black triangle is a distorted one.
Green sphere: inscribed sphere of the tetrahedron with radius r. The left tetrahedron is a regular one, while the tetrahedron is distorted one. A highly distorted tetrahedron will hurt the down-stream performance of the generated volumetric mesh.
Figure 16:
Flipped triangle For the surface mesh, consider it an oriented manifold. Hence, the vertices order each triangle face 'wound' to keep the normal vectors pointing outwards consistently. When the vertices are deformed or mapped improperly, flipped faces occur and cause flipped normals, which will heavily harm the downstream tasks. As in Figure 13 vertex and vertex b moves improperly, causing the triangle abc and its corresponding normal n flipped.
Flipped tetrahedron The orientation preservation for tetrahedra means that one vertex should stay on the same side of the plane determined by the other three vertices. And the sign of the determinant indicates if this same-side relation is satisfied. If the determinant is zero, the volume of the tetrahedron vanishes and the four vertices comes as co-planar. If the sign of the determinant becomes opposite, indicates that one vertex has crossed the plane to the opposite side. As in Figure 14 vertex crosses the plane determined by abc, so the right tetrahedron is flipped from the left.
Aspect ratio for triangle and tetrahedron For both surface mesh consisting of triangles and volumetric mesh consisting of tetrahedra, want to make acute cells as few as possible because they tend to cause the singular matrix and large rounding errors in numerical algorithms on meshes. The aspect ratio depicts how far cell is from the regular one. A high-quality mesh should keep most R. cells with low aspect ratio. Following industrial convention, the aspect ratio for the triangle is 2R, where R, is the circumradius for the triangle and RG is the inradius of the triangle. And we select one version of aspect ratio for tetrahedron as hmaz, where fmmaz is the largest edge length in one 2//r" tetrahedron and is the inradius of tetrahedron. By definition, both the perfect triangle(equilateraa) and tetrahedron(all four faces are equilateral) has aspect ratio of The larger the aspect ratio, the higher distortion occurs.
B ADDITIONAL EXPERIMENTS
B. ABLATION STUDY 1: NUMBER OF VOXELS
We study the setup different voxel resolutions against the final mesh quality. We compared our current resolution with 16 x 16 x 16 and 8 x 8 x 8 voxel resolutions in the whole procedure. Due to extend the configuration to higher resolutions. Thus
the cost of the 3D convolution layer,
16

---

Under review as a conference paper at ICLR 2023
show that 32 32 32 is our current best. We present the result by showing the unconditional generation scores of the chair point cloud in the INN metric with CD and EMD. We see result in 3Table3J that 32 x 32 x 32 is the optimal selection comparing with lower resolution. As we say in the limitation part, current network structure cannot fit in the GPU when scaling higher resolution. This may our future direction.
Resolution
CD
EMD
8 8 8 16 16 x 16
60.13
57.41
57.89
54.26
32 32 x 32
56.12
53.30
Table 3: Generation quality with different voxel resolutions. Higher resolution results in better generation quality.
B.2 ABLATION STUDY 2: THE SAMPLING STRATEGY IN TRAINING VOXEL-CONDITIONAL NEURAL CLOSEST POINT PREDICTOR
Since using the deformation method rather than the Marching cube to get the surface, the main idea of our proposed trajectory sample strategy is to make the query point efficiently distributed along the displacement trajectory. Thus, we compare our sample strategy with uniform sampling, which samples the query point uniformly from the voxel shape space and Gaussian samples, which sample the points with the Gaussian distribution whose mean is the ground truth surface and the IF-NET[Chibane variance is 0.1 and 0.01 IF-NET Chibane et al. (2020b) uses. We set up two choices number of
query points, 75,000 (75k) and 200,000 (200K), (200K), where 75,000 is the number of query points used in our main experiment and 200,000 is the number of points used in IF-NET Chibane et al. (2020b). We evaluate the performance of unconstrained chair point cloud generation described Section 4.1.
From Table we see that our method keeps the results both the number of query points during uniform sample and Gaussian sample performance drop in the 75,000 setups. Furthermore, from the observation, see that sampling from the trajectory between the voxel grid the surface closest. point for our mesh deformation tasks is more efficient.
Method
CD
EMD
Uniform 75k
56.88
54.93
Uniform 200k
56.39
54.12
Gaussian 75k
56.46
53.97
Gaussian 200k
56.14
53.31
Trajectory 75k
56.12
53.30
Trajectory 200k
56.11
53.31
Table 4: Generation quality under different query point sampling strategy.
B.3 ABLATION STUDY 3: THE COMPONENTS IN L(V)
We perform extensive ablation studies on our method with multiple combinations of components in L(V) to illustrate their effect. In each shape class, we randomly seleet 500 generated voxel- represented shapes as the initialization and then apply different optimization combinations as listed in the first column of Table 5S and Each final shape has its volumetric tetrahedron mesh and surface triangle mesh. For each volumetric mesh, we collect the aspect rshir for tetAR) and the number of flipped tetrahedra(shor, for tetFlip). For each surface mesh, we collect the arean/minimum/maximnn triangle aspect ratio(short for triAR), the number of flipped triangles for triFlip) and the number of self ielf-inntrseette triangles. In table 5 and table the average of mean triAR and mean tetAR overall meshes reported. And the median not average
tetrahedron
17

---

Under review as a conference paper at ICLR 2023
of min/max triAR and min/max tetAR overall meshes reported, for the sake of removing extreme numbers. Results show that the smooth term contributes lot to enhancing every aspect of mesh quality. The orientation term removes all the flipped tetrahedra and nearly all flipped triangles. The effective noise/stochastic term helps clean the mesh quality further. Though the one-step mapping has better quality than naive multi-step optimization, the former does not have space to improve.
Chair
tiAR (mean/min//max)
tetAR (mean/min//max)
triFlip
tetFlip
self-intersection
une-step clusest-point mapping
5.38 /1424.00
9,97/1.21/8321.54
69.77
364.83
61.54
Cpro
992.21/1.00/296660
111.43/.15/140042..66
763.97
2059.52
3181.81
Epro
05.69/1.00/1990 .04
24.26/1 1/25485.88
595.93
1009.10
841.54
Epruj Ryamooth
145/L.007455.6
35.22/1.09/2011 09/20134.61
294.81
642.85
109.68
Epros + Rocientation
5.01/1.00/75
4.42/1.09/5$
233.57
0.01
322.33
Opros + Remcooh + Recientacion
1.59/1.00/46.
2.28/1007/33044
2.82
0.01
2.74
All
1.45/1.00/27.39
2.19/1.07/25.49
0.00
0.48
Table 5: Ablation study of how different regularization terms affect the mesh quality in chair mesh generation. All: surface + smooth orientation + noise
Lump
triAR (meun/min/maxx)
tetAR (mean/min/maxx)
triFlip
tetFlip
seff-intersection
one-step closest-point mapping
3.73/1.00667.66
13.10/1.17/5260443
87.12
370.00
113.42
Cyroj
233.24/1.0066772..78
127.33/1.17/56608.6
505.39
1612.37
3530.89
Eyroj
15.17/..0/5785.62
14.38/111//18811 6.74
418.83
967.75
836.42
Cyroj Ramoosh
19/17/11000 17/1.09/10081.55
232.53
448.59
182.96
Cyruj + Rorientation
081/1.00/482.14
5.46/1.
216.07
0.02
467.91
Cyruj
Ramzooth
Rotientation
1.94//.00/29.11
3.43
0.00
1.63
All
1.78/1 1.78//.00/23.99
2.56/1 2/6/1..0722.99
0.44
0.00
0.18
Table Ablation study of how different regularization terms affect the mesh quality in lamp mesh generation. All: surface + smooth + orientation + noise
B.4 DO OUR MODEL REALLY GENERATE NEW SHAPES?
We show that our generated model has the ability to provide new shapes rather than memorize the training set. Given a generated mesh we retrieve the most similar mesh the training set by computing the Chamfer distance of the point cloud samples from the generated mesh and all the meshes in the training set. In Figure and Figure 18 provide some examples of the generated chairs and lamps in different view and compare them with the closest shapes the training set. We observe that we are generating new shapes.
C ADDITIONAL DETAILS ON EXPERIMENTS
C.I DETAILS ABOUT PHYSICAL ROBUST MESI DEFORMATION WITH REGULARIZED PROJECTION
Optimization step Empirically, observe that 80 steps of optimization are enough for voxel shape to converge to the volumetric mesh. Thus we choose 80 as our optimization step.
Noise schedule of Robust Projection for distortion suppression In Section 3.3, we introduce Robust Projection by adding the Gaussian noise with the ratio linear scheduler. We want add stronger noise at the beginning stage and gradually decrease it at the converge stage. So, starting with an initial ratio k 0.1, we decrease by 0.5 every 10 steps until 80 steps.
18

---

Under review as a conference paper at ICLR 2023
Figure 17: Generated chair samples v.s. its nearest neighbor in the training set from different views. In each column, the left one the generated mesh and the right one the nearest data point in the training set. We show that our model has ability to provide brand-new shape.
Figure 18: Generated lamp samples v.s. its nearest neighbor in the training set from different views. In each column, the left one is the generated mesh and the right one is the nearest data point in the training set. We show that our model has ability to provide brand-new shape.
19

---

Under review as a conference paper at ICLR 2023
Number of query point in sample strategy For the query point sample strategy described in Section 3.2, we sample 75,000 query points for each training data point using the strategy we mentioned.
C.2 NETWORK DESIGN
For DDPM model, build up 8-layer 3D convolution network the denoise function. The layer width are {32, 64, 128, 256, 256, 128, 64, 32). The first four layers gradually encode the voxel resolution from 32 to 4 by adding stride 2 in each layer. The last four layers are 3D convolution transpose layer which recovers the voxel resolution from 4 to 32 with stride in each layer.
For Voxel-conditional Neural Closest Point Predictor, similarly, we build up a 4-layer 3D convolution network to encode the voxel with width {32, 64, 1 128, 256) and gradually encode the voxel resolution from 32 4. As IF-NET Chibane al. (2020b), we apply a tri-linear interpolation on each query point based on each convolution layer's output as the multi-resolution features. After we encode the query point features based the input voxel, we use two-layer MLP to decode the feature into the predicted vector the closest point.
D ADDITIONAL EXPERIMENT RESULT
D.1 PERFORMANCE OF THE VOXEL GENERATOR MODEL
We also evaluate the unconditional mesh generation result in MMD and COV metrics and show the result in 8 We see a consistent result with the result the main table that our generator can get a comparable or higher performance comparing with the state-of-the-art point cloud generators.
D.2 MORE RESULTS ON ADDITIONAL CLASSES
We also evaluate lamp and bench classes and compared it with several baselines. Due to the lack of the result for some classes, we only report the result for ShapeGF and PVD and retrain these two models by ourself using the default configuration provided in the official codebases. As we shown in Table 7 get outstanding result compared with other methods.
Method
Lamp
Bench
CD
EMD
CD
EMD
GhapeGF (Cai et al. PVD Zhou al. 2021)
2020)
58.35
55.90
63.12
71.71
61.54
57.63
62.07
72.28
NVMG
56.89
57.01
61.99
70.32
Table 7: Generated samples performance Lamp and Bench classes
E MORE SHAPE EDITING EXAMPLE
We show that our intermediate representation, the voxel is friendly for various of editing in Figure 19 and We see that we can easily assemble or even directly mix two parts together generate the smooth volumetric mesh as input.
MMD:CD J MMD.EMD
CONCED
COVTED
MMD.CD,
MMD
COV.CDT 1
COV.EMD
L-GAN
0.3998
0,5832
38.52
21.23
2,589
2,007
41.99
29.31
PointFlow
0.2243
SofFlow
0.2309
0,3745
46.91
2.528
1,682
47.43
DeF-Net
0.2642
0.4086
46.17
48.39
2.536
1.632
44.71
48.79
ShapeGF
0.2703
0.6592
40.74
40.49
2,889
1.702
46.67
48.03
Nox-Diff
1.322
0.5610
17.52
21.75
PVD
0.2243
0,3803
48.88
52.09
2.622
1.556
49.84
$0.60
NVMG (w/ NDC)
0.2724
0.3814
47.84
50.35
2.491
1.613
48.23
48.38
NVMG
0,2314
0.3632
48.03
52.01
2/413
1.514
$1.34
49.78
Table 8: Addition Experiment Result evaluted in MMD and COV.
20

---

Under review as a conference paper at ICLR 2023
assembly
voxel2mesh
Figure 19: Parts assembly example for shape editing
mix
moxel2meshi
Figure 20: Parts mixture example for shape editing
21

---

Under review as a conference paper at ICLR 2023
F MORE GENERATED MESHES
We show more samples of our generated results in Figure 21 We take chair, lamp, and bench as an example.
Figure 21: More generated examples
22


Reviews:
[{"review_id": "cm41t0gKbbc", "reviewer": "Reviewer_E4fj", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a generative model for volumetric meshes (i.e., tetrahedral meshes). The generation process has two steps: a diffusion model (DDPM) first generates a voxelized shape, which serves as a template and is then deformed to the final volumetric meshes via optimization. The loss terms for the optimization are carefully chosen to get rid of of defects like flipping and high distortion. Experiments on ShapeNet dataset shows that the method can generate high quality volumetric meshes.", "summary_of_the_review": "Leveraging deep generative models to synthesize volumetric meshes is of course an interesting problem. Though the proposed method is far from the ideal way to do this (it's separated trained, limited by voxel template, fixed topology), I think this work is a solid step forward and the evaluation looks adaquate. ", "strength_and_weaknesses": "Strength:\n\n- The proposed method is technicaly sound and has its novelty. Genrateing voxels using a diffusion model and then deforming the voxelized shape via optimization to get the final tetrahedral mesh is new to this field. The optimization formulated using neural network term (i.e., data term) and regularization terms looks very effective.\n- The ablation study is pretty good, especially for the optimization part which contains many loss terms. As I read through the paper, I accumulated many questions in mind regarding the design choices. Most of them are well addressed in the ablation study.\n\nWeaknesses:\n\n- Though the method does achieve some state-of-the-art performance, technically it has some clear drawbacks including\n  - The quality of the generated volumetric meshes is limited by the quality and resolution of the initial voxelized shape. In addition, the topology is determined by the voxelized shape.\n  - Each module is trained/optimized separately, making the method less elegant.\n- The advantage of \"NVMG\" over \"NVMG (w/ NDC)\" seems small (Table 1). The numbers might be misleading, so Figure 6 is a good example to show the actual advantage. I'd like to see more example like Figure 6 and an explanation why NDC cannot handle these cases.\n- Missing timing statistics. Since the last step is an optimization process which needs to be carried out at inference, run time becomes important. How long does it normally take?\n- Main results (Table 1) only have numbers for Chair and Airplane categories. I see there are additional examples for Lamp and Bench in the appendix. Why not listing their numbers in the main text? I think it's better to include them for completeness.\n\nAdditional comments:\n\n- Sec. 3.2 first line, \"a coarse representation the shape of interest\" -> \"a coarse representation of the shape of interest\"\n- Sec. 4.1 the sentence \"Supp. material reports more evaluations.\" seems redundant.\n- Sec. 4.1.1, \" so in the hard cases\" -> \"In the hard cases\"\n- I'd like to see more discussion about DMtet and what's the unique advantage of the proposed method over it. DMtet also aims to generate tetrahedral meshes via learning and its representation seems to be successful in some downstream applications (https://nvlabs.github.io/nvdiffrec/).\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "BzEyysbL7ha", "reviewer": "Reviewer_nDNc", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method for synthesizing high-quality tetrahedral meshes. The method starts with a voxel grid generator and then produces a regualarized tet mesh from it. For the voxel grid, the authors use an existing diffusion-based model. For tet-meshing, the authors first employ a neural network trained to predict the closest point on the actual surface given the voxel grid and a query point, and then recover this surface from the closest-point function by optimizing a set of losses including regularizing losses. The authors claim that the resulting meshes are better than baselines on selected metrics, especially in terms of robustness and freedom from artifacts.\n", "summary_of_the_review": "I think the paper itself is reasonably sound. My main concern is regarding the scope/magnitude of the contribution, as detailed above.\n", "strength_and_weaknesses": "The method is straightforward (conditioned on the DDPM which is complex but not a contribution of this paper) and appears to work quite well.\n\nMy main concern is that the contribution is quite limited. The main feature is a different way of recovering a volumetric mesh from a voxel grid. The heavy lifting in the _generation_ part is done via a prior method, viz the voxel generator. It is clear that the results are better than the ones that directly produce meshes (e.g. Fig 8), and the point that converting voxels to meshes is a better choice than synthesizing meshes directly is well taken. But the magnitude of the technical contributions beyond this point is a bit less clear. There is clear utility in having a better voxel-to-tet-mesh method. But the paper conflates this with the generation aspect and makes it harder to evaluate. Is the voxel-to-mesh method really that much better than NDC or NMC? Is this properly evaluated in a wholistic way?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "BKPH5QGXkx", "reviewer": "Reviewer_KpX3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a generative model of meshes which are volumetric, ie. with an inner structure.\nAs opposed to recent implicit surfaces (based on SDF or occupancy), it directly outputs a mesh. It relies on a 2 steps process:\n- 1: a diffusion model generates voxel grids.\n- 2: a voxel grid refinement, using a neural network to project voxel vertices to their closest surface points.\n\nThe second step requires handcrafted regularization techniques to avoid flipped faces and preserve the mesh quality. Overall, the pipeline demonstrates good generative properties in terms of coverage.", "summary_of_the_review": "This paper proposes to use a recent deep learning technique (diffusion) to modernize an old representation (deep explicit meshes), but it still is not on par with more modern and simpler ideas (implicit fields). Since no motivation is given for sticking to explicit meshes, I vote for rejection.", "strength_and_weaknesses": "A strength of the paper is to employ a diffusion process to generate voxel grids. This is, to my knowledge, the first time this elegant framework is used to that end. The regularization techniques proposed for the refinement step are well ablated (smoothing, orientation, noisy reprojection)\n\n\nMy main concern is on the whole motivation for directly producing meshes with neural networks, as opposed to most state of the art methods (DeepSDF, DefTet, GET3D, Convolutional Occupancy Networks, …), to which authors do not compare. These methods also perform better qualitatively.\nAuthors only provide unclear and vague statements to support it, such as in the introduction’s first paragraph: “the implicit representation needs to be converted into explicit representations such as meshes, which by itself is not a completely solved problem”. I really don’t agree with the last part: Marching Cubes provides smooth and regular meshes off the shelf, and has been made differentiable for applications requiring to pass gradients from vertices to upstream implicit parameters (see Remelli et al. NeurIPS 202 (MeshSDF) or Atzmon et al. NeurIPS 2019).\nDirectly regressing meshes obviously has a very negative impact on shape quality, and requires many handcrafted regularization tricks which complexify the method. Authors need to suggest and demonstrate reasonable applications benefitting from the direct generation of meshes compared to implicit nets.\n\nSimilarly, I do not get the point of getting an inner mesh structure, whereas it is presented as a key feature of the presented pipeline - but never used for any concrete application. What is the benefit of having an inner mesh structure compared to just a surface? How to assess the quality of inner mesh structures, quantitatively? Why would a naive dense grid be bad?\n\nIf the main focus of the paper is about generative properties, then a comparison with randome latent code sampling of an implicit model (DeepSDF) is required. Authors would need to show better “sampling properties” of the proposed diffusion model.\n\n\nMinor concerns are:\n- the “Robust reprojection” consists in adding noise to points during reprojection. Authors show that it empirically helps, but provide no theoretical explanation for it.\n- all the regularizations employed during the reprojection step are termed as “physically robust” but do not involve any physics.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9YxtRBqWInx", "reviewer": "Reviewer_873W", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a generator, abbreviated as NVMG, for generating volumetric meshes. With the help of the diffusion model, the voxelized shape can be generated from a randomly sampled vector, which will be divided into tetrahedral mesh. Then the initial tetrahedral mesh is deformed to the final result under the guidance of neural closest point predictor and the regularization of several carefully designed terms, including smooth term for uniform structure, orientation term to prevent defects and data term for distortion suppression. The proposed method can generate tetrahedral mesh randomly or from an input image. It can also perform shape editing through editing the voxel representation. ", "summary_of_the_review": "This paper proposes a new method with good results, although some details of the method need to be further clarified. I tend to accept this paper.\n\n", "strength_and_weaknesses": "Strength:\n-- This paper proposes a novel pipeline for tetrahedral mesh generation.\n-- The paper has conducted adequate comparisons and evaluations, and showed generation results and editing results, which demonstrate the usability of the proposed method. \n--The results are good.\n\nWeaknesses\n-- Some details need to be verified. See my comments below.\n-- It seems that the voxel generation based on diffusion model adopts the existing method (Zhou et al., 2021). Is there any further improvement?\n-- As for the voxel-conditional neural nearest point predictor, given the voxelized shape generated in the first stage, it predicts the nearest point on the real surface for each voxel vertex, which is equivalent to predicting the correspondence of the initial tetrahedral mesh to the real surface for further deformation. So how to deal with the ambiguity, that is, some similar models have the similar voxel representation, or even the same voxel representation. How does the network deal with this situation?\n-- Although the authors give the optimization terms for mesh deformation, they do not explain how the optimization is carried out. For example, is the iteration required? what solver is used? What is the value of \\lambda_a, \\lambda_b and \\lambda_c in equation (7)?\n-- Minor issues:\nIn Section 3.3, ‘this naive projection does work well’ should be ‘this naive projection does not work well’\n-- Missing related work:\nAtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation, CVPR 2018\nLearning elementary structures for 3D shape generation and matching, NIPS 2019\nSDM-NET: deep generative network for structured deformable mesh, ACM TOG\nTM-NET: Deep Generative Networks for Textured Meshes, ACM TOG\nO-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis, ACM TOG\nAdaptive O-CNN: A Patch-based Deep Representation of 3D Shapes, ACM TOG\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject