Variational Autoencoding of Dental Point Clouds
Authors: Johan Ziruo Ye
Thomas Ørkild
Peter Lempel Søndergard
Søren Hauberg
Venue: Accepted by TMLR ()

Abstract: Digital dentistry has made significant advancements, yet numerous challenges remain. This paper introduces the FDI 16 dataset, an extensive collection of tooth meshes and point clouds. Additionally, we present a novel approach: Variational FoldingNet (VF-Net), a fully probabilistic variational autoencoder for point clouds. Notably, prior latent variable models for point clouds lack a one-to-one correspondence between input and output points. Instead, they rely on optimizing Chamfer distances, a metric that lacks a normalized distributional counterpart, rendering it unsuitable for probabilistic modeling. We replace the explicit minimization of Chamfer distances with a suitable encoder, increasing computational efficiency while simplifying the probabilistic extension. This allows for straightforward application in various tasks, including mesh generation, shape completion, and representation learning. Empirically, we provide evidence of lower reconstruction error in dental reconstruction and interpolation, showcasing state-of-the-art performance in dental sample generation while identifying valuable latent representations

Published in Transactions on Machine Learning Research (08/2024)
Variational Autoencoding of Dental Point Clouds
Johan Ziruo Ye
Johan. Ve@Bshapeecom com
Technical University of Denmark SShape
Thomas Ørkild
Thomas. Thomas.OrksiddiS Oww.idddrsaappp....ooo com
3Shape
Peter Lempel Søndergaard 3Shape
Pater.SenneerganndBBshpp.....
Søren Hauberg
sohau@ddtu. dk
Technical University of Denmark
Reviewed on OpenReview: /ttp:e///penrevieee net/ forum3 id= 16rLtat
Abstract
Digital dentistry has made significant advancements, yet. numerous challenges remain. This paper introduces the FD1 16 dataset, an extensive collection of tooth meshes and point clouds. Additionally, (VF-Net), fully probabilistic variational autoencoder for point clouds. Notably, prior latent variable models for point clouds lack one to one correspondence between input and output points. Instead, they rely on optimizing Chamfer distances, metric that lacks a normalized distributional counterpart, rendering it unsuitable for probabilistic modeling. We replace the explicit minimization of Chamfer distances with a suitable encoder, increasing computational efficiency while simplifying the probabilistic extension. This allows for straightforward application in various tasks, including mesh generation, shape completion, and representation learning. Empirically, provide evidence of lower reconstruction error in dental reconstruction and interpolation, showcasing stato-of-tho-art performance in dental sample generation while identifying valuable latent represenlations.
present novel approach: Variational FoldingNet
1 Introduction
Recent advancements and widespread adoption of intraoral scanners in dentistry have made micrometer- resolution 3D models readily available. Consequently, the demand for efficiently organizing these noisy scans has grown in parallel. To this end, we propose variational autoencoder (Kingma & Welling, 2014; Rezende et al., 2014) specifically designed for point clouds, enabling the identification of continuous representations. This approach effectively captures the continuous changes and degradation of teeth over time.
Our solution is a probabilistic latent variable model that ensures a one-to-one correspondence between points in the observed and generated point cloud. This one-to-one connection throughout the network allows for optimization of the original variational autoencoder objective. This is achieved by projecting the point cloud onto an intrinsic 2D surface representation, which allows for efficient sampling and also discourages storage information about the overall shape within this space. These 2D projections impart a strong inductive bias, proving highly beneficial when the imput point cloud and the 2D surface share topology. Notably, this also bottlenecks the model, preventing it from learning the identity mapping. Specifically, Variational Foldingnet (VF-Net) learns a projection from the 3D point cloud input down to 2D space, which then is deformed back to reconstruct the input point cloud. Finally, these projections facilitate mesh generation without further
1 Code available here

---

Published in Transactions on Machine Learning Research (08/2024)
Figure VE-Net teeth samples, generated by our probabilistic variational autoencoder for point clouds. Note the wide variety the samples which retain anatomical details in its cusps/fissuug composition.
training, well straightforward shape completion and shape extrapolation, all without compromising the quality of the learned representations (see Fig. 1 for samples). Previous point cloud models generally lack one to one correspondence throughout the network due to their invariant architecture design. Instead, they evaluate reconstruction error using Chamfer distances (CD) (Barrow et al., 1977) defined as
m
n
CHAMP-DIST(X.y)) X
| 85112 Ej
min
min |1V, z:ll2. FiE
(1)
where m and n are the number of elements of x and y respectively. This metric solves the invariance problem. However, it also poses a new one: The Chamfer distance does not readily lead to a likelthood, preventing ils use probabilistic modeling. For instance, when used in the Gaussian distribution, the function - 1/c (xp(- Disti (x, μ)) cannot be normalized to have unit integral due the explicit minimization in Eq. 1. Consequently, previous latent. variable models are closer to regularized antoencoders than the variational autoencoder. Since our model ensures one-to-one correspondence between points in the point clouds, we can easily build proper probabilistic model. Moreover, to encourage further research, we release a new dataset, the FDI 16 Tooth Dataset, providing a large collection of dental scans, available as both meshes and point clouds?. This dataset provides real-world reprosentations with planar topology. We consider this an excellent compromise between high-quality computer-aided design (CAD) models and sparse LiDAR scans (Chang al., 2015; 2017; Caesar et al., 2020; Armeni et al., 2016). In digital dentistry, significant challenges are found in diagnostics, tooth (crown) generation, shape completion of obstructed areas of the teeth, and sorting point clouds, etc.
In summary, we present the first fully probabilistic variational autoencoder for point clouds, VF-Net, charactorizod by a highly expressive decoder with statc-of-the-art generative capabilitics. All while learning compressed representations and being adaptable for shape completion tasks. Furthermore, we release a dataset of 7,732 tooth meshes to facilitate further research on world 3D data.
2 Related work
We focus on point cloud representations of 3D objects, but there are many alternative methods of representation including vexel grids (Zheng et al., 2021; Wu et al., 2018), multi angle inference (Wen et al., 2019; Han et al., 2019), and meshes (Alldieck et al., 2019; Wang et al., 2018; Groueix et al., 2018). major paradigm neural networks for point clouds is to remain permutation and cardinality invariant. terms of encoder-decoder models, this frequently leads designs without a one-to-one correspondence between inputs and outputs (Yang et al., 2018; Groueix et al., 2018). This becomes an obstacle in adapting the variational autoencoder to point clouds. Accordingly, other methods have become prominent, including GANs (Li et al., 2018; 2019), diffusion models (Zhou et al., 2021; Zeng et al., 2022; Zhou et al., 2023), and traditional autoencoders (Achlioptas et al., 2018; Groucix et al., 2018; Pang et al., 2021).
Existing Point Cloud Variational Autoencoders. Previous attempts to design a variational autoen coder for point clouds frequently relies on Chamfer distances as an approximation of the reconstruction
"Data available here
2

---

Published in Transactions on Machine Learning Research (08/2024)
term in the standard evidence lower bound. Consequently, these VAEs fail to evaluate a likelihood, key characteristic of Vs This includes works like Edi, which aims to disentangle each point cloud into smaller parts. For each disentangled part, they use the Chamfer distance individually and a superquadric loss that consists of another Chamfer distance term and a regularization term to prevent overlapping parts (Li et al., 2022). The Venatus Geometric Variational Anto-Encoder (VG-VAE) introduces a Geometric Proximity Correlator module to better capture local geometric signatures. However, their work also relies on the Chamfer distance as the reconstruction term. Another latent variable model for point clouds is Set (Kim et al., 2021), which uses transformers to process point clouds sets. Their primary novelty being the introduction of a latent space with an enforced prior inside the transformer block. These transformer blocks are then stacked to form a hierarchical variational autoencoder (Sønderby et al., 2016), which com- plicates evaluation of its representations. However, the Vet also approximates their reconstruction loss via Chamfer distances. Without explicit likelihood evaluation, these models become closer to a regularized autoencoder than the variational autoencoder.
R EPPRESENTATIOSS
Other Generative Models. On the other hand, LION (Zeng et al., 2022) is latent diffusion model (Rom- bach et al., 2022) that maintains one-to-one mapping throughout the network, allowing for probabilistic evalu- ation. However, they only implicitly utilize this by opti- mizing an L1-loss. Similar to our work, they encode their points in a separate space, but instead of bottlenecking this, they map them to a higher dimensional space. This, unfortunately, leads to information about the shape being stored here, preventing direct sampling, modificction to the embedded points in this space. Similarly to Set VAF, evaluating the quality of representations in LION, hier- archical latent variable model, poses challenges. Recently, Zhou et al. (2023) presented FrePolad, another latent dif- fusion model. Their primary novelty the introduction of the frequency rectification module that better captures frequeeyyy signals in point clonds. They train their model via modified VAE loss to account for frequency rectified distances. One fully probabilistic work is Point- Flow (Yang et al., 2019). PointFlow utilizes a continuous normalizing flow (CNF) both as a prior and decoder, similar to approaches previously applied to images (Kingma et al., 2017; Sadeghi et al., 2019). Intuitively, one CNF models the distribution of shapes, while the other models the point distribution given the shape. In a comparable way, VF-Net's encoder maps to a global latent space, with point encoding projections providing a latent mapping for each input point. However, PointFlow's two CNFs are trained separately, whereas VF-Net trains them simultancously, result- ing in a more integrated and efficient process. FlontFlow is unfortunately very slow to train (Kim et al., 2021). On our full proprietary dataset, PointFlow would have required 200 GPU days of training. Thus, we excluded it from our basclines. Diffusion models such as diffusion probabilistic model (DPM) (Luo & II, 2021) and point- voxel diffusion (PVD) (Zhou et al., 2021) present two diffusion models for the point clouds, especially PVD generates accurate new samples. However, diffusion models do not find compressed structured representations of the data as our VF-Net does; see table. 1 for model property overview.
GENERATIVE
COMPLETION
PROBABILISTIE
M
SetVAE
X
LION
X
X
FrePolad
/
X
XX
PointFlow
X
DPM PVD FoldingNet VF-Net (ours)
X
X
/
X
V
V
X
/
Table 1: VF-Net is generative model (GENERATIVE) for point clouds, but it can gen- crate meshes without additional training (MESH) and do simple shape completion (COMPLETION). It is also fully probabilistic (PROBABILISTIC) and can identify interpretable lower-dimensional representations (REPRESENIATIONS)
Digital Dentistry. In computational deutistry, extrapolating the tooth's obstructed sides is well-known task. Qin et al. (2013) presents an attempt to use classic computational geometry methods. They at tempt to reconstruct the missing parts of the distal and mesial sides of the tooth. This leads to a very smooth extrapolation, which performs well. Several works within dentistry take this a step further, e.g., attempting to extrapolate not just the sides but also the roots of the teeth (Wei et al., 2015; Zhou et al., 2018; Wu et al., 2016). We are optimistic that our model could adapt to such task given that dental cone beat computed tomography (CBCT) of the dental roots was available in the training data. Unfortunately, CBCT scans are expensive and rare; thus, we do not have large enough dataset for neural network training.
3

---

Published in Transactions on Machine Learning Research (08/2024)
Local feature vector (7 x 64)
Point encoding Mesh
Decoder (lodding-bssed)
PointNet encoder
~ p(2)
Variance network
Elbo optimization
Figure 2:
VF-Net is variational autoencoder with normalizing flow prior the shape latent. Individual points are projected to 2D space, establishing a one-to-one connection and facilitating mesh generation and shape completion. The decoder follows FoldingNet's with added residual connections, while the variance network consists of folding modules as introduced in FoldingNet.
3 Variational Point Cloud Inference
Background: FoldingNet. To handle varying sizes and arbitrary order in point clouds, common strategy is to employ neural networks exhibiting invariance to changes in cardinality and permutation, as proposed by Qi et al. (2017) in PointNet. FoldingNet employs very similar encoder, e, that operates independently on each point of the point cloud to identify latent code, Subsequently, the folding-based decoder, f Z x R2 - R2, "folds" a chosen constant base shape with points, c, according to the latent code. In our case, the base shape is a constant uniform grid in the two-dimensional planar patch [-1, (Yang et al., 2018). Both the encoder, e, and the decoder, f, are jointly trained to minimize the reconstruction error approximated via Chamfer distances (1);
E = CHAMH- (x, c))
2
This ensures invariance to cardinality and permutation changes, although it complicates variational inference extensions. A variational autoencoder yields a distribution for each input point (Kingma & Welling, 201 4: Rezende et al., 2014). However, FoldingNet and most current permutation-invariae neural networks do not have correspondent output for each individual input point in point cloud.
3.1 The Variational FoldingNet
Motivated by unsupervised probabilistic representation learning's benefits across many tasks, including generative modeling (Kingma & Welling, 2014; Rezende et al., 2014; Dinh et al., 2017: Ho et al., 2020), dndtof-distribution detection (Nalisnick et al., 2019; Havtorn et al., 2021), handling missing data (Mattei & Frellsen, 2019) etc, we introduce Variational FoldingNet (VF-Net). Architecturally, VF-Net closely resembles FoldingNet, employing a PointNet encoder, with the decoder structure mirroring that of FoldingNet. For a complete overview, consult Fig. 2.
The major technical innovation the introduction of a novel projection for each input point into the planar space, defined as each point 9.. in the set g. We will refer to these projections to note that the point encodings are not constrained by any prior distribution. Decoding these point encodings instead of a static planar patch establishes a one to one correspondence throughout the entire nelwork, a necessity for evaluating likelihoods using the classical variational autoencoder objective. As VF-Net. learns the point projections from x, the projected points, g, are now dependent on x. The folding of the point encodings, f2. g), continues to be governed by the latent parameter vector z predicted by the PointNet encoder, The optimal projections are thus given by
1J². Let x be point cloud of points 2,,..,, Each projection corresponds
point encodings. It is important
= arg min I|=, - f(z.g)111 g'eg
where 95 E g. We use a neural network amortize the calculation of g such that the encoder network outputs both g and the distribution of z. By enabling the model adjust the point encoding, we eircumvent

---

Published in Transactions on Machine Learning Research (08/2024)
Ground Truth
VF-Net (ours)
Reconstructon
Figure 3: Top: Mesh data samples from our released FDI 16 dataset and their corresponding VF-Net reconstructions. Note the large variety health conditions between the teoth.
the need for optimizing through costly Chamfer distances. Furthermore, the learned projections allow the point encodings to adapt to their input, mitigating common pitfalls observed in FoldingNet, see Fig. 4.
With a one-to-one point correspondence established across the network, we optimize our model using tra- ditional variational autoencoder methods. In this context, the variational extension aligns closely with traditional methods, yet with a notable adjustment, the evaluation of likelihood now also depends on the projected points p(x) = f p(x/z, g) p(z) dz. This integral remains intractable, and approximations are nec- essary. Following conventional variational inference (Kingma & Welling, 2014; Rezende et al., 2014), evidence lower bound (Elbo) on p(x) is given by
[(x) Eg(at) [lcelx))logxxxxe g)] KKLg(/))))
where (xx/x) an approximation to the posterior p(z|x), which assumed to follow a gaussian distribution. Note that Eq. 3 is implicitly optimized in the likelihood term of the ELBo. Most current point cloud models replace the likelihood with Chamfer distance, making the models closer regularized autoencoders (Yang et al., 2018; Groueix et al., 2018; Kim et al., 2021). This design loses one-to-one correspondences between input and output, making likelihood evaluation difficult. In particular, no suitable normalization constant can be derived for probabilistic distributions using Charnfer distances.
Our novel method for probabilistic evaluation for 3D reconstruction networks avoids the computationally expensive Chamfer distance (1). In supplementary Fig. S1, we empirically demonstrate that our projections can effectively replace Chamfer distances. We observe that the two metrics closely align, with Euclidean upper bound that tightens with improved reconstruction precision.
distances acting
During the evaluation of the Elbo loss we use multivariate student-t distribution with isotropic vari- ance and three degrees of freedom as the reconstruction term. This choice helps to decrease empha- sis on outliers and instead focus more on the majority of the data points (Takahashi et al., 2018). p(x|z,g) = Student-t(xIf(2, g), o2 v),, u)), where f Z x2 + R² and o² Z x IR + R2 are neu- ral networks. No major changes were made to the generative process. We let normalizing flow model the prior, p(z), which describes the shape of an object (Kingma et al., 2017). Note that this trained subsequently and no downscaling of the Kullback-leibler divergence term occurs. When the input, X, and the projections, G, share topology, the bias allows for uniform sampling in the planar patch [-1, 1Ja. As in FoldingNet, this grid is subsequently deformed according to New samples can thus be generated by first sampling z and then mapping the uniformly sampled grid points through f and a,
x f(z, g) + o g) t, t Student-t(z).
(5)
This also enables straightforward mesh generation as deformations are smooth points projected closely to each other correspond points close in output space. Consequently, we can generate meshes by simply defining the facets in the 2D planar space.
5

---

Published in Transactions on Machine Learning Research (08/2024)
4 The FDI 16 Tooth Dataset
To improve the state-of-thc-art modeling of dental scans, we will release an extensive new dataset alongside this paper under the CC BY-NC-SA 4.0 license. The FDI 16 dataset is a collection of 7,732 irregular triangle meshes of the right-side first maxillary molar tooth formally denoted as 'FDI 16' following ISO 3950 notation (see Fig. 3). These meshes were acquired from fully anonymized intraoral scams primarily scanned using 3Shape's TRIOS scanners. Each tooth in the FDI 16 Tooth datasct was algorithmically segmented from an upper jaw scan by 3Shape's Ortho Systems 2023. As the teeth are subsection of full intraoral jaw scan, there will be areas obstructed by the adjacent teeth. The teeth, therefore, constitute open meshes and have clear boundaries with no representatinn of interior object volume. All tooth meshes are from patients undergoing aligner treatment, and accordingly, aligner attachments will be present in a substantial number of scans. This introduces a bias towards younger individuals, who generally have fewer restorations and dental problems. The top row of Fig. shows examples of such meshes. All scans have been made publicly available fully anonymously as meshes and point clouds at millimeter scale. The teoth have been algorithmically rotated ensure that the r-axis is turned towards the neighboring tooth (FDI 17) while the y-axis points in the occlusal direction (dircction of the biting surfacc). Finally, the z-axis is given by the cross-product to ensure a right-hand coordinate system.
Dental scans have a diverse set of research applications. This study explores reconstruction, generation of new teoth, reprosentation lcarning, and shape completion. All of which have different but critical applications in digital dentistry. We believe that the FDI 16 dataset addresses a crucial niche within 3D datasets by offering a dataset that strikes balance between the highly detailed but idealized CAD scans (Chang et al., 2015) and sparser real-world LIDAR scans (Chang et al., 2017; Caesar et al., 2020; Armeni et al., 2016). Note that any method considered for deployment must be capable of running efficiently edge devices without a significant performance overhead. This is particularly important as intraoral scanners must function seamlessly even in areas with limited network connectivity.
5 Experimental results
We next evaluate VF Net's performance on point cloud generation, auto encoding, shape completion, and unsupervised representation learning. Note that. FrePolad (Zhou et al., 2023), EditVAR (Li et al., 2022), and VG-VAE (Anvekar et al., 2022) has been excluded from comparison as no public implementation is available.
Point cloud generation. To compare sampling performances, we deploy three established metries for 3D generative model evaluation (Yang et al., 2019). Namely, minimum matching distance (MMD) is a metric that measures the average distance to its nearest neighbor point cloud. Coverage (COV) measures the fraction of point clouds in the ground truth test set that is considered the ncarest test sample neighbor for a generated sample. 1-nearest neighbor accuracy (1-NNA) uses a 1-NN classifier to classify whether sample is generated or from the ground truth dataset, 50%, meaning generated samples are indistinguishable from the test set. Data handling and training details for FDI 16 experiments can be found in supplementary section S1.3 and S1.4, respectively.
Table 2: Across five seeds, VF-Net produces close to as large a variety of teeth as PVD and LION while generating samples much closer to real teeth. MMD has been multiplied by 100.
MMD(L)
COV(%)T
1-NNA(%4)
Method
CD
EMD
CD
EMD
CD
EMD
Train subsampled
21.00+0.00
51.53+0000
49.00
46.95-2.10
49.83
50.97
SetVAE
39.00
66, 66.6660..s
10.66
9.52+ 27
97.99
97.95 84
DPM
20.71 10.10
51.9442.oa
36.941n.6s
33.28=p.es
70.30 =..02
75.75
PVD
2..58+o..0
51 64.00
44. 41.
43. 43.23=o.92
62.85 0.78
60.70+1 06
LION
22.124t.15
52.75.20.12
45. 2 zo.00
43.32 2
68 68.56=um
66. 66.762uu
Ve-Net (Ours)
20.38
49.72
42.85
40.20
56.3 1 10.09
56.05

---

Published in Transactions on Machine Learning Research (08/2024)
Ground Truth
FoldingNet
VF-Net (ours)
Figure 4: FoldingNet's mesh reconstructions have gaps and highly distorted facets. Conversely, VF-Net's mesh facets even more regular than the input point cloud, and points in the reconstruction are placed closely resembling its input.
Sampling from VE-Net can be done by sampling a uniform grid in the latent point encodings space, akin FoldingNet. However, the corners of the uniform grid cause edge artifacts in the generated samples, evident in generated meshes in Fig. S2. This can also be observed in the generated meshes in Fig. 3 and Fig. 4, although it is more difficult to spot. The sampling metrics heavily punish such artifacts. Instead, we trained a minor network similar to the decoder of FoldingNet to predict the point encodings from the latent representation. We emphasize that this entirely unnecessary for regular sampling. The sampling evaluations across five different seeds can be found Table 2. The results demonstrate that VF-Net generates much more accurate samples, as evidenced by the significantly lower MMD and 1-NNA scores while being close in diversity PVD and LION (Zhou ct al., 2021; Zeng et al., 2022). Furthermore, sampling is much faster than PVD and LION FetNet does not depend on iterative diffusion process. Note that while MMD is very stable across seeds, the COV and 1-NNA scores may vary.
Outside of the FDI 16 dataset, we also train VF-Net on proprietary dataset, which includes the remaining teeth from the FDI 16 jaws; see supplementary section $1.5 for training details. However, we did not quantify sampling performance, as sampling evaluation on 40k test samples would be exceedingly computationally expensive. We observe that VF Net can sample from all major teeth types, incisors, canines, premolars, and molars, Fig. 1. Additional mesh samples may be found in supplementary Fig. S2.
Table 3: Reconstruction error measured in Chamfer distances (CD) and earth mover's distances (EMD). Note both values have been multiplied by 100.
FDI 16 Tooth
All FDIs
Method
CD
EMD
CD
EMD
DPM
10.04
43.98
5.67
35.8
Set
21.50
59.24
9.98
51.48
LION
5.35
22.85
3.02
9.66
FoldingNet
5.26
33.67
3.43
31.25
VF-Net (ours)
1.21
6.30
0.97
5.30
7

---

Published in Transactions on Machine Learning Research (08/2024)
Point cloud auto-encoding. We evaluate VT-Net's reconstruction quality to the previously mentioned generative models and FoldingNet. This evaluation was performed on both on FDI 16 dataset and the larger proprietary dataset. Please consult supplementary sections $1.3 and S1.5 for data handling and training details. We compared the reconstruction errors using Chamfer distance and earth mover's distance (Rubner et al., 2000),
EMD(x, min dix-t
1 1ar-0(aa ((;))l2 2iEx
Σ
(6)
The earth mover's distance measures the least expensive one-to-one transportation between two distri- butions. However, this is computationally expensive and thus rarely used for model optimization (Wu et al., 2021). The reconstruction errors are presented in Table 3. Point-Voxel Diffusion (PVD) (Zhou et al., 2021) was excluded from comparison due to not returning the same tooth upon reconstruction. VF-Net achieves significantly lower reconstruction error than our comparison methods on both the FDI 16 dataset and the proprietary dataset comprising 119,496 teeth, encompassing 32 distinct teeth. As shown in Fig, VF-Net's one-to-one correspondence is evident in its reconstruction. The point placements mimic those in the input point. cloud, while FoldingNet's 's points are evenly distributed. VF-Net and FoldingNet can both generate meshes without any additional training of the model.
However, NoldingNet folds the edge across the tooth to accommodate teeth of different sizes. Besides mesh gaps, this also leads to highly irregular facets that intersect one another. On the other hand, VF-Net can adjust the point encoding area to avoid such artifacts. However, VF-Net's s reconstructions often exhibit excessive smoothness and lack the desired level of detail. A common obscrvation variational autoencoders (Kingma & Welling, 2014; Vah- dat & Kautz, 2021: Tolstikhin et al., 2019).
Figure 5: Iutra-point cloud relative predicted variance (red is high, green is low). Notably, the carabelli cusp and aligner attachment areas ex- hibit high variance, two features only present in a subset of individuals.
Reconstructions and the Corresponding Latent Point Encoding
Variance estimation for point clouds. Predicted variances from the variance network are shown in Fig. 5, where red indicates higher variance and green indicates
Reconstruction
a lower variance wit each point cloud. Note that all variances shown are relative antra-point clond variances. Notably, the network assigus higher variance to the fifth cusp and aliguer attachments, features ouly present in a subset of samples. Furthermore, the border of the mesh tends to be assigned higher variance, likely due to a combination of data loading and segmentation artifacts. When the network is not in doubt about the previously mentioned two factors, the network assigns the highest variance to the occlusal surface. All of which aligns with expectations of areas of the teeth that have the most variance.
-2 Latent Point Encoding
Figure 6: Left: Red points removed from the point cloud. Right: Reconstruction and projected point encodings remain highly similar despite point deletion. Sampling the missing area is facilitated by sampling within the correspond- ing empty region of the latent point encoding.
Simulated shape completion. One significant benefit of the inductive bias from the point encodings is straight- forward shape completion and shape extrapolation. In computational deutistry, inferring the obstrd sides of tooth and reconstructing the tooth surface beneath ob- structions such as braces pose a key challenge. Paired data of obstructed and unobstructed surfaces exceedingly rare. Therefore, developing a model capable of extrapolating such surfaces without. explicit training highly desirable. To this end, we simulate the

---

Published in Transactions on Machine Learning Research (08/2024)
task by evaluating the interpolation performance of each model. This is done by sampling point the outward side of the tooth and deleting its nearest neighbors to total of 200 points. Selecting a mid-buccal point simulates bracket removal prediction ("Bracket sim') while opting for a lower buccal point simulates the obstructed side prediction ('Gap sim").
An example of a synthetic hole is depicted in Fig. 6, where the red points are to be removed. Both recon structions and latent point encodings remain highly similar despite the removal of the red points. Extrap- /lation/interoolann
Table 4: Unsupervised generative models in the top half are untrained interpolation, while the boltom half are trained models. All Chamfer distances have been multiplied by 100.
can be performed by sampling in the point encoding space. To quantify the interpolation performance, we calculate the distance from the deleted points to their nearest neighbor in the completed point cloud; soe supplementary Sec. S1.7 for more experiment details. To contextualize the performance, we trained several shape completion methods (PVD (Zhou et al., 2021), PoinTr (Yu et al., 2021), VRCNet (Pan et al.,
Method DPM
Bracket sim
Gap sim
Unanpervvied
15.88
38.00
Set
11.50
13.35
FoldingNet
16.42
20.14
VF-Net (ours)
4.35
3.55
Supurataed
PVD
2.23
2.37
2021)). Since these methods only predict the missing area, a completely fair comparison cannot be made. The results can be found in Table 4, under 'Bracket sim' and 'Gap sim,' simulating the removed bracket and the gap between teeth, respectively. Here, VF-Net outperforms its peers when it comes to untrained interpolation, and as expected there is gap in performance between the trained and untrained methods. Shape completion using LION's latent points from the original tooth contains information about the shape, rendering a fair comparison infeasible.
PoinTr
1.84
1.83
VRCNet
2.42
2.04
Table 5: Percentage teeth which had classification prediction increase according to expectation when moved in the tooth wear direction. L, M, H denotes light, medium, and heavy wear respectively.
Method
L + H
L
M
M
L
M
+
H
H
+
M
H
L
FoldingNet
91.77
91.77
95.02
94.89
97.80
97.80
VI-Net (ours)
92.11
99.31
97.04
96.37
98.24
99.12
Representation learning, We compare our latent representation to FoldingNet's, as it is the comparison model with the most interpretable latent variables. First, we follow FoldingNet's proposed evaluation method of classifying the input point cloud from the latent space. Using a linear support vector machine (SVM) to classify which tooth from the larger proprietary dataset is embedded, 32-class problem. Here, the SVM achieves 96.80% accuracy VF-Net's latent codes compared to 96.36% of FoldingNet. Indicating all global point cloud information is stored in the latent variables, meaning the latent point encodings exclusively contain information about specific points. No information pertaining to the overall point cloud shape is stored in the point encodings. For qualitative assessment, an interpolation between two FDI 16 teeth and an interpolation example between an incisor and a premolar can be found in Fig. 7. Both interpolations exhibit seamless transition in the latent space; for a more detailed view, see supplementary Fig. S3.
Figure 7: Interpolating between two teeth by intorpolating their latent codes using the same mesh decoding.
9

---

Published in Transactions on Machine Learning Research (08/2024)
Removed Tooth Wear
Medium Wear Reconstruction
Added Tooth Wear
Figure 8: Moving in the tooth wear direction in latent spacc. Left: Red areas have higher values than the original. Middle: The original reconstruction. Right: Blue areas lower than the original. As the level of tooth wear increases, we observe a gradual smoothing in the occlusal surface.
Next, attempt to add and remove toothwear; see Fig. 8 We navigate the latent space of VF-Net in the direction of tooth wear away from it. The direction was determined by calculating the average change in latent representations when encoding 10 teeth from their counterparts with synthetically induced tooth wear. These teeth were manually sculpted to simulate tooth wear; see supplementary Fig. S4. We observe behavior that closely aligns with our expectations of how the tooth would change when adding or subtracting tooth wear.
To quantify the performance, we train small Point model (Qi al., 2017) proprietary dataset of 1400 teeth annotated with right/mediumg /heavy tooth wear. Subsequently, validate whether change in the latent space yielded the expected change in classifier prediction. In Table 5, each class denotes the base class before
tooth wear. For light and heavy, we added and removed tooth wear, respectively, while medium tooth wear teoth were evaluated both when adding/renen graiin//rmovingggg.. wear. The findings presented in Table indicate that VF-Net's latent representations show greater robustness.
Limitations. Similar variational autoencoders in other domains, VF-Net tends to produce overly smooth samples. This characteristic could impact applications such as crown generation, where precise replication of the biting surface is crucial to prevent patient discom- fort. Moreover, the model's tendency towards smoothness suggests potential challenges capturing finer details of teeth, which are essential for comprehensive representa
Reconstructions and Corresponding Latent Point Encoding
Reconstruction
tion learning.
Latent Point Encoding
Until now, the inductive bias from folding 2D plane to point cloud has proven highly beneficial. This is only the case when the imput point cloud shares topology with the 2D plane. Unfortunately, this inductive bias not as ben- eficial when the two topologies differ. We trained VF-Net on ShapeNet data (Chang et al., 2015). The drawback is supplemen-
Figure 9: Left: While accurately reconstructed, the airplane forms a non-continuous distribution in the latent point encoding, posing challenges for sampling. Right: An incisor and its corre- sponding point encodings.
not evident. through the reconstructions; tary Table S1. VF-Net has low reconstruction error, but LION boasts the lowest. Issues arisc when attempting to generate new samples. Due to information of the shape being stored in the latent point encodings, as depicted in Fig. The latent point encodings form a non-continuous distribution, posing challenges for sampling new models. Note that for point clouds sharing topology, VF-Net is strongly biased towards generating contin- hous distribution; see Fig. 9. Addressing this issue could potentially be solved by training a flow or diffusion
10

---

Published in Transactions on Machine Learning Research (08/2024)
prior for the point encodings, similar to the approach used in LION (Zeng al., 2022). However, since this was not the focus of our model, we did not pursue this idea.
6 Conclusion
We have introduced the FDI 16 dataset and Variational FoldingNet (VF-Net),, fully probabilistic point cloud model in the same spirit as the original variational autoencoder (Kingma & Welling, 2014; Rezende et al., 2014). The key technical innovation is the introduction of point-wise encoder network that re- places the commonly used Chamfer distance, allowing for probabilistic modeling. Importantly, we have shown that VF-Net offers better auto-encoding than current state-of-the-art generative models and more realistic sample generation for dental point clouds. Additionally, VF-Net offers straightforward shape com- pletion and extrapolation due to its latent. point encodings. All while identifying highly interpretable latent representations.
Impact statement. This paper contributes a generative model that is particularly suitable for dental data. This translates into several positive use cases within clinical practice. However, previous generative models have shown to be useful for less positive use cases such as deep fakes and fake news. It unclear how this could take form in digital dentistry, but destructive minds tend to be creative.
This work was funded in part by the Novo Nordisk Foundation through the Center for Basic Machine Learning Research in Life Science (NNF200C0662666) SH was supported in part by research grants (15334, 42062) from VILLUM FONDEN. JY was supports by Innovation Fund Denmark (1014-00172B).
Acknowledgements.
References
Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning Representations and Generative Models for 3D Point Clouds, June 2018. URL http: arXiv:1707.02399 [cs].
http:://arxv.orrr/ass//T0...... wwtp://///////o/222.... 02392.
Thiemo Alldieck, Gerard Pons-Moll, Christian Theobalt, and Marcus Magnor. Tex2Shape: Detailed Full Human Body Geometry From Single Image. 19X:::190400845 [cs]. September 2019. URL http: arXiv: 1904.08645.
Tejas Anvekar, Ramesh Ashok Tabib, Dikshit Hegde, and Uma Mudengudi. VG-VAE: Venatus Geometry Point-Cloud Variational Auto-Fncoder. In 2022 IEFE/CVF Conference Compuler Vision and Paltern Recognition Workshops (CVPRW), pp. 2977-2984, New Orleans, LA, USA, June 2022. IEEE. ISBN 978 1 66548-739-9. doi: 2TL11///CCVPWW33377722000000 9857384/.
URL https://ieeexpler ieee org/document..
Iro Armeni, Ozan Sener, Amir R. Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 3D Semantic Parsing of Large-Scale Indoor Spaces. In 2016 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 1531 1543, Las Vegas, NV, USA, June 2016. IEEE. ISBN 978-1-4673- URL http://ieeexplore ieee org//bcument/7706339
8851-1. 1. doi: D2.100/CVVP..C
Harry G. Barrow, Jay M. Tenenbaum, Robert C. Bolles, and Helen C. Wolf. Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching. August 1977. URL https://
het//forummid=rkboottere
Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuScenes: A multimodal dataset for autonomous driving,
May 2020. URL http: //tpx//aivvvv///////// org/abs/1900 11027. arXiv:1903.1100 [cs, stat].
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Nießner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3D: Learning from RGB-D Data in Indoor Environments, September 2017. URL httt:///axxin http://aiii...rr/////////// 06158. arXiv:1709.066. [cs].
11

---

Published in Transactions on Machine Learning Research (08/2024)
Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An Information Rich 3D Model Repository, December 2015. URL http://arxiv/ http:///aaiiv..oggbbb/////ss /tt../////////1..00022222 arXiv:1512.0301 [cs].
Nicki S. Detlefsen, Martin Jorgensen, and Søren Hauberg. Reliable training and estimation of variance networks, November 2019. URL http://axxivv.oggbb////s/
03260. w:Xiv:1906.03200 [cs, stat].
Laurent Dinh, Jascha Sohl- Sohl-Dickstein, and Samy Bengio. Density estimation using Real NVP, February 2017.
URL http://arxiv orgg http: 000.///00.00000000...3 08803. arXiv:1605.08800 088800 [cs, stat].
Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, and Mathieu Aubry. AtlasNet: A Papier-Mache Approach to Learning 3D Surface Generation. arXiv:1802 180.:03888888 [cs], July 2018. URL arviv: 1802.05384.
http:/ / http///axxvvv.. http:/axxvvv.../ga//// wttp://.wrg/b////////// 05384.
Zhizhong Han, Xiyang Wang, Yu-Shen Liu, and Matthias Zwicker. Multi-Angle Point Cloud- Unsu- pervised Feature Learning for 3D Point Clouds from Multiple Angles by Joint Seff-Reconstruttion and Half- Half Prediction. arXiv:1907.1220 [es], July 2019. URL http://arxiv, wrg/ http://araav.org/bbb//00111111 12704.
arXiv: 1907.12704.
Jakob D. Havtorn, Jes Frellsen, Søren Hauberg, and Lars Maalge. Hierarchical VA Know What They Don't Know. arXiv:2102 08248 8 fes, statj, March 2021. URL 2102.08248.
08248. arXiv:
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models, December 2020. URL http://arxta: org/abs abs////6...../////// 11239. 2XXi:20006112399 [cs, stat].
Jinwoo Kim, Jaehoon Yoo, Juho Lee, and Seunghoon Hong. Set Learning Hierarchical Composition for Generative Modcling of Sct-Structured Data, March 2021. URL http: http:///axvvvvor//aaae/ axggabs//1033 15619. arXiv:2103.16619 [cs].
Diederik P. Kingma and Max Welling, Auto-Encoding Variational Bayes. arXiv:1312:6114 [es, stot], May
2014. URL http://arxiv. org wttp://...rrg///////////2 http///b///111111/// arXiv: 1312.6114.
Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, llya Sutskever, and Max Welling, Improving Variational Inference with Inverse Autoregressive Flow, January 2017. URL http://arxiv.ot /bs// 1606. 04934. arXiv:1606.0488 049944 [cs, stat).
Chun-Liang Li, Manzil Zaheer, Yang Zhang, Barnabas Poczos, and Ruslan Salakhutdinov. Point Cloud GAN. arXiv:1800. 05795 [cs, stat], October 2018. URL http://arxiv. wwtp/bs//10/....... 05795. 1810.05795.
arXiv:
Ruihui Li, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. PU-GAN: Point Cloud Upsampling Adversarial Network. vrXiv:1077.1 [esj, July 2019. URL 10844. arXiv: 1907.10841.
Shidi Li, Miaomiao Liu, and Christian Walder. Edit Unsupervised Part-Aware Controllable 3D Point Cloud Shape Generation, March 2022. URL http://arxiv. org/abs/2110. 06679. arXiv:2110.06670 [cs).
Shitong Luo and Wei Hu. Diffusion Probabilistic Models for 3D Point Cloud Generation, June 2021. URL http://arxiv. org/ attp////....//////////////aaaaaa 01458. arXiv:2103.01458 [cs].
Pierre-Alexandee Mattei and Jes Frellsen. MIWAE; Deep Generative Modelling and Imputation of Incomplete Data, February 2019. URL http://arxiv/ wwtp://g///ab//aeeeeao 1263..0033.1......33 arXiv:1812.02633 (cs, stat].
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do Deep Generative Models Know What They Don't Know?, February 2019. URL http://arxit: http//rrwwwi..r//////////// 09136. arXiv:1810.09133 (cs, stat).
12

---

Published in Transactions on Machine Learning Research (08/2024)
Liang Pan, Xinyi Chen, Zhongang Cai, Junzhe Zhang, Haiyu Zhao, Shuai Yi, and Ziwei Liu. Varia- tional Relational Point Completion Network, April 2021. URL http: http://aaxxivoorrr/b////10/44 wrg./bs////////.o/./b/b/////// 10154. arXiv:2104.00154 [cs).
Jiahao Pang, Duanshun Li, and Dong Tian. TearingNet: Point Cloud Autoencoder to Learn Topology- Friendly Representations. arXi::2006.1013 10187 [cs], September 2021. URL http:///rxtv.orr/abb///// arg/abs/206...
10187. arxiv: 2006.10187.
Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. arXiv:1612.0053 00593 [cs}, April 2017. URL http: http:///axvv..rr/bb.... 1612.00593. arXiv: 1612.00593.
Nina Qiu, Ran Fan, Lihua You, and Xiaogang Jin. An efficient and collision free hole-filling algorithm for orthodontics. The Visual Computer, 29(6):577 586, June 2013. ISSN 1/32-2315. doi: 10.1007/ 500371-013-0820-6 URL attps://dot htt////ii.og//////// 1007
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic Backpropagation and Ap proximate Inference in Deep Generative Models. arXiv:1401.4000 fes, stat], May 2014. URL http:
//t.xiv.ogg/abs//////// http:/org//bb//11...ee 1401.400822
arXiv: 1401.4082.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models, April 2022. URL http://arxi: wwtp://wwwr//b///11..11..00
arXiv:2112.10752 [cs].
Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. The Earth Mover's Distance Metric for Image Retrieval. Interrnational Journal of Computer Vision, 40(2):99-121, November 2000. ISSN 1573-1405. doi: 10.
URL https: /ttps://doi. //doi. org/1o///// 10. 1023/1: 1026543900554 10200.000022/11105550000444
Hossein Sadeghi, Evgeny Andriyash, Walter Vinci, Lorenzo Buffoni, and Mohammad H. Amin. PixeIVAE++: Improved PixeIVAE with Diserete Prior, August 2019. URL http://arxiv.or wtgg/as///00...../ 09948.
arXiv:1908.09948 [cs, stat].
Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, and Ole Winther, Ladder Varia- tional Autoencoders. arXiv:1602 02282 fcs, stat), May 2016. URL http://arxiv. org/ wwtpeiii.rgga/b///eeeeee httpa/ab////1222222 02282. arXiv: 1602.02282.
Hiroshi Takahashi, Tomoharu Iwata, Yuki Yamanaka, Masanori Yamada, and Satoshi Yagi. Student- Variational Autoencoder for Robust Density Estimation. In Proceedings of the Twenty-Seventh In- ternational Joint Conference on Artificial Intelligence, pp. 2696-2702, Stockholm, Sweden, July 2018. International Joint Conferences on Artificial Intelligence Organization. ISBN 978-0-9992411-2-7. doi: ppwpoeeeiingg//11/////0/
10. 224963/ijcai.20133
URL https://www.it
Ilya Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schoelkopf. Wasserstein Auto-Encoders December 2019. URL
arXiv:1711.01555 [cs, stat].
Arash Vahdat and Jan Kautz. NVAE: A Deep Hierarchical Variational Autoencoder. 0rXhv:2007.03898 fes, stat/, January 2021. URL http:://arxa hwwp:///bs///0..oo
arXiv: 2007.03898.
Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images. ar Xiv:1800 01654 [cs], August 2018. URL http://arxiv.
org/abs/ 1804. 01654. arXiv: 1804.01654.
Xiaomeng Wei, Li Chen, and Chaowei Gao. Automatic mesh fusion for dental crowns and roots in a computer-aided orthodontics system. In 2015 8th International Conference Biomedical Enginecring and Informatics (BMEI), pp. 280-290, October 2015. doi: 10.1109/BMEI 200...........1111166
Chao Wen, Yinda Zhang, Zhuwen Li, and Yanwei Fu. Pixel2Mesh++: Multi- View 3D Mesh Generation via Deformation. arxiv:1908. 100:1.00.04911 [cs], August 2019. URL http://arxix wwwpp////.rrabbbb////////// 01491. arXiv: 1908.01491.
13

---

Published in Transactions on Machine Learning Research (08/2024)
Chenglei Wu, Derek Bradley, Pablo Garrido, Michael Zollhöfer, Christian Theobalt, Markus Gross, and Thabo Beeler. Model-based teeth reconstruction. ACM Transactions on Graphics, 35(6):1-13, November 2016. ISSN 0730 0301, 1557 7368. doi: 10.114//298119.2996000000 2980233. URL https://dl.. acm org/doi/10. 1145/2980179. 2980233.
Jiajun Wu, Chengkai Zhang, Xiuming Zhang, Zhoutong Zhang, William T. Freeman, and Joshua B. Tenen- baum. Learning Shape Priors for Single- View 3D Completion and Reconstruction. 1r:: [cs], September 2018. URL http://arre
org/ abs/ 05068. arXiv: 1809.05068.
Tong Wu, Liang Pan, Junzhe Zhang, Tai Wang, Ziwei Liu, and Dahua Lin Density-aware Chamfer Distance Comprehensive Metric for Point Cloud Completion, November 2021. URL http://arxiv... org/ abs/
2111. arXiv:2111. 12702 [cs].
Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Yu Liu, Serge Belongie, and Bharath Hariharan. PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows, September 2019. URL http://arxiv. org/abs/1906. 12320. arXiv:1906.1222 12320 [cs).
Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation. arXiv:1712.0083 07262 /cs/, April 2018. URL 1712.07262.
arXiv:
Xumin Yu, Yongming Rao, Ziyi Wang, Zuyan Liu, Jiwen Lu, and Jie Zhou. PoinlI: Diverse Point Cloud Completion with Geometry-Awan Transformers, August 2021. URL 2108.08839. arXiv:2108. 0883999888899110 [cs].
org/abs/
Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojeic, Or Litany, Sanja Fidler, and Karsten Kreis. LION: Latent Point Diffusion Models for 3D Shape Generation, October 2022. URL http://arxiv.orrg abs/2210. 06978. arXiv:2210.06978 [cs, stat].
Zerong Zheng, Tao Yu, Qionghai Dai, and Yebin Liu. Deep Implicit Templates for 3D Shape Representation. In 2021 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 1429 1439, Nashville, TN, USA, June 2021. IEEE. ISBN 978-1-66544-509-. URL https://leeexplors leee... org/dbccument/9578111///
doi: 10. LLL.9//CVR6647...000 200
Chenliang Zhou, Fangcheng Zhong, Param Hanji, Zhilin Guo, Kyle Fogarty, Alejandro Sztrajman, Hongyun Gao, and Cengiz Oztireli. FrePolad: Frequency-Rectiffel Point Latent Diffusion for Point Cloud Cenera- tion, November 2023. URL http://arxii: http://org//b///////a/s 12090. arXiv:2311.12099 [cs].
Linqi Zhou, Yilun Du, and Jiajun Wu. 3D Shape Generation and Completion through Point- Voxel Diffusion, August 2021. URL http://arxixl orggabrg./.. 2164.03670......11101 arXiv:2104.0356 [cs].
Xinwen Zhou, Yangzhou Gan, Jing Xiong, Dongxia Zhang, Qunfei Zhao, and Zeyang Xia. Method for Tooth Model Reconstruction Based on Integration of Multimodal Images. Journal of Healtheare Engineering, 2018:1-8, June 2018. ISSN 2040-2295, 2040-2309.
doi:
URL
fttps:///wwr hindawi. http//juuumlll//////////////33.
14


Reviews:
[{"review_id": "A6NZjVvyIS", "reviewer": "Reviewer_b4Xv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- **Dataset Contribution:** The release of the FDI 16 Tooth Dataset is a valuable contribution to the field. This dataset provides a rich resource for future research in digital dentistry.\n- **Approach:** The introduction of Variational FoldingNet (VF-Net) effectively replaces the traditional Chamfer distance minimization with a probabilistic encoder, increasing computational efficiency and simplifying the probabilistic modeling.\n- **Empirical Performance:** The paper demonstrates lower reconstruction errors and state-of-the-art performance in dental sample generation, showcasing the superiority of VF-Net in dental reconstruction and interpolation tasks.\n\n## Weaknesses\n- **Reproducibility:** The dataset release is a valuable contribution; however, the use of a private dataset for certain experiments, particularly in the \"Representation Learning\" section, limits the study's reproducibility. Additionally, while annotations are provided for the proprietary dataset, they appear to be missing for the openly released dataset, which further hinders replication efforts.\n- **Discussion of diffusion models:** The authors discuss the non-compression of the space of diffusion models; however, they overlook approaches such as Latent Diffusion Model [1], which effectively addresses this issue. Including a discussion on these methods would enhance the study.\n\n[1] Rombach, Robin, et al. \"High-resolution image synthesis with latent diffusion models. 2022 IEEE.\" CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces Variational FoldingNet (VF-Net), a novel model that improves point cloud processing by utilizing a probabilistic encoder instead of traditional Chamfer distance minimization, enhancing computational efficiency and simplifying probabilistic modeling. It demonstrates state-of-the-art performance in dental reconstruction and interpolation tasks, achieving lower reconstruction errors. Additionally, the paper contributes to the field by releasing the FDI 16 Tooth Dataset, a comprehensive resource for future research in digital dentistry.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "My main requests regard the discussion of the reproducibility of some of the experiments and a more thorough discussion of the applicability of Diffusion Models in the field.", "broader_impact_concerns": "The authors already included an Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yhyM18AKoR", "reviewer": "Reviewer_HU4R", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The experiments of the paper look relatively comprehensive and several evaluations are covered, including point cloud generation, shape completion, representation learning, etc. The baselines seem to be recent advances in the area.\n\n2. The release of a new dataset is highly welcome, which can be valuable to the research area.\n\nWeaknesses:\n\n1. The focus of the paper is on the newly introduced dental point cloud dataset. It is a bit unclear about the unique characteristics of the new dataset compared with existing point cloud datasets. For example, does the new dataset bring new research challenges that existing method cannot address and can be addressed by the proposed method? If not, it might be interesting to evaluate the proposed method on other benchmark datasets as well.\n\n2. The paper claims that the proposed method is the first fully probabilistic VAE for point clouds. However, there are existing VAE methods such as in [1, 2]. Given the existing works, the claim might be bold to me. Moreover, the discussions and comparions to the following methods are also needed. \n\n[1] Anvekar, Tejas, et al. \"VG-VAE: a venatus geometry point-cloud variational auto-encoder.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.\n\n[2] Li, Shidi, Miaomiao Liu, and Christian Walder. \"EditVAE: Unsupervised parts-aware controllable 3D point cloud shape generation.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 2. 2022.\n\n3. The clarity of the paper needs to be improved significantly.\n- A few details are missing in the paper, including: what's the architectures of the encoder and decoder (I guess the paper reuses those from FoldingNet, but it is important to introduce the details)? What's the prior and variational posterior of $z$? \n- A few configurations of the method are not justified or motivated or supported with ablation study, including why the projection in Eq 3 and 4 is useful? Why student-t distribution is used as the likelihood function?\n- It is hard to understand the core part of the proposed method. Specifically, Eq 4 is a minimisation. How is this minimisation incorporated in ELBO of VAE? Eq 4 is also a reconstruction term of the input $x$. Why is a square error used here?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The contributions of this paper are summarised as follows:\n\n1. The paper proposes a new VAE method for representation learning and generation of 3D point clouds, with a focus on a dental point cloud dataset. The proposed method appears to achieve better performance than a few recent methods on the dataset.\n\n2. The paper introduces a new 3D dental point cloud dataset.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. More discussions and/or comparions to the methods in [1, 2] and adjustment to the claims of the contributions accordingly.\n\n2. Improvement to clarity.", "broader_impact_concerns": "Broader Impact Statement has been presented in the paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "v3siuMPEVz", "reviewer": "Reviewer_efj1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Weaknesses\n1. The main weakness in my view is that the method description (Section 3) is difficult to understand.\n     - For one, some of the notation is all over the place. For example, $\\mathcal{G}$ is a latent space, $\\hat{\\mathcal{G}}$ is a finite set of latent points, and $\\hat{\\mathcal{G}}(\\mathbf{x}, \\mathbf{z})$ is a specific \"projected\" point in the set $\\hat{\\mathcal{G}}$. These are confusing for a couple reasons.\n       - Referring to specific points as $\\hat{\\mathcal{G}}(\\mathbf{x}, \\mathbf{z})$ is misleading in light of the fact that a lowercase $\\mathbf{g}$ is otherwise used for generic points in the same set. \n       - The $\\hat{\\mathcal{G}}(\\mathbf{x}, \\mathbf{z})$ notation for a projection may or may not mean the same thing as a different notation introduced beforehand: $\\text{proj}_\\mathcal{G}(\\mathbf{x}): \\mathbb{R}^3 \\to \\mathcal{G}$.\n\n       These are just a couple of examples of where notational choices could be more consistent and suggestive of the meaning they signify. The fact that most of these notations are introduced in one paragraph and then never used again makes it even harder to infer what's going on.\n     - How are latent point encodings $\\mathbf{g}$ obtained? There seem to be two ways to do this:\n       1. Through the encoder (Figure 2 seems to suggest that point encodings are obtained through \"local feature vectors\").\n       2. Through the projection $\\text{proj}_\\mathcal{G}(\\mathbf{x})$ (or maybe $\\hat{\\mathcal{G}}(\\mathbf{x}, \\mathbf{z})$) given by Equation (4).\n     - No loss is given for this model. Equation (5) outlines a generic ELBO loss, but that doesn't apply to this model because, for example, $p(\\mathbf{x} | \\mathbf{z})$ is never defined. Only $p(\\mathbf{x} | \\mathbf{z}, \\mathbf{g})$ is defined. Is $p(\\mathbf{x} | \\mathbf{z}) := p(\\mathbf{x} | \\mathbf{z}, \\text{proj}_\\mathcal{G}(\\mathbf{x}))$?\n     - The structure of the encoder is discussed, but not how it is used to parameterized the approximate posterior $q(\\mathbf{z} | \\mathbf{x})$.\n     - Most of the discussion and notation in Section 3 pertains to individual points $\\mathbf{x}$, not entire point clouds $\\mathbf{X}$. The section never makes clear how the the full point cloud structure $\\mathbf{X}$ is incorporated into the optimization process.\n\n2. A key advantage boosted by the paper is that VF-Net computes the likelihood using the L2 loss rather than the Chamfer distance, which lacks a tractable normalizing constant but is used by competing methods. I question how much of an advantage this really is; it only really affects the coefficient of the reconstruction term in the loss, right? The authors put forward \"probabilistic evaluation\" as an advantage; I assume this means the ability to evaluate true densities/likelihoods/ELBOs. Does that actually have utility in this setting?\n\n3. As the authors point out, the model generates excessively smooth shapes (akin to VAEs in other domains).\n\n## Strengths\n- Outside of Section 3, the paper is clear and well-motivated. I like that the authors explain clearly how shape modelling is useful in computational dentistry: tooth generation, shape completion, diagnostics, etc. \n- By multiple metrics, VF-Net generates more realistic teeth than other shape generation algorithms. \n- VF-Net also demonstrates some interesting domain-specific capabilities, like point-wise uncertainty quantification and simulated shape completion. \n- The representation learning experiments were also cleverly designed, in my opinion. They clearly show that the learned representations are more robust than FoldingNet.\n- It's always nice to see a new dataset released alongside a paper. Hopefully this will open up future research in digital dentistry.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes a VAE-based generative model for dental point clouds. A single point cloud here corresponds to a single tooth. Its contributions are threefold:\n- The authors propose VF-Net, a variational autoencoder for point clouds. To my understanding (which is incomplete - see weaknesses), the unique idea here is to build a VAE using an autoencoding structure inspired by FoldingNet, which contains two types of latent variables: $\\mathbf{z}$, which captures shape structure, and $\\mathbf{g}$, which capture individual points' locations on the shape.\n- The authors also introduce a new dataset, FDI 16, of 7732 tooth meshes.\n- This appears to be the first application of generative shape modelling to digital dentistry. The proposed model is state-of-the-art at modelling teeth.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "## Critical changes\n- Please take some time to clean up the notation, especially in Section 3, to make it more understandable. Take any approach you find to be the most clear, but in my opinion, a bulletproof way to remove ambiguity would be to include all of the following:\n  - All the key variables (eg. $\\mathbf{x}$, $\\mathbf{g}$, $\\mathbf{z}$, $\\mathcal{G}(\\mathbf{x},\\mathbf{z})$), clearly defined, ideally including the set to which they belong.\n  - All the functions introduced in the section, including the encoder, decoder, and \"projection\" function, with clearly defined domains and codomains.\n  - The form of the approximate posterior $q(\\mathbf{z} | \\mathbf{x})$ (along with $p(\\mathbf{x} | \\mathbf{z})$ and $p(\\mathbf{z})$, which are already included)\n   - The loss used to train the model, expressed using the functions and key variables you've defined as per the previous three bullets.\n\n## Minor but necessary changes\n- Figure 1 caption: anatomically -> anatomical\n- The assertion that Chamfer Distances cannot be used in probabilistic modelling is overly strong - \"probabilistic modelling\" can be interpreted quite broadly and not every distribution used in \"probabilistic modelling\" needs to (1) be Euclidean-valued or (2) have a tractable normalizing constant.\n- Figure 2 caption: \"Figure 2\" appears twice\n- Figure 2 mentions that the encoder is \"graph-based,\" but the main text never mentions this anywhere.\n- Page 2: planer -> planar\n- Equation 2, RHS: this should read CHAMF-DIST$(\\mathbf{x}, f((e(\\mathbf{x}), \\mathcal{G})))$ , not CHAMF-DIST$(\\mathbf{x} - f((e(\\mathbf{x}), \\mathcal{G})))$\n- The meaning of the notation $\\mathbf{x}$ seems ambiguous. In equation (1) it represents an individual point in a point cloud, whereas in equation (2) it appears to represent an entire point cloud. Later it seems to represent an individual point again, though equation (5) is again ambiguous.\n- Weird line spacing on page 8 in the \"Simulated Shape Completion\" paragraph\n\n## Non-critical changes\n- The work doesn't contain any comparisons to previous algorithms designed specifically for digital dentistry. It would be really cool if VF-Net were to outperform more bespoke tooth modelling algorithms, such as the work of Qiu et al. (2013), which I think is applicable to the \"Simulated Shape Completion\" paragraph in Section 5. It would be helpful, then, to see Qiu et al. (2013) added to Table 4.\n- \"Probabilistic evaluation\" is a nonstandard term - either more precise terminology or some other added explanation would be helpful. \n- One of the model's limitations is that generated teeth are overly smooth. It would be nice to see some discussion about the downstream effects of this limitation in actual applications; I assume it would make the model less likely to capture the distribution of teeth with highly localized defects, for example.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The submission introduces a new dataset for digital dentistry and a novel method, Variational FoldingNet (VF-Net), as a probabilistic autoencoder for point clouds.  Reviewers originally had some concerns on the novelty of the method and the presentation quality. Post revision and discussion, all reviewers agreed that the revised paper has addressed their concerns and recommended acceptance. The AE agreed with the recommendation.
Decision: Accept (poster)