Point2SSM: Learning Morphological Variations of Anatomies from Point Clouds
Authors: Jadie Adams
Shireen Elhabian
Venue: ICLR 2024 spotlight (2024)
Keywords: Unsupervised learning
global correspondence
point cloud
statsitical shape modeling
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: An unsupervised approach to learn correspondence-based statistical shape models of anatomy directly from point clouds.

Abstract: We present Point2SSM, a novel unsupervised learning approach for constructing correspondence-based statistical shape models (SSMs) directly from raw point clouds. SSM is crucial in clinical research, enabling population-level analysis of morphological variation in bones and organs. Traditional methods of SSM construction have limitations, including the requirement of noise-free surface meshes or binary volumes, reliance on assumptions or templates, and prolonged inference times due to simultaneous optimization of the entire cohort. Point2SSM overcomes these barriers by providing a data-driven solution that infers SSMs directly from raw point clouds, reducing inference burdens and increasing applicability as point clouds are more easily acquired. While deep learning on 3D point clouds has seen success in unsupervised representation learning and shape correspondence, its application to anatomical SSM construction is largely unexplored. We conduct a benchmark of state-of-the-art point cloud deep networks on the SSM task, revealing their limited robustness to clinical challenges such as noisy, sparse, or incomplete input and limited training data. Point2SSM addresses these issues through an attention-based module, providing effective correspondence mappings from learned point features. Our results demonstrate that the proposed method significantly outperforms existing networks in terms of accurate surface sampling and correspondence, better capturing population-level statistics. The source code is provided at https://github.com/jadie1/Point2SSM.

Published as a conference paper at ICLR 2024
POINT2SSM: LEARNING MORPHOLOGICAL VARIATIONS OF ANATOMIES FROM POINT CLOUDS
Jadie Adams & Shireen Y. Elhabian Scientific Computing and Imaging Institute Kalhert School of Computing University of Utah, USA
{jadie, shireen]@sci. utah. edu
ABSTRACT
We present Point2SSM, a novel unsupervised learning approach for constructing correspondence-based statistical shape models (SSMs) directly from point clouds. SSM is crucial in clinical research, enabling population-level analysis morphological variation in bones and organs. Traditional methods of SSM con- struction have limitations, including the requirement of noise-free surface meshes binary volumes, reliance assumptions templates, and prolonged inference times due to simultaneous optimization of the entire cohort. Point2SSM over- comes these barriers by providing a data-driven solution that infers SSMs directly from raw point clouds, reducing inference burdens and increasing applicability as point clouds are more easily acquired. While deep learning on 3D point clouds has seen success in unsupervised representation learning and shape correspondence, its application to anatomical SSM construction is largely unexplored. We con- duct a benchmark of state-of-the-art point cloud deep networks on the SSM task, revealing their limited robustness to clinical challenges such as noisy, sparse, or incomplete imput and limited training data. Point2SSM addresses these issues through an attention based module, providing effective correspondence mappings from learned point features. Our results demonstrate that the proposed method significantly outperforms existing networks in terms of accurate surface sampling and correspondence, better capturing population-level statistics. The source code is provided at https:: //tp://///gthbb/// githubt.hh.u..c com/ ctwm//aadeel/ooont5555..
INTRODUCTION
Statistical shape modeling (SSM) enables quantifying and characterizing the morphological varia- tions in group of shapes. SSM captures the inherent characteristics of shape class or the under- lying parameters that remain when global geometric information (i.e., location and orientation) is factored out (Kendall, 1977). is a powerful tool in medical research, enabling population-Ievel analysis of anatomies such as bones organs, revealing correlations between shape variations and clinical outcomes. Despite being successfully applied
wide range of tasks (including pathol- ogy detection (Atkins et al.) 2017 Bischoff al. 2014 WWang al.] 2015). disease biomarker identification (Bruse et al. 2 Cates et al. [2014 Mendoza et al. [014]] Merle et al.) 2014), sur- gical/treatment planning (Bhalodia et 2020; Carriere et al. 2014; Zhang et 2015 Zachow 2015), and implant designs (Zadpoor & Weinans 2015)). practical limitations have prevented its widespread adoption. The conventional SSM approach entails analyzing a group of complete sur- face representations obtained from 3D medical images (e.g., CT or MRI) in the form of binary volumes or meshes. Correspondence-
SSM establishes sets of geometrically and semantically consistent landmarks correspondence points on the shape surfaces. Various optimization schemes have automated the correspondence point generation process (Davies et al. 2002 Ovsjanikov al.) 2012 Styner al. 2006) to provide dense sets of correspondence points for each shape, includ- ing particle-based shape modeling (PSM) (Cates et al. 2007] 2017). However, such optimization techniques have three significant linitations. Firstly, they require complete shape representations in the form of high-resolution meshes or binary volumes that are free from noise and artifacts, which difficult to acquire, prohibiting many cases. Secondly, the optimization process is

---

Published as a conference paper at ICLR 2024
time cime-consuming and must be performed on the entire cohort simultaneously, which greatly hinders inference when adding a new shape to the SSM. Lastly, these methods utilize metrics such as Gaus- sian entropy (Cates et al., (007) or parametric representations (Ovsjanikov et al. 2012 Styner al. 2006) to define optimization objectives, which may bias or restrict the types of variation captured by the SSM (i.e., via enforcing linearity inheriting the topology of the pre-defined template). Deep learning methods have been proposed to predict SSM from meshes (Bastian et al. [003] Lüdke et al., 2022 Iyer& & Elhabian 2023), addressing some of these limitations. However, such approaches rely on mesh connectivity, enforcing the same input restrictions as optimization based methods.
Point cloud deep learning has recently shown success in tasks such as unsupervised representation learning. shape generation, point cloud up- sammliiin and completion, and point-to-poin matching (Fei et al. 2022] Xiao et al., 2023] Akagic al. 2022). Point clouds can be readily obtained from full-shape segmentations such as meshes when available. They can, however, also be obtained from more lightweight shape acquisition methods, e.g., thresholding clinical images, anatomical surface scanning, and combining 2D contour representations (Timmins et al. 1022 Treleaven & Wells 2007]. Thus, generating SSM directly from point clouds would significantly expand the potential clinical use cases of shape analysis. Recently/AAdams [Adams & Elhabian (023b) demonstrated that existing point cloud mecoder-decoder-beet
completion networks perform reasonably well at SSM genera- tion out-of-the-box. Although such architectures were not designed for correspondence tasks, the bottleneck captures a population-specins shape prior, and the continuous-mapping decoder results in ordered output, providing correspondence as a by-product. However, such methods have not been benchmarked against point correspondence approaches or tested for robustness to noise, partiality, or sparse input (Adams & Elhabian 2023b). A myriad of challenges accompanies applying point cloud deep learning approaches to anatomical SSM most notably, the issue of data scarcity. Deep network training requires large cohort of representative anatomies defined from volumetric medi- cal images, which is difficult to acquire, especially if modeling an uncommon disease or pathology. In addition, point cloud shape representations obtained from medical images may suffer from vari- ous issues, such noise from the acquisition process, missing regions outside the scanner field of view, or sparse point clouds due to low image resolution (Razzak et al.) 2018).
In this paper, we introduce Point2SSM, an unsupervised deep learning framework for learning correspondence- based SSM of anatomy directly from point clouds. Point2SSM overcomes the lim- itations of existing optimization-based SSM methods and point cloud networks by providing data- driven solution that operates on unordered point clouds and infers SSMs that capture population- level statistics with good surface sampling. Point2SSM removes biases imposed by opti- mization assumptions and significantly relaxes the input shape requirement. Moreover, a trained Point2SSM provides a fast and efficient way to predict SSM from new unseen point cloud without re-optimization. We benchmark exist- ing ofate-of-the-art (SOTA) point cloud networks against the proposed method for the SSM ap- plication. This benchmark reveals that although these methods achieve some success on this new task, they have limited ability address the challenges that arise in clinical scenarios. We more robust to lim-
Model
Point Cloud Input
Scalable Fast Inference
Training Cohort
Small
Sparse Input
Partial Input
Noisy Input
X
x
AE
V
X
X
CPAE
V
X
x
X
ISR
X
Y
x
DPC
x
x
Point25 SM
Figure 1: Comparison of particle- based mod-
show that Point2SSM ited training budget as well as to sparse, noisy, and incomplete input than existing point meth- ods and achieves similar statistical compactness to an optimization based SSM method. Our con- tributions can be summarized as follows:
eling (PSM) (Cates et al.] 2017). point autoen- coders (AE) (Achiioptas et al. 20118) 8), canonical point autoencoder (CPAE) (Cheng et al. 2021). Chen et al. (2020) (ISR), deep point correspon- denCe(DPPC) (Lang et al. 2021). and Point2SSM.
(1) We introduce Point2SSM, a novel point cloud approach for anatomical SSM that removes the limitations of optimization- based SSM generation techniques without hindering statistical accuracy.
(2) We provide the first benchmark of SOTA point cloud networks on the anatomical SSM task.
(3) We demonstrate Point2SSM outperforms existing methods in terms of surface sampling and correspondence accuracy and is more robust to limited data and sparse, noisy, or incomplete input.
2

---

Published as a conference paper at ICLR 2024
2
RELATED WORK
2.1
OPTIMIZATION-BASEE SSM
Correspondence-baseee based SSM optimization techniques constrain points the shape surfaces and tablish intersubject correspondence via metrics such as entropy (Cates et al. (207 2017] Oguz et al.) 2016) or minimum description length (Davies et al. 2002), or via parametric representations (Ovs- janikov et al. 2012. Styner et al., 2006). These approaches require complete, faultless shape inputs (surface meshes or binary segmentations) and operate on the entire cohort simultaneously, prevent- ing the addition of a new shape without rerunning the optimization process. Convolutional deep learning approaches for predicting SSMs directly from unsegmented images have been proposed to reduce these burdens (Bhalodia et al. 2018 Adams et al. [0200] Adams & Elhabian 2022, 2023a] Bhalodia et al., [021; Ukey & Elhabian 2023 Tóthová et al. 2020). However, such approaches are supervised and thus require traditional optimization scheme for generating training data cohort,
limiting their accuracy potential based on the training set.
2.2 DEEP LEARNING ON POINT CLOUDS
PointNet (Qi et al. 2017a) was the first deep network designed to process point clouds. It employs multi-layer perceptrons (MLPs) and symmetric aggregation functions to learn permutation invariant features. PointNet++ (Qi et al. 2017b) extended this architecture have hierarchical structure for learning multiscale geometric information. Dynamic graph convolution (DGCNN) [Wang et al.) 2019) utilized nearest neighbors to construct point cloud graphs and apply edge con- volution. Transformer-bas
methods (i.e., Poin'Transformer (Zhao et al. 2021) and Point- Bert (Yu al. )022)) have also been developed, which apply self-attention to 3D point cloud processing to learn underlying structure. To date, there is no ubiquitous 3D backbone for point cloud networks, but the aforementioned networks are common (Xiao et al.] 2023). Supervised training of point cloud networks requires large-scale, densely labeled datasets. This annotation burden has inspired the exploration of unsupervised learning of robust feature representations (Xiao et al. 2023). Unsu- pervised tasks include self recnssrecntttitnnnn,, point cloud up-sampling, and completion of partial point clouds. |Achlioptas et al. (2018) proposed the first point cloud autoencoder (AE) and demonstrated the generative power of the learned latent space, which led to the subsequent development of gener- ative architectures (Xiao et al. 2023). Point completion networks have widely adopted an encoder and coarse-to-fine decoder architecture, allowing the network to learn general shape first and then refine it (uuan et al.] 2018 Fei et al.) 2022). Point cloud mecodr--decderr based networks such as these have been shown to perform reasonably well SSM generation (Adams & Elhabian, 2023b). Although these methods were not designed for SSM, the continuous mapping from the learned la- tent space to output space results in decoded ordered point clouds, providing correspondence as a by-product. However, such methods require sufficiently large and representative training dataset and have yet to be compared to point correspondence approaches (Adams & Elhabian, 2023b).
2.3 LEARNING 3D POINT CLOUD DENSE CORRESPONDENCE
While point cloud learning for anatomical SSM is largely unexplored, point networks have been de- veloped to establish shape correspondence. This task has been approached in a supervised manner through point cloud registration (Choy et al. 2019: Chen et al., 2019a| Gojeic al. 2019 Huang
et al. 2017). 7), via unified embeddings of multiple shape representations (Muralikrishnan et al. 2019). and via part labels (Bhatnagar et al.) 2020). Recently, unsupervised methods have been developed that formulate shape correspondence from either a pairwise or class-level, global standpoint. Pair- wise approaches seek to find a point-to-poit
mapping from a source shape to target shape. Mesh- based methods have been established for this task using functional maps (Ovsjanikvv et al.) [212]] Donati et al., 2020 Ginzburg& Raviv 2020 Eisenberger et al. 2020), which require connectivity. Functional maps have been extended to point clouds, using spectral matching to define correspon- dence (Marin et al. 2020). Other approaches extract correspondence by learned deformations to a
predefined template (Deprelle et al. 2019 Cosmo et al. 2016). Recent approaches utilize deep net- works to learn a matching permutation between a source and target point cloud. Zeng et al. (202T) utilize encoder-decoder architecture to regress the shape coordinates for permuted reconstruction, whereas Lang et al. (202T) opt to drop the decoder and use the original point cloud in reconstruction, achieving better performance by leveraging similarity in the learned feature space.
3

---

Published as a conference paper at ICLR 2024
Prior global correspondence approaches have focused discovering class-specific keypoints (a.k.a. ordered stable interest or structure points) from point clouds (Suwajanakorn et al. 2018 Fernandez-
Labrador et al. 2020: Jakab al., 202T). These semantically consistent points typically sparse. More relevant works have sought to define dense keypoints or structure points a task highly related to correspondence-basee hewenndenne--beeddSn SSM. Chen et (2020) developed network that reconstructs point clouds in a consistent way using an input subset and learned features, providing correspondence model with meaningful principal component analysis embedding. [Liu & Liu (2020) leveraged part features learned by branched AE (Chen et al. 2019b) to establish ittaclass correspondence. However, this approach requires additional knowledge of the shape surface compute the occupancy for training the implicit function. [Cheng et al. (2021) developed a self-supervised canonical point autoencoder that utilizes mapping to a canonical primitive (sphere) to establish order across classes of shapes. While these methods establish correspondence, they haven't been tested against anatomical SSM challenges, such as limited training budget or input noise, missingness, or sparsity.
METHODS
3.1 PROPOSED APPROACH: POINT2SSM
Let S denote an unordered point cloud of P points representing anatomical shape: S Si......pp) with S; E RRS. Given a subset of N points in S, the goal of Point2SSM is to predict a set of M correspondence points, denoted C. Point2SSM learns correspondence manner by estimating a set of points C that best reconstructs the full point clouds S. It is comprised of an encoder and a transformer- like attention module, as shown in figure
self-supervised
Input 3D
Point
Attention Module
Attention
Output Correspondence Points
Point Cloud
F Features
Map
DGCNN Encoder
SF. M)
SFA (M M)
SFA (M, M)
Softmax
Transpose
(N M)
(V 3)
(M x 3)
Figure 2: Point2SSM architecture
The encoder learns an L-dimensional feature vector for each point. We select DGCNN (Wang et al., 2019) encoder architecture to learn topological information via edge convolution. Unlike PointNet and other methods that treat each point independently, DGCNN incorporates local neigh- borhood information, enriching the representational power and capturing global semantic character- istics. This formulation is well suited for learning features for anatomical SSM because anatomies share typical characteristics that stem from the underlying mechanisms involved in their formation.
The attention module predicts an attention map from the feature representation via self-attention. The output correspondence points are then computed via matrix multiplication between the atten- tion map and the input point cloud. Similar to cxisting methods (Chen et al.) 2020 Lang et al.) 2021]. each resulting output point is weighted combination of all of the input points. This con- straint, combined with the attention module, increases surface sampling accuracy. The attention module must capture the structural characteristic of the shape from the point features to learn cor- respondence. Hence, we leverage the self-feature augment (SFA) blocks introduced in Wang et al.) 20022) The SFA blocks integrate the information from different point features and establish the spatial relationship among points by introducing oelf-attention. In this manner, SFA captures global information and reveals detailed shape geometry, generating semantically consistent attention maps for all shapes in cohort. Resulting attention maps are visualized in appendix B
The Point2SSM loss function is comprised of two terms one that encourages to reconstruct and one that regularizes C, encouraging better correspondence. Chamfer distance (CD) is used to measure the difference between the point clouds permutation-innarian way:
1 Σ eEC
CD(C, S)
min ||c
s||2 +
min cEC
cli
(1)
[C]
S
SE8
4

---

Published as a conference paper at ICLR 2024
A pairwise mapping error (ME), originally proposed in (Lag et al. 2021 is adapted to provide an output regularization loss term. The ME between two output point clouds C' and C" is defined as:
ME(C',C") M K im (c)
E
dy 11o di'l
where Ne (c() are the K indices of the K-nearest neighbors of point cf in C' (in Euclidean dis- tance). Here t/2 = - weights the loss elements according the proximity of the neighbor points. ME loss encourages the point neighborhoods spondence. ME is computed between all pairs of output in minibatch of size B. Poin22SSM is not sensitive to the choice of B, as demonstrated in appendix
C' to be similar to C promoting corre-
[F]
The Point2SSM loss thus defined as:
B
1 L -- I CD(C S') +a 1=1
(B-18
1
ME(C', MB(CI, (C) ,C')
(3)
(B-
where a is a hyperparameter that controls the effect the regularization. An ablation experiment that demonstrates the impact of the encoder architecture, attention module architecture, and ME loss is provided in appendix A
3.2 COMPARISON MODELS
We benchmark the following SOTA methods on the anatomical SSM tasks and compare their per- formance to that of Point2SSM:
PSM (Cates et al., 2017) denotes particle-based shape modeling, the SOTA optimization- based technique for generating correspondence points from complete shape surface representations GGparaju et al. 2022). We use the mesh-based implementation
PSM available in the open-source toolkit Works (Cates et al. 2017). We include this method to provide context regarding the expected modes of variation compactness of SSM for a given anatomy.
PN-AE (Achlioptas al. 2018 is the autoencoder formulated by Achlioptas al. (2018) with PointNet (Qi et al. (017a) encoder. The combination of bottleneck and MLP decoder results in consistent output ordering of C (Adams & Elhabian 2023b).
DG-AE (Wang ef al., 2019) is a variant PN-AE where the PointNet (Qi et al. 2017 encoder is replaced with DGCNN (Wang al. 2019) encoder. It is included to assist in comparing the AE framework with Point2SSM and DPC.
CPAE (Cheng et al. 2021) is the canonical point autoencoder that maps points to sphere template in the bottleneck and then reconstructs the points an ordered fashion.
ISR (Chnn et al., 2020) denotes the method for learning intrinsic structural representation (ISR)
points proposed by Chen et al. (2020). ISR utilizes Chamfer loss (equation [ a PointNet++ (Qi
al. 2017b) encoder, and an MLP point integration module that maps the features to probability
map. The probability map is multiplied by a learned subset of the input points to provide output.
DPC (Lang al., 2021 is the deep point correspondence model proposed by Lang al. (202
DPC is a pairwise correspondence method that takes the source and target point clouds as input and outputs the source reordered match the target. The architecture comprises DGCNN (Wang al, 2019) encoder and cross- and self cef-ccosstructon modules that utilize latent similarity. To adapt DPC to provide global correspondence, the same target point cloud or reference used for every shape in inference. In this work, the reference is selected as the point cloud with the minimum Chamfer distance to all other point clouds the training set.
3.3 EVALUATION METRICS
An ideal SSM accurately samples the shape surface via uniformly distributed points that are con- strained to lie on the surface. Simultaneously, it captures anatomically relevant mappings between shapes by establishing consistent, invariant points (i.e., correspondences) across diverse populations with varying forms. Consequently, the evaluation of an SSM should encompass both aspects: the accuracy of surface sampling and the efficacy of the extracted shape statistics.
5

---

Published as a conference paper at ICLR 2024
Three metries are used to define the point surface sampling accuracy: CD) Earth movers distance (EMD), and point-to- tonf-to-face distance (P2F). EMD requires both point clouds to have the same number of points; thus, subset of points in S is selected using farthest point sampling (Qi et al. 2017b). In this way, EMD captures whether the predicted correspondence points cover the entire shape. P2F distance is calculated as the distance of each point in C to the closest face of the ground truth mesh indicating how well points are constrained to the surface.
In an ideal SSM, point locations and neighborhoods are preserved across all shapes in the cohort, leading to a meaningful model with compact modes of shape variation. In SSM analysis, corre- spondence points are averaged to generate a representative mean shape. and principal component analysis (PCA) is performed to compute the sigmificant modes of variation. These modes can be used in medical hypothesis testing and visualized by deforming the mean shape along each basis of the linear subspace (Cates et al. 2014). Three statistical metrics are used to evaluate SSM corre- spondence accuracy: compactness, generalization, and specificity (Munsell et al. 2008). compact SSM represents the training data distribution using the minimum number of parameters; thus, com- pactness is quantified as the number PCA modes required to capture 95% of the variation in the correspondence points. Moreover, good SSM should generalize well from training examples to unseen examples. The generalization metric quantifies how well the SSM generalizes from training examples to unseen examples via the reconstruction error (I.2) between held-out correspondence points and those reconstructed via the training SSM. The specificity metric measures the degree to which the SSM generates valid instances of the shape class presented in the training set. It is computed as the average distance between correspondences sampled from the training SSM and the closest existing training correspondences. The equations of these metrics are available in Munsell et al. (2008 and Appendix C Additionally, ME (Equation 2 is included as an SSM metric since it captures how well point neighborhoods correspond across the cohort.
Both categories of metrics must be used to evaluate the accuracy of SSM, as one does not imply the other. For example, a network that yields identical output given any input will perform very well on statistical metrics but poorly on sampling metrics. Conversely, a network that perfectly reconstructs the input in an unordered fashion will succeed at point sampling metries but fail at statistical metrics.
EXPERIMENTS AND ANALYSIS
We utilize three challenging organ mesh datasets of various sample sizes to benchmark the per- formance of Point2SSM and the comparison methods: spleen (Simpson et al. (2019) (40 shapes), pancreas (Simpson et al., 2019) (272 shapes), and left atrium of the heart (1096 shapes). We elect to acquire point clouds from meshes to enable benchmarking against PSM, which requires mesh connectivity. The spleen dataset represents a typical SSM scenario with limited data and consid- erable variation in shape and curvature. The pancreas dataset consists of cancer patients, resulting in increased shape variability due to varying tumor sizes and morphologies. The left atrium is a notoriously difficult shape to model because it exhibits significant variations in volume, appendage size. and pulmonary vein configuration, number, and length. Appendix provides a visualization of the organ cohorts. Note that the nonanatomical shape datasets previously used to benchmark comparison methods are much larger (i.e., tens of thousands).
Meshes are pre-aligned to factor out global geometric information via iterative closest points (Besl & McKay 1992) utilizing the Shape Works (Cates et al. (017)) toolkit. The aligned, unordered mesh vertices serve as ground truth complete point clouds, S. In all experiments, we set N 1024, L= 128, M = 1024, and batch size B- = 8, unless otherwise specified. Point2SSM is not sensitive to batch size (appendix F.. During training, imput point clouds are generated by randomly selecting N points from S each iteration and uniformly scaling them to be between -1 and 1. The datasets are randomly split into a training, validation, and test set using an 80%, 10%, 10% split. Adam opti- mization with constant learning rate of 0.0001 is used, and model training is run until convergence via validation assessment. Specifically, a model is considered converged if the validation CD has not improved in 100 epochs. Models resulting from the epoch with the best validation CD are used in the evaluation. A 4x TITAN GPU was used to train all models. For Point2SSM loss (equation B a is set to 0.1 based on tuning using the validation set. For comparison models, the originally proposed loss is used with the reported tuned hyperparameter values. Appendix(pprovidde all parameters and appendix [ provides comparison of model memory footprint.

---

Published as a conference paper at ICLR 2024
4.1 RESULTS
Figure 3 provides an overview of the results on all datasets. Point2SSM significantly outperforms the existing point methods on the surface sampling metrics. This is further illustrated in figure 4 which provides a point-level visualization of the test example with median P2F distance output by each model. F additionally demonstrates that Point2SSM performs comparably with respect to statistical metrics to PSM and provides the best compactness on the left atrium dataset. Of the point-based methods, Point2SSM achieves the best compactness on datasets with the exception of the spleen DG-AE, which greatly suffers in terms of point sampling metrics. The AE methods aggregate features into global (L X 1) feature in the bottleneck. This restriction enforces shape prior, providing compactness, but it greatly limits model expressivity, hindering accurate surface sampling. The CPAE output reconstructs the point cloud to a degree but does not provide corre- spondence. The resulting CPAE SSM is not compact interpretable, likely because learning canonical mapping is too complex given a small training set. Like Point2SSM ISR and DPC pre- diet output weighted combination of input points, thus providing good performance surface sampling metrics. However, they do not provide SSM that is as compact, generalizable, or specific as Point2SSM. While ISR learns map a learned subset of input points to correspondence points via a point integration model, Point2SSM makes use of the full input information. Additionally, it suffers from its individual treatment of points, whereas Point2SSM incorporates neighborhood in- formation via edge convolution. DPC is comprised of only an encoder and utilizes latent similarity alone to compute an affinity matrix for establishing correspondence. In contrast, Point2SSM learns correspondence in global manner by incorporating an attention module, which allows for captur- ing population shape statistics without bias induced by reliance on a reference shape. Point2SSM combines the strengths of the comparison methods, providing the best overall accuracy.
Model
Point Accuracy Metrics (mm)
SSM Metrics
CD
Avg
EMD
P2F Dist.
Avg.
Compactness Plot
Comp.
Spec.
PN-AE
43.7
3.99
5.59
39.71
Spleen
DG-AE
43.5
4.64
4.11
2.66
4.95
42.58
CPAE
61.3
3.97
3.43
29
35.5
27.6
1101
ISR
17.6
2.86
7.01
4.77
54.11
DPC
10.6
5.26
Point25 SM
3.42
1.53
0.363
4.19
6.31
3.25
PSM
4.06
5.61
6.46
CD
Avg
EMD
P2F Dist
Avg.
Compactness Plot
Comp.
Spec.
Pancreas
PN-AE
22.0
7.02
4.71
13.90
DG-AE
21.0
3.06
2.48
28
1.78
4.19
9.91
CPAE
10.8
2.55
1.93
189
36.0
28.7
1126
ISR
7.39
1.03
1.01
2.42
4.93
9.26
DPC
Point25
2.72
1:42
0.203
24
2.15
4.55
3.36
PSM
16
2.12
3.83
5.40
CD
Avg.
EMD
P2F Dist.
Avg.
Compactness
Comp.
Gen.
pec.
Left Left Atrium
PN-AE
11.2
1.68
Plot
DG-AE
10.6
1
2.25
1.64
23
1.71
3.93
20.98
CPAE
8.00
1.77
1.10
721
23.6
33.9
623.9
ISR
3.75
1.46
0.651
4.75
7.67
DPC
34.73
Point25SM
2.03
1.25
0.221
20
1.90
4.13
3.45
PSM M
33
1.90
4.28
8.85
Figure 3:
Accuracy metrics are reported with best values outlined. Boxplots show the distribution across test sets and averages are reported to the right, Compactness plots show cumulative popula- tion variation captured by PCA modes, larger area under the curve indicates a more compact model.
Figure 5 displays the SSM resulting from Point2SSM on the pancreas dataset. The mean shape is plausible and interpretable, and the individual point locations are geometrically consistent across shapes. The primary modes of variation in the pancreas are semantically similar to those resulting from the PSM method, suggesting they accurately capture population statistics and could be sim- ilarly used in downstream tasks. Evaluation one such task (pancreatic tumor classification) is provided in appendix H demonstrating the superior predictive power of Point2SSM output.

---

Published as a conference paper at ICLR 2024
Median Cases
Spleen
Pancreas
Left Atrium
Point2SSM Pancreas
Example 0 ullpul Correspondecee Points
PN-AE
Mean Shape
Recolored to highlight one point
DG-AE
Recolored to highlight another point
PSM Modes of Variation
Point25SM Modes
Variation
ISR
Mode
Mode 1
Mode
Mode 2
DPC
Mode 3
Mode 3
PoInt25SM
Mode 4
Mode
P2F Distance:
-15
Mean
SD
-D
Mean
SD
126
Figure 4: The test examples with median P2F distance output from each model are shown over ground truth meshes. P2F distance is displayed via a color map.
Figure The pancreas SSM from Point2SSM is displayed. Point color denotes correspondence. Recoloring according to the distance to a selected point provided for further illustration. The first four modes of variation are shown for the PSM and Point2SSM model at +1 standard deviation from the mean. The heatmap and vector arrows display the distance the mean.
Appeedii displlys the first two modes of variation captured by Point2SSM and comparison models on the spleen and left atrium. Point2SSM the only point-based method that provides similar, not more smooth and interpretable, modes to the PSM method. Not only does Point2SSM allow for faster inference than PSM, but also more scalable. While PSM accuracy not largely affected by the shape cohort size, the optimization process is much slower given large cohort. Fitting the ShapeWork PSM model to the large left atrium cohort required running optimization incrementally over four days, whereas, the Point2SSM model required under eight hours to train GPU.
Appendix[[]
4.2 ROBUSTNESS EVALUATION
We perform experiments to demonstrate model robustness using the medium- sized pancreas dataset. These results are plotted in figure 6 In all robustness experiments, the training and testing point clouds corrupted in the same manner. Appendix[Jprovide
visualization of example input.
Robustness against input noise. To analyze the effect of noise on performance, we apply random Gaussian noise input point clouds at various levels: 0.25, 0.5, 1, and 2 mm standard devia- tion. Point2SSM and the comparison models are not significantly impacted by input noise, and Point2SSM achieves the best accuracy at all noise levels. These results indicate that by adding the denoising task in training, Point2SSM can easily be made robust to input noise.
Robustness against partial input. To analyze the impact of input with missing regions, we remove continuous regions at random locations of various sizes (5%, 10%, and 20% of the total points) from the input. The CD and EMD metrics capture how well the output fills in the missing regions to provide full coverage. The autoencoder architectures (Æann DG-AE are the least impacted by partial input, which logical given this is the architecture used point completion networks. Not only does Point2SSM perform similarly or better than all models regarding the distance metrics, but it also preserves compactness better than DPC and ISR with increasingly partial input.
Robustness against sparse input. To test the impact of input density, train the models with input size N 128, 256, 512, 1024, 2048, and 4096, keeping fixed at 128 and output size M fixed at 1024. This experiment benchmarks the network's ability both upsample and provide SSM. The CPAE model and DPC model require = M and are thus excluded. Point2SSM achieves the best overall accuracy; however, performance declines given very sparse input (N 128).
Impact of training sample size. The final experiment benchmarks the effect training size on model performance. We consistently define random subsets of the 216 training point clouds of size
8

---

Published as a conference paper at ICLR 2024
100, 50, 25, 12, and 6. DPC and Poin12SSM perform the best on the distance metries, demonstrating impressive robustness (generalizing to the test set with only six training examples). Compactnesss results are excluded since compactness depends the variation in the training cohort.
The robustness experiments demonstrate that Point2SSM combines the strengths of all existing mod- els. It performs well Æand DG-AE given large missing regions and as well DPC given small training cohort. These comparisons are compiled in figure
CD (mm)
EMD (mm)
P2F (mm)
(mm)
Compactness
Noisy Input
P Partial Input
S parse
Input
Training Size
Fiaar BSAE
DPC
CASE
PaRKZ55M
Figure 6: Robustness experiment results on the pancreas test set. Distance metrics are shown with standard deviation error bands. Compactness is calculated at 95% variability.
4.3 LIMITATIONS AND FUTURE DIRECTIONS
Deterministic deep learning frameworks like Point2SSM have some limitations. They produce over- confident estimates that could potentially misrepresent shapes, especially when dealing with noisy, partial, and sparse inputs. Incorporating uncertainty quantification would enhance the reliability of deploying Point2SSM in sensitive clinical docision-making scenarios. Additionally, Point2SSM re- quires roughly aligned input point clouds and is designed to produce SSM of a single anatomy (i.e.. bone or organ). While Point2SSM can be trained on multiple anatomies (see appendix L. multi- anatomy training does not notably improve accuracy. Broadening the scope of Point2SSM to handle misalignment and leverage multiple anatomies directly in training would increase applicability.
CONCLUSION
We introduced Point2SSM, the first deep learning method designed to produce 3D anatomical SSM directly from point clouds in an unsupervised manner. Point2SSM provides substantial advantages over traditional SSM generation approaches, offering a data-driven solution free of biases stem- ming from reference selection, metrics, or parametric representations used in classical methods. By redueing the input requirement from complete, noise-free shape representations to point clouds, Point2SSM significantly broadens the potential use cases of SSM. Additionally, the scalability and fast inference distinguish Point2SSM from optimization-based SSM generation methods, which are slow given large cohorts and necessitate complete reoptimization to incorporate new shapes. Deep learning approaches such as Point2SSM also enable incremental model updating through sequential or online learning. This adaptability is crucial to real-world clinical scenarios where shape data accumulates over time. Furthermore, our approach enables concurrent SSM prediction and supple- mentary tasks such as up-sampling, completion, or noise removal. Traditional methods cannot be applied when dealing with sparse, partial, noisy observations. In rigorous evaluations, Point2SSM outperforms state-of-the-art point cloud networks in surface sampling and correspondence accuracy. Moreover, it exhibits robustness in challenging clinical modeling situations, deftly managing lim- ited data and handling noisy, incomplete, and sparse shape representations. Ultimately, our proposed methodology enhances the feasibility of SSM generation and broadens its possible applications, po- tentially accelerating its adoption an invaluable tool in clinical research.
9

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work was supported by the National Institutes of Health under grant numbers NIBIB- U24EB029011 and NIAMS-ROAAR07612
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The authors would like to thank the University of Utah Division of Cardiovascular Medicine for providing left atrium MRT seans and segmentations from the Atrial Fibrillation projects.
REFERENCES
Panos Achlioptas, Olga Diamanti, loannis Mitliagkas, and Leonidas Guibas. Learning representa- tions and generative models for 3d point clouds. In International conference on machine learning, pp. 40-49. PMLR, 2018.
Jadie Adams and Shireen Elhabian. From images to probabilistic anatomical shapes: A deep vari- ational bottleneck approach. In Medical Image Computing and Computer Assisted Intervention- MICCAI 2022: 25th International Conference, Singapore, September 18-22, 2022, Proceedings, Purt II. pp. 474-484. Springer, 2022.
Jadie Adams and Shireen Y Elhabian. Fully bayesian vib-deepssm. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp- 346-356. Springer, 2023a.
Jadie Adams and Shireen Y Elhabian. Can point cloud networks learn statistical shape models of anatomies? In International Conference Medical Image Computing and Conputer-Assisted Intervention, pp. 486-496. Springer, 2023b.
Jadie Adams, Riddhish Bhalodia, and Shireen Elhabian. Uncertain-deepssmn From images to proba- bilistic shape models, Shape in Medical Imaging: : International Workshop, ShapeMI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 2020, Proceedings, 12474:57-72, 2020.
Amila Akagic. Senka Krivić, Harun Dizdar, and Jasmin Velagić. Computer vision with 3d point cloud data: Methods, datasets and challenges. In 2022 XXVIII International Conference Infor- mation, Communication and Automation Technologies (ICAT), pp. 1-8. IEEE, 2022.
Penny R Atkins, Shireen Y Elhabian, Praful Agrawal, Michael D Harris, Ross Whitaker, Jeffrey A Weiss, Christopher L Peters, and Andrew E Anderson. Quantitative comparison of cortical bone thickness using correspondence-
shape modeling patients with cam femoroacetabular im- pingement. Journal of Orthopaedic Research, 35(8): 743- 1753, 2017.
Lennart Bastian, Alexander Baumann, Emily Hoppe, Vincent Bürgin, Ha Young Kim, Mahdi Saleh, Benjamin Busam, and Nassir Navab. S3m: scalable statistical shape modeling through unsuper- vised correspondences. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 459-469. Springer, 2023.
Paul J Besl and Neil D McKay. Method for registration of 3-d shapes. In Sensor fusion IV: control paradigms and data structures, volume 1611, pp. 586-606. Spie, 1992.
Riddhish Bhalodia, Shireen Y. Elhabian, Ladislav Kavan, and Ross T. Whitaker. Deepssm: A deep learning framework for statistical shape modeling from raw images. In Shape In Medical Imaging at MICCAI, volume 11167 of Lecture Notes in Computer Science, pp. 244-2 257.. Springer, 2018
Riddhish Bhalodia, Lucas A Dvoracek, Ali M Ayyash, Ladislav Kavan, Ross Whitaker, and Jesse A Goldstein. Quantifying the severity of metopic craniosynostosis: A pilot study application of machine learning in craniofacial surgery. Journal of Craniofacial Surgery, 2020.
Riddhish Bhalodia,
Shireen Elhabian, Jadie Adams, Wenzheng Tao, Ladislav Kavan, and Ross Whitaker. Deepssm: A blueprint for image-to-shape deep learning models. arXiv preprint arXiv:21 152, 2021.
Bharat Lal Bhatnagar, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. Combin- ing implicit function learning and parametric models for 3d human reconstruction. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceed- ings, Part I1 16, pp. 311-329. Springer, 2020.
10

---

Published as a conference paper at ICLR 2024
Jeffrey E Bischoff, Yifei Dai, Casey Goodlett, Brad Davis, and Mare Bandi. Incorporating population-level variability in orthopedic biomechanical analysis: a review. Journal of biome- chanical engineering, 136(2):02104,, 2014.
Jan Bruse, Kristin McLeod, Giovanni Biglino, Hopewell Ntsinjana, Claudio Capelli, Tain- Yen Hsia, Maxime Sermesant, Xavier Pennec, Andrew M Taylor, Silvia Schievano, et al. A statistical shape modelling framework extract 3d shape biomarkers from medical imaging data: assessing arch morphology of repaired coarctation of the aorta. BMC medical imaging, 16:1-19, 2016.
Nicolas Carriere, Pierre Besson, Kathy Dujardin, Alain Duhamel, Luc Defebvre, Christine Del- maire, and David Devos. Apathy in parkinson's disease is associated with nucleus accumbens atrophy: magnetic resonance ímaging shape analysis. Movement disorders, 29(7):897-903, 2014.
Joshua Cates, P P'oomas Fletcher, Martin Styner, Martha Shenton, and Ross Whitaker. Shape mod- eling and analysis with entropy-based particle systems. In IPMI, pp. 333-345. Springer, 2007.
Joshua Cates, Erik Bieging, Alan Morris, Gregory Gardner, Nazem Akoum, Eugene Kholmovski, Nassir Marrouche, Christopher McGann, and Rob MacLeod. Computational shape models characterize shape change of the left atrium in atrial fibrillation. Clinical Medicine Insights: Cardiology, 8:CMC-$15710, 2014.
Joshua Cates, Shireen Elhabian, and Ross Whitaker. Shapeworks: particle-based shape correspon- dence and visualization software. In Statistical shape and deformation analysis, pp. 257-298. Elsevier, 2017.
Mingjia Chen, Qianfang Zou, Changbo Wang, and Ligang Liu. Edgenet: Deep metric learning for 3d shapes. Computer Aided Geometric Design, 72:19-33, 2019a.
Nenglun Chen, Lingjie Liu, Zhiming Cui, Runnan Chen, Duygu Ceylan, Changhe Tu, and Wenping Wang. Unsupervised learning of intrinsic structural representation points. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9121-9130, 2020.
Zhiqin Chen, Kangxue Yin, Matthew Fisher, Siddhartha Chaudhuri, and Hao Zhang, Bae-net: Branched autoencoder for shape co-segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8490-8499 8499, 2019b.
An-Chich Cheng, Xueting Li, Min Sun, Ming- Hssan Yang, and Sifei Liu. Learning 3d dense corre- spondence via canonical point autoencoder. Advances in Neural Information Processing Systems, 34:6608-666 2021.
Christopher Choy, Jaesik Park, and Vladlen Koltun. Fully convolutional geometrie features. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 8958-8966, 2019.
Luca Cosmo, Emanuele Rodola, Jonathan Masci, Andrea Torsello, and Michael M Bronstein. Matching deformable objects in clutter. In 2016 Fourth international conference on 3D vision (3DV), pp. 1-10. IEEE, 2016.
Rhodri Davies, Carole J Twining, Timothy F Cootes, John C Waterton, and Christopher J Taylor. A minimum description length approach to statistical shape modeling. IEEE transactions on medical imaging, 21 21(5):525-533 2002.
Theo Deprelle, Thibault Groueix, Matthew Fisher, Vladimir Kim, Bryan Russell, and Mathieu Aubry. Learning elementary structures for 3d shape generation and matching. Advances in Neural Information Processing Systems, 32, 2019.
Nicolas Donati, Abhishek Sharma, and Maks Ovsjanikov. Deep geometric functional maps: Ro- bust feature learning for shape correspondence. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8592-8601, 2020.
Marvin Eisenberger, Aysim Toker, Laura Leal-Taixé and Daniel Cremers. Deep shells: Unsuper- vised shape correspondence with optimal transport. Advances Neural information processing systems, 33:10491-10501 2020.
11

---

Published as a conference paper at ICLR 2024
Ben Fei, Weidong Yang, Wen-Ming Chen, Zhijun Li, Yikang Li, Tao Ma, Xing Hu, and Lipeng Ma. Comprehensive review of deep leamning-baae based 3d point cloud completion processing and analysis. IEEE Transactions Intelligent Transportation Systems, 2022.
Clara Fernandez-Labrador, Ajad Chhatkuli, Danda Pani Paudel, Jose J Guerrero, Cédric Demon- ceaux, and Luc Van Gool. Unsupervised learning of category-specific symmetric 3d keypoints from point sets. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXV 16, pp. 506-546633 Springer, 2020.
Dvir Ginzburg and Dan Raviv. Cyclic functional mapping: Self-supervised correspondence between non-isometric deformable shapes. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part V 16, pp- 36-52. Springer, 2020.
Zan Gojeic, Caifa Zhou, Jan D Wegner, and Andreas Wieser. The perfect match: 3d point cloud matching with smoothed densities. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 5545-5554, 2019.
Anupama Goparaju, Krithika Iyer, Alexandre Bone, Nan Hu, Heath Henninger, Andrew An- derson, Stanley Durrleman, Matthijs Jacxsens, Alan Morris, Ibolya Csecs, al. Benchmarking off-the-shelf statistical shape modeling tools in clinical applications. Medical Image Analysis, 76: 102271, 2022.
Haibin Huang, Evangelos Kalogerakis, Siddhartha Chaudhuri, Duygu Ceylan, Vladimir Kim, and Ersin Yumer. Learning local shape descriptors from part correspondences with multiview convolutional networks. ACM Transactions on Graphics (TOG), 37:1-14, 2017.
Krithika Iyer and Shireen Y Elhabian. Mesh2ssm: From surface meshes to statistical shape models of anatomy. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 625..65.000 Springer, 2023.
Tomas Jakab, Richard Tucker, Ameesh Makadia, Jiajun Wu, Noah Snavely, and Angjoo Kanazawa. Keypointdeformer: Unsupervised 3d keypoint discovery for shape control. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2021.
David G Kendall. The diffusion of shape. Advances in applied probability, 903)428-430,, 1977.
Itai Lang, Dvir Ginzburg, Shai Avidan, and Dan Raviv. Dpe: Unsupervised deep point correspon- dence via cross and self construction. In 2021 International Conference on 3D Vision (3DV), pp.
1442-1451, IEEE, 2021.
Feng Liu and Xiaoming Liu. Learning implicit functions for topology-varying dense 3d shape correspondence. Advances in Neural Information Processing Systems, 33:4823-4834, 2020.
David Lüdke, Tamaz Amiranashvili, Felix Ambellan, Ivan Ezhov, Bjoern H Menze, and Stefan Za- chow. Landmark- -free statistical shape modeling via neural flow deformations. In Medical Image Computing and Computer Assisted InterventionMMCCAA/ 2022: 25th International Conference, Singapore, September 18-22, 2022, Proceedings, Part II, pp. 453-463. Springer, 2022.
Riccardo Marin, Maric-Julie Rakotosaona, Simone Melzi, and Maks Ovsjanikov. Correspondence learning via invaart-invariant embedding. Advances in Neural Information Processing Systems, 33:1608-1620, 2020.
Carlos S Mendoza, Nabile Safdar, Kazunori Okada, Emmaric Myers, Gary F Rogers, and Mar- ius George Linguraru. Personalized assessment of craniosynostosis via statistical shape modeling. Medical Image Analysis, (8(4):6356646 2014.
C Merle, W Waldstein, IS Gregory, SR Goodyear, RM Aspden, PR Aldinger, DW Murray, and HS Gill. How many different types of femora are there primary hip osteoarthritis? an active shape modeling study. Journal of Orthopaedic Research, 32(3):413 2014.
Brent C Munsell, Pahal Dalal, and Song Wang. Evaluating shape correspondence for statistical shape analysis: A benchmark study. IEEE Transactions on Pattern Analysis and Machine Intelligence, (o(1)):023-2039) 2008
12

---

Published as a conference paper at ICLR 2024
Sanjeev Muralikrishnan, Vladimir G. Kim, Matthew Fisher, and Siddhartha Chaudhuri. Shape uni- code: A unified shape representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Ieek Oguz, Josh Cates, Manasi Datar, Beatriz Paniagua, Thomas Fletcher, Clement Vachet, Mar- tin Styner, and Ross Whitaker. Entropy-based particle correspondence for shape populations. International journal of computer assisted radiology and surgery, 11:1221-1232, 2016.
Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Func- tional maps: flexible representation of maps between shapes. ACM Transactions on Graphics (ToG), 31(4):1-11, 2012.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652 660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical fea- ture learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017b.
Muhammad Imran Razzak, Saeeda Naz, and Ahmad Zaib. Deep learning for medical image pro- cessing: Overview, challenges and the future. Classification in BioApps: Automation of Decision Making. pp. 323-350, 2018.
Anjany Sekuboyina, Malek E Husseini, Amirhossein Bayat, Maximilian Löffler, Hans Liebl, Hong- wei Li, Giles Tetteh, Jan Kukačka, Christian Payer, Darko Štern, al. Verse: vertebrae labelling and segmentation benchmark for multi-detector ct images. Medical image analysis, 73:102166, 2021.
Amber L Simpson, Michela Antonelli, Spyridon Bakas, Michel Bilello, Keyvan Farahani, Bram Van Ginneken, Annette Kopp-Schneider, Bennett Landman, Geert Litjens, Bjoern Menze, al. A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv: wwt.:/0220999 902.09063, 2019.
Martin Styner, Ipek Oguz, Shun Xu, Christian Brechbühler, Dimitrios Pantazis, James J Levitt, Martha E Shenton, and Guido Gerig. Framework for the statistical shape analysis brain struc- tures using apharm-pdm. The insight journal, pp. 242, 2006.
Supasorn Suwajanakorn, Noah Snavely, Jonathan J Tompson, and Mohammad Norouzi. Discov- ery of latent 3d keypoints via end-to-end geometric reasoning. Advances in neural information processing systems, 31, 2018.
Lucas Timmins, Habib Samady, and John N Oshinski. Effect of regional analysis methods on assessing the association between wall shear stress and coronary artery disease progression in the clinical setting. In Biomechanics of Coronary Atherosclerotic Plaque, pp. 203-223. Elsevier, 2021.
Katarína Tóthová, Sarah Parisot, Matthew Lee, Esther Puyól-Antón, Andrew King, Mare Polle- feys, and Ender Konukoglu. Probabilistic 3d surface reconstruction from sparse mri information. In Medical Image Computing and Computer Assisted Intervention-MICCA. 2020: 23rd Interna- tional Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part 23, pp. 813-823, Springer, 2020.
Philip Treleaven and Jonathan Wells. 3d body scanning and healtheare applications. Computer, 40 (7):28 -1228,3,,75 2007. doi: 10. 00.1109/M..2007.222.
Janmesh Ukey and Shireen Elhabian. Localization-awar deep learning framework for statistical shape modeling directly from images. In Medical Imaging with Deep Learning, 2023.
Jun Wang. Ying Cui. Dongyan Guo, Junxia Li, Qingshan Liu, and Chunhua Shen. Pointattn: You only need attention for point cloud completion. arXiv preprint arXiv:2203.0842 08485, 2022.
13

---

Published as a conference paper at ICLR 2024
Li Wang, Yi Ren, Yaozong Gao, Zhen Tang, Ken- Ken-Chung Chen, Jianfu Li, Steve GF Shen, Jin Yan, Philip KM Lee, Ben Chow, et al. Estimating patient-specific and anatomically correct reference model for cramiomaxilllffciall deformity via sparse representation. Medical physies, 42(10))5809- 5816, 2015.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph enn for learning on point clouds. Acm Transactions On Graphics (tog), 38(5): 1-12, 2019.
Aoran Xiao, Jiaxing Huang, Dayan Guan, Xiaoqin Zhang, Shijian Lu, and Ling Shao. Unsupervised point cloud representation learning with deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19313-19322, 2022.
Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, and Martial Hebert. Pen: Point completion network. In 2018 international conference on 3D vision (3DV), pp. 728 737. IEEE, 2018.
Stefan Zachow. Computational planning in facial surgery. Facial Plastic Surgery, 31(05):446446. 2015.
Amir A Zadpoor and Harrie Weinans. Patient-specific bone modeling and analysis: the role of integration and automation in clinical adoption. Journal of biomechanics, 4855)7700-760, 2015.
Yiming Zeng. Yue Qian, Zhiyu Zhu, Junhui Hou, Hui Yuan, and Ying He. Cormet3d: Unsupervised end-to-end learning of dense correspondence for 3d point clouds. In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 6052-6061, 2021.
Kun Zhang, Wee Kheng Leow, and Yuan Cheng. Performance analysis of active shape reconstruc- tion of fractured, incomplete skulls. In Computer Analysis of Images and Patterns: 16th Interna- tional Conference, CAIP 2015, Valletta, Malta, September 2-4, 2015 Proceedings, Part I 16, pp. 312-324. Springer, 2015.
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 16259-16268, 2021.
A POINT2SSM ABLATION EXPERIMENT
We perform ablation experiment on the pancreas dataset to analyze the impact of each aspect of the Point2SSM model. To illustrate the impact of the DGCNN (Wang et al.) 2019) encoder, design PointSSM variant with the PointNet (Qi et al. 20 decoder used in PN-AE The Point2SSM attention module (denoted ATTN) is comprised of attention-based SFAWang et al. (202) blocks. To analyze this impact, design variation of Point2SSM where the attention module is replaced with an MLP-based architecture. For the MLP-based architecture, we elect to use the Point Integration Module proposed in ISR with three ID convolution blocks. Finally, to we test without by setting = O.
test the impact of the ME loss in equation
The results of this ablation shown in table T. with the full proposed Point2SSM in the final row. The DGCNN-encoder and ATTN attention module both provide surface sampling and correspon- dence accuracy improvements. The addition of the ME loss (a 0.1) improves the correspondence accuracy without reducing the surface sampling accuracy.
14

---

Published as a conference paper at ICLR 2024
Table I: Poin(2SSM ablation experiment on the pancreas dataset. Average set values are
ported for point accuracy distance metrics in mm. SSM metrics are calculated at 95% variability.
Pancreas Point2SSM Ablation
Point Accuracy Metrics (mm)
SSM Metrics
Encoder
Attention Module
CD
EMD
P2F
Comp.
Gen.
Spec.
PointNet
MLP
7.35
1.78
0.833
52
2.96
4.48
PointNet
ATTN
3.00
1.44
0.306
27
2.24
4.67
DGCNN
MLP
3.40
1,46
0.378
31
2.2
4.52
DGCNN
ATTN
2.87
1,43
0.283
26
2.32
4.80
DGCNN
ATTN
0.1
2.72
1.42
0.283
2.15
4.55
B ATTENTION MAP VISUALIZATION
Figure illustrates the attention map weights learned by the Point2SSM attention module the pancreas dataset. Output correspondence points are a weighted combination of the input points, where the learned attention map defines the weights. Figure F highlighhs two output correspondence points across shapes. The attention maps show the weights on the input points (via color map) that generated the selected output point. The maps illustrate which input points were most important for defining given output point. Note the maps highlight similar anatomical regions across samples for given output corresponding point.
Selected Point 1
Selected Point 2
Output Point
Attention Map
Output Point
Attention Map'
Shape
Weight
0.25
Shape
0.01
0.002
0.0001
Shape
Figure 7: Two output points (highlighted in red boxes) across three pancreas shapes are shown with the corresponding weights (blue log scale color map) on the input point clouds.
C SSM EVALALUATION METRICS
As is standard (Munsell al. 2008), utilize three statistical metrics to evaluate correspondence accuracy: compactness, generalization, and specificity. A compact SSM represents the training data distribution using the minimum number of parameters. We quantify compactness as the number of PCA modes required to capture 95% of the total variation in the output training cohort correspon- dence points, where fewer modes indicate more compact model. The compactness plots figure show the cumulative explained variance as the number of modes increases to provide a full picture.
A good SSM should generalize well from training examples unseen examples and be able to describe any valid instance of the shape class. Given an unseen test cohort of correspondence point sets, denoted Drest, the generalization metric defined as:
Gen.
Σ CED
||C (C-CCl CI2
iDees
where è is the point reconstructed via the training cohort PCA eigenvalues and vectors that preserve 95% variability. A smaller average reconstruction enror indicates that the SSM generalizes well to the unseen test set.
15

---

Published as a conference paper at ICLR 2024
Finally, effective SSM is specific, generating only valid instances of the shape class presented in the training set. This metric is quantified by generating a set new sample correspondence points, denoted D sample: from the SSM generated the training cohort, denoted Drain The specificity metric quantified as:
Spec.
min CEDiraiv:
||C'
C
(5)
D sample
C'eD Dample
The average distance between correspondence points sampled from the training SSM and the closest existing training correspondence points provides the specificity metric. small distance suggests the samples match the training distribution well, indicating the SSM is specific.
D MODEL HYPERPARAMETERS
Model hyperparameters are provided in the following tables and the configuration files proved with the code. Table the hyperparameters that consistent across all models. Note in the sparsity robustness experiments, the value of varies. Tables 3 and display the hyperparameters specific the CPAE and DPC models. These values match those originally tuned/reported. Finally, table [5] displays the hyperparameters specific to our Point2SSM model. The number of neighbors used the ME loss is set 10, as it for DPC. The value of a is determined via tuning based on the validation set performance.
Table Hyperparameters shared by all models.
Shared Hyperparameters
Parameter
Description
Value
Number input points
1024
L
Number of per-point features output by encoder
128
M
Number of output points
1024
B
Batch size
8
LR
Learning rate
0.0001
ES
Early stopping patience (epochs)
100
31
Adam optimization first coefficient
0.9
Adam optimization second coefficients
0.999
Table 3: Hyperparameters specific to the CPAE model.
Additional CPAE Hyper Parameters
Parameter
Description
Value
AMSE
MSE loss weight
1000
ACD
CD loss weight
10
AEMD
EMD loss weight
Cross-construction loss weight
10
Aunfold
Unfolding loss weight
10
Adaptive loss epoch
100
Table 4: Hyperparameters specific to the DPC model.
Additional DPC Hyperparameters
Parameter
Description
Value
K
Neighborhood size for loss calculation
10
Mapping loss neighbor sensitivity
8
Cross-construction loss weight loss weight Mapping loss weight
Ase
10
Am
16

---

Published as a conference paper at ICLR 2024
Table Hyperparamelers specific our Point2SSM model.
Additional Point2SSM Hyperparameters
Parameter Description
Value
Neighborhood size for ME loss
10
ME loss weight
0.1
E
MODEL MEMORY COMPARISON
Table 6 shows comparison of the memory footprint of each model.
Table 6: Model memory footprint comparison. Size is reported in MB.
Model
Total params
Forward/backward pass size
Params size
Total size
PN-AE
3,832,576
11.04
14.62
25.68
DG-AE
4,702,336
609.54
17.94
627.5
CPAE
156,652
19.58
0.60
56.18
ISR
1,962,208
4.00
7.49
11.51
DPC
962,176
609.50
3.67
613.21
Point2SSM
22,098,560
633.69
84.3
718.01
Point2SSM has many more parameters than the other models due to the attention module. When this module is replaced with simple MLP, is done in the ablation experiment in appendix A the number parameters reduced from 22,098,560 to 2,707,328. It is worth noting that this reduced model still provides more accurate SSM than the comparison models, can be seen by comparing the metrics reported in table row and figure 3 This suggests that the performance improvement provided by Point2SSM not simply result increased model capacity.
F BATCH SIZE ABLATION EXPERIMENT
The Point2SSM loss (equation 3 contains ME equation 2). which is computed pairwise within batch. To analyze the impact of batch size on model performance, we perform an ablation on the pancreas dataset with batch sizes: 2, 4, 6, 8, 10, and 12. The results provided in table 7 The batch size does not have a large impact on accuracy.
Table Effect batch size on Point2SSM performance on the pancreas dataset. across test set are reported with best values marked in bold.
Average values
Point Accuracy Metrics (mm) J
SSM Metrics $
Batch Size
CD
EMD
P2F
Comp.
Gen.
Spec.
ME
2
2.74
1.42
0.279
24
2.12
4.41
3.38
4
2.61
1.41
0.256
23
2.22
4.61
3.22
6
2.64
1.41
0.270
23
2.20
4.65
3.25
8
2.72
1.42
0.283
24
2.15
4.55
3.36
10
2.67
1.42
0.269
24
2.14
4.66
3.32
12
2.71
1.42
0.280
24
2.14
4.59
3.34
G SHAPE DATASET VISUALIZATION
Figure S displays example shapes from each organ dataset (spleen, pancreas, and left atrium) from multiple views, illustrating the large amount of variation in these shape cohorts.
17

---

Published as a conference paper at ICLR 2024
Organ
Top View
Anterior View
Spleen
Pancreas
Left Atrium
Figure 8: Example shapes are shown from each the datasets from the top and anterior views.
H DOWNSTREAM EVALUATION: PANCREAS TUMOR SIZE CLASSIFICATION
As SSM is useful in many downstream applications, have performed an additional evaluation of such a task. The pancreas dataset (Simpson al. 2019) comprised cancerous cases, where each pancreas shape has tumorous masses of various sizes. As downstream classification task, analyze whether the tumor region of a given pancreas larger or smaller than 20% of the pancreas size. In particular, PCA embeddings then serve as input into a random forest ensemble comprised of decision tree classifiers. The classifier is fit and evaluated using the original panereas train/test split. The accuracy of the classifier built from each SSM is reported in table 8 The PSM and the comparison methods all performed similarly in this task, but Point2SSM provided a slight improvement. This evaluation demonstrates that Point2SSM provides effective, usable SSM estimation.
compute PCA embeddings of the SSM generated from each method. These
Table 8: Pancreas Tumor Mass Classification Accuracy Percentage of test cases correctly classi- fied having tumor regions are larger than 20% of the total pancreas size. Random forest ensem- ble classifiers were trained in the PCA embedding of the correspondence points provided by each method.
Model
PSM
PN-AE
DG-AE
CPAE
ISR
DPC
Point2SSM 89.28%
Accuracy
85.71%
85.71%
85.71%
82.29%
85.71%
85.71%
1 SPLEEN AND LEFT ATRIUM MODES OF VARIATION
Figure 9 displays the first two modes of variation captured by Point25SM and all comparison mod- els the spleen and left atrium datasets. The CPAE results excluded from this visualization because the resulting SSM is uninterpretable. The meshes in figure are constructed using the out- put correspondence points. Mesh artifacts implausible morphologies miscorrespondence.. Point2SSM is the only point-based method that provides similar, not more smooth and interpretable, modes the PSM method.
indication of output
18

---

Published as a conference paper at ICLR 2024
PN-AE
DG-AE
S Splenn
Modee
Mode: 2 Mode
15.5
Mode 2
7.8
-1.50
+1.0
-155
+1.D
-1
41 SD
0.0
DPC
SM
Mode 2 Mode
Modee
Mode:
-150
Mean
+D
-15D
Mean
+1.50
155
Mcan
+1 +150
Left
PN-AE
DG-AE
ISR
Moodl Mozaa
Made
(odde Mode
Atrium
13.0
45
-1
Mean
41
-155D
Mean
+1.00
-15
Mean
+1.5D
DPC
PS M
Point255M
0.0
Model
2 Mode
Mode 2 Mode
445
Mode 2
Mean
Mean
Mean
+1.D
Figure 9: Primary and secondary modes of variation captured by spleen and left atrium SSMs output by each model. Point2SSM provides the smoothest and most plausible mean and modes of variation.
J ROBUST EVALUATION INPUT EXAMPLES
Figure 10
shows examples of input point clouds from the robustness experiments described in sec-
tion
Noisy
Input
= 0
= 0.25
a = 0.5
a =
Partial Input
0% Missing
5% Missing
10% Missing
20% Missing
Sparse
Input
128
256
N
1204
2048
4096
Figure 10: Robustness evaluation input point cloud examples are displayed over the semi-transparent ground truth shape.
K VERTEBRAE EXPERIMENT
In addition to the three organ experiments presented in section 4
provide experiment complex bone: the fourth lumbar (L4) vertebrae. This dataset is selected from the publicly available labeled and segmented data for human vertebrae the vertebrae segmentation challenge (VerSe)
19

---

Published as a conference paper at ICLR 2024
Sekuboyina al. 2021). The L4 vertebrae data comprises 160 complete bone shapes. A visualiza- tion of subset of the bones is provided in figure II.
Top View
Posterior View
L4 Vertebrae
Figure 11: Example shapes are shown from each of the datasets from the top and anterior views.
The experiment was conducted using same steps and parameters as described section The results are summarized in table While PN-AE and DG-AE provide the most compact SSM's. s. they do not perform well on point accuracy metrics. CPAE performs poorly on all metrics, likely because the vertebrae topology is difficult map to sphere. ISR and DPC perform slightly better than the autoencoder models in terms of point accuracy, but suffer on the SSM metrics. Point2SSM performs best overall and provides interpretable shape model with good correspondence and smooth modes of variation, as shown in figure 12
Table 9: Average results the 1.4 vertebrae test set. Best values marked in bold.
Point Accuracy Metrics (mm)
SSM Metrics
Model
CD
EMD
P2F
Comp.
Gen.
Spec.
ME
PN-AE
5.22
1.82
0.897
24
0.606
1.70
2.03
DG-AE
4.90
1.79
0.836
22
0.615
1.67
2.09
CPAE
8.13
1.84
0.946
117
27.4
24.7
531.42
ISR
4.66
1.69
0.714
54
1.24
2.59
3.02
DPC
4.82
1.63
0.573
94
1.83
3.04
4.71
Point2SSM
2.61
1.48
0.304
34
0.879
2.06
1.98
L4 Vertebrae
Point255M Modes of Variation
Example Output Correspondence Points
Mode
Recolored to highlight one point
Mode 2
Ma
Mode 3
Recolored to highlight another point
Shape
The first three modes of variation shown for the Point2SSM model at +1 standard deviation from the mean. The heatmap and vector arrows display the distance to the mean. Right: Example predictions are shown where the point color denotes correspondence. Recoloring according the distance to selected point is provided for further illustration.
Figure 12:
The L4 vertebrae SSM from Point2SSM is displayed. Left:
20

---

Published as a conference paper at ICLR 2024
L MULTI-ANATOMY TRAINING
Table TO compares the results of training and testing Point2SSM on single anatomy versus multi- anatomy training. The results similar, indicating multi-anatomy training neither helps nor hinders Point2SSM. Future work could leverage anatomy labels to increase the training signal and provide improved performance given multiple anatomies.
Table 10: Multi-anatoona Experiment
Anatomy
Point Accuracy Metries (mm)
SSM Metrics
Train
Test
CD
EMD
P2F
Comp.
Gen.
Spec.
ME
Spleen
Spleen
3.42
1.53
0,363
8
4.19
6.11
3.25
Spleen, Pancreas, Left Atrium
Spleen
3.33
1.54
0.310
8
4.42
6.13
2.94
Pancreas
Pancreas
2.72
1.42
0.283
24
2.15
4.55
3.36
Spleen, Pancreas, Left Atrium
Pancreas
3.10
1.44
0.352
24
2.11
4.21
3.45
Left Atrium
Left Atrium
2.03
1.25
0.221
20
1.90
4.13
3.45
Spleen, Pancreas, Left Atrium
Left Atrium
2.15
1.25
0.258
20
1.86
4.09
3.47
21


Reviews:
[{"review_id": "OEvZiz2QGb", "reviewer": "Reviewer_x8Fq", "summary": "This paper proposes a method, Point2SSM, for the task of correspondence-based statistical shape modeling(SSM), This refers to mapping a shape to a set of $M$ keypoints (correspondence points (CPs) in the paper). The CPs must accuractely summarize a shape and correspond to consistent anatomical features such that they can be used, e.g., with PCA, to compute population level statistics.\n\nPoint2SSM maps $N$ input points to $M$ CPs. It consists of a dynamic graph CNN (DGCNN) encoder which outputs a feature vector for each input point. Then, self-feature augment (SFA) attention is applied on the feature vectors. Softmax is then applied to produce an $M x N$ attention matrix that maps the input points to the CPs. Point2SSM is trained in an unsupervised fashion. A Chamfer loss is used to ensure the CPs accurately summarizes the input shape and an adapted pairwise mapping error (ME) is used to ensure the CPs correspond to consistent features. While each components of Point2SSM does not seem novel, their combination is.\n\nThe authors compare Point2SSM with several recent and relevant methods on three datasets of spleens, pancreases, and left atriums. They show that Point2SSM summarizes shapes significantly better than the compared methods, while being competitive regarding the consistency of the CPs. They perform additonal experiments showing improved robustsness and include ablation experiments in the supplementary that shows the benefit of each component.", "questions": "- Point2SSM has many more parameters than other models (App. E, Table 6.). Is comparing it with the much smaller models fair? Would \"upscaling\" the other models narrow the performance gap? I would like the authors to clarify why/why not.\n- Top of page 5: please specify what \"Euclidean neighbors\" mean. As I can tell, it is the K-nearest neighbors - if so I suggest using that. While Lang et al., 2021 also used \"Euclidean neighborhood\", I think  it would make the text in this paper easier to read with another term.\n- The text in the figures is very small. Also, blue text on blue background in Fig. 5 is hard to read. Improving this would make the results much easier to interpret.\n- For the robustness experiments, it would be nice to illustrate how the pertubations look like. This can just be in the supplementary.\n- For the sparse input experiment, how are the points sampled? Random, farthest point samping or other?\n- When illustrating modes of variation, I find it more clear to deform the mean shape instead of showing deformation vectors. However, this is minor for me and the authors may choose to change it or not.\n- Section 3.1, first line: second use of \"unordered\" seems redundant.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well written. It is well motivated and it is clear how the method works and how the experiments are set up.\n- The results show clear improvements over the compared approaches.\n- The experiments are comprehensive enough to demonstrate the benefits of Point2SSM.\n- The use of the attention module provides clear benefits - this is a nice finding.", "weaknesses": "- The method seems to have significant overlap with the ISR method from Chen et al., 2020. Specifically, it seems to me that replacing PointNet++ with DGCNN + SFA in ISR would almost be the same method. This is not an issue for me, but: if true, I think the inspiration/similarities should be mentioned and Chen et al., 2020 cited in sec. 3.1. If false, I would like the authors to clarify the differences.\n- It is stated that enforcing the CPs to be a convex combination of the input points \"increases surface sampling accuracy\". It is not fully clear to me why this must be true. From the attention maps in App. B, Fig. 7, right, CPs could be placed very inacurately even if they are convex combinations of the attended points. I would like clarification on this point. If it is an empircal observation, I would like it clearly stated (ideally, with an ablation experiment but this is not crucial to me).\n- It is not clear to me why Point2SSM has such a benefit when it comes to adding new samples. How would this work? Would one \"simply\" continue training from previous weights, or does the architecture enable something smarter?\n- Minor point: At the end of sec. 2.3, it is stated that: \"These methods establish correspondence but are prone to overfitting given a limited training budget and are not robust to noise, missingness, and sparsity in the input point cloud.\" I find this to be a strong statement. Could it be softened or further justified?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MSg5cceQc1", "reviewer": "Reviewer_PV7h", "summary": "The author introduces Point2SSM, a novel unsupervised learning approach designed for constructing statistical shape models (SSMs) from point clouds, a method crucial for analyzing morphological variation in anatomy in clinical research. Traditional SSM creation methods have limitations, such as the need for noise-free meshes, reliance on assumptions, and long inference times. Point2SSM addresses these issues by inferring SSMs from raw point clouds, a process that is more efficient and widely applicable due to the ease of acquiring point cloud data. Despite recent advances in deep learning for 3D point clouds, the construction of SSMs for anatomy faces challenges such as noisy or incomplete input and limited training data. Point2SSM overcomes these by using an attention-based module for correspondence mappings, demonstrating superior performance in surface sampling and correspondence, thus enhancing the capture of population-level anatomical statistics.", "questions": "How could Point2SSM be modified to model multiple anatomical structures concurrently and understand their interrelationships?\nWhat approaches could be integrated into Point2SSM to allow for automated pre-alignment or to make the model resilient to unaligned data from diverse orientations?\nAre there methods to incorporate uncertainty quantification into Point2SSM's outputs to aid risk assessment in clinical decision-making?\nHow could Point2SSM be optimized for improved performance with sparse data, ensuring more accurate and robust statistical shape models?\nIn what ways could Point2SSM be developed to handle both misaligned data and the modeling of multiple anatomies within a single framework?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Point2SSM introduces a novel unsupervised learning framework that directly constructs SSMs from point clouds, a significant advancement over traditional methods. Additionally, this method addresses the limitations of classical SSM generation methods, such as the need for noise-free surface meshes and reliance on predefined templates. By incorporating local information, it utilizes a DGCNN encoder to learn features by incorporating local neighborhood information, capturing global semantic characteristics of anatomical shapes. Moreover, the attention module in Point2SSM predicts correspondence maps in a self-supervised manner, eliminating the need for labeled data. The results show robustness against noisy, partial, and sparse inputs, which is critical for clinical modeling.", "weaknesses": "Single Anatomy Modeling: Point2SSM's current limitation to model only one anatomical structure at a time could be expanded upon. How might the method be adapted to accommodate the study of multiple anatomical structures simultaneously, and could this expansion improve the model's understanding of the interrelationships between various anatomical features?\n\nPre-alignment Requirement: The necessity for pre-aligned input point clouds could restrict Point2SSM's use in scenarios with non-aligned data sources. What methods could be integrated into Point2SSM to automate the alignment process, or to make the model robust to variations in data orientation due to patient movement or diverse scanning protocols?\n\nUncertainty Quantification: The lack of uncertainty quantification in the model's outputs is a notable concern for clinical decision-making. Are there potential strategies for integrating uncertainty estimation into Point2SSM to make it more suitable for clinical environments where risk assessment and error margins are vital?\n\nPerformance with Sparse Data: While Point2SSM has demonstrated promising results with limited datasets, its performance with sparse data is not fully explored. Investigating this could provide insights into how the model might be optimized to better capture the variability and complexity of anatomical shapes, leading to more accurate and robust SSMs.\n\nModel Expansion for Multiple Anatomies: Enhancing Point2SSM to handle misaligned inputs and model multiple anatomies could greatly increase its clinical relevance. How could the model be developed to not only process various anatomical structures within a single framework but also address the variations and interconnections between them?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fef5wMZeiI", "reviewer": "Reviewer_gxDE", "summary": "This paper addresses a clinically relevant problem of statistical shape modeling from point cloud data of different anatomical structures. Specifically, the authors learn to reconstruct the point cloud and learn correspondence in a self-supervised manner. The results show improved SSM on three anatomical structures.", "questions": "see weaknesses", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Strength:\n\n+ The paper is well-motivated and has done a thorough literature review.\n\n+ The results improve SSM performance over previous methods.\n\n+ The method shows robustness against noisy, partial, and sparse input.", "weaknesses": "Major comments:\n- Although the paper addresses a clinically relevant application, the technical novelty of this paper is limited since it is based on existing architecture and loss without any new insights or analysis. This remains a major weakness and is hard to address during rebuttal.\n- Since the corresponding promoting loss acts between the minibatch samples, what is the effect of minibatch size on the model performance?\n- Only three anatomical structures have been studied. The inclusion of more anatomical structures is needed to support the generalizability of the method.\n- The methods describe the objective as reconstructing a sparse set of point reconstruction but, in practice, uses N=M=1024, which raises concern about the initial objective.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ldqSxzcgBW", "reviewer": "Reviewer_ARFH", "summary": "The paper proposes a new model for learning statistical shape models (SSM) from unregistered 3D point cloud data for modelling anatomical shapes. The main contributions lie within the newly proposed architecture that enables attention based correspondences and later on more compact models, even from noisy or partial inputs.", "questions": "I wonder whether DPC is the optimal choice for a pairwise/groupwise correspondence model, as it does not consider instance optimisation which was shown to improve point registration nor does it create an unbiased mean for the shape model (but simply selects one reference cloud).", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper addresses an interesting and relevant problem that has not received enough attention in research yet. Shape models are very important for interpretable segmentation outputs.\nThe motivation and differentiation to related work is clear and the numerical and visual evaluation convincing. The visualisations are well done.\nThe paper discusses different mapping algorithms in detail and experimentally validates the positive aspects of the proposed global correspondence finding.", "weaknesses": "While the used metrics cover a good range of aspects from (Chamfer) surface distance to compactness and noise robustness, as well as a shape classification task in the supplement, other point distances e.g. density or rasterisation-based could have been explored.\nThe (compared) models have very many different aspects making a pin-pointing of the decisive difference harder. E.g. why did the authors not compare always the same geometric convolutional backbone / encoder for both correspondence and auto encoder-based solutions? \nThe method requires the computation losses of all pair-wise alignments within one mini-batch, which is computationally expensive but also raises the question why the authors did not directly compare to groupwise point registration as another good baseline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)