$\texttt{NAISR}$: A 3D Neural Additive Model for Interpretable Shape Representation
Authors: Yining Jiao
Carlton Jude ZDANSKI
Julia S Kimbell
Andrew Prince
Cameron P Worden
Samuel Kirse
Christopher Rutter
Benjamin Shields
William Alexander Dunn
Jisan Mahmud
Marc Niethammer
Venue: ICLR 2024 spotlight (2024)
Keywords: Shape Modeling
Medical Shape Analysis
Interpretable Representation
AI4Science
Primary area: visualization or interpretation of learned representations
TL;DR: We propose $\texttt{NAISR}$, the first shape representation method to investigate an atlas-based representation of 3D shapes in a deformable, disentangleable, transferable and evolvable way.

Abstract: Deep implicit functions (DIFs) have emerged as a powerful paradigm for many computer vision tasks such as 3D shape reconstruction, generation, registration, completion, editing, and understanding. However, given a set of 3D shapes with associated covariates there is at present no shape representation method which allows to precisely represent the shapes while capturing the individual dependencies on each covariate. Such a method would be of high utility to researchers to discover knowledge hidden in a population of shapes. For scientific shape discovery purpose, we propose a 3D Neural Additive Model for Interpretable Shape Representation ($\texttt{NAISR}$) which describes individual shapes by deforming a shape atlas in accordance to the effect of disentangled covariates. Our approach captures shape population trends and allows for patient-specific predictions through shape transfer. $\texttt{NAISR}$ is the first approach to combine the benefits of deep implicit shape representations with an atlas deforming according to specified covariates. We evaluate $\texttt{NAISR}$ with respect to shape reconstruction, shape disentanglement, shape evolution, and shape transfer on three datasets, i.e. 1) $\textit{Starman}$, a simulated 2D shape dataset; 2) ADNI hippocampus 3D shape dataset; 3) pediatric airway 3D shape dataset. Our experiments demonstrate that $\texttt{NAISR}$ achieves competitive shape reconstruction performance while retaining interpretability. Our code is available at https://github.com/uncbiag/NAISR.

Published as a conference paper at ICLR 2024
NAISR: A 3D NEURAL ADDITIVE MODEL FOR INTERPRETABLE SHAPE REPRESENTATION
Yining Jiao', Carlton Zdanski', Julia Kimbell', Andrew Prince', Cameron Worden', Samuel Kirse", Christopher Rutter', Benjamin Shields', William Dunn', Jisan Mahmud', Mare Niethammer': for the Alzheimer's Disease Neuroimaging Initiative* 'University of North Carolina at Chapel Hill, Wake Forest School of Medicine, 'hhe Ohio State University College of Medicine
ABSTRACT
Deep implicit functions (DIFs) have emerged as a powerful paradigm for many computer vision tasks such as 3D shape reconstruction, generation, registration, completion, editing, and understanding. However, given a set 3D shapes with as- sociated covariates there is at present no shape representation method which allows to precisely represent the shapes while capturing the individual dependencies on each covariate. Such method would be of high utility to researchers to discover knowledge hidden in a population of shapes. For scientific shape discovery, we pro- pose 3D Neural Additive Model for Interpretable Shape Representation (NAISR) which describes individual shapes by deforming a shape atlas in accordance to the effect of disentangled covariates. Our approach captures shape population trends and allows for patient-specific predictions through shape transfer. NA NAISRI is the first approach to combine the benefits of deep implicit shape representations with an atlas deforming according to specified covariates. We evaluate NAISH SR with respect shape reconstruction, shape disentanglement, shape evolution, and shape transfer on three datasets: 1) Starman, a simulated 2D shape dataset; 2) the ADNI hippocampus 3D shape dataset; and 3) a pediatric airway 3D shape dataset. Our experiments demonstrate that NAISR achieves excellent shape reconstruction performance while retaining interpretability. Our code is publicly available.
INTRODUCTION
Deep implicit functions (DIFs) have emerged as efficient representations of 3D shapes (Park et al., 2019; Novello et al., 2022; Mescheder et al., 2019; Yang et al., 2021), deformation fields (Wolterink et al., 2022b), images and videos (Sitzmann et al., 2020), graphs, and manifolds (Grattarola & Vandergheynst, 2022). For example, DeepSDF (Park et al., 2019) represents shapes as the level set of a signed distance field (SDF) with neural network. In this way, 3D shapes are compactly represented as continuous and differentiable functions with infinite resolution. In addition to representations of geometry such as voxel grids (Wu et al., 2016; 2015; 2018), point clouds (Achlioptas et al., 2018; Yang et al., 2018; Zamorski et al., 2020) and meshes (Groueix al., 2018; Wen et al., 2019; Zhu et al., 2019), DIFs have emerged as a powerful paradigm for many computer vision tasks. DIFs used for 3D shape reconstruction (Park et al., 2019; Mescheder al., 2019; Sitzmann al., 2020), generation (Gao et al., 2022), registration (Deng et al., 2021; Zheng et al., 2021; Sun et al., 2022a; Wolterink et al., 2022b), completion (Park et al., 2019), editing (Yang et al., 2022a) and understanding (Palafox et al., 2022).
Limited attention has been paid shape analysis with DIFs. Specifically, given set of 3D shapes with set of covariates attributed to each shape, a shape representation method is still desired which
Corresponding author.
1 Data used in preparation of this article were obtained from Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni. . 1 ioni. use. edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http:/ / a .ucccede wp- app-conten//pp oads hwww/hhwwwhhheeoooeooeeeeeeeeeee wtwpoootttoppppppeeeeeeeeeeepppe wwwaapppppyypa. //ADN _Acknow edgement 1ri pdd

---

Published as a conference paper at ICLR 2024
can precisely represent shapes and capture dependencies among set of shapes. There is currently no shape representation method that can quantitatively capture how covariates geometrically and temporally affect a population of 3D shapes; neither average nor for an individual. However, capturing such effects desirable as it is often difficult and sometimes impossible to control covariates (such as age, sex, and weight) when collecting data. Further, understanding the effect of such covariates is frequently goal of medical studies. Therefore, it is critical to be able to disentangle covariate shape effects on the individual and the population-level to better understand and describe shape populations. Our approach is grounded in the estimation of a shape atlas (i.e., template shape) whose deformation allows to capture covariate effects and model shape differences. Taking the airway as example, a desired atlas representation should be able to answer the following questions:
Given an atlas shape, how can one accurately represent shapes and their dependencies?
Given the shape of an airway, how can one disentangle covariate effects from each other?
Given covariate, e.g., age, how does an airway atlas change based on this covariate?
Given a random shape, how will the airway develop after a period of time?
To answer these questions, we propose a Neural Additive Interpretable Shape Representation (NAISR), interpretable way of modeling shapes associated with covariates via shape atlas. Table compares NAISR to existing shape representations with respect to the following properties:
Implicit relates to how shape is described. Implicit representations are desirable as they naturally adapt to different resolutions of a shape and also allow shape completion (i.e., reconstructing complete shape from partial shape, which is common in medical scenarios) with no additional effort.
Deformable captures if a shape representation results in point correspondences between shapes, e.g., via a displacement field. Specifically, we care about point correspondences between the target shapes and the atlas shape. A deformable shape representation helps to relate different shapes.
Disentangleable indicates whether a shape representation can disentangle individual covari- ate effects for shape. These covariate-specd speciffc effects can then be composed obtain the overall displacement of an atlas/template shape.
Evolvable denotes whether a shape representation can evolve the shape based on changes of a covariate, capturing the influence of individual covariates on the shape. An evolvable atlas statistically captures how each covariate influences the shape population, e.g., scientific discovery scenarios.
Transferable indicates whether shape changes can be transferred to a given shape. E.g., one might want to edit an airway based simulated surgery and predict how such surgical change manifests later in life.
Interpretable indicates a shape representation that simultaneously deformable, disen- tangleable, evolvable, and transferable. Such an interpretable model is applicable to tasks ranging from the estimation of disease progression to assessing the effects of normal aging weight gain on shape.
NAISR is the first implicit shape representation method to investigate an atsad-based representation of 3D shapes in deformable, disentangleable, transferable and evolvable way. To demonstrate the generalizability of NAISR, we test NAISR on a simulated dataset and two realistic medical datasets :: 1) Starman, simulated 2D shape dataset (Bône al., 2020); 2) the ADNI hippocampus 3D shape dataset (Jack Jr et al., 2008); and 3) a pediatric airway 3D shape dataset. We quantitatively demonstrate superior performance over the baselines.
2 RELATED WORK
We introduce the three most related research directions here. A more comprehensive discussion of related work available in Section S. of the supplementary material.
"Medical shape datasets are our primary choice because quantitative shape analysis is common need for scientific discovery for such datasets.
2

---

Published as a conference paper at ICLR 2024
Methad
Impleit
Deformable
Disentangicable
Evelvable
Trunsterable
Intesretuble
Coodinonal Tomplle (Diica al., 20199
X
K
X
3DAunilow (Wen 2022)
x
x
x
X
DeepSty (Park at sl,, 20199
x
x
A-DDF
2021)
x
x
X
DIT (Zaeng et al, 2021), DUP (Leng et tl, 2021), NDF (Sun et al., 2022g)
x
x
X
NASAM (Wes al,, 2022)
x
X
Ours (88.28)
7
7
/
Table I:
Comparison of shape representations based on the desirable properties discussed in Section 1. A / 'ndicates that model has property; a X indicates that does not. Only NAT SR has all the desired properties.
Deep Implicit Functions. Compared with geometry representations such as voxel grids (Wu et al., 2016; 2015; 2018), point clouds (Achlioptas al., 2018; Yang al., 2018: 88; Zamorski et al., 2020) and meshes (Groueix et al., 2018; Wen et al., 2019; Zhu et al., 2019). DIFs are able to capture highly detailed and complex 3D shapes using a relatively small amount of data (Park et al., 2019; Mu et al., 2021; Zheng et al., 2021; Sun et al., 2022a; Deng et al., 2021). They are based on classical ideas of level set representations (Sethian, 1999; Osher & Fedkiw, 2005); however, whereas these classical level set methods represent the level set function function, c.g., by a neural network. Hence, DIFs are not reliant on meshes, grids, or a discrete set of points. This allows them to efficiently represent aatural-looking surfaces (Gropp et al., 2020; Sitzmann et al., 2020; Niemeyer et al., 2019). Further, DIFs can be trained on a diverse range of data (e.g., shapes and images), making them more versatile than other shape representation methods. This makes them useful in applications ranging from computer graphics, to virtual reality, and roboties (Gao et al., 2022; Yang al., 2022a; Phongthawee al., 2022; Shen et al., 2021). We therefore formulate NAISR based on DIFs.
grid, DIFs parameterize it continuous
Neural Deformable Models Neural Deformable Models (NDMs) establish point correspondences with DIFs. In computer graphics, there has been increasing interest in NDMs to animate scenes (Liu et al. 2022; Bao et al., 2023; Zheng et al., 2023), objects (Lei & Daniilidis, 2022; Bao et al., 2023; Zhang et al., 2023), and digital humans (Park et al., 2021b; Zhang & Chen, 2022; Niemeyer et al., 2019). Establishing point correspondences is also important to help experts to detect, understand, diagnose, and track diseases. NDMs have shown to be effective in exploring point correspondences for medical images (Han et al., 2023b; Tian et al., 2023; Wolterink et al., 2022a; Zou et al., 2023) and shapes (Sun et al., 2022a; Yang et al., 2022b) Among the NDMs for shape representations, ImplicitAtlas (Yang et al., 2022b). DIF-Net (Deng et al., 2021), DIT (Zheng al., 2021), and NDF (Sun et al., 2022a) capture point correspondences between target and template shapes within NDMs. Currently, no continuous deformable shape representation which models the effects of covariates exists. NAISR provides a model with such capabilities.
Explainable Artificial Intelligence. The goal of eXplainable Artificial Intelligence (XAI) is to provide unmaannddddttt explanations for decisions and actions of an Al model. Various flavors of XAI exist, including counterfactual inference (Berrevoets et al., 2021; Moraffah et al., 2020; Thiagarajan et al., 2020; Chen et al., 2022), attention maps (Zhou et al., 2016; Jung & Oh, 2021; Woo et al., 20, feature importance (Arik & Pfister, 2021; Ribeiro et al., 2016; Agarwal et al., 2020), and instance retrieval (Crabbe et al., 2021). NAISR inspired by neural additive models (NAMs) (Agarwal et al., 2020) which in turn are inspired by generalized additive models (GAMs) (Hastie, 2017). NAMs are based on a linear combination of neural networks each attending to a single input feature, thereby allowing for interpretability. NAISR extends this concept to interpretable 3D shape representations. This is significantly more involved as, unlike for NAMs and GAMs, we no longer dealing with scalar values, but with 3D shapes. NAISR will provide interpretable results by capturing spatial deformations with respect to an estimated atlas shape such that individial covariate effects can be distinguished.
3 METHOD
This section disccuses our NAISR model and how we obtain the desired model properties of Section 1.
3.1 PROBLEM DESCRIPTION
Consider a set of shapes S } where each shape S has associated vector of covariates c= <=[c [C,...., C....,, CN] (e.g., age, weight, sex). Suppose {S*} are well pre-aligned and centered (c.g.. based on Iterative Closest Point (ICP) registration (Arun et al., 1987) landmark registration; see
3

---

Published as a conference paper at ICLR 2024
Training
p....... C.,
covnrisboeiee dleglacemen::
SDF & R
Diglicement Appragation
MLP
P
p
temuplate shape
ovecrall displacement
oovariate-wise disglacemmntt
3
Testing
Inference Generation
Disentanglement
Transfer
evolution of
estimate & tand
2 years Inter?
shapes
4 genemae shapes
transfer to d DG
Figure I: Neural Additive Implicit Shape Representation. During training we learn the template T and the multi-layer perceptrons (MLPs) [9:} predicting the covariate-wise displacement fields {d.). The displacement fields are added to obtain the overall displacement field d defined in the target space: provides the displacement between the deformed template shape T and the target shape. Specifically the template shape queried not its original coordinates p. but at p = p + d effectively spatially deforming the template. At test time we evaluate the trained MIPPs for shape reconstruction, evolution, disentanglement, and shape transfer.
Section S.2 for details). Our goal is to obtain representation which describes the entire set S while accounting for the covariates. Our approach estimates a template shape, (the shape atlas), which approximates S. Specifically, T is deformed based on set of displacement fields {D* }. such that the individual shapes (8*) are approximated well by the transformed template.
A displacement field D* describes how a shape is related to the template shape T. The factors that cause this displacement may be directly observed or not. For example, observed factors may be covariates such as subject age, weight, or sex; i.c., C₁ for subject k. Factors that not directly observed may be due to individual shape variation, unknown or missing covariates, or variations due differences in data acquisition observed factors by a latent code z. The covariates c and the latent code z then contribute jointly to the displacement DD with respect to the template shape T.
data preprocessing errors. We capture these directly
Inspired by neural additive (Agarwal et al., 2020) and generalized additive (Hastie, 2017) models, assume the overall displacement field is the individual covariates: D = E,D, Here, D, is the displacement field controlled by the i-th covariate. ca This results by construction in an overall displacement D that is disentangled into several sub-displacement fields (D:))
of displacement fields that are controlled by
3.2 MODEL. FORMULATION
Figure gives an overview of NAISR. To obtain a continuous atlas representation, we use DIFs to represent both the template 'T and the displacement field D. The template shape 'I is represented 0} captures the desired function
by signed distance function, where the zero level set {p E RR3 (T()) template shape. A displacement D; of particular point p can also be represented
d; d (p, C., z) E RR. We use S SIREN (Sitzmann et al., 2020) as the backbone for T(-) and {-)..)) Considering that the not directly observed factors might influence the geometry of all covariate- specific networks, we make the latent code, visible to all subnetworks {f((-)). }. We normalize the covariates so that they are centered at zero. To assure that zero covariate value results in a zero displacement we parameterize the displacement fields as d; (i G, 2) where
9i (p. C, z) = f. c, z) fp 0,)).
(I)
The sub-displacement fields are added to obtain the overall displacement field
N
= g(p. = Σ gi cr 2)
(2)
4

---

Published as a conference paper at ICLR 2024
We then deform the template shape T to obtain implicit representation of a target shape
= T(p Σ (p,)),,,2))
where p is point in the source shape space, e.g.. point on the surface of shape S; and p represents a point in the template shape space, e.g., point on the surface the template shape T. To investigate simply extract the zero level set of
how an individual covariate c; affects a shape
$ (P, Ci 2) = ((p - d.) = (p + 9: Ci, 2)).
3.3 TRAINING
Losses. All our losses ultimately summed over all shapes of the training population with the appropriate covariates C₁ and shape code 3₁ for each shape S For ease of notation, we describe them for individual shapes. For each shape, we sample on-surface and off- points. On-surface points have zero signed distance values and normal vectors extracted from the gold standard mesh. Off-surface points have non-zero signed distance values but no normal vectors. Our reconstruction losses follow Sitzmann et al. (2020); Novello et al. (2022). For points on the surface, the losses are
Lon c, 2) {S
+
1 (p. c, c2 +Ag (1 (V L c, n()))) dp,
Cpicaier
(5)
where n(p) is the normal vector at p and! (-) denotes cosine similarity. For off-surface points, we use (P))
/
Cott ( c, c, = Jnys
A4 |.(pp,.., ( + |I xp. c. 2)) - dp.
(6)
Coinass
Ceneeall
where stge (P) is the signed distance value at p corresponding to given target shape. Similar to (Park et al., 2019; Mu et al., 2021) we penalize the squared L norm of the latent code
Liat (z) A o² 1|
(7)
As result, our overall loss (for given shape) is
C( z) = (z) + Lon(4 (0,,c, z) + (o c, c.
(8)
regularizer
for roconstrution
where the parameters of + and are trainable.
3.4 TESTING
As shown in Figure our proposed NAISR is designed for shape reconstruction, shape disentangle- ment, shape evolution, and shape transfer.
Shape Reconstruction and Generation. As illustrated in the inference section in Figure 1, shape Stgt is given and the goal to recover its corresponding latent code z and the covariates c. To estimate these quantities, the network parameters stay fixed and we optimize over the covariates and the latent code which are both randomly initialized (Park et al., 2019; Mu et al., 2021). Specifically, solve the optimization problem
è, غ = arg min L(D, z).
In clinical scenarios, the covariates might be known (e.g., recorded age weight at imaging time). In this case, we only infer the latent code z by the optimization
z arg min L(Ø, c.z).
(10)
new patient shape with different covariates can be generated by extracting the zero level set of T (p, Cnow: : 2).
In medical imaging, there is typically no groundrruth. We use gold standard to indicate shapes based off of manual or automatic segmentations, which our targets for shape reconstruction.
5

---

Published as a conference paper at ICLR 2024
Shape Evolution. Shape evolution along covariates {4} is desirable in shape analysis to ob- tain knowledge of disease progression or population trends in the shape population S. For time-varying covariate (c,, .... c. ... ).), we obtain the corresponding shape evolution by (#.(p, c,, 2), 2).,.. (, c. 2). (p, of 2)). If some covariates are correlated (c.g., age and weight), we can first obtain a reasonable evolution of the covariates (c" ...., c'...., and the corresponding shape evolution as (p( c", 2).... @(p, c', ž).... $(p, c .2)).. By setting 2 to one can observe how a certain covariate influences the shape population on average.
Shape Disentanglement. As shown in the disentanglement section in Figure 1, the displacement for a particular covariate o displaces point p in the source space to p + d, in the template space for given or inferred ž and c.. We obtain the corresponding signed distance field
8; P. Cr, 2) (p d:) T(p + ((
(11)
As a result, the zero level sets of {+)( represent shapes warped by the sub-displacement fields controlled by G.
Shape Transfer. We use the following clinical scenario to introduce the shape transfer task. Suppose a doctor has simulated surgery on an airway shape with the goal of previewing treatment effects on the shape after a period of time. This question can be answered by our shape transfer approach. Specifically, as shown in the transfer section in Figure 1, after obtaining the inferred latent code 2 and covariates è from reconstruction, one can transfer the shape from the current covariates è to new covariates è + Ac with ( è + Ac, 2). As a result, the transferred shape is a prediction of the outcome of the simulated surgery; is the zero level set of (pp è + Ac, 2). In more general scenarios, the covariates are unavailable but it is possible to infer them from the measured shapes themselves (see Eqs. 9-10). Therefore, in shape transfer we are not only evolving shape, but may also first estimate the initial state to be evolved.
EXPERIMENTS
We evaluate NA SRR in terms of shape reconstruction, shape disentanglement, shape evolution, and shape transfer three datasets: 1) Starman, a simulated 21) shape dataset used in (Bône et al., 2020); 2) the ADNI hippocampus 3D shape dataset (Petersen et al., 2010); and 3) pediatric airway 3D shape dataset. Starman serves as the simplest and ideal scenario where sufficient noise-free data for training and evaluating the model is available. While the airway and hippocampus datasets allow for testing on real-world problems of scientific shape analysis, which motivates NAISR.
We can quantitatively evaluate NAISR for shape reconstruction and shape transfer because our dataset contains longitudinal observations. For shape evolution and shape disentanglement, we provide visualizations of shape extrapolations in covariate space to demonstrate that our method can learn reasonable representation the deformations governed by the covariates.
Implementation details and ablation studies are available in Section S.3.1 and Section S.3.2 in the supplementary material.
4.1 DATASET AND EXPERIMENTAL PROTOCOL
Starman Dataset. This is a synthetic 2D shape dataset obtained from predefined model as illustrated in Section 1 without additional noise. As shown in Fig. S.4. each starman shape is synthesized by imposing random deformation representing individual-level variation to the template starman shape. This followed by covariate-controle
deformation to the individualized starman shape, representing different poses produced by starman. 5041 shapes from 1000 starmen are synthesized as the training set; 4966 shapes from another 1000 starmen are synthesized as a testing set.
ADNI Hippocampus Dataset. These hippocampus shapes were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. The dataset consists of 1632 hippocampus segmentations from magnetic resonance (MR) images. We an 80%-20% train-test split by patient (instead of shapes); i.c., a given patient cannot simultancously be the train and the test set, and therefore information can leak between these two sets. As result, the training consists of 1297

---

Published as a conference paper at ICLR 2024
Searman
ADNI Dipporampps
Pediutric Airoay
CD
UMD'1
HD
CD
EMDD
I101
CD7
EMD
I1D
M
M
M
M
M4
M
M
M4
M
DespSUF
0.117
0,113
1.541
1.337
6.302
6.271
01157
0.140
2198
2.035
9,702
3,276
0.077
02002
1401
1.256
10.700
9,236
A SDF
0.173
0.092
2,010
1.668
5.805
6.949
1,294
1,162
7.156
7.667
25
25.092
25.938
2,647
1.178
10.302
9.088
47.172
57,835
A D.SDP10 mc)
0,089
0.043
1.298
1.361
5.388
4.964
0.311
0.294
3,136
3.099
13:852
13,005
0.552
0:225
4.890
2.890
30,863
21.965
DIT
0.281
0.181
2.727
2.497
10.295
3,442
0.15%
0.142
2,0%6
2.054
9.465
9.123
0.096
0.699
1.44
1.262
11.324
10.228
NDP
1.06
0.736
5.364
4.821
21.098
19,205
0.253
0.213
2,699
2,580
11.328
10,947
0.236
0.117
2.174
1.237
14.950
12.516
Ours
0.111
0.072
1.709
1.515
7.951
7.341
0.174
0.153
2,258
2.191
10.019
9.521
0.867
0,839
1.233
1.132
10.333
5.484
O2
0.089
0.036
1,376
1,356
5.851
4,666
0.136
0.116
1.847
1,800
8.586
8.153
0.384
0.844
1.345
1,190
10.719
5,577
Table 2: Quantitative evaluation of shape reconstruction. CD Chamfer distance. EMD Earth mover's distance. HD = Hausdorff distance. All metries are multiplied by 10 Bold red values indicate the best scores across all methods, Bold black values indicate the 2nd best scores all methods. Ours means the covariates were inferred during testing (sce Equation 9). Ours(c) means the covariates are used as input during inference (scc Equation 10), y and M indicate the mean and median of the mcasurements on the testing shapes respectively. NAISR SR performs well for all three reconstruction tasks while allowing for interpretability.
shapes while the testing set contains 335 shapes. Each shape is associated with 4 covariates (age, sex, AD, education length). AD is a binary variable indicating whether a person has Alzheimer disease.
Pediatric Airway Dataset. This dataset contains 357 upper airway shapes to evaluate our method. These shapes are obtained from automatic airway segmentations of computed tomography (CT) images of children with radiographically normal airway. These 357 airway shape are from 263 patients, 34 of whom have longitudinal observations and 229 of whom have only been observed once. We use a 80%-20% train-test split by patient (instead of shapes). Each shape has 3 covariates (age, weight, sex).
More details, including demographic information, visualizations, and preprocessing steps of the datasets are available in Section S.2 in the supplementary material.
Comparison Methods. For shape reconstruction of unseen shapes, we compare our method on the test set with DeepSDF (Park et al., 2019), A-SDF (Mu et al., 2021), DIT (Zheng et al., 2021), and NDF (Sun et al., 2022a). For shape transfer, we compare our method with A-SDF (Mu et al., 2021) because other comparison methods cannot model covariates as summarized Table
Metrics. For evaluation, all target shapes and reconstrueted meshes are normalized to a unit sphere (i.e., centered at the origin and uniformly scaled so that the furthest point is at unit distance from the origin) to assure that large shapes and small shapes contribute equally error measurements. We use the Hausdorff distance, Chamfer distance, and earth mover's distance to evaluate the performance of our shape reconstructions. For shape transfer, considering that a perfectly consistent image acquisition process is impossible for different observations (e.g., head positioning might slightly vary across timepoints for the airway data), we visualize the transferred shapes and evaluate based on the difference between the volumes of the reconstructed shapes and the target shapes the hippocampus and airway dataset.
4.2
SHAPE RECONSTRUCTION
The goal of our shape reconstruction experiment is to demonstrate that NAISR can provide compet- itive reconstruction performance while providing interpretabilityy Table 2 shows the quantitative evaluations and demonstrates the excellent reconstruction performance of NAISR. Figure visualizes reconstructed shapes. We observe that implicit shape representations can complete missing shape parts which can benefit further shape analysis. A-SDF (Mu et al., 2021) works well for representing Starman shapes but cannot reconstruct real 3D medical shapes successfully. The reason might be the time span of our longitudinal data for each patient is far shorter than the time span across the entire dataset, mixing individual differences and covariate-controllin differences. A-SDF may fail to disentangle such mixed effects (from individuals and covariates). but instead memorizes training shapes by their covariates c. In contrast, the additive architecture of NA SSR prevents the model from memorizing training shapes through covariates More discussions are available in Section S.3.3 of the supplementary material.
4.3 SHAPE TRANSFER
Table 4 shows an airway shape transfer example for a cancer patient who was scanned 11 times. We observe that our method can produce complete transferred shapes that correspond well with the measured shapes. Table 3 shows quantitative results for the volume differences between our transferred shapes and the gold standard shapes. Our method performs best the real datasets
7

---

Published as a conference paper at ICLR 2024
Gold Stankee
DempSDF
A-SDF
DIT
NDF
Our
Ours (c)
Figure 2: Visualizations of airway and hippocampus reconstruction with different methods. The red and blue circles show the structure in the black circle from two different views. Hippocampus shapes are plotted with two 180"-ffipped views. NAISR produce detailed and accurate reconstructions as well as impute missing airway parts. More visualizations are available in Section S.3.3 of the supplementary material.
Starman FMD
ADNI Hippocampus
Pediatric Airway VD
CD 1
HD
VD. .
W.C.
M
M
M
M
A-SDF
X
0 0.009
0.008
0.036
0.034
0.518
0.488
81.074
82,918
A-SDF
0.009
0.009
0.036
0,035
0.215
0,177
41.460
40,956
Ours
0.003
0.002
0.025
0.023
0.094
0.077
0,086
0.063
12,820
8.837
Ours
0.009
0.002
0,031
0.025
0.116
0.083
0.089
0.071
11.227
9.653
Table 3: Quantitative evaluation of shape transfer. Statistics of Volume Difference (VD, cm between transferred
shapes and gold standard shapes. w.C. abbreviates with covariates. / in w.C. indicates the inference follows Equation 10. A in w.C. indicates the inference follows Equation 9. Red bold scores indicate the best performance across all methods and bold scores indicate the 2nd best. NAISR results significantly improved volume estimates for real medical shapes.
while A-SDF (the only other model supporting shape transfer) works slightly better on the synthetic Starman dataset. Our results demonstrate that NAISK SR is capable of transferring shapes to other timepoints S from given initial state S
#time 0
2
3
4
5
7
8
9
10
(S))
time
0
2
ago
154
155
137
139
163
164
167
170
194
227
233
weight
55.2
60.9
64.3
65.25
59.25
59.2
65.3
68
77.1
75.6
75.6
M
M
M
M
M
M
M
M
M
M
M
p-vol
92.50
93.59
94.64
95.45
96.33
96.69
98.40
99.72
109.47
118.41
118.76
m-vol
86.33
82.66
63.23
90.65
98.
84.35
94.14
127.45
98.81
100.17
113.84
Table 4: Airway shape transfer for patient. Blue: gold standard shapes; red: transferred shapes with NAISR.
The table below lists the covariates (age/month, weight/kg, sex) for the shapes above. P-vol(predicted volume) the volume (cm²) of the transferred shape by NAISR covariates following Equation M-vol (measured volume) is the volume (cm²) of the shapes based on the actual imaging. NAISR can capture the trend of growing volume with age and weight while producing clear, complete, and topology-consisten shapes. Note that measured volumes may differ depending on the CT imaging field of view. More visualizations are available Section S.3.4 the supplementary material.
4.4 SHAPE DISENTANGLEMENT AND EVOLUTION
Figure 3 shows that NAISR ISR is able to extrapolate reasonable shape changes when varying covariates. These results illustrate the capabilities of NAISR for shape disentanglement and to capture shape evolutions. A-SDF and NAISR both produce high-quality Starman shapes because the longitudinal data is sufficient for the model to discover the covariate space. However, for real data, only NA ISR can produce realistic 3D hippocampi and airways reflecting the covariates' influences on template shapes. Note that when evolving shapes along single covariate the deformations from other covariates are naturally set to 0 by our model construction (see Section 3.2). As a result, the shapes in the yellow and green boxes in Figure 3 represent the disentangled shape evolutions along different covariates respectively. The shapes in the other grid positions can be extrapolated using ((-). By inspecting the volume changes in the covariate space in Figure 3, we observe that age is more important for airway volume than weight, and Alzheimer disease influences hippocampal volume. These observations from generated shapes consistent with clinical expectations (Luscan et al., 2020; Gosche et al., 2002), suggesting that NAISR is able to extract hidden knowledge from data and is able to generate interpretable results directly as 3D shapes.
8

---

Published as a conference paper at ICLR 2024
weight kk
954
A-SDF, Starman
A-SDE, ADNI Hippocampus
A-SDF, Pediatric Airway
S84
woight
10
129
month
Ours, Starman
Ours, ADNI Hippocampus
Ours, Pediatric Airway
Figure 3: Template shape extrapolation covariate space using A-SDF NAISR on three datasets. the Starman shape extrapolations, the blue shapes are the groundtruth shapes and the red shapes are the reconstructions. The shapes in the middle white circles the template shapes. The template shape is generated with zero latent code and is used to create template covariate space. The shapes in the green and yellow boxes are plotted with { (#.), J. representing
disentangled shape evolutions along each covariate respectively. The purple shadows over the space indicate the covariate range that the dataset covers. Cyan points represent male and purple points female patients in the dataset. The points represent the covariates of all patients in the dataset. The colored shades at the boundary represent the covariate distributions stratified by sex. Example 3D shapes in the covariate space visualized with their volumes (cm²) helow. shapes in the covariate space given either individualized latent code z or template latent code O, whereas A-SDF struggles. The supplementary material provides more visualizations of individualized covariate spaces in Section S.3.5. (Best viewed zoomed.)
is able extrapolate the
LIMITATIONS AND FUTURE WORK
Invertible transforms are often desirable for shape correspondences but not guaranteed in NA SS.. Invertibility could be guaranteed by representing deformations via velocity fields, but such parameter- izations are costly because of the required numerical integration. In future work, we will develop efficient invertible representations, which will ensure topology preservation. So far only indirectly assess our model by shape reconstruction and transfer performance. Going forward we will include patients with airway abnormalities. This will allow to explore if our estimated model of normal airway shape can be used to detect airway abnormalities. Introducing group sparsity (Yin et al., 2012; Chen et al., 2017) NATSR for high-dimensional covariates is also promising future work.
CONCLUSION
We proposed NA WSRS,, 3D neural additive model for interpretable shape representation. We tested NAISK on three different datasets and observed particularly good performance on real 3D) medical datasets. Compared other shape representation methods, NAISE SR 1) captures the effect of individual covariates on shapes; 2) can transfer shapes to new covariates, e.g., to infer anatomy development; and 3) can provide shapes based on extrapolated covariates. NAISR the first approach combining deep implicit shape representations based on template deformation with the ability to account for covariates. We believe our work an exciting start for a new line of research: interpretable neural shape models for scientific discovery.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENT
The research reported in this publication was supported by NIH grant 1R01HL154429. The content is solely the responsibility the authors and does not necessarily represent the official views of the NIH. Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department ADNI is funded by the National Institute on Aging. the National Institute of Biomedical Imaging and Bioengineering. and through generous contributions from the following: AbbVie, Alzheimer's s Association; Alzheimer's Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company: CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaccuticals, Inc.; Eli Lilly and Company: Eurolmmun; F. Hoffmann- La Roche
of Defense award number WXX--H2-2200111
Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Hcalthcare; IXICO JJanseen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical
Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.:Meso Scale Diagnostics,
LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer
Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company: and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org) The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer's Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.
10

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
We are dedicated to ensuring the reproducibility of NAISR to facilitate more scientific discoveries on shapes. To assist researchers in replicating and building upon our work, made the following efforts.
Model & Algorithm: Our paper provides detailed descriptions of the model architectures (see Section 3), implementation details (see Section S.3.1), and ablation studies (see Sec- tion S.3.2). We have submitted our source code. The implementation of NATSR will be made publicly available.
Datasets & Experiments: We provide extensive illustrations and visualizations for the datasets we used. To ensure transparency and ease of replication, the exact data processing steps, from raw data to processed input, are outlined in Section S.2 of the supplementary materials. We expect our detailed supplementary material to ensure the reproducibility of our method and the understandability our experimental results. We also have submitted the code for synthesizing the 2D Starman dataset so that researchers can easily reproduce the results.

---

Published as a conference paper at ICLR 2024
REFERENCES
Panos Achlioptas, Olga Diamanti, loannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3d point clouds. International conference on machine learning, pp. 40-49. PMLR, 2018.
Rishabh Agarwal, Nicholas Frosst, Xuezhou Zhang, Rich Caruana, and Geoffrey E Hinton. Neural additive models: Interpretable machine learning with neural nets, arXiv preprint arXiv:2004.13912. 2020.
Sercan Ô Arik and Tomas Pfister. Tabnet: Attentive interpretable tabular learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 1679-6687, 2021.
K. Somani Arun, Thomas S Huang, and Steven D Blostein. Least-squares fitting of two 3-d point sets. IEEE Transactions on pattern analysis and machine intelligence, (5):698-700, 1987.
Chong Bao, Yinda Zhang, Bangbang Yang. Tranxing Fan, Zesong Yang, Hujun Bao, Guofeng Zhang, and Zhaopeng Cui. Sine: Semantic-driven image-based nerf editing with prior-guided editing field. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp-
20919-20929, 2023.
Cosmin Bercea, Benedikt Wiestler, Daniel Rueckert, and Shadi Albarqouni. Federated disentangled representation learning for unsupervised brain anomaly detection. Nature Machine Intelligence, 4 (8):685-695, 2022.
Jeroen Berrevoets, Ahmed Alaa, Zhaozhi Qian, James Jordon, Alexander ES Gimson, and Mihaela Van Der Schaar. Learning queueing policies for organ transplantation allocation using interpretable counterfactual survival analysis. In International Conference on Machine Learning. pp. 992-802. PMLR, 2021.
Alexandre Bône, Olivier Colliot, Stanley Durrleman, and Alzheimer's Discase Neuroimaging Initia- tive. Learning the spatiotemporal variability in longitudinal shape data sets. International Journal of Computer Vision, 2):277)) 2896, 2020.
Agisilaos Chartsias, Thomas Joyce, Giorgos Papanastasiou, Scott Semple, Michelle Williams, David E Newby, Rohan Dharmakumar, and Sotirios A Tsaftaris. Disentangled representation learning in cardiac image analysis. Medical image analysis, 58:101535, 2019.
Hong Chen, Xiaoqian Wang, Cheng Deng, and Heng Huang. Group sparse additive machine. Advances in Neural Information Processing Systems, 30, 2017.
Kan Chen, Qishuo Yin, and Qi Long. Covariate- balancing aware interpretable deep learning models for treatment effect estimation. arXiv preprini arXiv:2203.03188. 2022.
Dovariateballaccigg..o
Xu Chen, Yufeng Zheng, Michael Black, Otmar Hilliges, and Andreas Geiger. Snarf: Differentiable forward skinning for animating non-rigid neural implicit shapes. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 115941 1604, 2021.
Jiebin Chu, Yaoyun Zhang, Fei Huang, Luo Si, Songfang Huang, and Zhengxing Huang. Disentangled representation for sequential treatment effect estimation. Computer Methods and Programs in Biomedicine, 226:107175, 2022.
Jonathan Crabbe, Zhaozhi Qian, Fergus Imrie, and Mihacla van der Schaar. Explaining latent representations with corpus of examples. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 12154-12166. Curran Associates, Inc., 2021. URL https: wtt.//ee//////////eee /ttp:////2.222....aa.aaaa2222222
neurips. pdf.
Adrian Dalca, Marianne Rakic, John Guttag, and Mert Sabuncu. Learning conditional deformable templates with convolutional networks. Advances in neural information processing systems, 32, 2019.
12

---

Published as a conference paper at ICLR 2024
Boyang Deng, John P Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, and Andrea Tagliasacchi. Nasa neural articulated shape upproximation. In Computer Vision-ECee ECCV 2020: 16th European Conference, Glasgow. UK, August 23-28. 2020. Proceedings, Part VII 16, pp. 612-628. Springer, 2020.
Yu Deng, Jiaolong Yang, and Xin Tong. Deformed implicit field: Modeling 3d shapes with learned dense correspondence. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pp. 10286-10296 2021.
Zheng Ding, Yifan Xu, Weijian Xu, Gaurav Parmar, Yang Yang, Max Welling, and Zhuowen Tu. Guided variational autoencoder for disentanglement learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7920-7929, 2020.
Shivam Duggal and Deepak Pathak. Topologically-awan deformation fields for single-view 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1536-1546, 2022.
Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojeic, and Sanja Fidler. Get3d: generative model of high quality 3d textured shapes learned from images. In Advances In Neural Information Processing Systems, 2022.
KM Gosche, JA Mortimer, CD Smith, WR Markesbery, and DA Snowdon. Hippocampal volume as an index of Alzheimer neuropathology: findings from the nun study, Neurology, 2002.
Daniele Grattarola and Pierre Vandergheynst. Generalised implicit neural representations, arXiv preprint arXiv:2205. 15674, 2022.
Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regular- ization for learning shapes. arXiv preprint arXiv:2002. 10099, 2020.
Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. A papier-måché approach learning 3d surface generation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 216-224, 2018.
Kun Han, Shanlin Sun, and Xiaohui Xie. Hybrid neural diffeomorphic flow for shape representation and generation via triplane. arXiv preprint arXiv:2307.01957 2023a.
Kun Han, Shanlin Sun, Xiangyi Yan, Chenyu You, Hao Tang. Junayed Naushad, Haoyu Ma, Deying Kong, and Xiaohui Xie. Diffeomorphic image registration with neural velocity field. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1869-1879, 2023b.
Trevor Hastie. Generalized additive models. Routledge, 2017.
Clifford Jack Jr, Matt A Bernstein, Nick C Fox, Paul Thompson, Gene Alexander, Danielle Harvey, Bret Borowski, Paula J Britson, Jennifer L. Whitwell, Chadwick Ward, et al. The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine, 27(4): 685-691, 2008
Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. Disentangled representation learning for non con-parlllll text style transfer. arXiv preprint arXiv:1808.0433 2018.
Hyungsik Jung and Youngrock Oh. Towards better explanations of class activation mapping. In Proceedings of the IEEÉ/CVF International Conference on Computer Vision, pp. 1336-1344, 2021.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.
Jiahui Lei and Kostas Daniilidis. Cadex: Learning canonical deformation coordinate space for dynamic surface representation via neural bomeomorphism. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6624 6634, 2022.
13

---

Published as a conference paper at ICLR 2024
Di Liu, Anastasis Stathopoulos, Qilong Zhangli. Yunhe Gao, and Dimitris N Metaxas. Lepard: Learn- ing explicit part discovery for 3d articulated shape reconstruction. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Jia- Wei Liu, Yan-Pei Cao, Weijia Mao, Wenqiao Zhang, David Junhao Zhang, Jussi Keppo, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Devrf: Fast deformable voxel radiance fields for dynamic scenes. Advances in Neural Information Processing Systems, 35:36762-36775, 2022.
Lingjie Liu, Mare Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, and Christian Theobalt. Neural actor: Neural free-view synthesis human actors with pose control. ACM transactions on graphics (TOG), 40(6): 2021.
William Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. ACM siggraph computer graphics. 21(4):163-169, 1987.
Romain Luscan, Nicolas Leboulanger. Pierre Fayoux, Gaspard Kerner, Kahina Belhous, Vincent Couloigner, Erea-Noël Garabedian, François Simon, Françoise Denoyelle, and Briac Thierry. Developmental changes of upper airway dimensions in children. Pediatric Anesthesia, 30(4): 435 445, 2020.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVÉ conference on computer vision and pattern recognition, pp. 4460-4470, 2019.
Raha Moraffah, Mansoorch Karami, Ruocheng Guo, Adrienne Raglin, and Huan Liu. Causal inter- pretability for machine learning- dearning-problems, methods and evaluation. ACM SIGKDD Explorations Newsletter, 22(1):18 1011-333 33, 2020.
Jiteng Mu, Weichao Qiu, Adam Kortylewski, Alan Yuille, Nuno Vasconcelos, and Xiaolong Wang. A-sdf: Learning disentangled signed distance functions for articulated shape representation. In Proceedings of the JEEE/CVF International Conference on Computer Vision, pp. 13001-13011, 2021.
Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Occupancy flow: 4d reconstruction by learning particle dynamics. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5379-5389, 2019.
Tiago Novello, Guilherme Schardong, Luiz Schirmer, Vinícius da Silva, Hélio Lopes, and Luiz Velho. Exploring differential geometry in neural implicits. Computers & Graphics, 108:49-60, 2022.
Stanley Osher and Ronald P Fedkiw. Level set methods and dynamic implicit surfaces, volume 1.
Springer New York, 2005.
Pablo Palafox, Aljaž Božič, Justus Thies, Matthias Nießner, and Angela Dai. Npms: Neural parametric models for 3d deformable shapes. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 26655-17755. 2021.
Pablo Palafox, Nikolaos Sarafianos, Tony Tung, and Angela Dai. Spams: Structured implicit parametric models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12851-12860, 2022.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. Proceedings of the IEEF/CVF conference on computer vision and patlern recognition, pp. 165-174, 2019.
Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 5865-5874, 2021a.
Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualll and Steven Seitz. Hypernerf: A higher--dimensional representation for topologically varying neural radiance fields. arXiv preprint arXiv:2 / 13228, 2021b. lb.
14

---

Published as a conference paper at ICLR 2024
Sida Peng, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, and Hujun Bao. Animatable neural radiance fields for modeling dynamic human bodies. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14314-14323, 2021.
Ronald Carl Petersen, Paul Aisen, Laurel A Beckett, Michael C Donohue, Anthony Collins Gamst, Danielle Harvey, Clifford R Jack, William J Jagust, Leslie M Shaw, Arthur W Toga, et al. Alzheimer's 's disease neuroimaging initiative (adni): clinical characterization. Neurology, 74(3):
201-209, 2010.
Pakkapon Phongthawee, Suttisak Wizadwongsa, Jiraphon Yenphraphai, and Supasorn Suwajanakorn. Nex360: Real-time all-around view synthesis with neural basis expansion. IEEE Transactions Pattern Analysis and Machine Intelligence, 2022.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why should trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1 1135-1144 2016.
James Albert Sethian. Level set methods and fast marching methods: evolving interfaces in compula- tional geometry, fluid mechanics, computer vision, and materials science, volume 3. Cambridge university press, 1999.
Tianchang Shen, Jun Gao, Kangxue Yin, Ming- Yu Liu, and Sanja Fidler. Deep marching tetrahedra: hybrid representation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems, 34:6087-6101, 2021.
Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, and Gerard Medioni. GAN-control: Explicitly controllable GANs. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 14083-14093, 2021.
Qingyao Shuai, Chi Zhang, Kaizhi Yang, and Xuejin Chen. Dpf-net: Combining explicit shape priors deformable primitive field for unsupervised structural reconstruction of 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14321-14329, 2023.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems, 11:7462-74733 2020.
Wolfgang Stammer, Marius Memmel, Patrick Schramowski, and Kristian Kersting. Interactive disen- tanglement: Learning concepts by interacting with their prototype representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10317-10328, 2022.
Shanlin Sun, Kun Han, Deying Kong, Hao Tang, Xiangyi Yan, and Xiaohui Xie. Topelogy-preserving shape reconstruction and registration via neural diffeomorphic flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20845-20855, 2022a.
Shanlin Sun, Kun Han, Deying Kong, Chenyu You, and Xiaohui Xie. Mirnf: Medical image
registration via neural fields. arXiv preprini NXNi:220060031111 2022b.
Ayush Tewari, Xingang Pan, Ohad Fried, Maneesh Agrawala, Christian Theobalt, et al. Disen- Learning 3d generative model with disentangled geometry and appearance from monocular images. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 1516-1525, 2022.
tangled3d:
Jayaraman Thiagarajan, Prasanna Sattigeri, Deepta Rajan, and Bindya Venkatesh. Calibrat- ing healthcare AI: Towards reliable and interpretable deep predictive models. arXiv preprint arXiv:2004. 14480, 2020.
Lin Tian, Soumyadip Sengupta, Hastings Greer, Raúl San José Estépar, and Mare Niethammer. Nephi: Neural deformation fields for approximately diffeomorphic medical image registration. arXiv preprint 2rXiv:2309.07322. 2023.
15

---

Published as a conference paper at ICLR 2024
Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Carsten Stoll, and Christian Theobalt. Patchnets: aatch-bascd generalizable deep implicit 3d shape representations. In Computer ECConnECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVI 16, pp. 293-309. Springer, 2020.
Stefan Van der Walt, Johannes L Schönberger, Juan Junez-Iglesian François Boulogne, Joshua D Warner, Neil Yager. Emmanuelle Gouillart, and Tony Yu. scikit-image: image processing in python. PeerJ, 2:e453, 2014.
Fangyin Wei, Elena Sizikova, Avncesh Sud, Szymon Rusinkiewicz, and Thomas Funkhouser. Learn- ing to infer semantic parameters for 3d shape editing. In 2020 International Conference on 3D Vision (3DV), pp. 434 442. IEEE, 2020.
Fangyin Wei, Rohan Chabra, Lingni Ma. Christoph Lassner, Michael Zollhoefer, Szymon Rusinkiewicz, Chris Sweeney, Richard Newcombe, and Mira Slavcheva. Self-supervised neural articulated shape and appearance models. In Proceedings IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Chao Wen, Yinda Zhang, Zhuwen Li, and Yanwei Fu. Pixel2mesh++: Multi-view 3d mesh generation via deformation. In Pruceedings of the IEEE/CVE international conference on computer vision, pp. 1042-1051, 2019.
Xin Wen, Junsheng Zhou, Yu-Shen Liu, Hua Su, Zhen Dong, and Zhizhong Han. 3d shape reconstruc- tion from 2d images with disentangled attribute flow. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3803-3813, 2,0.3811,,0000 2022.
Jelmer Wolterink, Jesse Zwweeneneerg, and Christoph Brune. Implicit neural representations for deformable image registration. In International Conference on Medical Imaging with Deep Learning, pp. 1349-1359, PMLR, 2022a.
Jelmer M Wolterink, Jesse C Zwienenberg, and Christoph Brune. Implicit neural representations for deformable image registration. In International Conference on Medical Imaging with Deep Learning, pp. 1349-1359. PMLR, 2022b.
Sanghyun Woo, Jongchan Park, Joon- Young Lee, and In So Kweon. Cbam: Convolutional block attention module. Proceedings of the European conference on computer vision (ECCV), pp. 3-19, 2018.
Jiajun Wu, Chengkai Zhang. Tianfan Xue, Bill Freeman, and Josh Tenenbaum. Learning a proba- bilistic latent space of object shapes via 3d aaveeraiivvadeessrriilll modeling. Advances in neural information processing systems, 29, 2016.
Jiajun Wu, Chengkai Zhang, Xiuming Zhang, Zhoutong Zhang. William T Freeman, and Joshua B Tenenbaum. Learning shape priors for single-view 3d completion and reconstruction. In Proceed- ings of the European Conference on Computer Vision (ECCV), pp. 646-662, 2018
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang. and Jianxiong Xiao. 3d shapenets: deep representation for volumetric shapes. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1912-1920, 2015.
Mutian Xu, Junhao Zhang, Zhipeng Zhou, Mingye Xu, Xiaojuan Qi, and Yu Qiao. Learning geometry--disentaggle representation for complementary understanding of 3d object point cloud. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 3056-3064, 2021.
Bangbang Yang, Chong Bao, Junyi Zeng. Hujun Bao, Yinda Zhang, Zhaopeng Cui, and Guofeng Zhang, Neumesh: Learning disentangled neural mesh-based implicit field for geometry and texture editing. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVI, pp. 597-614. Springer, 2022a.
Guandao Yang, Serge Belongie, Bharath Hariharan, and Vladlen Koltun. Geometry processing with neural fields. Advances in Neural Information Processing Systems, 34:22483-2244 2021.
16

---

Published as a conference paper at ICLR 2024
Jiancheng Yang, Udaranga Wickramasinghe, Bingbing Ni, and Pascal Fua. ImplicitAtlas: learning deformable shape templates in medical imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15861-15871, 2022b.
Jie Yang, Kaichun Mo. Yu-Kun Lai, Leonidas Guibas, and Lin Gao. Dsm-net: Disentangled structured mesh net for controllable generation of fine geometry. arXiv preprint urXiv:2008.054440 2(3), 2020.
Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 206-215, 2018.
Junming Yin, Xi Chen, and Eric P Xing. Group sparse additive models. Proceedings of the... International Conference Machine Learning. International Conference Machine Learning. volume 2012, pp. 871. NIH Public Access, 2012.
Maciej Zamorski, Maciej Zięba, Piotr Klukowski, Rafal Nowak, Karol Kurach, Wojciech Stokowiec, and Tomasz Trzciñssi. Adversarial autoencoders for compact representations of 3d point clouds. Computer Vision and Image Understanding. 193:102921, 2020.
Baowen Zhang, Jiahe Li. Xiaoming Deng, Yinda Zhang, Cuixia Ma, and Hongan Wang. Self- supervised learning of implicit shape representation with dense correspondence for deformable objects. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 14268- 2023.
Ruiqi Zhang and Jie Chen. Ndf: Neural deformable fields for dynamic human modelling, In European Conference on Computer Vision, pp. 37-52. Springer, 2022.
Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Josh Tenenbaum, Bill Freeman, and Jiajun Wu. Learning to reconstruct shapes from unseen classes. Advances in neural information processing systems, 31, 2018a.
Yuting Zhang, Yijie Guo. Yixin Jin, Yijun Luo, Zhiyuan He, and Honglak Lec. Unsupervised discovery of object landmarks structural representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2694-2703, 2018b.
Chengwei Zheng, Wenbin Lin, and Feng Xu. Editablenerf: Editing topologically varying neural radiance fields by key points. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8317-8327, 2023.
Zerong Zheng, Tao Yu, Qionghai Dai, and Yebin Liu. Deep implicit templates for 3d shape represen- tation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1429-1439, 2021.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Computer Vision and Pattern Recognition, 2016.
Hao Zhu, Xinxin Zuo, Sen Wang, Xun Cao, and Ruigang Yang. Detailed human shape estimation from a single image by hierarchical mesh deformation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 4491-4500, 2019.
Jing Zou, Noémie Debroux, Lihao Liu, Jing Qin, Carola-Bibian Schönlieb, and Angelica I Aviles- hyperelastic regularisation.
Rivero. Homeomorphic image registration via arXiv preprint 28Xiv:2303.081 13, 2023.
17

---

Published as a conference paper at ICLR 2024
SUPPLEMENTARY MATERIAL FOR NAISR
S.1 RELATED WORK
Deep Implicit Functions. Compared with geometry representations such as voxel grids (Wu et al., 2016; 2015; 2018), point clouds (Achlioptas et al., 2018; Yang et al., 2018; Zamorski et al., 2020) and meshes (Groueix et al., 2018; Wen et al., 2019; Zhu al., 2019). DIFs are able to capture highly detailed and complex 3D shapes using a relatively small amount of data (Park et al., 2019; Mu et al., 2021; Zheng et al., 2021; Sun et al., 2022a; Deng et al., 2021). They are based on classical ideas of level set representations (Sethian, 1999; Osher & Fedkiw, 2005); however, whereas these classical level set methods represent the level set function on a grid, DIFs parameterize it as a continuous function, c.g., by neural network. Hence, DIFs are not reliant meshes, grids, or a discrete set of points. This allows them to efficiently represent natural- -ouurlllooking surfaces (Gropp et al., 2020; Sitzmann et al., 2020; Niemeyer al., 2019). Further, DIFs can be trained data (e.g., shapes and images), making them more versatile than other shape representation methods. This makes them useful in applications ranging from computer graphics, to virtual reality, and robotics (Gao et al., 2022; Yang et al., 2022a; Phongthawee et al., 2022; Shen et al., 2021). We therefore formulate NAISR based on DIFs.
diverse range of
Neural Deformable Models Neural Deformable Models (NDMs) establish point correspondences with DIFs, computer graphics, there has been increasing interest in NDMs to animate or edit scenes (Liu et al., 2022; Park et al., 202 la; Bao et al., 2023; Zheng et al., 2023), objects (Duggal & Pathak, 2022; Lei & Danilidis, 2022; Bao et al., 2023; Niemeyer et al., 2019; Zhang et al., 2023; Shuai et al., 2023), and digital humans (Liu et al., 2021; Chen et al., 2021; Niemeyer et al., 2019; Park et al., 2021b;a; Peng et al., 2021; Zhang & Chen, 2022; Zhang et al., 2023).
Establishing point correspondences is also important to help experts to detect, understand, diagnose, and track diseases. NDMs have shown to be effective in exploring point correspondences for medical images (Han et al., 2023b; Tian et al., 2023; Sun al., 2022b; Wolterink et al., 2022a; Zou et al., 2023) and shapes (Sun et al., 2022a; Yang et al., 2022b; Han et al., 2023a)
Among NDMs for shape representations, ImplicitAtlas (Yang et al., 2022b), DIF-Net (Deng et al., 2021), DIT (Zheng et al., 2021), and NDF (Sun et al., 2022a) were proposed to capture point corre- spondence between target and template shapes within NDMs. Currently, no continuous deformable shape representation which models the effects of covariates exists. NAISR provides a model with such capabilities.
Disentangled Representation Learning. Disentangled representation learning (DRL) has been explored in a variety of domains, including computer vision (Shoshan et al., 2021; Ding et al., 2020; Zhang et al., 2018b;a; Xu et al., 2021; Yang et al., 2020), natural language processing (John et al., 2018), and medical image analysis (Chartsias et al., 2019; Bercea et al., 2022).
For example, Wei et al. built a mesh editing model based on disentangled semantic parameters. Their model learns from simulated datasets which are based on parameterized models and pre-defined templates (Wei et al., 2020). DRL has also emerged in the context of implicit representations promising approach for 3D computer vision. By disentangling the underlying factors of variation, such as object shape, orientation, and texture, DRL. can facilitate more effective 3D object recognition, reconstruction, and manipulation (Stammer et al., 2022; Zhang et al., 2018b;a; Xu et al., 2021; Yang al., 2020; 2022a; Gao al., 2022; Tewari et al., 2022).
Besides DRL in computer vision, medical data is typically associated with various covariates which should be taken into account during analyses. Taking (Chu et al., 2022) as an example, when observing tumor's 's progression, it is difficult to know whether the variation of a tumor's ' progression due to time-varying covariates or due to treatment effects. Therefore, being able to disentangle different effects is highly useful for representation to promote understanding and to be able to quantify the effect of covariates on observations. NA ISR provides a disentangled representation and allows us capture the shape effects of covariates.
Articulated Shapes. There is significant research focusing on articulated shapes, mostly on hu- mans (Palafox al., 2021; Chen al., 2021; Tretschk et al., 2020; Deng al., 2020). There is also
18

---

Published as a conference paper at ICLR 2024
line of work articulated general objects, e.g., A-SDF (Mu et al., 2021), NASAM (Wei et al., 2022) and LEPARD (Liu et al., 2023). A-SDF (Mu et al., 2021) uses articulation as an additional input to control generated shapes, while NASAM (Wei et al., 2022) and LEPARD (Liu et al., 2023) learns the latent space of articulation without articulation as supervision.
The aforementioned works articulated objects assume that each articulation affects separate object part. This is easy to observe, e.g., the angles of the two legs of a pair of eyeglasses. Hence, although A-SDF (Mu et al., 2021) and 3DAutriFlow (Wen al., 2022) can disentangle articulations from geometry, they do not disentangle different covariates and their disentanglements are not composable. However, in medical scenarios, covariates often affect shapes complex way, for example, shape might simultaneously be influenced by sex, age, and weight. Dalca al. (Dalca al., 2019) use templates conditioned on covariates for image registration. However, they did not explore covariate-specific deformations, shape representations or shape transfer. NA SARSR allows us to account for such complex covariate interactions.
more entangled and
Explainable Artificial Intelligence. The goal of eXplainable Artificial Intelligence (XAI) is to provide e explanations for decisions and actions of an Al model. Various flavors of XAI exist, including counterfactual inference (Berrevoets al., 2021; Moraffah et al., 2020; Thiagarajan et al., 2020; Chen et al., 2022), attention maps (Zhou et al., 2016; Jung & Oh, 2021; Woo et al., 2018), feature importance (Arik & Pfister, 2021; Ribeiro al., 2016; Agarwal et al., 2020), and instance retrieval (Crabbe et al., 2021). NAISR is inspired by neural additive models (NAMs) (Agarwal et al., 2020) which in turn are inspired by generalized additive models (GAMs) (Hastie, 2017). NAMs are based on a linear combination of neural networks each attending to single input feature. NAISR& extends this concept to interpretable 3D shape representations. This is significantly more involved as, unlike for NAMs and GAMs, we are no longer dealing with scalar values, but with 3D shapes. NA ISR provides interpretable results by capturing spotial deformations with respect to estimated atlas shape such that individual covariate effects can be distinguished.
S.2 DATASET
S.2.1 STARMAN DATASET
Figure S.4 illustrates how each sample in the dataset is simulated. 5041 shapes from 1000 different starmen simulated as the training set. 4966 shapes from another 1000 starmen simulated as testing set. The number of movements for each individual comes from a uniform distribution 10
The deformation for arms can be represented as
d,(p) - exp( (p c) ci)) mo i 0,1;0= 0.5.
(S.12)
20
The deformation for legs can be represented as
dp(p) = B exp( (p
c.) Ci)
m = 2, = 0.5.
(S.13)
20
We sample the covariates a and 3 from a uniform distribution 11111 The overall deformation D is the sum of the covariates-controllingg deformations {d,} imposed on the individual starman shape, as
D = (d) +p)
(S.14)
S.2.2 ADNI HIPPOCAMPUS
The ADNI hippocampus dataset consists of 1632 hippocampus segmentations from magnetic resonance (MR) images from the ADNI dataset, 80% (1297 shapes) of which are used for training
'Dat used in the preparation of this article were obtained from the Alzheimer's Discase Neuroimaging Initiative (ADNT) database (adni. coni -us..ee. edu). The ADNT was launched in 2003 public-private
19

---

Published as a conference paper at ICLR 2024
m7
Figure S.4: Visualization of the Starman dataset simulation. The template shape is in solid black, the control points {c:] are the five points in magenta. The grey dashed lines represent the individual random deformation d, the template and those control points {c.}. yielding an individualized template X.. The moving direction mo controlling the two arms shown as bold blue arrow. The moving direction m. controlling the two legs is shown as bold green arrow. The velocity fields du and d, control the upward/downward movement of two arms correspondingly. The velocity fields d2 and ds controls the splits of two legs correspondingly. The individualized template shape X, is deformed by d = E,d; to Y, (the red shape), representing person moving their arms and legs. The covariates and B decide how much the arm is lifted and how much the legs are split.
and 20% (335 shapes) for testing. Each shape associated with covariates (age, sex, AD, education length). AD is binary variable that represents whether a person has Alzheimer disease. AD=1 indicates person has Alzheimer disease. Table S.5 shows the distribution of the number of obser- vations across patients. Table S.6 shows the hippocampus shapes and the demographic information of an example patient. Table S.7 shows the shapes and demographic information at different age percentiles for the whole data set. We observe that the time span of our longitudinal data for each patient is far shorter than the time span across the entire dataset, indicating the challenge of capturing spatiotemporal dependencies over large time spans between shapes while accounting for individual differences between patients.
# observations patients
2
3 410
4
6
10
5
54
Table S.5: Number of patients for given number of observations for the ADNI dataset. For example, the 1st column indicates that there are patients who were only observed once.
S.2.3 PEDIATRIC AIRWAY
The airway shapes extracted from computed tomography (CT) images. We use real CT images of children ranging in age from month to ~9 years old. Acquiring CT images is costly. Further, CT uses ionizing radiation which should be avoided, especially in children, due cancer risks. Hence, is difficult to acquire such CT's for many children. Instead, our data was acquired by serendipity from children who received CTs for reasons other than airway obstructions (c.g., because they had cancer). This also explains why it is difficult to acquire longitudinal data. E.g., one of our patients has 11 timepoints because very sick child had be scanned 11 times. Note that our data is very different from typical CV datasets which can be more readily acquired at scale or may even already
partnership. led by Principal Investigator Michael Weiner, MD. The primary goal ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCT) and early Alzheimer's disease (AD). For up-to-date information, see www adni info org.
20

---

Published as a conference paper at ICLR 2024
3
(
0
3
4
5
age
75.6
75.7
76.2
76.2
76.7
76.7
AD
No
No
No
No
No
Sex
F
edu
20.0
20.0
20.0
20.0
20.0
m-vol
2.26
2.38
2.2
2.35
2.27
2.16
Table S.6: Visualization and demographic information of observations of a patient the ADNI hippocampus dataset. Shapes are plotted with their covariates (age/yrs, AD, sex, edu length)/rr) printed in the table. M-vol (measured volume) is the volume (cm") of the gold standard shapes based the actual imaging.
P-
20
30
50
60
80
90
100
(S*)
0
3
4
7
9
age
55.2
68.4
71.2
72.6
74.2
76,2
77.9
79.8
82.0
85.2
90.8
No
Yes
No
Yes
No
F
F
F
M
F
edu
18.0
16.0
16.0
15.0
18.0
18.0
17.0
20.0
16.0
7.0
15.0
m-vol
1.91
1.48
2.08
1.66
2.21
Table S.7: Visualization demographic information our ADNI Hippocampus shape dataset. Shapes [0, 10, 20, 20,30, 40, 50, 60, 70, 80, 90, 10h)-th age percentiles are plotted with their covariates (age/yrs, AD, sex, edu( education length)/yrs) printed in the table. M-vol (measured volume) is the volume (cm, 2) of the gold standard shapes based on the actual imaging.
21

---

Published as a conference paper at ICLR 2024
exist based on internet photo collections. This is impossible for task because image acquisition risks always have to be justified by patient benefits.
Our dataset includes 229 cross-sectional observations (where patient only imaged once) and 34 longitudinal observations. Each shape has covariates (age, weight, sex) and annotated anatomical landmarks. Errors in the shapes {8*) may arise from image segmentation error, differences in head positioning. missing parts of the airway shapes due to incomplete image coverage, and dynamic airway deformations due to breathing. Table S.8 shows the distribution of the number of observations across patients. Most of the patients in the dataset only have one observation; only 22 patients have 3 observation times. Table S.9 shows the airway shapes and the demographic information of an example patient. Table S.10 shows the shapes and demographic information at different age percentiles for the whole data set. Similar the ADNI hippocampus dataset, the time span of the longitudinal data for each patient is far shorter than the time span across the entire dataset, which poses a significant shape analysis challenge for realistic medical shapes.
# observations
2
11
# patients
229
12
6
8
3
2
Table S.8: Number of patients for given number of observations for the pediatric airway dataset. For example, the column indicates there are 229 patients who were only observed once.
Data Processing. For the ADNI hippocampus dataset and the pediatric airway dataset, the shape meshes are extracted using Marching Cubes (Lorensen & Cline, 1987; Van der Walt et al., 2014) 14) to obtain coordinates and normal vectors of on-surface points. The hippocapus shapes are rigidly aligned using the ICP algorithm (Arun et al., 1987). The airway shapes are rigidly aligned using the anatomical landmarks. The true vocal cords landmark is set the origin. We follow the implementation in (Park et al., 2019) to sample 500,000 off-surface points. During training, is important to preserve the scale information. We therefore scale all meshes with the same constant.
S.3 EXPERIMENTS
Section S.3.1 describes implementation details Section S.3.2 describe the ablation study. Section S.3.3, Section S.3.4, and Section S.3.5 show additional experimental results for shape reconstruction, shape transfer, and disentangled shape evolution, respectively.
S.3.1 IMPLEMENTATION DETAILS
Each subnetwork, including the template network T and the displacement networks {fr), are all parameterized with an Ni-layer MLP using sine activations. We use N for Starman and the
#time
2
4
5
6
7
8
{
#time
0
2
3
4
5
6
7
8
age
84.00
85.00
87.00
91.0
95.00
98.00
101.00
104.00
120.00
weight
20.40
20.40
21.00
21.9
22.80
22.90
23.50
24.90
28.50
sex
M
M
M
m-vol 30.07
32.18
48.95
33.8
44.87
42.29
28.42
40.92
61.36
Table S.9: Visualization and demographic information of observations of a patient in our 3D airway shape datasct. Shapes plotted with their covariates (agc/month, weight/kg, sex) printed in the table. M-vol (measured volume) the volume (cmi) of the gold standard shapes based on the actual imaging.
22

---

Published as a conference paper at ICLR 2024
70
P-
10
20
30
40
50
60
70
80
90
100
age
1.00
23.00
55.00
71.00
89.00
111.00
129.00
161.00
179.00
199.00
233.00
weight
3.90
14.20
20.10
21.80
19.70
32.85
44.80
21.30
59.00
93.90
75.60
sex
M
M
F
F
M
M
M
F
F
F
M
m-vol
4.56
16.84
29.53
28.91
27.31
70.90
71.23
43.34
78.63
102.35
113.84
Table S.10: Visualization and demographic information of our 3D airway shape dataset. Shapes of [0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 10h age percentiles are plotted with their covariates (age/month, weight/kg, sex) printed in the table. M-vol (measured volume) is the volume (cm²) ) of the gold standard shapes based the actual imaging.
Ours ADNI Hippocampus Pediatric Airway
Methods
DeepSDF
A-SDF
DIT
NDF
Starman
#params
2.24M
198M
1.92M
0.34M
1.33M
2.26M
1.26M
Table S.11: Number of parameters of the different models.
ADNI hippocampus dataset; we use Nı=6 for the pediatric airway dataset. There are 256 hidden units in each layer. The architecture of the {fi) follows DeepSDF (Park al., 2019), in which skip connection is used to concatenate the input of (p, c:) to the input of the middle layer, as shown in Fig. S.5. We latent code of dimension 256 (L 256).
We follow SIREN (Sitzmann et al., 2020) for the network architecture. SIREN uses periodic activation functions for implicit neural representations and demonstrates that networks which use periodic functions (such as sinusoidal functions) as activations, are well suited for representing complex natural signals and their derivatives. We also follow SIREN's initialization draw weights according to uniform distribution W U (-v
D. W. (Din is the input dimension and wg is the scaling factor of the SIREN layers, which is set to 30).
6
Table S.11 lists the number of model parameters.
For each training iteration, the number of points sampled from each shape is 750 (N 750), of which 500 are on-surface points (N... 500) and the others are off- points (Noff = 250). We train NATSR for 3000 epochs for the airway dataset and 300 epochs for the ADNI hippocampus and St ataran datasets using Adam (Kingma & Ba, 2014) with learning rate 5e - 5 and batch size of 64. Also, we jointly optimize the latent code z with NAISR using Adam (Kingma Ba, 2014) with learning rate of -.
During training, A = As = 10; A2 = 3 10; A3 = 10. A = 1- 10a. For Ltats As = 7. 0.01 (following DeepSDF (Park et al., 2019)). During inference, the latent codes optimized for N, iterations with a learning rate 5e N, is set to 800 for the pediatric airway dataset; N, is to 200 for the Starman and ADNI Hippocampus datasets.
Computational Runtime The model is trained on an Nvidía GeForce RTX 3090 GPU for approximately 12 hours for 3000 epochs for the airway dataset. For each shape, takes around 130 seconds for 800 iterations and 30 seconds for 200 iterations to infer the latent code z and the covariates с respectively. However, we observed that 200 iterations are sufficient to produce a reasonable reconstruction. Then, takes approximately 30 seconds to sample the SDF field and apply Marching Cubes to extract the shape mesh. Regarding shape transfer, evolution, and disentanglement for specific case, first need to optimize the latent code for the shape reconstruction. Once
23

---

Published as a conference paper at ICLR 2024
code
layer
deformetion
point coordinate (Ky22)
point coordinate (K,32)
covatisee 6
covariate
Figure S.5: Construction of
CD 4
EMD
HD +
Methods
variants
influenced term
M
M
M
Ours
A Aς
Exikonal
0.072
0.047
1.447
1.323
10.426
8.716
Ours
M 0
4.323
4.481
1.374
1.244
68.527
69.715
Ours
Àg 0
L NNeumnnn
0.081
0.051
1.449
1.307
10.269
8.546
Ours
A2 0
Dirishhled_on_sw
0.124
0.077
1.912
1.682
10.803
8.916
Ours
Àσ 0
Liat
0.045
0.023
0.980
0.890
8.920
7.028
Ours
Ag* 0.01
Liut
0.049
0.023
1.053
0.924
9.064
7.041
Ours
Ag = 0.1
Liat
0.056
0.031
1.126
1.015
9.578
7.831
Ours
Ag*
Liat
0.067
0.039
1.251
1.143
10.333
8.404
Ours
Ag 10
Lial 0.075
0.049
1.368
1.270
11.111
9.176
Ours
Ag* 100
Liat
0.100
0.073
1.607
1.532
13.456
11.853
Table S.12: Ablation study: quantitative evaluation of shape reconstruction. Ours means covariates used additional input NAISR. The shadowed line is what we report in the main text.
the latent code z is optimized or assigned (c.g.. to 0 as template), it will take around 30 seconds to produce new mesh controlled by the covariate c.
Comparison Methods. For shape reconstruction of unseen shapes, we compare our method on the test set with DeepSDF (Park et al., 2019) A-SDF (Mu et al., 2021) DIT (Zheng et al., 2021) and NDF (Sun et al., 2022a). For shape transfer, we compare our method with A-SDF (Mu et al., 2021) because other comparison methods cannot model covariates summarized in Table 1. The original implementations of the comparison methods did not produce satisfying reconstructions on dataset. We therefore improved them by using our reconstruction losses and by using the S SIREN backbone (Sitzmann et al.. 2020) in DeepSDF (Park et al., 2019), A-SDF (Mu et al., 2021), and the template networks in DIT (Zheng al., 2021) and NDF (Sun al., 2022a).
S.3.2 ABLATION STUDY
We conduct an ablation study on the loss terms on the pediatric airway dataset. Airway shapes are more complicated than the Starman shapes and the hippocampi. Further, the number of shape samples is smallest among the three datasets. On this challenging dataset, our aim is to observe the model robustness when using varying loss terms and our goal is also to determine which loss terms are necessary and what suitable hyperparameter settings are.
Table S.12 and Table S.13 show the shape reconstruction evaluation for different hyperparameter settings. We see that Diriichet for off-surface points is the most important term. A lower Ag for the latent code regularizer Liat yields better reconstruction results. Table 1. shows an ablation study for shape transfer. We observe that the reconstruction losses Dirichlet and Neumann important for shape transfer.
To up, removing any of the reconstruction losses (Leskomal, Dirrihhett LNeumann) ) hurts performance. A smaller Àg yields better reconstruction performance, but may hurt shape transfer performance.
24

---

Published as a conference paper at ICLR 2024
CD 4
EMD +
HD +
Methods
variants
influenced term
M
M
P
M
Ours (c)
A À 0
L Eskonol
0.097
0,052
1.559
1.344
11.178
9,426
Ours (c)
A 0
of
4.029
4.485
1.454
1.239
68.272
69.356
Ours (c)
A 0
Nesmman
0.089
0.055
1.494
1.313
10.52
8.625
Ours (c)
À O
DDirichlet_om_sur
0.151
0.09
2.058
1.756
11.354
9.569
Ours
AS
Liat
0.041
0.019
0.936
0.834
8.677
7.335
Ours
Āg 0.01
Liat
0.051
0.025
1.061
0.923
9.301
7.139
Ours (c)
Āg = 0.1
Liat
0.061
0.031
1.154
1.043
9.875
8.151
Ours
Liat
0.084
0.044
1.344
1.182
10.719
8.577
Ours (c)
Ag 10
Liat
0.109
0.058
1.554
1.336
11.933
9.705
Ours (c)
Āg 100
Liat
0.152
0.088
1.943
1.715
14.37
12.043
Table S.13: Ablation study: quantitative evaluation of shape reconstruction. Ours (c) means covariates used as additional input to NA NAISK, / indicates the mean value of the measurements; M indicates the median of measurements. The shadowed line is what we report in the main text.
ablations
Volume Difference 4
Methods
without covariates
with covariates
variarants
influenced term
1
M
M
Ours
A As
Lisonat
13.977
10.925
8.324
7.214
Ours
A4
Divichhlet, Laur
3736.527
3709.001
3693.514
3612.394
Ours
A O
LNNeumann
24.717
24.877
26.094
25.311
Ours
A2 =
Diriihhlet DDerichlet_ommsur (Dirichlet_om_su//
55.579
57.766
64.845
63.442
Ours
AG O
Liat 14.679
10.306
9.465
7.096
Ours
Ag* = 0.01
Liat
8.766
6.629
8.518
5.105
Ours
Ag 0.1
Liat 12.861
8.307
11.518
8.950
Ours
AG*
Liat
12.820
8.837
11.227
9.653
Ours
Ag 10
Liat
8.644
4.676
9,464
5,756
Ours
dg 100
Liat 11.959
8.857
11.939
8.113
Table S.14: Ablation study: quantitative evaluation of shape transfer. μ indicates the value of the measurements; indicates median measurements. The shadowed line what we report the main text.
25

---

Published as a conference paper at ICLR 2024
CD J
EMD
HD
EpirieMss
Nessuam
Eikomal
List
M
M
M
100
0.01
X 10
X 100
x
5.486
5.386
1,093
0.981
69.792
69.779
1
0.1
10
x1
x 0.00
2,046
1.657
9.576
8.835
33.179
30.757
100
X1
0.175
23.102
3
X 0.1
x 0.1
X 0.01
X 0.1
x 100
0.168
0.131
1.697
1.641
15.570
14.062
0.01
10
0.011
0.210
0.128
1,828
1.458
17.472
17.574
X
0.01
0
X
X
3.815
4,483
1,538
1.437
61.785
68. 420
6
X 10
0.01
0.1
0.1
0.1
1.598
1.416
3.357
2.898
37.793
39 948
X
X
X
33.090
892
0.027
8
X
X 100
X 1
0.01
X 0.01
1.382
0.047
1.461
1.368
14.455
12.120
9
0.01
100
0.202
0.103
1.837
1.453
20.760
19.042
10
0.1
X
X 1
0.01
0.1
0.066
0.038
1.309
1.150
10.494
8.781
11
0.1
0,1
1.662
1.405
5.776
5.282
26.966
24.977
10
0.01
X
0.193
0.126
2.662
2.419
13,704
11
13
X 0.1
X 1
X 100
X 10
100
33.090
25.892
0.030
0.029
36.9668
80.081
14
100
0.064
1.335
9,859
8.240
15
100
X 0.01
X 100
X 10
X
4,302
4.304
1.216
1.020
63.564
62.475
16
0.1
0.01
2,210
1.844
6.875
6.230
38.485
30.460
17
X 0.01
x 0.1
X
100
x
5. 5.560
5.916
6.561
6.09%
960
67.416
18
0.01
x10
x 0.01
0,053
0.031
1.196
1.077
8.565
7,080
Ours
1
X
X1
X1
X1
0.067
0.039
1.239
1,138
10.333
8.404
Table S.15: Ablation study: quantitative evaluation of shape reconstruction on random hyperparameter settings. y indicates the mean value the measurements; M indicates the median of the measurements. The shadowed line is what we report in the main text. Bold text indicates the hyperparameter settings which produce a relatively low chamfer distance (<0.1).
ablations
Volume Difference |
Methods
CDirichles.omusr
Diirchte ruf
Nesmmmn
Deccma!
Ciat
without covariates
with covariates
M
M
10
x 0.1
X
X
x 0.01
0.1
10.891
8.929
11.80
9.095
14
x
100
x
15,260
13.312
12.424
10.632
18
0.01
x10
X
1
x 0.01
18.164
17.261
20.980
19.932
Ours
x
x
x x1
12,820
8.837
11.227
9.653
Table S.16: Ablation study: quantitative evaluation of shape transfer on successful randomized hyperparameter settings for reconstruction task. z indicates the mean value of the measurements; indicates the median of the measurements. The shadowed line what we report in the main text.
Doing a full grid search over all 5 or 6 hyperparameters would be prohibitive. Instead, we test based on random configurations from the full hyperparameter grid. We tested 20 random configurations to 1) demonstrate our setting is reasonable; and to 2) provide some possible guidance from failure settings. Specifically, we created grid by multiplying our coefficients by [0.01, 0.1, 1, 10, 100). Then, we randomly chose 20 grid points to test our model's 's robustness for different hyperparameter settings, as shown Table and Table 1.1 Our randomized grid-search analysis shows that our chosen hyperparameters provide good performance.
S.3.3 SHAPE RECONSTRUCTION
Figure S.6 and Figure S.7 visualize more reconstructed hippocampi and airway shapes respectively. We observe that NAISR produces detailed and complete reconstructions from noisy and incomplete observations.
Each shape reconstruction method, except for A-SDE, successfully reconstructs airways and hip- pocampi. As discussed in Section 4.2, we suspect A-SDF overfits the training set by memorizing shapes with their covariates. We investigate this by evaluating shape reconstruction on the training set for A-SDF and NAISR as shown in Table S.17. From Table S.17, we can see that A-SDF overfits the training set.
S.3.4 SHAPE TRANSFER
Table S.18 shows the transferred airways using NAISR without covariates as input (following Equation 10). The predicted shapes from Equation 9 and Equation 10 look consistent in terms of appearance and development tendency. Table S.19 and Table S.20 show the transferred hippocampi. Due to the limited observation time span of patients the ADNI hippocampus dataset, the volume stays almost constant.
26

---

Published as a conference paper at ICLR 2024
DeepSDF
Ours
Figure S.6: Visualizations of airway shape reconstructions with different methods. The and blue circles show the structure in the black circle from two different views. NAISR produces detailed and accurate reconstructions and imputes missing airway parts.
Gold Standand
DeySDDF
A-SDF
DIT
NDF
Ours
Ours
Figure S.7: Visualizations of hippocampus shape reconstructions with different methods. The red and blue circles show the structure in the black cirele from different views. All methods except for A-SDF are able reconstruct well.
S.3.5 SHAPE DISENTANGLEMENT AND EVOLUTION
Fig. S.9 shows an example of airway shape extrapolation in covariate space for a patient in the testing set. Fig. $.10 shows example of hippocampus shape extrapolation in covariate space for patient in the testing set. We observe that in the range of observed covariates (inside the purple shade), shape extrapolation produces realistic-looking and reasonable growing/shrinking shapes in accordance with clinical expectations. Further, NAISR is able to extrapolate shapes outside this range, but the quality lower than within the range of observed covariates.
S.3.6 VISUALIZATION OF TEMPLATE LEARNING
Fig. S.11 shows the learned Starman, airway and hippocampus templates across epochs. We did not use a fixed atlas. Instead, the atlas is learned as the best shape explaining the shape population given the covariate-speciff deformations. This estimated template shape is by-design expected be the a population-average shape at the average age and average weight in the dataset. NAISR quickly converges and capture reasonable template shape already within the first epochs.
27

---

Published as a conference paper at ICLR 2024
Training Set
Testing Set
Methods
CD
EMD
HD
CD J
EMD
HD +
M
M
M
M
M
M 37.835
A-SDF
0.014
0.010
0.770
0.699
5.729
4.762
2.647
1.178
10.307
8,992
47.172
Ours
0.038
0.025
0.975
0.883
8.624
7.538
0.067
0.039
1.246
1.128
10.333
8.404
Comparison of NAISR and A-SDF on the training set. A-SDF performed well on the training set Table S.17: but failed on the testing set.
#time
0
2
3
5
6
8
9
{S*}
# time
5
6
7
8
9
10
age 154
155
157
155
163
164
167
170
194
227
233
weight
55.2
60.9
64.3
65,25
59.25
59.2
65.3
68
77.1
75.6
75.6
sex
MM
M
M
M
M
M
M
M
M
p-vol 91.08
92.47
93.57
94,26
94.35
94.59
96.28
97.34
102.59
104.75
104.51
m-vol
86.33
82.66
63.23
90.65
98.11
84.35
94.14
127.45
98.81
100.17
113.84
Table S.18: Airway shape transfer without covariates as input for the patient shown in the main text. Blue: gold standard shapes; red: transferred shapes with NA SR. The table below lists the covariates (age/month, weight/kg, sex) for the shapes above. P-vol(predicted volume) is the volume (cm²) of the transferred shape by NA SR with covariates following Eq. equation 9. M-vol (mcasured volume) is the volume (cm²) of the shapes based the actual imaging. The transferred shapes show similar growth trends pediatric airways as shown Table. 4.
# time
0
2
3
C
{8*)
# time
0
2
3
4
age
64.7
65.2
65.2
65.7
65.7
AD
No
No
No
No
No
F
F
F
F
F
edu
14
14
14
14
14
p-vol
1.55
1.55
1.55
1.55
1.55
In-vol
1.49
1.56
1.55
1.45
1.55
Table 5.19: Hippocampus shape transfer with covariates as input. Blue: gold standard shapes; red: transferred shapes with IRR. The table below lists the covariates (age/yrs, AD, sex, eduíeducation length)/yrs for the shapes above. P-vol(predicted volume) is the volume (cm 3) of the transferred shape by NAT I SR with covariates following Equation 10. M-vol (measured volume) the volume (cm of the shapes based the actual imaging. transferred shapes stay almost the same the one-ye meay time period for this patient.
28

---

Published as a conference paper at ICLR 2024
# time
0
1
2
3
4
{S*}
# time
0
2
3
age
64.7
65.2
65,2
65.7
65.7
AD
No
No
No
No
No
sex
F
edu
14
14
14
14
14
p-vol
1.6
1.6
1.6
1.6
1.6
m-vol
1.49
1.56
1.55
1.45
1.55
Table S.20: Hippocampus shape transfer without covariates input. Blue: gold standard shapes; red: transferred shapes with NA The table below lists the covariates (age/yrs, AD, sex, edu(education length)/yrs for the shapes above. P-vol(redicctd volume) is volume (cm 3) the transferred shape by NAISR with covariates following Equation 9. M-vol (measured volume) is the volume (cm of the shapes based on the actual imaging. The transferred shapes stay almost the same in the year period space for this patient.
29

---

Published as a conference paper at ICLR 2024
(4,3)
arm
(1.1)
leg
(-1,1)
arm
(1,4)
A A-SDF, Starman
(4,1)
arm
(1,1)
leg
(1,-1)
(1,41)
Ours, Starman
Figure S.8: Individualized Starman shape extrapolation in covariate space. The blue shapes are the groundtruth shapes and the red shapes are the reconstructions. The purple shadows over the space indicate the covariate range that the dataset covers. The latent code kept constant to create individualized covariate shape space. The shapes in green and yellow boxes are plotted with {Ø.) (see Section 3.4), representing disentangled shape evolutions along the arm and leg respectively. Shapes extrapolated from z; look realistic and smooth across different covariates.
30

---

Published as a conference paper at ICLR 2024
153.3
124.8
MCAD
96.4
weight kg
67.9
39.4
10.9
M01471
-17.6
-12.2
29.1
70.4
111.6
152.9
194.2
235.4
age month
A-SDF Pediatric Airway
153.3
Ai MM-M01
124.8
96.4
weight kg
67.9
39.4
10.9
-17.6
-12.2
29.1
70.4
111.6
152.9
194.2
235.4
age month
Ours, Pediatric Airway
Figure S.9: Individualized airway shape extrapolation in covariate space. Example shapes in the covariate space are visualized with their volumes (cma") below. Cyan points represent male and purple points female children in the dataset. The points represent the covariates of all children in the dataset. The purple shadows the space indicate the covariate range that dataset covers. The colored shades covariate distributions stratified by sex. The latent code z is kept constant to create an individualized covariate shape space. The shapes in the green and yellow boxes are plotted with { (see Section 3.4), representing the disentangled shape evolutions along weight and age respectively. Shapes extrapolated from look realistic and smooth different covariates.
boundary represent
31

---

Published as a conference paper at ICLR 2024
1.1 Mi-
PM 6.144.1
WELTO
0.6
AD
0.4
0.0
-0.2
55.7
62.4
69.1
75.8
82.5
89.2
95.9
age/ / yrs
A-SDF, ADNI Hippocampus
1.1
94:15
TM L4T
SELLA
VA
0.8
0.6
WiL13
AD
0.4
0.0
.0.2
55.7
62.4
69.1
75.8
82.5
89.2
95.9
age/ yrs
Ours, ADNI Hippocampus
Figure S.10: Individualized hippocamiuu shape extrapolation in covariate space. Example shapes in the covariate space are visualized with their volumes (em²) below. Cyan points represent male and purple points female patients in the dataset. The points represent the covariates of all patients in the dataset. The purple shadows over the space indicate covariate range that the dataset covers. The colored shades the boundary represent the covariate distributions stratified by sex. The latent code z is kept constant to create an individualized covariate shape space. The shapes in the green and yellow boxes are plotted with { (see Section 3.4), representing the disentangled shape evolutions along AD and age respectively. Shapes extrapolated from z, look realistic and smooth across different covariates.
32

---

Published as a conference paper at ICLR 2024
epoch,
final
Opocs 0000
CPeCT 0100
epech 0000
opect,
epoch 0600
opech, 1000
opech 2000
Final
epoch, 0000
epoct, 0010
spoch, D020
spoct, 0030
spoch, 0050
epoca, 0100
spost, 0200
final
Figure S.I I: Learned Starman, airway, and hippocampus templates across epochs. The Starmans are ground truth while the red ones are our learned templates across epochs.
33


Reviews:
[{"review_id": "pm1lYxL1Fa", "reviewer": "Reviewer_ScLH", "summary": "This paper proposes a 3D Neural Additive Model for Interpretable Shape Representation (NAISR) which describes individual shapes by deforming a shape atlas in accordance to the effect of disentangled covariates. The proposed approach captures shape population trends and allows for patient-specific predictions through shape transfer. Moreover, NAISR is the first approach to combine the benefits of deep implicit shape representations with an atlas deforming according to specified covariates. Sufficient experiments demonstrate that NAISR achieves excellent shape reconstruction performance while retaining interpretability.", "questions": "1. It would be better if the authors give more details of how to train the additive model $g_i$?\n2. Does the inclusion of group structure contribute to the improvement of model performance? Please refer to the Group Sparse Additive Model (Yin et al 2012; Chen et al 2017).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea is ok\n2. This paper is well-written and easy to follow\n3. The experiment is sufficient", "weaknesses": "1. The research survey on the investigated model is relatively limited.\n2. The construction details of the algorithm need to be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TAK8snM2rc", "reviewer": "Reviewer_juiL", "summary": "The paper proposes a data model for representing shapes in a shape dataset as a function of given co-variates (i.e., age, weight, sex, etc.). It uses deep implicit representations to learn templates and deformations of shape specific to each co-variate whereas both the templates are deformations are not available apriori. The authors evaluated their method in terms of various shape analysis tasks (shape reconstruction, transfer, disentanglement, and evolution) by applying them to one simulated and two real human organ shape datasets. The experimental results support the claim of uniqueness and usefulness of the work.", "questions": "- All the shapes in the dataset seem to be pre-aligned and centered. Is it a requirement that the shapes need to be pre-aligned and centered? In such a case, it should be clearly mentioned in Problem Description (Section 3.1)", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "+ The paper is a very good-read and tells a story in an elegant manner. It touches upon a new and potentially very impactful area of research in the domain of shape analysis of medical data. \n\n+ The method is reasonable and the proposed claims are backed up by quantitative and qualitative evidence as experimental results\n\n+ All the related works and their limitations have been comprehensively mentioned. The differences and similarities between the related works and the proposed work are clearly intelligible.", "weaknesses": "- The loss function is made up of 6 components, each associated with a scalar co-efficient to constitute the final loss. Though an ablation study on the loss components is presented in the supplementary table S.12 and S.13, they do not clearly discuss the effects of these co-efficients on the final result and what should be a reasonable set of values for them.\n\n- Some of the limitations (invertibility problem) should be mentioned in the main manuscript instead of the supplementary. Ending the conclusion of the paper with a reference to the supplementary does not seem like a good approach to me.\n\n- In Section 3.2, the authors simply mentioned that they used SIREN as the backbone of their template and deformation prediction network. I think a little bit more details on SIREN method needs to be added here.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ft7LfzmCV6", "reviewer": "Reviewer_7YZF", "summary": "The paper proposes a concept disentangled deep implicit representation learning method for 3D shapes, with a focus on medical imaging application. Given a set of shapes from the same class represented by a point cloud, the proposed method identifies a template, and learns a set of displacement maps with respect to both known and unknown covariates.  \n\nThe technique is compared to five other implicit shape representations (based on implicit functions), and qualitatively and quantitatively outperforms the best competitor A-SDF (Mu 2021), which also learns disentangled implicit shape representation. \n\nExperiments are conducted on Starman, a simulated 2D shape dataset (5041 training + 4966 testing shapes), ADNI hippocampus 3D shape dataset (1632 hippocampus shapes segmented from magnetic resonance images (MRI)), and pediatric airway 3D shape dataset (357 airway shapes segmented from computed tomography (CT)). Qualitative and quantitative results conducted using shape reconstruction, transfer, disentanglement and evolution experiments and demonstrate competitive performance.\n\nEDIT: I have updated my rating after reading authors' comments.", "questions": "In section 4.2, authors describe experiments with shape reconstruction, and indicate that the the proposed method and other SDF techniques can help complete shapes. It isn’t clear whether the technique is trained for shape completion, and if so, where do the partially complete shape come from? Does the fact that a shape is partial negatively influence method training?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "The paper is well-written and easy to follow. Experiments are thorough and conducted with well-established metrics (Chamfer distance (CD), Earth mover’s distance (EMD), Hausdorff distance (HD)), all established metrics for evaluating 3D shape methods. Code is provided in supplementary materials.\n\nVisualizations are very clear, and lots of additional details are provided in supplementary material.", "weaknesses": "It is not clear how SD based techniques will be beneficial for medical images. Majority of medical imaging techniques use voxel-based representations. The proposed implicit shape representation (as other baselines) smoothed out details (compare ‘’gold standard’’ to all methods in Figure 2, for instance) and it not clear why a method with such artifacts would be beneficial in practice? Authors should provide more clear motivation of why the proposed method will be useful for medical imaging. \n\nAs described in Section 3.1, it is assumed that “the overall displacement field is the sum of displacement fields that are controlled by individual parameters”, plus there is a contribution of the unknown covariate (z) which cannot be controlled. However, it isn’t clear what happens if a key covariate is unknown (e.g., not provided in metadata) or a not useful covariate is present? The authors should clearly explain implications of such assumptions. \n\nThere are several weaknesses in related work:\n \n* Relationship to other methods (Mu 2021) is not clearly described. What is the methodological advantage that the proposed technique brings?\n* Related work is limited to representations focusing on implicit functions. There is other work in allowing parametrized shape editing, e.g., see for instance: Learning to Infer Semantic Parameters for 3D Shape Editing (Wei, 3DV 2020)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vaRmDMTbKt", "reviewer": "Reviewer_ftBT", "summary": "The paper introduces NAISR, a 3D neural additive model that aims to provide an interpretable shape representation. This approach combines deep implicit shape representations with an atlas that deforms in response to specified covariates. The utility of NAISR is demonstrated through evaluations on simulated and real medical datasets. The paper discusses the advantages of NAISR over other shape representation methods, particularly in capturing individual covariates' effects on shapes, shape transfer capabilities, and the ability to generate shapes based on extrapolated covariates.", "questions": "- It is unclear how the atlas is initialized. It would be very helpful to show the learned atlas across epochs.\n\n- The paper does not discuss the computational complexity and runtime for various processes, including training, inference, shape transfer, evolution, and disentanglement. Given that neural implicit representation works at the point coordinate level, understanding these metrics is crucial.\n\n-  What are the computational and time requirements to reconstruct a full shape using the proposed implicit representation?\n\n- The paper does not adequately justify the use of a non-amortized latent code, particularly the latent code encapsulating shape parameters not observed or captured by the covariate. The latent code is being optimized during training and inference as opposed to using a neural network to estimate such latent codes.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well-structured, making it easy to follow. The flow of content is logical, facilitating a clear understanding of the proposed model and its implications.\n\n- The paper tackles a significant gap in the shape modeling field: quantitatively capturing shape changes concerning problem-specific covariates. This is a significant contribution to the domain of shape representation and analysis.\n\n- Leveraging neural additive deep implicit for shape representation offers interpretability and shape reconstructions that are resolution agnostic.\n\n- The proposed approach's efficacy is showcased on both simulated data and medical datasets.\n\n- The paper compares NAISR with existing state-of-the-art implicit shape representation methods, further establishing its superior performance and potential for broader applications.", "weaknesses": "- The paper assumes the given population to be unimodal and representable fully by a learned atlas. However, the methodology's ability to handle multimodal distributions and topological variations remains unclear.\n\n- The experiments do not adequately demonstrate the model's performance under limited training sample sizes, a typical scenario in medical shape analysis.\n\n- While deformations from the learned atlas provide a means to statistical shape analysis, the paper does not evaluate or showcase the learned models' statistical aspects. This omission limits the paper's depth and breadth in understanding the model's holistic performance.\n\n- The paper does not discuss the sensitivity of the proposed model to hyperparameters, which could be crucial for replication and application in various contexts.\n\nMinor: Authors provide a discussion of limitations in the supplementary material, however this should be part of the main paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)