Parametrizing Product Shape Manifolds by Composite Networks
Authors: Josua Sassen
Klaus Hildebrandt
Martin Rumpf
Benedikt Wirth
Venue: ICLR 2023 notable top 25% (2023)
Keywords: shape spaces
product manifolds
nonlinear statistics
low-dimensional data manifolds

Abstract: Parametrizations of data manifolds in shape spaces can be computed using the rich toolbox of Riemannian geometry. This, however, often comes with high computational costs, which raises the question if one can learn an efficient neural network approximation. We show that this is indeed possible for shape spaces with a special product structure, namely those smoothly approximable by a direct sum of low-dimensional manifolds. Our proposed architecture leverages this structure by separately learning approximations for the low-dimensional factors and a subsequent combination. After developing the approach as a general framework, we apply it to a shape space of triangular surfaces. Here, typical examples of data manifolds are given through datasets of articulated models and can be factorized, for example, by a Sparse Principal Geodesic Analysis (SPGA). We demonstrate the effectiveness of our proposed approach with experiments on synthetic data as well as manifolds extracted from data via SPGA.

Published as a conference paper at ICLR 2023
PARAMETRIZING PRODUCT SHAPE MANIFOLDS BY COMPOSITE NETWORKS
Josua Sassen
Klaus Hildebrandt
Martin Rumpf University of Bonn
Benedikt Wirth University of Münster
University of Bonn
TU Delft
ABSTRACT
Parametrizations of data manifolds in shape spaces can be computed using the rich toolbox of Riemannian geometry. This, however, often comes with high computa- tional costs, which raises the question if one can learn efficient neural network approximation. We show that this is indeed possible for shape spaces with a spe- cial product structure, namely those smoothly approximable by direct sum of wwwdiiensioonll manifolds. Our proposed architecture leverages this structure by separately learning approximations for the oow-dimesiional factors and a subse- quent combination. After developing the approach as general framework, we apply it to shape space of triangular surfaces. Here, typical examples of data manifolds are given through datasets of articulated models and can be factorized, for example, by Sparse Principal Geodesic Analysis (SPGA). We demonstrate the effectiveness of our proposed approach with experiments on synthetic data well as manifolds extracted from data via SPGA.
INTRODUCTION
Modeling collections of shapes as data on Riemannian manifolds has enabled the usage of rich set of mathematical tools in areas such as computer graphics and vision, medical imaging, compu- tational biology, and computational anatomy. For example, Principal Geodesic Analysis, general- ization of Principal Component Analysis, can be used to parametrize submanifolds approximating given data points while preserving structure of the data such as its invariance to rigid motion. The evaluation of such a parametrization, however, typically comes at high computational cost as the Riemannian exponential, mapping infinitesimal shape variations shapes, has be evaluated. This motivates trying to learn an efficient approximation for these parametrizations. Direct applica- tion of deep neural networks (NNs), however, proves ineffective for high-dimensiona, spaces with strongly nonlinear variations. Therefore, we consider more structured shape manifolds, namely, we assume that they can be approximated by an affine sum of oow-dimesiional submanifolds. com- puter graphics, typical examples of data manifolds are given through datasets of articulated models, e.g. human bodies, faces or hands. Then, the desired structure of an affine sum of factor manifolds can be produced, for example, by Sparse Principal Geodesic Analysis (SPGA), Motivated by this, we exploit the data manifolds' approximability with such affine sums: We separately approximate the exponential map the factor manifolds by fully connected NNs and the subsequent combina- tion of factors by a convolutional NN to yield our approximate parametrization. In formulas, based on judiciously chosen decomposition nian exponential exp_(v) by & (1 (v,),,...... u5 (v,)), where & is NN and the v5 are further NNs approximating the Riemannian exponential exp, on the low-dimensional factor manifolds.
+ .... + U., our aim is approximate the Rieman-
We develop our approach focusing on the shape space of discrete shells, where shapes are given by triangle meshes and the manifold is equipped with an clasticity-based metric. In principle. our approach is also applicable other shape spaces such as manifolds of images, and we will include remarks on how we propose this could work. We evaluate our approach with experiments on data manifolds of triangle meshes, both synthetic ones and ones extracted from data via SPGA, and we demonstrate that the proposed composite network architecture outperforms monolithic fully connected network architecture as well as an approach based on the affine combination of the factors. We see this work as first step to use NNs to accelerate the complex computations of shape manifold parameterizations. Therefore, we think that our approach has great potential to stimulate further research in this direction, which could in turn advance the applications of Riemannian shape spaces.

---

Published as a conference paper at ICLR 2023
Contributions In summary, the contributions of this paper are
combining the Riemannian exponential map on shape spaces and neural network methodol- ogy for the efficient parametrization of shape space data manifolds,
demonstrating the applicability of such approach for data manifolds which can be
smoothly approximated via direct sums of low-dimensional submanifolds,
using a combination of fully connected neural networks for the factorwise Riemannian ex- ponential maps and convolutional network to couple them,
verifying that such setup works well with existing methods to construct product manifolds, such as Sparse Principal Geodesic Analysis, and
showing that the composite network architecture outperforms alternative approaches.
RELATED WORK
Shape Spaces Shape spaces are manifolds in which each point is shape, e.g., triangle mesh or an image. A Riemannian metric on such a space provides means to define distances between shapes, to interpolate between shapes by computing shortest geodesic paths, and to explore the space by constructing the geodesic curve starting from point into a given direction. Shape spaces have proven useful for applications in areas such computer graphies (Kilian al., 2007: Heeren al., 2012: Wang et al., 2018) 8) and vision (Heeren et al., 2018; Xie et al., 2014), medical imaging (Kurtek al., 201 lb; Samir al., 2014; Kurtek et al., 2016; Bharath et al., 2018), computational biology (Laga et al., 2014). and computational anatomy (Miller et al., 2006; Pennec, 2009; Kurtek et al., 201 la). For an introduction to the topic, we refer to the textbook of Younes (2010).
Shape Space of Meshes Triangle meshes are widely used to represent shapes in computer graphics and vision. Riemannian metrics on shape spaces of triangle meshes can be defined geometrically, using norms on function spaces on the meshes (Kilian al., 2007), or physics-based, considering the meshes as thin shells and measuring the dissipation required to deform the shells (Heeren al., 2012: 2014). The computation of geodesic curves in these spaces requires numerically solving high-dimensional nonlinear variational value problems, which can be costly. For shape interpolation problems, model reduction methods can be used to efficiently find approximate solutions (Brandt al., 2016; von Radziewsky al., 2016).
Statisties in Shape Spaces Data in Riemannian shape spaces can be analyzed using Principal Geodesic Analysis (PGA) (Fletcher et al., 2004; Pennec, 2006). Analogous to principal component analysis (PCA) for data Euclidean spaces, PGA can construct low-dimensional latent represen- tations that preserve much of the variability in the data. This is achieved by mapping the data with a non-linear mapping, the Riemannian logarithmic map, from the manifold to a linear space, the tangential space at the mean shape, and computing a PCA there. Latent variables of the PCA are then mapped with the inverse mapping, the Riemannian exponential map, onto the manifold, so that the latent space describes a submanifold of the shape space. A PGA in shape spaces of meshes was introduced in (Heeren et al., 2018) and used to obtain a low-dimensional, nonlinear, rigid body motion invariant description of shape variation from data.
While PCA modes involve all variables of the data, Sparse Principal Component Analysis (Zou et al., 2006) constructs modes that involve just few variables. This is achieved by integrating a sparsity encouraging term the objective that defines the modes. Based on this idea, Neumann et al. (2013) proposed a scheme for extracting Sparse Localized Deformation Compo- nents (SPLOCS) from a dataset of non-rigid shapes. Since SPLOCS are linear modes, they are well-suited to describe small deformations such as face motions accurately. To increase the range of deformations and to compensate linearization artifacts Huang et al. (2014) integrated SPLOCS with gradient-domain techniques and Wang et al. (2017; 2021) with edge lengths and dihedral angles. In (Sassen et al., 2020b), a Sparse Principal Geodesic Analysis (SPGA) was introduced. Similar to PGA, the SPGA modes are nonlinear and rigid motion invariant. On top of that, however, the SPGA modes describe localized deformations. Due to the localization, many pairs of SPGA modes have disjoint support and therefore independent of each other. We want to take advantage of this prop- erty to effectively learn the reconstruction of points in the manifold from their latent representation through adapted network structure.
Sparse PGA
2

---

Published as a conference paper at ICLR 2023
Product Manifolds This work is focused on the parametrization of the product manifold structures obtained from SPGA. Altemattvv approaches for extracting product structures of data manifolds include the approach of Fumero et al. (2021), which finds product structure of manifold based on a geometric notion of disentanglement, the Geometric Manifold Component Estimator presented in (Pfau al., 2020), which uses Lie group of transformations to generate symmetry- disentangiement of data manifolds, and the approach of Zhang et al. (2021), which decomposes the eigenfunctions of the Laplace-Beltrami operator of a manifold in order to find product structure in the manifold. To the best of our knowledge, there are no works focusing on using networks to approximate the parametrization of such product manifolds in Riemannian shape spaces.
PRELIMINARIES AND NOTATION
We introduce step by step the background necessary understand the context of our work. We also provide an overview of our notation in Appendix E.
shape space is a manifold S C IR whose elements are shapes. These could, for example, be images, curves, surfaces described in various ways. Endowing such a shape manifold with the structure of Riemannian manifold, i.e. a (smoothly) z-dependent inner product 9= the tangent space T,S each point S, provides us with rich set of geometric tools. For example, we can use geodesics c: [0, 1] + S, i.e are length parametrized locally shortest paths, as mathematical formulation of shape interpolation. The Riemannian logarithm log, z E T is then defined as the time derivative è(0) of the geodesic e interpolating between c(0) = 2 and c(1) 2. This allows to interpret the tangent space T.S as the linear space of infinitesimal shape variations. Lastly, the Riemannian exponential map is the inverse of the logarithm, which means that for tangent vector v E T&S one 'shoots' a geodesic in its direction, i.e. constructs geodesic curve with initial velocity to obtain exp, v == c(1) E S. The exponential map allows to transfer operations from infinitesimal shape variations back to actual shapes. We provide more details on Riemannian operators and their discretization in Appendix C.
Riemannian Shape Space
Principal Geodesic Analysis With these tools at hand, one can use Principal Geodesic Analysis (PGA) to compute submanifolds of the shape space approximating given data points {-²} S: One first computes their Riemannian center of mass Σ, i.e the point with minimal sum of squared distances all data points. Then one computes the logarithms 2" = log, z² and thus linearizes the approximation problem at the center of mass by passing to the tangent space S.S. In T&S, one uses classical Principal Component Analysis (PCA) to compute the m dominant modes {w² }, whose span 2/ is the best m-dimensional subspace approximating the logarithms v². Then the submanifold M approximating the data points is parametrized by the exponential map, i.e. M == e.
Sparse Principal Geodesic Analysis The coordinates on Riemannian shape spaces, e.g. pixel values, often correspond to different spatial locations of the shape, e.g. pixel positions, such that it makes sense to consider their support and sparsity: Sassen et al. (2020b) introduced Sparse Princi- pal Geodesic Analysis (SPGA) to compute spatially localized dominant modes with widely disjoint supports. Those modes are easier to interpret semantically and in addition allow for efficient approx- imations of the exponential map. SPGA is performed by adding an appropriate sparsity- -inducin regularization functional R to the variational formulation of PCA to compute such sparse deforma- tion modes. The concrete choice of R depends on the specific shape space. Hence, for given set V E B K of K logarithms, the SPGA problem to compute the first m dominant modes U E 12 reads
minimize UERaXm
||V -V-Uw13 + +RR(U)
W
subject to Uj T3S and [wylso S 1 for j E m1.
The bound on the magnitude of the weights ensures that the coordinates of the uj do not shrink while the weights grow inverse proportionally. As before, the manifold approximating the data points is then parametrized using the exponential map. It can be equipped with a product structure by grouping the modes and applying the exponential map to the corresponding subspaces. This will be elucidated more in Section 4.
NRIC Manifold As shapes we will consider triangular surfaces with fixed connectivity, i.e. with shared sets of vertices V, edges E, and faces F. For vertex positions X E | we denote by
3

---

Published as a conference paper at ICLR 2023
(XX (Le( the vector of edge lengths and by ((X) (8)( the vector of dihedral
angles. We consider the vectors z xxX) (L(X), ((x)) Ra combining both. The manifold of all E [2 corresponding to immersed triangular surfaces is given by
= {= | T(2) > ((2) =),
where T(2) encodes the triangle inequalities and (()) 0 are the discrete integrability conditions from Wang et al. (2012), see also Sassen et al. (2020a) for more details. S is called the NRIC manifold, short for Nonlinear Aatation-Invarian Coordinates, and can be equipped with an elasticity- baseed Riemannian metric (Heeren et al., 2014). To evaluate the logarithm and exponential map, we use the dineeticcttinationn developed by Rumpf & Wirth (2015).
To obtain vertex positions for given lengths and angles, we use the nonlinear least-squares method from Fröhlich Botsch (2011) based on -n-norms to measure the difference between arbitrary 2,
||20 z" =>pp =
Ewe, weallee -21 cEE
121 + Σ EEE
1 10 021".
The weights are computed from a reference shape 2 with vertex positions X, e.g. the center of mass from above, as We,l Le (x)== and we,0 = (x)),,(8)) (x))1/2, where ae is (one third of) the area of both faces adjacent to The bending weight 1/ is the same as used in the elasticity- metric.
To apply SPGA to this shape space, we need to specify the sparsity-inducing regularization R. Sassen et al. (2020b) observed that the simple mode-wise T.--norms
R(U)=
|uyll
suffice due to the natural connection of NRIC variation with elastic distortions. We provide more details on NRIC in Appendix D.
Neural Networks We indicate functions that are implemented as neural networks by the super- script Ç as in s This ( represents the network parameters and is the same for all occurring networks, with the implicit convention that different networks depend on different subsets of these parameters.
We denote by ML Nr) fully-connected network with layer sizes , the non- linear activation function IR - I after each layer, and parameters Ç. For graph convolutional networks, we adapt the approach by Kipf & Welling (2017) to data 2 E R.No on edges e: The tth layer is given by
2 =
W Witt 2e W
Σ BEA(()
2/ + b-+1
(5)
where Wh++1 E M and W.+1 E M are small matrices with learnable parameters and 6t+1 E Ne+1 is the bias, all stored in the parameters Ç. We define the neighborhood of an edge in a triangle mesh as M(e) == {ē E e and share vertex]. We used the Exponential Linear Unit (Clevert et al., 2016) as activation function in all our experiments.
4 COMPOSITE NETWORK APPROXIMATION
As previously explained, we want to learn an efficient parametrization +: Rd + M of an m- dimensional Riemannian data manifold M. Below, we will explain our structural assumptions to achieve this, detail how we include them in the network architecture and training, and finally discuss their applicability to practical examples.
Structural Assumptions One can think of the parametrization as the decoder part an autoen- coder, however, for the purpose of this article we want to be more specific: We would like the parametrization to have a geometric meaning, so we will focus on the situation which I should encode the Riemannian exponential map at some point be combined with alternative concepts in which no ground truth parametrization is available, but this setting allows particularly simple quantification of the results.
M. In principle, our approach can also
4

---

Published as a conference paper at ICLR 2023
If no additional structure of the data manifold is known, one of course not improve over com- puting the Riemannian exponential directly or approximating it by some standard neural network. In contrast, consider the where specific structure is known. For the purpose of our article, this structure is a priori given, that is, we do not study how such a structure can be found or obtained, though we give an example for corresponding procedure on Riemannian shape spaces later on. We require that the data manifold M to be parametrized is embedded in some IR" for possibly large n. The structure of M we want exploit has three components:
(0) Correlation We assume a structural correlation between the different coordinates, e-g. graph- neighbour relations for triangular meshes or pixel-neighbour relations for image data, so that convolutional networks can be applied.
(1) Factorization We assume that the manifold can be smoothly approximated by a product of much lower-dimensional manifolds Mi, MM..... MM,, which we will parametrize separately. Thereby, we exploit that the necessary network size as well as the required training effort de- crease with smaller manifold dimension: For m-dimensional manifolds the network size should scale at least linearly in m, while the required training set and thus also training time will scale exponentially in m.
(2) Combination It is not sufficient that the single factor manifolds are easy to parametrize since generic point on M has components in all factors. Thus, we assume that the direct sum of all factors
M1 +2 m....... +2 M, = f=+ (z) - 2) .... + (z) 2) 21 E Mı...., E My}
for some 2 E MM already approximates the actual manifold M with (possibly large, but) very smooth approximation error. This will ensure that a suitable map from (2) 2,...,. 2) to M can be efficiently learned.
The toy shape manifold M of tori from Figure has the flat metric of St S x T (the factors representing the orientation of the longitudinal and latitudinal ellipsoidal cross-sections as well as bump position, see subsection 5.1) and thus satisfies condition (I). Condition (2) holds since each torus is represented as NRIC: For instance, the creation of the bump (which corresponds to changing the position in T2) is well described by simply adding fixed numbers in the right places of the edge length vector 1(X). However, the direct sum is indeed only an approximation otherwise there would be no error in Figure 1 (green), but one clearly sees a remnant of the bump from the reference shape.
Figure 1: Approximation of the Freaky Torus. We show the reference shape in grey, the approxi- mation of exp, by affine combination of exact factors in green, by a monolithic network in blue, and by our composite network in yellow. The correct vertex positions of exp, are shown as purple points. These purple points should ideally lie on the shaded surface, which would indicate good fit. Indeed, for the approximation by our composite network, this is mostly the case, while for the other two approaches the approximation does not match the point cloud in many places.
Network Representation and Training Before we discuss the previous conditions, let us describe how we exploit them in our network architecture and training procedure. To learn the parametriza- tion Φ: Rm - M, we decompose it as
v (WiS...., ..)).
where 15. Rmg + 1n the parametrization of the myddinensional factor manifold M, and s (R") + R= is the combination of the single factors, behaving approximately like (a) (31,-.-,,%l Ra) z (x1 2) +... +xn, 2), see Figure 2.
5

---

Published as a conference paper at ICLR 2023
M2
TpS
M
Figure 2: Structure of composite network.
The maps 5 will be fully connected neural networks. Because they approximate the smooth expo- nential map rather low-dimensional spaces, they expected achieve high approximation quality that is typically stable under variation of the concrete network architecture (number and size of layers). For the maps y exploit that they operate on structured domain, e.g. mesh, and thus they will be convolutional neural networks. This allows for efficient training and storage even though they operate on digensinnsii data. One could alternatively also use fully connected networks for V/S, but the observed quality of the results was similar despite significantly increased memory requirements to train, store, and evaluate.
These networks are trained separately: To train the map v5.. we consider a parametrization
wy: R + T,M, wy(a²) = ag + + am, Cn,
with coefficient vector a² = (af...., a and basis ei, ei...., e of the Riemannian logarithm of M, at point linear subspace of the tangent space T.M. We then consider a set of random samples S C mm training data (for instance normally distributed or uniformly on ball) and minimize the loss function
J,(()
(lexp,(w,(a)) 35 Σ a'ES!
S
in some norm || depending on the application. To train the map V we subsequently consider a random training set S Im
Rms 1 and minimize the loss function
I()
Σ
(wi +... w)(a")) - w ('
S
(at JES
where we may or may not keep the maps S fixed.
Applicability of Assumptions While condition (0) essentially has hold for any approach em- ploying neural networks, conditions (1) and (2) are specific to our approach. Condition (1) expresses that the intrinsic geomelry of the data manifold M has simplifying structure, while condition (2) is about the extrinsic geometry of how M embedded in R² only both conditions together charac- terize structure that can efficiently be learned. Typical situations where conditions hold include the following two examples:
The different factor manifolds correspond different spatial regions. For instance, images may sometimes be partitioned into different regions that can vary more or less independently of each other. If the regions are fully disjoint, the manifold M exactly be written as a direct sum of factor manifolds; if on the other hand the regions slightly overlap, then this is only fulfilled approximately. Examples in shape spaces are where the single factor manifolds correspond to shape variations with disjoint support such as articulation of the different extremities of character. Such a manifold structure ubiquitous in computer graphics applications, and our computational examples mostly belong to this category.
On shapes one can also often observe variations at different length scales that are independent of each other, for example, geometric texture small length scales versus global shape variations at large length scales. The above toy example of the torus, in which the small bump is or less independent of the global shape variations, belongs this category.

---

Published as a conference paper at ICLR 2023
As mentioned before, assume the product manifold structure to be given priori (in the torus example, for instance, was given by design of the data manifold). The product manifold struc- ture would typically be a result of some data manifold analysis, for example from disentanglement learning. In the setting of Riemannian shape spaces, one way to obtain this structure is the SPGA introduced earlier: It makes of the decomposition into independent product manifolds due to disjoint support of the corresponding shape variations.
5 EXPERIMENTS AND APPLICATIONS
In this section, we present experimental results on the aforementioned synthetic shape manifold of deformed tori and manifolds extracted via SPGA. We will compare our method to approaches based on (Sassen et al., 2020b) and a straightforward approximation of the parametrization by single fully connected network. To quantify the approximation quality of different approaches, we use the coefficient of determination R2_ For approximations & of NRIC 2; with mean ž, it is defined as
Σ, >,
12.5
(2, =1
2.5
From statistical point of view, it quantifies the proportion of variation of the data that is explainable by a given model. This means an R² of one is optimal, the smaller it is the worse is the approxima- tion, and negative li2 means that the model is worse than simply using the mean.
Training& Implementation We used Adam (Kingma & Ba, 2015) as descent method for training all networks, where the initial learning rate was 10-3 and was reduced by factor of 10 every time the loss did not decrease for multiple iterations. For regularization, we used batch normalization after each layer and moderate dropout regularization (p = 0.1) after each convolutional layer. We implemented the neural networks in PyTorch (Paszke et al., 2019) using the PyTorch Geometric library (Fey & Lenssen, 2019). The tools for the NRIC manifold were implemented in C++ based on OpenMesh (Botsch et al., 2002), where we use the Eigen library (Guennebaud et al., 2010) for numerical linear algebra. We follow approach similar to Kilian al. (2007) perform all computations on coarsened mesh and prolongate solutions to fine one only for visualizations.
5.1 SYNTHETIC DATA: FREAKY TORUS
For the Freaky Torus dataset, construct synthetic shape space with factors S x S x T2, where 2² refers to the flat 2-dimensional torus. realized NRIC by (i) deforming the two cross- sectional circles of torus to ellipses of fixed aspect ratio and orientation controlled by the first two S² factors and (ii) growing a bump in normal direction whose position is controlled by the last T factor. These torus deformations are applied and the deformed meshes' NRIC are extracted to obtain our datapoints. We used a mesh with 2048 vertices and uniformly drew 1000 samples from S' x S T More details on the data generation can be found in Appendix A.
regular mesh of regular torus embedded R&3
Figure 1 shows that single fully connected network struggles with approximating the high fre- quency detail of the bump, while our composite network is able to handle this well. This can also be observed in the approximation quality quantified using the R², The composite network achieves an R2 of 0.99 and the monolithic network one of 0.95. This difference may sound small, however, this because the bump a detail and the error is dominated by the overall shape of the torus.
5.2 APPLICATION: SPGA MANIFOLDS
Lastly, we report the results of applying our method to shape manifolds whose approximate product structure found with the help of SPGA. To this end, we repeat three of the examples discussed by Sassen et al. (2020b) and consider one new dataset. The repeated examples are a humanoid dataset from (Anguelov et al., 2005), a dataset of face meshes from (Zhang et al., 2004), and set of hand meshes from (Ych et al., 2011). For the new example, examine humanoid dataset based on SMPL-X (Pavlakos et al., 2019), where we consider their expressive hands and faces (EHF) dataset,
'The data and the generating code can he found at https:/ //gs::///ttt tl com cow/srssssen/ freaky-torus.
7

---

Published as a conference paper at ICLR 2023
containing 100 shapes, and 49 additional shapes from the SMPL+H dataset, which feature more expressive arm and leg movements, adding to a total of 149 imput shapes. We interpret all shapes elements of the nonlinear NRIC space that this small amount of data points already suffices to span a high-dimensional, nonlinear NRIC submanifold that serves as our data manifold.
Based this data, follow the numerical approach of Sassen et al. (2020b) solve (1) and thereby compute the sparse tangent modes. We report the chosen number m of included modes in Table I, where we used the same number Sassen et al. (2020b) for the repeated examples. To factor the resulting data manifold, we again follow the approach outlined by Sassen et al. (2020b), which is clustering based on the spatial overlap the modes. For the hand and face examples, we choose the same number of factors J, while for the SCAPE example we decreased the number to account for the possibility handle nigher-dimensional factors with our method. Our choices are again documented in Table 1. Each cluster then spans exactly one of the factor manifolds and the range of their dimensions is also reported in said table.
Figure 3: Examples from the face and hands datasets. We use the same colors as in Figure 1.
Data Generation Next, we sample the exponential map on the space spanned by the SPGA modes generate the training (and test) data. To this end, we consider the hypercube in R given by the minimal and maximal coefficients of projections of the input data onto the SPGA subspace. Then we draw our parametrization coefficient samples S с R" uniformly from this hypercube, To create the samples S for the factor manifolds, we simply take the corresponding subcomponents of coefficient vectors from S. The corresponding shapes were then computed by evaluating the exponential map for each of them. Overall, we sampled approximately |S] 4000 points for each of the considered examples. The dataset was split randomly into a training (80 %) and test (20 %) set, with the training set being used for the descent method of the loss functionals and the test set being used to evaluate the performance of the networks.
Figure 4: Two examples from the SCAPE dataset. We use the same colors in Figure 1.
Composite (Ours)
Example
J
mj
Affine
Monolithic
Sassen et al. (2020b)
SMPL+X
80
10
3 3-26 24
0.78
0.85
0.93
SCAPE
40
6
5
0.77
0.60
0.91
Hands
12
4
2-
0.88
0.95
0.98
0.80 0.95
Faces
10
6
1 - 0.96
0.95
0.99
Table 1: Approximation quality R² on SPGA examples.
Comparison to Affine Approximation Sassen et al. (2020b) also proposed a scheme based on multilinear interpolation of precomputed exponentials for each of the factor manifolds and subse- quent affine combination of the results to approximate the exponential map and parametrize M. A natural question is how network-based approach compares to this.
8

---

Published as a conference paper at ICLR 2023
For the humanoid examples, it was not possible precompute Riemannian exponentials on a regular grid for all factor manifolds due to their high dimensionality. Hence, instead of multilinearly inter- polating precomputed exponentials we simply compute the exponentials within each factor mani- fold exactly before combining them affinely. We dub this method simply 'affine combination' and present its results in Figure 4 and Table 1; it is computationally heavy, but yields upper bound on the quality of the method by Sassen et al. (2020b). The limitation does not apply to our new ap- proach, which for example allows to learn an efficient parametrization for the SMPL+X dataset, where the expressive movements of hand and face require figher-dimensione data manifold.
In all examples, our composite network approach achieves higher approximation accuracy than the 'affine combination'. This shows that the network yf is able to correct the approximation errors of the direct sum structure. For the lower-dimensional examples, this difference not as pronounced since their sparse modes have a better support separation.
Furthermore, storing our retwork-based approximation requires less memory than the approach by Sassen et al. (2020b). For example, on the SCAPE dataset storing grids with approx. 20000 samples as reported by Sassen et al. (2020b) requires about 1.7 GB of storage, while our networks only require 0.6 GB (without optimizing for a small memory footprint).
Figure 5: Two examples from the SMPL-X dataset. We the same colors in Figure 1.
Comparison to Monolithic Network Another obvious question is whether our composite ap- proach shows any benefit over training simple, single network. For evaluation, we also trained one fully connected network T5: Rm + .M to approximate the parametrization at once, dubbed 'monolithic' approach. The corresponding approximation qualities are reported in Table 1. One sees that for the lower-dimensional examples this monolithic approach achieves an approximation quality close to the one of our composite network. However, for the higher-dimensional, humanoid examples the approximation quality of the monolithic approach is noticeably lower.
CONCLUSION
Our results suggest that the most fundamental geometric operation on Riemannian data manifolds, the parametrization of the manifold via the Riemannian exponential map, can in principle be learned. We illustrated this on shape manifolds of triangular meshes, for which the exponential map is com- putationally expensive that approximation is attractive. However, while naive implementations via deep neural networks proved7 ineffective, we achieved consistently satisfying results by match- ing our training and network architecture to a typical structure of shape manifolds: that they can be approximated by an affine sum of submanifolds. We thus learned both, the lower-dimensional Riemannian exponential map on each submanifold well as the (close affine) composition of the different submanifolds. We furthermore illustrated in our examples that such manifold structures arise from basic principles like support or scale separation of different shape variations.
While we implemented the above concept of composite networks only for shape manifolds, it should also be applicable to image manifolds. However, first the corresponding tools to identify approx- imate product manifold structures (such as the SPGA) would have to be developed for images. It also conceivable to replace the SPGA by learning approaches akin to disentanglement learning. Furthermore, we did not touch upon further optimization for the specific setting of our shape man- ifolds: The single factor manifolds typically describe localized shape variations so that a sparsity regularization of the corresponding networks would make sense and could substantially reduce the parameter size. Furthermore, one could add a regularization favoring those NRIC that correspond to an immersed mesh, thereby reducing postprocessing.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work was supported by the Deutsche Forschungsgemeinschaff (DFG, German Research Foundation) via project 211504053 - Collaborative Research Center 1060, project 212212052 "Geodesic Paths in Shape Space" (part of the NFN Geometry Simulation), and via Germany' Excellence Strategy project 390685813 Hausdorff Center for Mathematics and project 390685587
Mathematics Münster:
REFERENCES
Dragomir Anguclov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. SCAPE: shape completion and animation of people. ACM Transactions on Graphics, 24 19:408-416, 2005.
Karthik Bharath, Sebastian Kurtek, Arvind Rao, and Veerabhadran Baladandayythapani Radiologic image-based statistical shape analysis of brain tumours. Journal of the Royal Statistical Society: Series C (Applied Statistics), 67(5):1357-13788 2018.
Mario Botsch, Stephan Steinberg, Stephan Bischoff, and Leif Kobbelt. Openmesh generic and efficient polygon mesh data structure. 1st OpenSG Symposium, 2002.
Christopher Brandt, Christoph von Tycowicz, and Klaus Hildebrandt. Geometric flows of curves in shape space for processing motion of deformable objects. Computer Graphics Forum, 35(2), 2016.
Djórk-Améé Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs). In International Conference on Learning Repre- sentations, 2016.
Matthias Fey and Jan Lenssen. Fast graph representation learning with PyTorch Geometrie, In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
P. Thomas Fletcher, Conglin Lu, Stephen M. Pizer, and Sarang Joshi. Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transaction on Medical Imaging, 23(8):995-1000 2004.
Stefan Fröhlich and Mario Botsch. Example-driven deformations based on discrete shells. Computer Graphics Forum, 30(8):2246-25) 2257, 2011.
Marco Fumerr, Luca Cosmo, Simone Melzi, and Emanuele Rodolà. Learning disentangled repre- sentations via product manifold projection. In Marina Meila and Tong Zhang (cds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Ma- chine Learning Research, pp. 3530-3540, 2021.
Gaël Guennebaud, Benoît Jacob, and Others. Eigen v3. http://cigen.tuxfaailyyoo 2010.
Behrend Heeren, Martin Rumpf, Max Warderzky, and Benedikt Wirth. Time-discrete geodesics in the space of shells. Computer Graphics Forum, 31(5):1755-1764 2012.
Behrend Heeren, Martin Rumpf, Peter Schröder, Max Wardetzky, and Benedikt Wirth. Exploring the geometry of the space of shells. Computer Graphics Forum, 3355):247-256 2014.
Behrend Heeren, Chao Zhang, Martin Rumpf, and William Smith. Principal geodesic analysis in the space of discrete shells. Computer Graphics Forum, 37(5), 2018
Zhichao Huang. Junfeng Yao, Zichun Zhong, Yang Liu, and Xiaohu Guo. Sparse localized decom- position of deformation gradients. Computer Graphics Forum, 33(7):239-248, 2014.
Martin Kilian, Niloy J. Mitra, and Helmut Pottmann. Geometric modeling in shape space. ACM
Transactions on Graphics, 26(64):1-8, 2007.
Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In International
Conference Learning Representations, 2015.
10

---

Published as a conference paper at ICLR 2023
Thomas N. Kipf and Max Welling. Semi- classification with graph convolutional net- works. In International Conference on Learning Representations, 2017.
Sebastian Kurtek, Eric Klassen, Zhaohua Ding, Sandra Jacobson, Joseph B. Jacobson, Malcolm Avi- son, and Anuj Srivastava. Parameterization invariant shape comparisons of anatomical surfaces. IEEE Transactions Medical Imaging, 30(3):849- 888,, 201 la.
Sebastian Kurtek, Eric Klassen, John C. Gore, Zhaohua Ding, and Anuj Srivastava. Classification of mathematics deficiency using shape and scale analysis of 3d brain structures. In Benoit M. Dawant and David R. Haynor (eds.), Medical Imaging 2011: Image Processing, volume 7962 of SPIE Proceedings. SPIE, 201 1b.
Sebastian Kurtck, Qian Xie, Chafik Samir, and Michel Canis. Statistical model for simulation of deformable clastic endometrial tissue shapes. Neurocomputing, 173:36-41, 2016. ISSN 0925-
2312.
Hamid Laga, Sebastian Kurtek, Anuj Srivastava, and Stanley Miklaveic. Landmark-fiee free statistical analysis of the shape of plant leaves. Journal of theoretical biology, 363:41-52, 08 2014.
Michael L. Miller, Alain Trouvé, and Laurent Younes. Geodesic shooting for computational anatomy. Journal of Mathematical Imaging and Vision, 24(2):209-25 228, 2006.
Thomas Neumann, Kiran Varanasi, Stephan Wenger, Markus Wacker, Marcus Magnor, and Chris- tian Theobalt. Sparse localized deformation components. ACM Transactions on Graphics, 32(6): 1-10, 2013.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Veto,o, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, nigh-performance deep learning library. H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buu -uc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024-8035, 2019,
Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dim- itrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from single image. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10975-10985, 2019.
Xavier Pennec. Intrinsic statistics on riemannian manifolds: Basic tools for geometric measure- ments. Journal of Mathematical Imaging and Vision, 25())12771154, 2006.
Xavier Pennec. Statistical Computing on Manifolds: From Riemannian Geometry to Computational Anatomy, pp. 347-386. Springer Berlin Heidelberg, Berlin, Heidelberg. 2009.
David Pfau, Irina Higgins, Aleksandar Botev, and Sébastien Racanière. Disentangling by subspace diffusion. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florine Balcan, and Hsuan-Tiee Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Con- ference Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Martin Rumpf and Benedikt Wirth. Variational time discretization of geodesic caleulus. IMA Journal of Numerical Analysis, 35(3):1011-130) 2015.
Chafik Samir, Sebastian Kurtek, Anuj Srivastava, and Michel Canis. Elastic Shape Analysis of Cylindrical Surfaces for 3D/2D Registration in Endometrial Tissue Characterization. IEEE Trans- actions on Medical Imaging, 1335 2014.
Josua Sassen, Behrend Heeren, Klaus Hildebrandt, and Martin Rumpf. Geometric optimization using nonlinear rotation-invariant coordinates. Computer Aided Geometric Design, 77, 2020a.
Josua Sassen, Klaus Hildebrandt, and Martin Rumpf. Nonlinear Deformation Synthesis via Sparse Principal Geodesic Analysis. Computer Graphics Forum, 39(5), 2020b. ISSN 1467-8659.

---

Published as a conference paper at ICLR 2023
Philipp von Radziewsky. Elmar Eisemann, Hans-Peter Seidel, and Klaus Hildebrandt. Optimized subspaces for deformation-base shape editing and interpolation. Computers & Graphics 58: 128-138, 2016.
Guan Wang, Hamid Laga, Ning Xie, Jinyuan Jia, and Hedi Tabia. The shape space of 3d botanical tree models. ACM Transaction on Graphics, 37(1):7, 2018.
Yuanzhen Wang, Beibei Liu, and Yiying Tong. Linear surface reconstruction from discrete funda- mental forms on triangle meshes. Computer Graphics Forum, 31(8)22277-2287, 2012.
Yupan Wang, Guiqing Li, Zhichao Zeng, and Huayun He. ized decomposition. Computer Graphics Forum, 36(8):4772259 2017.
sparse local-
Yupan Wang, Guiqing Li, Huiqian Zhang, Xinyi Zou, Yuxin Liu, and Yongwei Nie. Panoman: Sparse localized components-basee based model for full human motions, ACM Transactions on Graph- ics, 40(2) 2021.
Qian Xie, Ian H. Jermyn, Sebastian Kurtek, and Anuj Srivastava. Numerical inversion of SRNFs for efficient elastic shape analysis of star-shaped objects. In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars (eds.), ECCV, volume 8693 of Lecture Notes in Computer Science, pp. 485-499, Springer, 2014.
I-Cheng Yeh, Chao-Hung Lin, Olga Sorkine, and Tong- Yee Lee. Template-basee 3d model fitting using dual-domain relaxation. IEEE Transactions on Visualization and Computer Graphics, 17 (8):1178-1199) 1190, 2011.
Laurent Younes. Shapes and Diffeomorphisms. Springer, 2010.
Li Zhang, Noah Snavely, Brian Curless, and Steven M. Seitz. Spacetime faces: High resolution capture for modeling and animation. ACM Transactions on Graphics, 23(3):548-558, 2004. ISSN 0730-0301.
Sharon Zhang, Amit Moscovich, and Amit Singer. Product manifold learning. In Arindam Baner- jee and Kenji Fukumizu (eds.), The 24th International Conference Artificial Intelligence and Statistics, AISTATS 2021, April 13-15, 2021, Virtual Event, volume 130 of Proceedings of Ma- chine Learning Research, pp. 3241-3249, PMLR, 2021.
Hui Zou, Trevor Hastie, and Robert Tibshirani. Sparse principal component analysis. Journal of Computational and Graphical Statistics, 15(2)2665-28) 2006.
12

---

Published as a conference paper at ICLR 2023
g
g1 x
T
Figure 6: Factors of the Freaky Torus. We visualize our synthetic shape space by demonstrating the effect of moving along the individual factors to the final shape. In the first row, we see how the first factor, an St, controls the deformation of the latitudinal cross-section. We show the deformed torus from the top. In the second row, of the longitudinal cross-section. Here, we show the torus cut in half to better highlight the cross- section's 's shape. In the last row, show how the third factor, two-dimensional flat torus T controls the position of a bump on the deformed torus.
how the next factor, another S controls the deformation
A FREAKY TORUS
In this appendix, we provide more details on the construction our shape space Freaky Torus of deformed tori, synthetic shape space with factors S X x T whose action is also summarized in Figure 6. We will first derive its continuous version leading to a family of parametrization and then arrive at the discrete version via simple spatial discretization. To begin, recall the parametriza- tion f: [0, 2m > R of a standard torus of revolution with radii R and r of the latitudinal and longitudinal circular cross-sections respectively, which is given by (os(()) cos(v) sin(u) cos(v) sin(v)
/cos(u)
v(u = R
sin(u) +r
The first two factors S S of our shape space control the deformation of these cross-sections into ellipses and hence want to replace the parametrizations of the cross-sectional circles by ones of appropriate ellipses. this end, a circle deformed into an ellipse with semi-axes' lengths a and rotated by n/2 against the coordinate axis is parametrized by
aa cos( cos(t ") bsin(3) 3) sin(t - 7) sin ! cos(t 2) b cos sin(t /)
T T (t)
The phase-shift in the parametrization comes from the fact that the rotation of the semi-axes is also achieved by warping of the circle instead of rotating it. We introduced this parametrization with the half-scaling of for cosmetic reasons so that later all deformations are parametrized over the same interval [0, 2T).
Now, this parametrization as replacement for the circular cross-section in the parametrization of the torus. We fix the semi-axes' lengths aR, bR and a,, by of the latitudinal and longitudinal cross-sections respectively and introduce their rotation parameters a, B [0, 2T). This leads to the parametrization
Ti a.ag.b (u)
cos(u) T (aaaabrr sin(u)= T B.ar.bo (v)
3 (u, R
ba (u)
+r
(8)
T2
Brar (v)
T2
The last factor T controls the position on a bump on such a torus. To describe the corresponding deformation, first need the normal of the surface in which direction the bump will point. It is
13

---

Published as a conference paper at ICLR 2023
given by the usual formula for smooth surfaces namely
a.f x fa n n (u, v)
f x fe (u, v).
Then the bump is a deformation in the direction of this normal around the position determined by (7,5) E [0, 2Tr² with maximal height h. To limit the support of the deformation, we use a simple Gaussian with parameter & on the distance to the center point.
We finally arrive at the parametrization of the continuous version of our synthetic shape space
F: 00 2m) x [0, 2x) x 2T²² (10, 2T)² - IR"))
+ far uu, v) + he 11 fa. (T,C)] no (7.50))))))
(10)
(,) (u.
where the images are parametrizations of embedded surfaces. To obtain discrete surfaces, we apply these to a fixed triangulation of the torus, which yields the discrete version of the shape space.
For the dataset we used in our experiments, we chose ag = a, 1, bR = b, 2. 0.375, r 0.125, h = 0.075, and E = 0.05. It is available along with the code at https: //ggtlabl...////// com/ jrsassen/ hatae//ereaaayrrraaee
B ADDITIONAL RESULTS
Exemplary SPGA modes extracted from the SCAPE dataset. We show a selection of modes generating two of the factor manifolds (indicated by the blue frames; not all modes of each factor manifold are shown as indicated by the purple dots). Note that while these deformations move many nodal positions, their support in NRIC is indeed localized. For example, the fifth mode is supported mainly in the hip region of the shape. This also links it with the other modes in the same group even though they might move a different leg.
Figure 7:
Sparsity In Section 4, we postulated several assumptions on the structure of our data manifold, which raises the question if they are actually satisfied in our experiments. Since we work with NRIC, assumption (0) is fulfilled as they are given as data on the edges of a mesh (see also Appendix D). Then, for the freaky torus example, we explicitly constructed a product man- ifold (hence assumption (1) is fulfilled) and chose the factors such that they act on different length scales, which entails the fulfillment of assumption (2). For the examples using decom- position via SPGA, we rely on it producing sparsely supported modes that can be grouped by their spatial overlap. As already Sassen et al. (2020b) observed, the resulting factorization of the data manifold fulfills our assumptions. In Figure 7, we show ex- emplary modes from the SCAPE dataset highlighting that this is indeed the case.
Figure 8: Exemplary PGA modes extracted from the SCAPE dataset showing the global support typical for such modes.
The sparsity of the SPGA modes is a result of the regularization R. in the SPGA problem (1). To still achieve a good approximation of the input data, the modes
14

---

Published as a conference paper at ICLR 2023
typically localize in different regions of the shape. This allows to group them explained before to obtain the factor manifolds that fulfill assumptions (1) and (2). In contrast, this is not the case for PGA modes, i.e. if do not any regularization there is no reason to expect localized support, which is also illustrated in Figure 8. This means our method is not applicable for such modes.
Animation One possible application of our composite network is efficient animation of shapes. In this context, we can consider shape interpolation and extrapolation problems, which correspond to the evaluation of the Riemannian logarithm and exponential map as explained in Section 3 and Appendix C. For the case of shape interpolation, we are given two shapes by their latent coordinates a(O) E R and a(1) I respectively. Then, the latent coordinates of intermediate shapes are obtained by linear interpolation, i.e. we define a(t) = t ta(1) + (1 t))(0) for E [0. 1]. By evaluating our composite network these coordinates, we obtain the approximate NRIC 2(t)
V (w (a (t))..... (a5 (t)) of these shapes, where a² (t) E mRm are the factorized coordinates as before. This leads to a smooth interpolation between shapes as demonstrated in Figure 9 and the supplementary video. Shape extrapolation can be equivalently phrased by considering linear extrapolation in the latent space R"
8 =
w5 (a (1))
(1))
(1)
(a)(())
v (a)
Figure 9: For two given shapes with latent coordinates a0)) and a(1), we compute interpolating NRIC z()) using our composite network. In the top row, we see the surfaces reconstructed from these NRIC for intermediate time steps exhibiting smooth deformations. Below, we also show the elements (a (1)) from the factor manifolds Mj which lead to the final shape by applying the combination network VSS These individual factors lead primarily deformations of the legs for 4/1 and 4/3, of the arms for dy and dss of the wrists for 122, and of the head for to.. See also the supplementary video.
observed that our composite network can also trained with smaller amounts of samples than used in subsection 5.2. For example the SCAPE dataset, if we only use 20 % of the data as training set (about 800 samples) then we still achieve an R² 0.86. Even use mere 5 % (200 samples) still reach an R² of 0.77. We observed similar behavior the SMPL+X dataset with an R² of 0.85 at 20 % training data and of 0.74 at 5 % training data.
Number Samples
Runtimes Our network- based approach enables runtime efficient approximation of the exponen- tial map. For example, on the SCAPE dataset, we used K 16 time steps to evaluate the time- discrete exponential map (see Appendix C) when generating the training samples. The computation for each such evaluation required around 8 seconds. In contrast, evaluating the networks takes about 10 milliseconds.
To render the result, we have reconstruct the nodal positions of the triangle mesh from the NRIC, for which we use the nonlinear least-squares method from Fröhlich Botsch (2011). This requires small number (e.g. 2 to 3 in Figure 9) of about 20ms each. Overall the performance is comparable to the approach by Sassen et al. (2020b), which in contrast approach is limited in the amount of latent dimensions it can handle, though.
iterations taking
15

---

Published as a conference paper at ICLR 2023
C RIEMANNIAN OPERATORS AND THEIR DISCRETIZATIONS IN BRIEF
The tangent space T.S of a manifold S in the point = E S is the vector space of all velocities a path in S can have when passing through 2. A Riemannian manifold equips each of these tangent spaces T, with an inner product (=(-,-) so that norms of velocity vectors and angles between them can be measured. The length of path 7 [0, 1] - S in the Riemannian manifold then is the time integral of the norm of its velocity 5, [[7] = Ju v (+)())((())))))))))))))))))))( (()) dt. Given two points 20, 21 E &,, the shortest connecting path 7 with (0)) Zo. 7(1 = 21 is called a geodesic, and the Riemannian distance dist( cu. 21) between both points is defined its length. It essentially an application of Jensen's ' inequality that the geodesic connecting 2y and 21 can equivalently be found by minimizing
the path energy
(,00000000000000 f de
Eh1 -
instead of [(y]. Yet another view of geodesics, following from the optimality conditions of the above minimization, is that they are paths that always go straight and at constant speed, i.e. they do not accelerate (neither do they change the motion direction, this direction; on the earth's surface, for instance, geodesics are great circles). This last viewpoint suggests associate to every v in the tangent space T,S at an arbitrary is defined as follows: Given v E T&S, just start out at z with initial velocity and continue straight (i.e. along a geodesic) for total time 1. The map which maps v to the arrival point y is the so-called Riemannian exponential map
do they change the velocity along
S a point y E S, which
exp,: : -
exp= V = y-
Via this exponential map exp, is injective, otherwise 7.S can be identified with a multiple covering of S). The inverse map the Riemannian logarithm log,: U + T,S, defined for a small enough neighborhood U of z. For a point ž E U, it tells us the initial velocity log, ž of the geodesic from to 21.
identify the tangent space T_S with the manifold S (at least if
Riemannian geodesics, logarithms and exponentials are expensive to approximate computationally. The computation typically has to be performed in charts or local parametrizations of the manifold, i.e. S is identified with (an open subset of) R and the Riemannian metric with a symmetric positive definite matrix G² B=*n depending on E R A variational discretization by Rumpf & Wirth (2015) approximates continuous paths y by time-discrete paths (7o. ...KK) to be interpreted as polygonal paths in R² with vertices "u..... 7K at times 0 approximated by a discrete path energy
K. The path energy then K K: K
K
F(( 10......... Tk) = K
Σ k=1
W (11611176)
where W22,, 21) is second order accurate approximation to the squared Riemannian distance dist?(zp, 21) (for instance, W(20, 21) (en -0,0-2,700 T G2. - 21 )). E may be viewed as a Riemann sum approximation of the integral E. Minimizing under fixed end points then yields a discrete K K-geodesic, i.e. discrete approximation (7o...., . to geodesic between "o and 7x- The initial velocity K(11 7ro of this polygonal path is the discrete approximation of the Riemannian logarithm log,. 7K. The discretization of the Riemannian exponential works the other way round: Given a velocity vector v E IR" we approximate expx,, V E R² by that point УKK E R" such that the discrete K K-geodesic (10.... 7k) ) has initial velocity K(11 -%) = v. This point can be found by a time stepping procedure one first sets 71 YYo + g and then iteratively computes "2, 73...... as follows: Since each triplet - 7/: 7k+1 of a discrete K K-geodesic forms a discrete 3-geodesic (much like any subsegment of continuous geodesic is itself again geodesic), 7 must minimize Whe--, 7k] + W [Tk, 1] The corresponding nonlinear optimality condition
Whw-h mi + 3 W[7k, 78+1]
is then solved via Newton's 's method for 7h+11 given 7 and 7k.
Instead of working on charts, one can also consider an implicitly defined manifold M {2 R. ((2) = 0}. for suitable smooth functions Q: R" + R". In this case, one proceeds analo- gously constraiiing the search for points on the manifold via a Lagrangian approach.
16

---

Published as a conference paper at ICLR 2023
D A RECAP OF NONLINEAR ROTATION- COORDINATES
Let us briefly review the tools introduced by Wang et al. (2012) for a discrete version of the funda- mental theorem of surfaces. Consider simply connected, triangular surface with the of vertices V. edges& & C V x V. and faces F C V V x V. For given vector of vertex positions X RRM we introduce the vector of all edge lengths ((X) (le()))e and the vector of all dihedral angles ((X)= = (0. (eeE. As discussed by Wang et al. (2012), ensure that the edge length and dihe- dral angle data (1. 0) E TR2 actually corresponds triangular surface immersed in R two admissibility conditions have to be fulfilled.
The obvious first condition is the triangle inequality for the edge lengths on all triangles. We write this condition in formulas as (T) > O for all I E F where Ty(() (4+11- 1g. 4 11 + 1x -4 + 1x) for a face f E F with edge lengths 4, lj. lg. and the above inequality is meant componentwise. Fulfillment of these conditions guarantees that we can construct individual triangles from given edge lengths. However, we need a second condition assur- ing that these triangles fit together with the given dihedral angles to form surface, i.e. to guarantee integrability of z. For simply-connected discrete surfaces, this can be broken down to individual conditions for the fans of triangles surrounding any vertex v in the set of interior vertices Vo. For- mally, we express this individual condition as (z) 0, which guarantees that we can construct the geometry of this fan from 2. The explicit formula for this was introduced by Wang et al. (2012). If this condition is fulfilled for all interior vertices, one can show that it is indeed possible to construct the geometry of the entire surface. Sassen al. (2020a) demonstrated that 2n and its derivatives can be robustly and efficiently computed using quaternions. The conditions can be extended to higher- genus surfaces by including integrability conditions along non-contractible paths that generate the fundamental group on the triangular surface, but this is not used here.
+1
The manifold of all z E 2 corresponding to immersed triangular surfaces of the given mesh connectivity can be given by
M {= E R T(z) 0, 2(2) =0.
where we collect all constraints in vector-valued functionals T= (Tr) and Q = ve.) As Sassen al. (2020a), we call the manifold M the NRIC manifold (Nonlinear Rotation-Invariant Coordinates). The differential structure of this manifold is at first described in terms of the tangent space, which is given at position E . by T&M = ker DQ(=) == {w E TR DQ(=) 0}- Here the matrix (=) E IR3 Val x2|8| is the Jacobian of Q. The triangle inequalities define an open set of 2 and are thus not needed to define the tangent space.
The advantage of NRIC is that they allow local description of shell deformations based on the local variation of the edge lengths, which encodes membrane distortions, and the local variation of the di- hedral angles, which encodes bending distortions. We use Riemannian metric g the manifold M that reflects the physical dissipation caused by these infinitesimal variations of the discrete surface. In detail, the metric coincides with the Hessian of an elastic energy W, i.e. 9= R2 x 2 R with 9a = 1/ Hess [(=, restricted to T&M x T.M. We use an elastic energy describing de- formation from configuration to a configuration ă that decomposes into a membrane energy and a bending energy, i.e. W(2, 2] W [2, ž Weenl [2, 22. The bending energy is given by Woenl (2, 2] (0e 2d 2.-dd112 122 where de 3 (a, + a..) for the two faces f and f' adja- cent to E E (a, is the area of f) Furthermore, the membrane energy is given by W Wmeml [,mul=, E| Esero, ay W Wmem(GG/ (G[2, E) where W (A) = fr A A+ 2 det A (++ }) det A μ 3 The constants μt and À are positive material constants, and G[z, ž] denotes the Cauchy- tensor of the deformation - describing the face-wise distortion as a function of the edge lengths on each face. Let us remark that the logarithmic term in the energy density W mem acts as a barrier ensuring the triangle inequalities for finite-energy deformations.
strain
17

---

Published as a conference paper at ICLR 2023
E
LIST OF SYMBOLS
Riemannian Shape Spaces (see also Appendix C)
Shape space, p. 3
Point of a shape space, p.
T.8
Tangent space of 8 at point p. 3
9=
Riemannian metric at point
log,
Riemannian logarithmic map at point 2,
exp=
Riemannian exponential map at point 2, p.
Principal Geodesic Analysis
Riemannian center of mass, p.
R
Regularization functional, p. 3
Triangle Meshes and NRIC (see also Appendix D)
Vertices of triangular surface, 3
Edges of a triangular surface, p. 3
F
Triangles triangular surface, p.
((e)
Neighbors of edge in triangle mesh, p.
X
Vector storing the vertex positions of all vertices of triangular surface, p.
1
Vector of edge lengths, 3
4.
Length edge e, p. 3
Vector of dihedral angles, p.
0.
Dihedral angle triangles meeting at edge e, p. 3
11-1 Weighted LP-norm for NRIC with reference shape p.
Neural Networks
N
Layer sizes of a network, 4
Number of layers of a network, Nonlinear activation function, p.
P
<
Network parameters, p.
W
Matrix of learnable weights, 4
g
Learnable bias, p. 4
J,J,
Loss functions used for training, p. 6
45
Learnt parametrization of the factor manifolds, p.
Learnt map combining the outputs of the v5, p. 5
Data Manifolds
Riemannian data manifold, p. 4
M; Factors of product manifold, p. 5
Parametrization of data manifold, 4
18


Reviews:
[{"review_id": "xTBe2pHzBM", "reviewer": "Reviewer_SjVS", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper explores a learning-based approach for computing exponential maps for shape spaces modeled as Riemannian manifolds. Previous works have shown that statistical shape spaces represented as meshes can be modeled effectively using the NRIC  (Non-Rigid Invariant Coordinates)  system, allowing for operations like Principal Geodesic Analysis that encapsulate the dominant factors of variation in the corpus of shapes. However, the more meaningful tasks like shape interpolation require a computation of the exponential map which is arguably costly. \n\nThe authors propose a framework for computing the exponential map of specific shape manifolds - namely those that can be factored and written as product manifolds of much smaller dimensions (as required by conditions 0,1,2). The architecture is decomposed to look structurally identical to an affine sum of exponential maps of the submanifolds. By training the composite network to follow the idealistic exponential map in the training dataset, the authors demonstrate that there is merit in their approach, and show R2 confidence scores compared with a straightforward network and the direct axiomatic competitor (Sassen2020b). Overall the perception is that the network generates impressive models of shape variation at a (possibly?) much lower memory and compute cost. \n", "summary_of_the_review": "Overall I find this to be a very interesting work, mainly for the novelty of the idea and clarity of discussion. However, the results seem to give an impression of “proof of concept” rather than convincing practical use cases. Put together I am weighing for a weak accept.  ", "strength_and_weaknesses": "Strengths \n\n- I find the problem setup very original, novel, and interesting. The arguments that motivate the use of neural networks to model computationally expensive exponential maps are very refreshing \n- Overall - the writing of this paper is fantastic despite some missing necessary details (see weaknesses) and occasional lack of background\n\nWeaknesses, and Questions \n\n- It is unclear what the role of the sparsity is in the process? Can it be said that the proposed approach cannot work for shape spaces having factors of variation that are not necessarily spatially localized? If so this seems restrictive. To ask a more direct question - does the proposed approach only work for some conceptually similar version of the freaky torus? \n- I find the background on NRIC coordinates to be very short. To that aid, I think it is wise to provide a little more background on the time-discretization by Rumpf&Wirth (2015) for the generation of the exponential and logarithmic maps and hence to have at least some insight as to why that is a computationally expensive affair. \n- On the practical side, it would be much more convincing to see interpolation/smoothly varying factors like in Fig 6 for more practical manifolds shown earlier. \n- Figures 1,3 and 4 merit a better explanation. How does a more uniform density of points be indicative of a better model fit? And it is also important I think to also see the output of Sassen2020b for these figures (probably mention runtimes as well). In Table 1 what is the methodical difference between column 5 (Affine) and column 8 (Sassen 2020b) ?\n- There is very little evaluation reported on the computational gains of the proposed method \n- (suggestion) It would be interesting to explore some way to measure the generalization of the proposed approach, especially as a function of the size of training data. How does the quality of the fit change as I progressively increase the training samples from say 100 to 1000? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "snD_AdZbvt", "reviewer": "Reviewer_ADhs", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper seeks to approximate a given high-dimensional manifold with a product of some smaller, simpler manifolds. It learns this approximate mapping using neural network based optimization. The paper motivates this approach by discussing the concept of PGA (which is essentially Tangent PCA as stated here) and a sparse Tangent PCA. The idea is to approximate the range space of exponential map from the dominant tangent subspace by a composition of some smaller maps \\psi_l. Both the smaller maps and their composition \\Psi are learnt from the data using fully connected CNNs. The paper provides some experimental validation using shape space of surfaces. It generates training data using space PGA or TPCA and exponential maps of the individual components. It then compares the quality of approximation with some other methods. ", "summary_of_the_review": "\nTo the extent I understand the presentation, the paper is trying to find efficient ways to explore complex manifolds. The approach is ambitious (learn the overall exponential map using a CNN). However, it is not so clear when the approach will be work and when it will not. It would have been useful to take some simpler intuitive examples where the approach succeeds and fails, and discuss those cases. Then, move on to more complicated shape manifolds. The validity assumptions need to be discussed further. ", "strength_and_weaknesses": "Strengths\n\nIf the idea and approximation hold, then there is an efficient way to parameterize elements of a complex manifold. \n\nWeakness: \n\nThere are a number of questions that arise from the paper. Perhaps it is my lack of understanding of the paper. \n\n--I am not quite sure when the approximation is feasible and meaningful? Especially given how the training data is generated. I get the impression that the paper is trying to approximate: \nExp_z(v1 + v2 + …+ vn) \nBy the quantity\nPsi(Exp_z(v1) \\times Exp_z(v2) \\times …. \\times Exp_z(vn)). \nExcept that the latter factors are learnt from neural networks. \nIf this is correct, perhaps there is a simpler way to state the problem. \n\n--- Can the authors provide an analytical nontrivial example where the factorization along principal directions is feasible? \n\n-- I assume that in most interesting manifolds this factorization is not valid but perhaps one can make a good approximation. Can one always do that? When should we expect the approximation to be good? This brings us to the three assumptions. \n\n-- The paper states several assumptions which seem plausible but not verified in the experimental setup. In the set up – “We assume that the manifold can be smoothly approximated by a product of much lower dimensional manifolds….” When does this work and not work?\n\n-- For the shape manifolds studied here, do these assumptions hold? How can we verify them? For instance, the statement that “The different factor manifolds correspond to different spatial regions”. Is this forced or is it a fortuitous outcome of SPGA? Also, the local shape variations are claimed to be “independent” of global variations. Again, why? \n\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Ib5w8hjr6jJ", "reviewer": "Reviewer_VcAe", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper works on learning the exponential map of a shape space using deep neural networks.  The proposed method adopts a product structure to approximate shape manifolds using a sum of multiple low-dimensional submanifolds. The method is evaluated on synthetic data and manifolds extracted from data using sparse principal geodesic analysis. ", "summary_of_the_review": "The experimental results look good; however, the current shape of this paper is not ready for publishing. More efforts are desired to make this paper easy to understand and use by other researchers. ", "strength_and_weaknesses": "Strength:\n\n+) This paper targets an interesting and challenging problem and the proposed method seems to make sense. \n\n+) The qualitative and quantitative results demonstrate the effectiveness of the proposed method. \n\nWeaknesses:\n\n-) Missing a good literature review.  The related work should provide a summary of existing works that tackle the same or similar problem; while the related work in this paper looks like a background. A missing discussion on existing methods makes it unclear where this paper stands in the literature. \n\n-) The equations in this paper are not clearly explained, which brings difficulty in understanding the details of this paper.  For example, in Eq. 1, what is \\bar{z}? Is it the mean of a set of shapes? If so, how to calculate it? The same question for the defintion of R2 score. Constructing an atlas for shapes is also non-trivial. In Eq. 3, what is l_e^b, l_e^a, etc.? In Eq.5, what is Z_e^0? It is unclear what is the input and output of the neural networks. The readers have to guess, perhaps explicitly pointing this out would make the paper more understandable and readable. \n\n-) In the loss function J_l(\\theta), what is exp operator? Is \"a\" a vector? then what is S^l? a set? but it says \"a random sample\"? In J(\\theta), the \\sum_\\omega_i(a^i) is the sum of tangent vector. Should it be the composition of several exp operators? When a point moves on manifolds, it goes step by step and each is a small step to make sure it stays on manifolds. How to understand the definition of J(\\theta)? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%