Learning Riemannian metric for disease progression modeling
Authors: Samuel Gruffaz
Pierre-Emmanuel Poulet
Etienne Maheux
Bruno Michel Jedynak
Stanley Durrleman
Venue: NeurIPS 2021 Poster (2021)
Keywords: Riemannian Geometry
RKHS
mixed-effect model
Disease progression modelling
Longitudinal data

Abstract:  Linear mixed-effect models provide a natural baseline for estimating disease progression using longitudinal data. They provide interpretable models at the cost of modeling assumptions on the progression profiles and their variability across subjects. A significant improvement is to embed the data in a Riemannian manifold and learn patient-specific trajectories distributed around a central geodesic. A few interpretable parameters characterize subject trajectories at the cost of a prior choice of the metric, which determines the shape of the trajectories. We extend this approach by learning the metric from the data allowing more flexibility while keeping the interpretability. Specifically, we learn the metric as the push-forward of the Euclidean metric by a diffeomorphism. This diffeomorphism is estimated iteratively as the composition of radial basis functions belonging to a reproducible kernel Hilbert space. The metric update allows us to improve the forecasting of imaging and clinical biomarkers in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) cohort. Our results compare favorably to the 56 methods benchmarked in the TADPOLE challenge.

Learning Riemannian metric for disease progression modeling
Samuel Gruffaz
Pimrne-Emmanuel Poulet
Inria Paris
Paris Brain Institute
Ecole normale supérieure Paclay
Inria Paris
samuel. -ww.ffareens--prri--aaaalaaeeeee sacis-saalaall.l......a fr
pierre-enmanuel
poulet@inria.ff. hwwee--mmmaeelpooleeeenrr.......
Etienne Maheux
Bruno Jedynak
Paris Brain Institute
Maseeh Professor Mathematical Sciences
Inria Paris
Fariborz Maseeh Hall, room 464P
etienne. waahuux@icmminstiieee
Portland, OR 97201
bruno jedynak@dddxxxxx.omo edu
Stanley Durrleman'
Paris Brain Institute
for the Alzheimer's Disease Neuroimaging Initiative
Inria Paris
Inserm, CNRS, Sorbonne University
stanley.o durrleman@inria..
Abstract
Linear mixed-effect models provide a natural baseline for estimating disease pro- gression using longitudinal data. They provide interpretable models the cost modeling assumptions on the progression profiles and their variability across subjects. A significant improvement is to embed the data in a Riemannian manifold and learn matient-specife trajectories distributed around central geodesic. A few interpretable parameters characterize subject trajectories at the cost of a prior choice of the metric, which determines the shape of the trajectories. We extend this approach by learning the metric from the data allowing more flexibility while keeping the interpretability. Specifically, we learn the metric as the push-forward of the Fuclidean metric by diffeomorphism. This diffeomorphism is estimated iteratively as the composition of radial basis functions belonging to a reproducible kernel Hilbert space. The metric update allows us to improve the forecasting of imaging and clinical biomarkers in the Alzheimer's Disease Neuroimaging Initia- tive (ADNI) cohort. Our results compare favorably to the 56 methods benchmarked in the TADPOLE challenge.
"his work partially supported by NIH RO1AG027 and R01EY032284,
This research has also received funding from the program "Investissements l'avenir" ANR-10-IAIHU-06
This work was also funded in part by the French government under management of Agence Nationale de la Recherche as part of the "Investissements d' l'avenni" program, reference 19P-19--31 (PRAIRIE 31A Institute) and by ANR under the joint programme in neurodegenerative diseases (JPND) ANR-19-JPW2-000 (E-DADS).
*Data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNT) database fadni.loni.uss euus.. As such, the investigators within the ADNI contributed design and implementation
ADNI and/or provided data but did not participate analysis or writ- ing this report. A complete listing of ADNI investigators can be found at: http://addi loni. lssi.usc.edu/wpp conten/uploadsshow...o to to_appll/AADaI Attp:/AADDDAoooooweeeeeeeee
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Introduction
Understanding the progression of diseases is essential to accurately monitor, diagnose and predict patients' state of health. Disease progression modeling analyses longitudinal data to capture common and subject-specific progression patterns. Longitudinal data analysis has usually been addressed in the framework of parametric mixed-effects models []. The population parameters express the characteristics of the disease, and the individual parameters encode the specificities each patient. represents the diversity of pathology. The challenge is to construct models flexible enough to learn the disease heterogeneity across the population and sufficiently interpretable to derive practical conclusion.
To understand the trade-off between flexibility and interpretability, we review these models through the prism of time and space variability of patients' trajectories. We refer to time variability when all the patients follow the same trajectory but not at the same speed and not with the same onset and to space variability when each patient follows a different trajectory but with the same speed and the same onset. For instance, to observe the different stages of a disease, event event-based models have been introduced 22) the timeline of disease progression is seen as succession of stages followed by each patient but not same order. Time variability is encoded into this subject-specific ordering. Extension to mixture model includes space variability as set of possible trajectories 33. Increasing the number of stages and mixture components improves both the space and time granularity but will result in a lack of interpretability of cach stage.
To express more finely the space granularity, we can assume the patients' observations follow continuous parametric curve, the dataset providing discrete, noisy observations along this curve.
1 6). These curves can be defined with mechanistic approaches using Ordinary Differential Equations (ODE), the sources of variability (time and space) across patient being in the Cauchy conditions (to and X (fn)) and the common pattern of progression in the vector field as derivative 5 This family of models is interesting to study the correlation between time and space variability (4) but it is double-edge sword since these two factors cannot be analysed independently. Probabilistic approaches to represent parametric curves can solve this point. Gaussian Processes (GP) have been applicd for disease progression successfully 7 to study correlation in the pathological progression while keeping time and space variability separated. Their semi comi-parametric paradigm allows for significant flexibility by encoding the apace-variability in the Gaussian kernel, but this last part is not easy interpret. More generally in Bayesian models, it is difficult disentangle the different random effects to obtain meaningful parameters. To address this point, geometric approaches have proved to be efficient for scalar biomarkers 6 and brain shapes [8]
One instance of this family of geometric models 66 assumes that each subject follows a curve on Riemannian manifold which is a translation from a common geodesic (Disease Course Mapping, DCM). The space variability is encoded in the translation and the time variability by a time reparametrization. Though the Riemannian formalism provides interesting tools for modeling trajectories, it is usually constrained to well-studied manifolds with simple geodesics such as sigmoid curves for biomarkers or straight lines for cortical atrophy. In order to increase the model flexibility, some contributors propose to learn the metric by playing on the representation of trajectories [9] They rely porh-forward method to learn metric with seeudo-diffemoong 10]. Observations' trajectories are seen as transformations of straight lines evolving in a latent space learned by an auto-encoder. Though this semi- parmmeeraettii representation enables multiple progression profiles, it is inappropriate for disentangling space and time variability.
transformations
Another method for trajectory learning is to use the theory of shape analysis and more especially the deformation of shapes in time. Authors commonly learn the deformation as a diffco- morphism with Reproducing Kernel Hilbert Space (RKHS), the kernel regularity encoding the smoothness of the deformation through time [8] [T]. We believe that this deformation method is an interesting alternative to the the auto-encoders to increase model flexibility while keeping control on space and time variability.
In this work, following 9 but taking step back, we propose semi semi-parametie parametric method using a RKHS to learn the Riemannian metric of DCM models. We thereby retain the possibility to learn the inter-variability of patient trajectories thanks to the mixed-effect framework, keeping the advantages of the geometric and Bayesian approach presented in [2]. First, we recall the structure of the mixed-effect model in section 2.1 then we present the method to learn the metrics step by

---

step in section 2.2 We validate the presented method on synthetic data in 3.1 and on a real dataset
(Adpoe 3.2 by comparing it with previous models on the task of predicting patient's biomarker progression. Finally, we discuss limitations and possible future works in the discussion section 4
1.1
Related works
Recurrent neural networks have been proposed to predict a sequence of biomarker values. Such methods, however, need cope with missing values, limited and variable number of observations per patient, a variable time interval between observations, and a very variable pattern of change depending disease stage and subject.
Mixed-effect models make several assumptions about the pattern of progression. In particular, they assume that the progression falls within parametric family of curves of small dimension, c.g. linear curve or a logistic curve with unknown parameters. This is the case for instance of the DCM approach where the linear or logistic curve seen as the solution of an Hamiltonian flow (a geodesic a particular case of an Hamiltonian flow) with given Hamiltonian function. Our main contribution to learn the metric of the Riemannian manifold, and therefore the Hamiltonian, to better learn the dynamics of progression of the biomarker. In this respect, our contribution relates to the techniques of Hamiltonian and normalizing flows that are becoming popular in the machine learning community, see e.g. 13 or
Methodology
All proofs are given in the Supplementary material A. Notations in all the following are: |.| for the usual Euclidean norm, (..-) the Euclidean scalar product, ||| suPze(2<)> |=| for all linear function L and III d fl sup,, |l| fx)]]]] for all differentiable function f.
L(=)]
2.1 Riemannian geometric setting of the mixed-effect model.
In this part, present the geometrical construction of the mixed-effect models [6] with further explanations Supp. material A. Let M be an open subset of IR (d is the dimension of the observations) equipped with the Riemannian metric g. We make now explicit the dependency on the metric g that will be optimized.
The mixed-effect model. Based on 12] observations are seen as point in Riemannian manifold MI, denoted Vi.j corresponding to the data of the i-th subject at its j-th visit at time 4.j- The points are seen as noisy samples along an individual trajectory 7 namely a curve on the manifold, which result of random spatiotemporal transformation of reference geodesic g on the manifold. The general formula states:
Vi.j ny. (38) (1a(()) telgs eig ~N i. a>0
(1)
=x (4.5)
1/s :t t- at(t to Ti) - to is a time-reparametriiing function, with to being a reference time for the population. The individual variability in time is captured by the two subject- specific parameters in the time-shift т; and the acceleration factor a;, The time-shift т; represents the delay at onset relative to for the individual accounting for early late onset. The ag models the speed at which the trajectory of individual is traveled, thus accounting for fast or slow progressors of the disease.
n : Expy.. (vo), the population average trajectory. is the geodesic passing at point Po with velocity up at time lo. It represents the fixed effects of the mixed-effects model. This supposedly summarizes the characteristic progression of the disease at hand.
nl, (7o) s - Exp" (Pa (P(.o... (w:)) is the exp paaplaaaleliisatoon of the geodesic %o in the subject-specific direction w; called space-shift. We denote p2 ( (w,) the parallel transport of the vector w; along the curve To from 70(ta) (o(s). The notion of exp- parallelisation generates the spatial inter-varts variabiiti among the population (disentangled

---

Normalized biomarker value 2
1.0
Myy (ro.c) c)
0.8
pa.
0.6
Y1
No (ro.))
Po(t)
7,le)) = Po Yo(tg)
0.4
0.2
0.00
0.2
0.6
1.0
Normalized biomarker value
Figure 1: a) sketch of ssppparalellisation in abstract manifold. b) exp-parallelisation for the unit square as manifold with the metric ((F1, (2), (2)) == p + 2222
21th
The
12(1-p2)2
average trajectory in red and an individual trajectory in blue or purple.
from the time- variability encoded in 1/s), it is depicted in T and further developed in the Supp. material A. The space-shift encodes the specificity of the individual biomarkers' evolution, to help distinguish between different patterns of disease progression.
In the mixed-effects model framework, we identify the random effects as the individual parameters (wa, Tı, az)) for the patient i. They induce spatiotemporal transformation of the average trajectory parametrized by the fixed effects On = (po: to, lo). To ensure unique decomposition of the temporal and spatial effects, the wy are chosen orthogonal to vo in the tangent space at po- Normal distributions are chosen as priors for T2,00; and &, with a exp(E.). All the model effects can be estimated by maximizing the likelihood with the algorithm CCCCC-SAMM [15 providing the convergence guarantces (see [12] for the associated statistical specification).
The metric. Traditionally, the metric g is selected using empirical arguments. In contrast, propose to estimate g from data, thus relieving from a potentially tricky modeling task. This last parameter is the most important since the whole model depends it: it encodes the shape of geodesics with Exp" and the variability due to exp-paralletisation with 179. In [1), (6], the authors choose (M. 9) as manifold product of dimension- manifold have closed form for Exp" and 178 For example,
choose M = Rd and g (.,-) recover straight lines geodesics or M - (0, 1)d and 9p (Jp(), with [n(v) = (p.(1-p.)): ): recover logistic curves as geodesics. Product metrics assume that the geodesics coordinates true when they describe neurodegenerative bio-markers progression [7]] Moreover, due to the product-metric, the coordinates follow geodesics which enforces bijectivity and thus monotonicity of the trajectories, Thus, we assume that the noise effect fi.j in (T) is in reality different from the real data noise 2,3 with e, the error due to the modelling. The goal is to minimize ef.--
independent which is not necessarily
noise because of the miss-specification of the metric g. in such a way that Ei,5 = i
noise
To state the problem, we suppose that the population and individual parameters (6o. (2,)4) estimated and want minimize the squared reconstruction error [.(g) according g:
L(g) Σ 12
Σ
+
noise
Iuus - (tig))
To prevent the risk of overfitting since noise is present in I., we can add regularizing term or constrain the set of admissible metrics g. Next, we describe a method to solve this optimization problem.
2.2 Learning the metric.
The task of learning a Riemannian metric has already been addressed in the literature 10 and most of the time, it is reduced to the task of learning a diffeomorphism using the following proposition (method coming from [10]:
9
18

---

Proposition 1 (Pushforward metrics), Provided (M. g) Riemannian space, N a manifold and Φ + a Ct diffeomorphism, we can equip N with the Riemannian metric g defined as:
pp E N, Vw, v E TNp, (w, u) 90- (d Φ- (p)d) / (p).o)
Moreover, & is an isometry, which implies that for all (m.v) in TM and (-1,1) 7 M differentiable curve:
Edp o Exp". (v), (d dé(()(s)) 7.
Thanks to this property. we can start from an initial metric g in M. and reach wide variety of Riemannian metrics by considering (+(M), ) with di in F a set of C'-diiffeomorhhism Moreover, the ap-paraaletiiation 178 can be expressed in closed form as soon as and n/ are expressed in closed form as well. Nevertheless, it restricts the possibilities because the curvature is preserved. For example if g is the Euclidean metric and M = Rd, the space (o(M), go) will be always flat (curvature equals zero) since (MI, g) is flat.
Learning diffeomorphism a more common task than learning a metric especially in the field of shape analysis with the LDDMM algorithm [1I] [8] and even in deep learning with the invertible networks [19] for different applications 9 20 22 Inspired from the LDDMM framework, we
chose to use the following method to construct the set of C' -diffeomorphism F: Proposition
db i. with f bounded function in C2 (R3, Rd) such that ||ld (HI), then & is a C² diffeomorphism.
In all the following we choose M = Rd which is the most flexible choice. For example if we want subsequently work on J0. 1f, consider ø' = 6o with o 1+exp(-t)* To respect the condition of the previous proposition, we choose to take f in an RKHS denoted H to encode its regularity in its norm. Moreover, the RKHS functions handy in non-parametric optimization thank to the representer theorem (for a comprehensive review kernel methods (22). We derived the following lemma fit the constraints:
Lemma 1. If is kernel such that k(x, y) g(x y) Id with g E CRR ,R) and h(x, y) = g(x yy) is bounded kernel, then ff E H s.1 [|fll Vivata
(IRRRR)
|l|d fllbso < 1 and f is bounded (H1).
Examples:
The Gaussien kernel; g(x) == exp( 2 o
Vd
The Sobolev kernel or generalized T-student kernel, g(z) = (1+
0 > 0,0 > d,
avd
In all the following we choose a kernel respecting the previous conditions such that for all f in the closed ball Bx c) (c positive constant), (HI) is verified. Note that we can prevent overfitting thanks to this constraint. Now, we are equipped to address the optimization problem (
Lemma 2. If the population and random effects = (0o, (z:)) are estimated, the minimization of the error of reconstruction ( can be performed by solving the following problem:
f" argminneBB((..cc L(f), L(f) = Σ lys.5 ma(tes) - f((r(tts)))
(3)
1.j
Thanks to the representer theorem, we have:
Yx E (*)) = Σ
k k(mass JJWs,, = (1..).))
6,j
With this parametric formfor f*, the previous problem is equivalent to:
mènémize W
- KxWWI
(4a)
subject to WF S Co.
(4b)
where the previous variables are vectorized:
W = (wis)iss U (yi.s m((ts))sss Kx 2k)),.)))
5

---

Algorithm 1 Geodesics Bending (Alternating maximization algorithm)
Require:
Gmits Imii N Nevmpp ki Ne. RMCMC
- Pimits Φ <- id: <- mmit
for 1 = to Ncomp do
Run the MCMC-SAEM for RMCMC iterations with metric & to estimate 0j
0 <- O₁.
Solve the optimization problem (4a) with parameters 0 to estimate /7
Φ - (fd+ Φ
g 3 9
end for return 0,g
The minimization of a quadratic function under constraints (4a can be solved numerically within a reasonable time soon as the matrix Kx is not in high dimensions. The total number of visits in a medical cohort can reach up to dozens of thousands and the points vi,j are close to each other, which results in the matrix being ill-conditioned. To overcome these obstacles, we assume that Vi.j E [0.1] and we create a grid of control points in [0, 1] with step-size a if a is the kernel variance (there exists other methods [23]. but this one is more close the LDDMM framework (T]) and gives satisfying results (24)). We only keep the grid points which are a-close to the observations, thus removing the useless ones. We note N₂ the number of control points, which can increase exponentially with the dimension according to the previous construction: Na(a)) R 1 If d is too ga. large, we can choose another method to obtain a reasonable number of control points, a simple way to do so is to subsample the points (zin) [23]. In all the following, the grid method is used because of its practical upsides (no randomness, no risk of ill-conditioning the matrix Kx).
Now, we have method to improve the choice of the metric when keeping the mixed- effect parameters 9 fixed. Then we need to both optimize the metric g and the model parameters 0 = (fo. (z4)). We choose to maximize the likelihood with algorithm 1. As we know how to optimize g when 0 is fixed and reciprocally, we implement an alternating scheme of optimization. We start with simple metric Iini (usually the Euclidean metric or the metric resulting from the push-forward of the logit), and we estimate the mixed-effect parameters 0 with the MCMC-SAEM algorithm [6) Then, with 0 fixed, we estimate the metric by numerically solving the optimization problem (4a) with the chosen kernel and a chosen number of control points N.. Each iteration is a repetition of these two steps. Each step with a metric optimization amounts to composition of the previously estimated diffeomorphism é with the newly estimated diffeomorphism. The number of iterations thus amounts to the number of compositions Ncomp- The hyperparameters k Neomp and Ne are tuned to make compromise between loss minimization, computation time and number of parameters.
For Neomp compositions, we have g 9 Nouup with D @Neup .... Φ1 where di id + En= resembles the deep neural networks considering N, as the width and Ncomp as the depth. The total time complexity is OnMmcMd comp) which reduces in practice the choice of Neomp (see Supp. material B to more details on these considerations). We called this method Geodesics Bending (GB).
No -wuj, (ar)), the Ne control points and (wf)) their associated weights. This structure
3 Experiments
All the methods are developed in Python by extending the open-source Leaspy library hhtps://leasyyrradttecooss created for DCM models and run on a 2.80GHz CPU with 16 GB RAM. All code will be available on Github in the near future.
3.1 Synthetic data
In this part, study the learning capacity of the method and its stability synthetic data depending on the values of the depth Neomp. the width (o) and the initial metric Jinit- We study the algorithm in the context of neurodegenerative diseases. The parameters are selected to be realistic.
In all the following, to perform the metrie estimation, we choose i to be the Gaussian kernel (experiments with the Sobolev kernel gives similar results), RMCMC = 200 (лмMмC 10000 at the

---

Convergence
Stabilty
Reference
step
Step
Stop
Second coordinate
0.6
Step
Stop
Second courdinate
0.8
Step
Step
Stop 12
Step
Last step 15
0.5
0.4
0.3
0.0
68
70
72
Alzelhmmer ago
Alzeihmer age
Figure 2: (a) The average trajectory on the second coordinate is displayed for each value of Noomp in the Convergence experiment. (b) The average trajectory on the second coordinate is displayed for each value of comp in the Stability experiment.
very first step since the mixed-effects are not necessarily well initialized whereas for subsequent iterations the MCMC-SAEM starts closer to the optimum), 0m is computed empirically from patients features 25 leaves Neump: Ne(o) and Jan to select according the situation.
Data generation Once we have selected the number of patients Npat and the observations' dimension d, we generate data according to the DCM model (T) selecting the fixed ef- fects tu, Up. Po: Igen o7, og. Tuiser and sampling the random effects parameters such that ~ NN0, of),
((0,06) G.j ~ ~N(0. ) principal directions of variation for space-shifts (Am). s!" NN (), uc Σm s. Am and time parameters ot, Dinin nă generate the patient time- toee-points (4,,5 ISSTT randomly: T, n ō,, 6, B(ns, 2), 4.5 (No,, oe). Vh. - 7 (Li,) + Ei,y For both experiments, we set pa 500, d 2, n 2, n 6, oe 4.
Convergence First, we experiment whether learning the metrics with kernels enables the regression of exotic trajectories. We choose Onoise 0.01, Igen 9 with Φ(x) = exp(0.07(c: sin( 1) and = (-.-) for the data generation and No(a = 0.08) ~ 90 for the metric estimation (more details in Supp. material C).
Increasing the number of compositions improves the capacity to recover the true average trajectory
in figure 2 The estimated noise variance oh decreases linearly and plateau around the true value of noise o noise = 0.013 which shows that the model is a little under-fitting, The kernel regularization impacts the smoothness of the function, making it harder to fit the fast oscillations. By testing different values of the width (o)... we observed that the lower the kernel variance a, the higher the risk to overfit. Conversely, the higher the higher the risk to underfit. As for deep neural networks, there an optimum to find regarding the hyper-parameters. In practise with datasets neurodegenerative disease, normalized data very noisy moise 0.05. To avoid over-fitting, we select the kernel variance o in [0.1.0.5]. The choice of Jinit influences the number of compositions required reach good approximation of the "true" metric.
Stability Secondly, we experiment whether the method is stable: beginning from the metric which has generated the data with Gimit, observe the effect of supernumerary compositions. We choose Igen عاوي with sig(æ) : T- and nooss = 0.05 for the data generation, Jinit. = S and
No 0.2) s 31 for the metric estimation.
The compositions on produce local fluctuation on the sigmoid trend as pictured in figure There are less observations near the inflexion point which causes over-fitting. This phenomenon is also present in datasets on neurodegenerative disease: areas of the trajectory where data are scarce can be over-fitted by GB. Nevertheless, the value of the effects 0 are nearly constant after its first estimation in the algorithm (see Supp. material C), which strengthens the model interpretability and encourages the reduction of тMMMMC
7

---

Table 1: Experiment performances recorded with the MAE. The proposed method is compared with the alternative using a paired, two-sided Wilcoxon signed rank test. *: p<0.05, **: p<0.01, ***; p<0.001.
MMSE
HIPP
VENTS
ABETA
Experiment
GB
DCM
GB
DCM
GB
DCM
GB
DCM
Pred(nn f- 1):
0.140
0.146
0.046*
0.056
0.042
0.056
0.124
0.114
Pred(ay = 2):
0.132 T*
0.146
0.037 kix
0.046
0.038
0.049
0.123
0.109
Imp(n,- =1):
0.080
0.088
0.029
0.31
0.017
0.020
0.128
0.123
Imp(n, = =2): 0.081 w*r
0.089
0.028 Kook
0.030
0.023
0.027
0.120
0.111 **
The problem of generalization The method reveals be flexible and quite stable provided the hyper-parameters are wisely selected. To assess whether the learning complexity of the method makes sense practice, we propose to measure its capacity of generalization by recording its performance on prediction tasks within a 5-folds cross validation framework: running the algorithm Ion 4-folds, predicting on the last fold the last patient visit from its first visits, or predicting the previous visits from the last visits (as for data imputation). In the next part, these experiences are carried out on reference dataset of neurodegenerative diseases with different types of data (cognitive scores, sub-cortical volumes from MRI, cerebro-spinal fluid biomarkers) to show the potential of a flexible method such as the one we introduce in this paper.
3.2 Real data
Introduction ADNI The data used in this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database uddi.Joni.usc.ede launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer's 's disease (AD).
First, the algorithm was tested on the subset of AD patients. For each visit, we jointly model their Mini-Mental State Examination (MMSE, a cognitive score), the volume of their hippocampi and lateral ventricles normalized by their intracranial volume (HIPP and VENTS respectively) and the concentration of amyloid 31-12 in their cerebro-spinal fluid (ABETA). The goal was to understand the impact of the metric estimation compared to the sfate-of-the-ar with The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE E)) challenge (26). These methods include penalized regression, linear mixed-effect models, recurrent neural networks, and multi-task learning. Since the challenge was closed, more algorithms have been presented for the same prediction task, using the same dataset, including (27) and (28 using deep RNNs, and (29] using random forests. The forecast of ventricle volumes (VENTS) and cognitive decline (ADAS-cog) presented in this paper have smaller mean absolute errors than all these competing methods.
these typical AD biomarkers. Then, the model was
An Alzheimer's disease cohort The cohort is composed of patients from all ADNI phases (ADNII,
ADNI GO, ADNI2, ADNI3), having been diagnosed with AD, with a minimum of 2 visits (4.15 visits on average and 1.5 visits for standard deviation) and ABETA 977 pg/mL. cutoff (so to get amyloid-posinee poddt-osiivee patients only (30 31). Sub-cortical volumetric segmentations of MRI were performed with the Freesurfer image analysis suite v6.0.0, which documented [32 and freely edu/). All scores were normalized in [0.1] (to motivate the logistic prior for trajectories), using preprocessing tools from the scikit-learn library 33: MMSE was affinelymapped using the score bounds, other biomarkers were Box-Cor transformed
available for download online (htp://surfer.mnl hww.mmh..aawee.
34 to un-skew their distribution, clipped between their 0.1 and 99.9 percentiles and into [0.1]. When needed, scores were reversed such that they all increase during disease progression. The models being generative, we did not drop out missing values. We compare the DCM models using the Riemannian metric tit (baseline) and GB (giit = - Neomp and No = 0.24)), both using 2 principal directions of variation for space-shifts. Note that the bascline (DCM in yellow in figure 3 is the very first iteration of GB optimization.
finally
8

---

Disease Course Mapping
Geodesics Bending
Compositions impact MMSE
1.0
Stop3
87 years
Gtop
0.0
0.5
0.8
Stop
87 years
Stop
0.6-
HIPP
MISE
0.4
0.4
0.4
60 years
[+]
years
[S]
a
b
0.0
3G
1.0 0.0
NMSE
Alzelimerr
Figure 3: (a, b) The estimated tubular coordinate system is shown for the HIPP and MMSE scores, showing the distribution of the spatial variability around the average geodesics in red by changing the space-shifts on principal direction (w;- = 8A2, E [-1, (c) The effects of each learning step on the average trajectory "Yu is shown. 60 and 87 years are the 0.05 and 0.95 quantile for the ages (t..)-
Experiments: (1) Future visits prediction and (2) data imputation. For both experiments, mod- els are trained with a 5-folds vols-vaaii.tion. In the prediction experiment, we estimate, in the out-of-train fold, the individual parameters 2; on the first n visits of patients and predict on their very last visit. In the imputation experiment, in the out-of-train fold, we estimate the individual parameters z; on all the patients visits except n, visits drawn uniformly in its set of observations and predict on these n, removed visits. We evaluate the performance of estimations with the Mean Absolute Errors (MAE) on the 5 test-sets, depending on the value of nt and ny varying from to 2.
Results. The Iable demonstratess that GB outperforms DCM except on ABETA 3.2 no matter the value of ny and ny. Looking at the figure B]. the MMSE trajectory learnt by GB has a more exponential shape compared the DCM and the HIPP trajectory has been transformed from a nearly linear curve into piece-wise linear curve (see Supp. material C for all figures), the inter-patien variability is changed correspondingly (see sub-figure a and b). shows that there are different trajectories behaviors that DCM cannot take into account. With GB, at the beginning of the disease we observe that HIPP volume decreases while MMSE stays constant, which is clinically coherent. However, GB fails to generalize on ABETA because of the small number of data (576 missing values out of 909 visits). When few information is available logistic prior for trajectories' shape is already almost optimal. It has been remarked that GB makes better predictions than DCM especially on patients with high scores, this fact is worth mentioning if we want to later improve predictions with ensemble methods [35]. We observe that the algorithm begins to stabilize after 4 steps which motivates that Neump = 6 is a good trade-off between under-fitting and over-fitting Now, we should assess the proposed method on a more general cohort including MCI and cognitively normal patients.
TADPOLE Challenge. 266 The TADPOLE training set is composed of data from the first three ADNI phases (ADNI 1, ADNI GO and ADNI 2). It includes approximately 1500 features acquired from 1737 subjects (957 males and 780 females) during 12.741 visits for most 22 distinct time points, between 2003 and 2017, Challengers were evaluated on their prospective predictions on enrolled individuals rolling-over for ADNI next phase, regarding the Alzheimer's Disease A ssessment Scale Cognitive Score (ADAS-Cogl)) and the volume of lateral ventricles normalized by the intracranial volume (VENTS), with the MAE metric. The final test set, disclosed in 2019 (time to prediction is about 2 years), is composed of 223 follow-up visits. This challenge allows for fair comparison of the prediction performances presented here against 56 alternative methods. These methods include penalized regression, linear mixed-effect models, recurrent neural networks, and multi-task learning (26). Since the challenge was closed, more algorithms have been presented for the same prediction task, using the same dataset, including [36] and [37] using deep RNNs, and (38) using random forests. The forecast of ventricle volumes (VENTS) and cognitive decline (ADAS-cog presented in this paper have smaller mean absolute errors than all these competing methods.
We compared GB and DCM to the best Challenger. For the training, we select all patients having at least 2 visits and being amyloid-positivy (ABETA < 977 pg/mL [31]) it left us 765 patients

---

Compaition GB DCM on the MMSE
TADPOLE Drallnge
0.42
G8
DCW
DOM
0CM
challnger
(booablap dizzbbution
Figure Boxplots after bootstrapping the prediction errors on the different scores. (For the boxplot: Whiskers=(2.5.97.55 %, 5ox=125,50,75]9%
with 5.5 visits average. We use the observations of ADAS-Cog13, MMSE, HIPP, VENTS and CSF P-TAU to calibrate the models with a 5-folds vross-validation. We apply the same transformation as previously to pre-process HIPP, VENTS and to CSF P-TAU (Box-Cox transform, quantile clipping and min-max rescaling) as well as MMSE and ADAS-Cog13 (only affine-rescaling since scores are already bounded). For GB, take N(a 0.24) Neomp and finit d with principals directions for space-shifts, but 3 for DCM.
To know whether our results statistically significant in the absence of other challengers' errors sample, we derive confidence intervals of DCM and GB's s MAE by bootstrapping our error sample (10000 discounted draws). On figure 4 observe that DCM and GB outperform the best challenger on ADAS-Cop Co and stay competitive on VENTS. This fact shows the potential of the geometric approach to offer interpretable and flexible models. Regarding the effect of learning the metric, GB seems to be better than DCM on ADAS-Cogl1 and MMSE but not on VENTS. Comparing with the previous experience, is likely that GB focuses more on the AD patients profile compared to MCI and controls. Indeed AD patients are representative of the disease progression whereas MCI and control subjects are seen by the model as slower disease progressors (delayed in time with large time-shift and slow-paced with low acceleration factor oz)).
Results
4 Conclusion
Riemannian metric learning applied to mixed-effect models enables to reach a sensible trade-off between flexibility and interpretability in disease progression modeling. The proposed approach allows us to disentangle time and space variability while learning the inter-patient variability and the average trajectory from the data. Without data scarcity, it proves to be efficient on homogeneous cohort for improving predictions on imaging and clinical biomarkers and suggests promising perspectives to handle heterogeneous cohorts with ensemble methods and mixture models. This work be pursued by searching theoretical guarantees for the optimization process and practical guidelines for selecting prper-parameters with empirical estimators. This model has the potential to better monitor patients and select them in clinical trials, although we need to pay particular attention to the representativity of minority groups the training data
10

---

References
[1] Nan M Laird and James H Ware. Random-effects models for longitudinal data. Biometrics, pages 963-974, 1982.
[2] Hubert M Fonteijn, Marc Modat, Matthew J Clarkson, Josephine Barnes, Manja Lehmann, Nicola Z Hobbs, Rachael Scahill, Sarah J Tabrizi, Sebastien Ourselin, Nick Fox, et al. An event-based model for disease progression and its application in familial alzheimer's disease and huntington's 's disease. Neurolmage, 60(3): 1800 1889, 2012.
[3] Alexandra L Young, Razvan V Marinescu, Neil P Oxtoby, Martina Bocchetta, Keir Yong, Nicholas C Firth, David M Cash, David Thomas, Katrina M Dick, Jorge Cardoso, al. Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with subtype and stage inference. Nature communications, 9(1):1-16, 2018.
[4] Bachirou O Taddé, Hélène Jacqnin-Gadda, Jean-François Dartigues, Daniel Commenges, and Cécile Proust-Lima. Dynamic modeling of multivariate dimensions and their temporal relationships using latent processes: Application to alzheimer's disease. Biometrics, 76(3):886 899, 2020.
[5] Murat Bilgel, Bruno M Jedynak, and Alzheimer's Disease Neuroimaging Initiative. Predicting time to dementia using a quantitative template of disease progression. Alzheimer's Dementia: Diagnosis, Assessment & Disease Monitoring, 11(1):205-215, 2019.
[6] Jean-Baptiste Schiratti, Stéphanie Allassonniere, Olivier Colliot, and Stanley Durrleman. Learn- ing spatiotemporal trajectories from manifold-valued longitudinal data. In Neural Information Processing Systems, number 28, 2015.
[7] Clement Abi Nader, Nicholas Ayache, Philippe Robert, Marco Lorenzi, Alzheimer's 's Dis- case Neuroimaging Initiative, et al. Monotonic gaussian process for spatio-temporal disease progression modeling brain imaging data. Neurolmage, 16266, 2020.
[8] Alexandre Bône, Maxime Louis, Benoît Martin, and Stanley Durrleman. Deformetrica 4: an open-source software for statistical shape analysis. In International Workshop on Shape in Medical Imaging, pages 3-13. Springer, 2018.
[9] Maxime Louis, Raphaël Couronné, Igor Koval, Benjamin Charlier, and Stanley Durrleman. Riemannian geometry learning for disease progression modelling. In International Conference on Information Processing in Medical Imaging, pages 542-553. Springer, 2019.
[10] Guy Lebanon. Learning riemannian metrics. arXiv preprint arXiv: 1212:2111 2012.
[11] Sarang Joshi and Michael Miller. Landmark matching via large deformation diffeomor- phisms. IEEE transactions image processing, 98)):1377-330. 2000.
[12] -aa-tBaptstte Schiratti, Stéphamie Allassonnière, Olivier Colliot, and Stanley Durrleman. A bayesian mixed-effects model learn trajectories of changes from repeated manifold-valued observations. The Journal of Machine Learning Research, 18(1):4840-4872, 2017.
[13] Peter Toth, Danilo Jimenez Rezende, Andrew Jaegle, Sébastien Racanière, Aleksandar Botev, and Irina Higgins. Hamiltonian generative networks. arXiv preprint arXiv: 13789, 2019.
[14] Ivan Kobyzev, Simon Prince, and Mareus Brubaker. Normalizing flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[15] Estelle Kuhn and Mare Lavielle. Coupling a stochastic approximation version of em with an meme procedure. ESAIM: Probability and Statistics, 8:1 15-131,, 2004.
[16] Stéphanie Allassonnière, Estelle Kuhn, Alain Trouvé, et al. Construction of bayesian deformable models via a stochastic approximation algorithm: a convergence study. Bernoulli, 16(3):641- 678, 2010.
11

---

[17] Bachirou O Taddé, Hélène Jacqmin hacqmin-Gadda, Jean-Françooi Dartigues, Daniel Commenges, and Cécile Proust-Lima. Dynamic modeling of multivariate dimensions and their temporal relationships using latent processes: Application to alzheimer's disease. Biometrics, 76(3):886 899, 2020.
[18] Søren Hauberg, Oren Freifeld, and Michael Black. A geometric take on metric learning. In NIPS, volume 25, pages 2033-2041, 2012.
[19] Jöm--enrik Jacobsen, Arnold Smeulders, and Edouard Oyallon. i-revnet: Deep invertible networks. arXiv preprint urXiv:/8022 1802.80028..8 2018.
[20] Laurent Younes. Diffeomorphic learning. arXiv preprint arXiv: 2018.
[21] Muhammad Asif Rana, Angi Li, Dieter Fox, Byron Boots, Fabio Ramos, and Nathan Ratliff. Euclideanizing flows: Diffeomorphic reduction for learning stable dynamical systems. In Learning/ for Dynamics and Control, pages 330-639, PMLR, 2020.
[22] Thomas Hofmann, Bernhard Schölkopf, and Alexander J Smola. Kernel methods in machine learning. The annals of statistics, pages 1171-1220, 2008
[23] Fanghui Liu, Xiaolin Huang, Yudong Chen, and Johan AK Suykens. Random features for kernel approximation: A survey in algorithms, theory, and beyond. arXiv preprint arXiv:2004. 1144, 2020.
[24] Tri Dao, Christopher De Sa, and Christopher Ré. Gaussian quadrature for kernel features. Advances in neural information processing systems, 30:6109, 2017.
[25] Igor Koval. Learning Multimodal Digital Models of Disease Progression from Longitudinal Data: Methods & Algorithmss for the Description, Prediction and Simulation of Alzheimer's Disease Progression. PhD) thesis, Institut polytechnique de Paris, 2020.
[26] Razvan V Marinescu, Neil P Oxtoby, Alexandra L Young, Esther E Bron, Arthur W Toga, Michael W Weiner, Frederik Barkhof, Nick C Fox, Stefan Klein, Daniel C Alexander, al. Tadpole challenge: Prediction of longitudinal evolution in alzheimer's 's disease. arXiv preprint arXiv:1805.03909 2018.
[27] Minh Nguyen, Tong He, Lijun An, Daniel C Alexander, Jiashi Feng, BT Thomas Yeo, Alzheimer's Disease Neuroimaging Initiative, et al. Predicting alzheimer's disease progression using deep recurrent neural networks. Neurolmage, 222: 17203, 2020.
[28] Wonsik Jung, Ahmad Wisnu Mulyadi, and Heung-ll Suk. Unified modeling of imputation, forecasting, and prediction for ad progression. International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 168-177 176. Springer, 2019.
[29] PJ Moore, TJ Lyons, John Gallacher, and Alzheimer's Disease Neuroimaging Initiative. Random forest prediction of alzheimer's 's disease using pairwise selection from time series data. PloS one, 14(2):c0211 (BD):e22115555 1558,, 2019.
[30] Clifford R Jack, David A Bennett, Kaj Blennow, Maria C Carrillo, Howard H Feldman, Gio- vanni B Frisoni, Harald Hampel, William J Jagust, Keith A Johnson, David S Knopman, al. A/tm: an unbiased descriptive classification scheme for alzheimer disease biomarkers. Neurology, 87(5):539-55) 547, 2016.
[31] Oskar Hansson, John Seibyl, Erik Stomrud, Henrik Zetterberg, John Trojanowski, Tobias Bittner, Valeria Lifke, Veronika Corradini, Udo Eichenlaub, Richard Batrla, et al. Csf biomarkers of alzheimer's 's disease concord with amyloid-3 pet and predict clinical progression: study of fully automated immunoassays in biofinder and adni cohorts. Alzheimer's 's & Dementia, 14(11)14400-4811 2018.
[32] B. Fischl, D. H. Salat, E. Busa, M. Albert, M. Dieterich, C. Haselgrove, A. van der Kouwe, R. Killiany, D. Kennedy, S. Klaveness, A. Montillo, N. Makris, B. Rosen, and A. M. Dale. Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Newron, 33:341-355,
12

---

[33] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
[34] George EP Box and David R Cox. An analysis of transformations. Journal of the Royal Statistical Society: Series B (Methodologicall, (666):211112333 1964.
[35] Thomas G Dietterich. Ensemble methods in machine learning. In International workshop multiple classifier systems, pages 1-15. Springer, 2000.
[36] Minh Nguyen, Tong Hc, Lijun An. Daniel C. Alexander, Jiashi Feng, and B. T. Thomas Yeo. Predicting Alzheimer's disease progression using deep recurrent neural networks. Neurolmage. 022:117203, November 2020.
[37] Wonsik Jung, Ahmad Wisnu Mulyadi, and Heung-ll I1 Suk. Unified Modeling of Imputation, Forecasting, and Prediction for AD Progression. In Dinggang Shen, Tianming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, and Ali Khan, editors, Medical Image Computing and Computer Assisted Intervention MICCAI 2019, Lecture Notes in Computer Science, pages 168-176, Cham, 2019. Springer International Publishing.
[38] PJ Moore, TJ Lyons, J Gallacher, and for the Alzheimer's Disease Neuroimaging Initiative. Random forest prediction of Alzheimer's ' disease using pairwise selection from time series data, 2019.
[39] peerre-Emmanul) Poulet, Sean-Christophe Corvol, and Stanley Durrleman. A latent mixed- effects model for longitudinal categorical data in parkinson disease. In CompAge 2020- Computational approaches for ageing and age-related diseases 2020, 2020.
[40] Yuchen Wang, Matthieu Chan Chee, Ziyad Edher, Minh Due Hoang, Shion Fujimori, Sornnujah Kathirgamanathan, and Jesse Bettencourt. Forecasting black sigatoka infection risks with latent neural odes. arXiv preprint arXiv:2012.00752, 2020.
[41] Emilien Dupont, Arnaud Doucet, and Yee Whye Tch. Augmented neural odes. arXiv preprint 6181 2019.
[42] Steven Diamond and Stephen Boyd. Cvxpy: A python-embedded modeling language for convex optimization. The Journal of Machine Learning Research, 17(1):9009-2913, 2016.
[43] Peter Petersen, S Axler, and KA Ribet. Riemannian geometry, volume 171. Springer, 2006.
[44] Annette N Markham, Katrin Tridenberg, and Andrew Herman. Ethics as methods: doing ethics in the era big data research- introdductio....iionn.. Social Media+ Society, 1 18784502,, 2018. 8.
13


Reviews:
[{"review_id": "vBJjVyh0CuY", "reviewer": "Reviewer_nBtD", "summary": "In this work, the authors proposed a mixed effect model in a Riemannian manifold to estimate disease progression. The proposed model is built upon previous work with an improvement to estimate a Riemannian metric from data. An alternating maximization algorithm is proposed to learn a sequence of diffeomorphisms, represented in the RKHS, to bend the initial metric toward optimal. The method was evaluated on synthetic data and a real-world dataset to demonstrate improvement upon previous work. ", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work brings several ideas from previous studies together and combines them into an improved mixed effect model in a Riemannian manifold. The method was validated on synthetic data and a real dataset. The improvement was demonstrated on the ANDI dataset, but only one baseline was compared.  Evaluation on synthetic data could be further improved since only two qualitative examples were provided. The proposed method seems to be general and can be used in other applications, but no evidence was provided to justify that.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "_-gUICTUDR1", "reviewer": "Reviewer_w1hQ", "summary": "This paper presents a method for disease trajectory modeling that embeds patient trajectories in a Riemannian manifold to allow for more flexible models than simpler linear mixed effects models.   The primary methodological contribution appears to be a technique for learning the Riemannian metric from data, rather than using a fixed prior choice of a metric. This technique is applied to real data from patients with Alzheimer’s.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:  This paper appears to make a relatively straightforward extension of existing work in disease trajectory modeling by learning the Riemannian metric directly, rather than using a fixed metric. Although previous work is well cited,  it was hard to determine whether this idea to learn the metric from data had been previously attempted, and whether that was the key piece of methodological novelty.  I am not an expert in this subarea and did not track down all of the references cited, but some of the references cited in the introduction imply that this had already been done before, so the specific technical contribution remains unclear to me.\n\nQuality:  The submission appears technically sound and the methods appear appropriate for the task.  However, there is limited empirical evaluation of the methods, and no comparison with simpler baselines such as linear mixed effects models to motivate the improvements that these nonlinear models offer. The synthetic experiments in section 3.1 were also hard to follow, and it was unclear what the important takeaways were. \n\nClarity: The paper was hard to follow, and heavily relies on references for some of the details and background.  It was hard to read as a standalone paper.  I found the methods section overly mathy and lacking in basic intuition as to what was going on, and it assumed a lot of prior knowledge about Riemannian geometry and related works. \n\nSignificance:  It’s hard to tell if the results are impactful, and in general some additional motivation on why this particular class of methods is a good idea was lacking.  It’s unclear from reading this paper in what situations both this specific method and this broader subarea of Riemannian-manifold-embedded nonlinear models for disease trajectories are helpful, and what their failure modes are.  Although the proposed GB improves over the existing DCM, a simpler baseline like a linear mixed effects model would be helpful to get a sense for how much these more complex methods improve over something far simpler.  Although predictive performance is generally improved for GB vs DCM it’s hard to tell how to generalize this information to other problems and contexts.  \n\nOther comments:\n-\tDoes this approach assume that all d variables are observed at each observation time? Can this handle settings where only a subset of variables are observed at each time point?\n-\tSome individual examples of fitted trajectories along with the observed data would be helpful to get a better sense of how well GB, and DCM, fit the observed data.  A lot can be lost in just showing overall MAEs\n-\tDid you do any subgroup analyses to try to better understand which types of patient trajectories are easier/harder to predict?  \n-\tHow do these classes of approaches fare in the presence of clusters of trajectories, since they assume that all trajectories are subject-specific deviations from a global mean?  Eg if there are a small number of unknown disease subtypes, and each subtype exhibits a specific type of trajectory shape.\n-\tHow many iterations does it take for convergence? What is the rough runtime per iteration? \n-\tWhat is the main take-home message from the comparison with the TADPOLE challenge?  \n-\tHow good are the reported MAEs? What level of error might be low enough in order to be clinically impactful, if you were to back-transform the variables back to their original scale?  Even if some of these mean differences are statistically significant, it is hard to contextualize how much better an error reduction of .002 is in practice\n-\tFor the Imp(n=1) row of Table 1, the HIPP DCM column, is this a typo? Should that 0.31 really be 0.031?\n-\tCould use some copy-editing, as there are a fair number of typos throughout.\n-\tNot all of the notation is fully defined in the methods\n-\tThere were several places where ] -1, 1 [ brackets were used, was this just a typo and these were supposed to be [-1,1]?\n\n\n----------\nEdit: after reading through the author response, I have bumped my score up to a 5.  More detailed comments below in thread.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There was little discussion of limitations of the proposed approach.  A short section mentioning limitations in one place would be helpful.\n\nSocietal impact was only briefly mentioned in passing in the end of the conclusion around the need to pay attention to diversity and representativeness of the training data.  Were you able to do any subgroup analyses to see if your model performs roughly as well on different demographics?  If it does not, is there a chance that if your model were operationalized that it could exacerbate inequalities?  How might your model be useful in practice, if there is little we can do to slow or reverse progression of Alzheimer’s?  Could this lead to increased stress and anxiety for afflicted individuals?  \n", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "SiWnGGQFoRS", "reviewer": "Reviewer_wbfL", "summary": "This paper uses Riemannian metric learning within the context of mixed-effect models for modelling distributions of patient trajectories through a disease process.  Disease progression modelling is an active area with high relevance to NeurIPS and this is a novel take on its implementation.  Experiments demonstrate efficacy in simulation, illustrate the benefits of the trade-off between flexibility and interpretability the proposed approach provides, and show strongly competitive performance on a standard Alzheimer's disease data set (from the TADPOLE challenge) even against purely black-box data-driven models that offer much less in terms of understanding disease and its heterogeneity.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a well thought-out and novel formulation for disease progression modelling that is well motivated in terms of balancing flexibility and interpretability.  Experiments are clear and compelling and I believe these ideas will be of broad interest to the NeurIPS community, as well as computational neurology and disease progression modelling researchers.  The presentation bridges those fields nicely.  \n\nI don't have much feedback to give on this paper, but would say that it would benefit from a clearer specification from the start of what benefits the authors expect from proposed formulation over existing implementations such as refs [6-8] in the manuscript.  In essence the ideas are similar - estimating trajectories as well as time shift and acceleration factors.  Is the Riemannian formulation primarily a mathematical nicety, or are there genuine performance issues with earlier implementations that the authors expect it to improve?  With more careful specification of these motivations and expectations, the experimental work could be more targeted to highlighting those hypothesised benefits.\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Fine", "review_text": "", "extra_scores": {"ethical_concerns": "Not mentioned, but I see no issues.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "OItJ4hrTy0C", "reviewer": "Reviewer_Wxum", "summary": "This paper proposes a mixed effects model for modeling disease trajectories. The method builds on prior work that considers geometric models (which represent trajectories as geodesics on prespecified Riemannian manifolds), by allowing for the Riemannian metric to be learned from data. The proposed method combines desirable aspects of different disease trajectory modeling approaches: interpretable individual-specific and population specific parameters governing space and time variability and a metric that can be learned from data. On synthetic experiments the authors study learning properties of the proposed method. Experiments on an Alzheimer's dataset challenge show that the method has strong and competitive performances (in addition to desirable interpretable properties from the mixed effects modeling structure).", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While I have experience with time-series modeling of disease trajectories, unfortunately, I must admit that I am not familiar with the literature on geometric trajectory models. As such, I cannot speak much to the novelty or technical soundness of the proposed method.\n\nThat said, the problem of disease progression modeling is important as motivated by the authors in the introduction, and these methods are very useful for, e.g., brain imaging data. They do a good job of identifying desirable features of disease progression models and then walk through the development of their modeling approach. The empirical results on the TADPOLE challenge appear to be good evidence of the strong performance of the proposed approach relative to existing alternatives.\n\nI have a few suggestions to make the paper more accessible to a broader audience:\n- The introduction reviews various strategies for disease progression modeling, but it does not clearly draw attention to the problems that this paper is intended to address until Line 67. I think streamlining the introduction to be more focused on the challenges addressed by and the benefits of the proposed approach will help readers more quickly understand what they will find in this paper.\n- I think this paper would benefit from a dedicated and broad related work section. The introduction provides a somewhat narrow (and hasty) review of some methodology for disease trajectory modeling. A dedicated related work section would help a broader set of readers understand where the proposed method (and related works like [6,9]) fits in the landscape of work on disease trajectory modeling.\n\nFor example, I am familiar with work like the following on modeling longitudinal disease trajectory data:\n- Schulam, P., & Arora, R. (2016). Disease Trajectory Maps. Advances in Neural Information Processing Systems, 29, 4709-4717.\n- Schulam, P., Wigley, F., & Saria, S. (2015, February). Clustering longitudinal clinical marker trajectories from electronic health data: Applications to phenotyping and endotype discovery. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 29, No. 1).\n- Proust-Lima, C., Séne, M., Taylor, J. M., & Jacqmin-Gadda, H. (2014). Joint latent class models for longitudinal and time-to-event data: A review. Statistical methods in medical research, 23(1), 74-90.\n- Lasko, T. A., Denny, J. C., & Levy, M. A. (2013). Computational phenotype discovery using unsupervised feature learning over noisy, sparse, and irregular clinical data. PloS one, 8(6), e66341.\n\n\nUPDATE:\nThanks the authors for their response. I will be maintaining my score. I think this paper will be greatly improved by broadening the perspective in the introduction, including a dedicated related work section, and discussing some of the other competitor methods that were used in the TADPOLE challenge to provide more context for readers. Reviewer w1hQ has suggestions that I also believe would improve the paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Thanks to the authors for their engaging submission on an important topic.  There was a lot of positive response in the reviews to this work, and all reviewers were positive about the underlying idea.  However, there are areas in which this submission could be strengthened. 

A major reviewer concern is understanding the strengths and weaknesses of the proposed method.  An explicitly stated motivating factor is that the parametric assumptions of standard mixed effects models are too restrictive, and that the flexibility afforded by learning the Reimannian manifold will allow models to better fit the data and form better trajectories. And intuitively this makes sense. But the empirical evidence of this is lacking.  As reviewer w1hQ points out, a standard linear mixed effects model would make for an informative baseline to empirically justify these claims.  As the abstract introduces this method as an improvement upon (and alternative to) a standard linear mixed effects model, having a direct empirical comparison would help identify what parts of the proposed method are most performant, and in what situations the proposed method may be inappropriate.  Further, a simulation study where aspects of the data generating process are known and manipulated could enhance such a direct comparison to a standard mixed effects model.
Decision: Accept (Poster)