Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper learns Riemannian metric from data for disease progression. The proposed method has been validated on two datasets. This paper also demonstrates several theoretical results of the proposed methods.
main_review: This paper proposed a novel method for learning metric from data. The paper provided some proofs of the proposed methods. But, this paper should give more details of the experimental setting, the explanation of the Figures, which will make the paper more clarity. The experimental results on two dataset shows the results for disease progression prediction. Some comments and suggestions are listed as follows:
1.In line 97: There are some typo.
2.In Figure 2: The authors should give the more explanation of the figure. 
3.In Line 224 – 225: How to choose the σ. I guess the author select the best value
for the proposed method by using the cross-validation.  But, I think it is a bad choice. The dataset in your experiments on the ADNI dataset just separate the ADNI dataset into training and validation sets. The choice in the validation sets may produce bad results on the real test data.
4.In Line 237-245: Author should give more rigorous experiments to demonstrate the generalization of the proposed methods.
5.Line 260-361: There are some same subjects in the ADNI1, ADNI GO, ADNI2, and ADNI3 datasets.  Is there a data leakage?  Are you only using AD data?
6.There are many methods proposed for metric learning and disease progression prediction. Please compared with the state-of-the-art methods. 
7.The claimed contribution of this paper is the interpretability. But the paper lacks the descriptions of the model interpretability and result interpretability. 
8.Please give the time and space complexity analysis of the proposed method.

limitations_and_societal_impact: In term of novelty and performance improvement, this paper meets the publication requirement.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Linear mixed-effect models provide a natural baseline for estimating disease progression using longitudinal data. This paper applies Riemannian metric learning to mixed-effect models to improve their flexibility while keeping the interpretability. Specifically, the metric is learned as the push-forward of the Euclidean metric by a diffeomorphism that is estimated iteratively as the composition of radial basis functions belonging to a reproducible kernel Hilbert space. The metric update allows to improve the forecasting of imaging and clinical biomarkers in the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. Experimental results on synthetic data and on a real dataset shows that it performs better than or comparably to the 56 methods benchmarked in the TADPOLE challenge on the task of predicting patient's biomarker progression.
main_review: Comments:
1. Figure 1 needs to be explained in more detail.

2. Language issues and others
Line 96: There is a redundant article "the".
Line 103: "depicted on 1" should be "depicted in (1)" (using LaTex command $\eqref{}$).
Line 103: $]0,1[$ should be written as $[0,1]$.
Line 120: "not necessary true" should be "not necessarily true".
Line 123: "in 1" should be "in (1)".
Eq. (2): The middle term is missing the Euclidean norm symbol $| |$.
Line 132: The word "time" is repeated twice.
Line 133: $]-1,1[$ should be written as $[-1,1]$.
Line 148: $]0,1[$ should be written as $[0,1]$.
Line 149: "a RKHS" should be "an RKHS".
Line 153: What is the symbol "Id"?
Line 157: Should $|x|$ be $|x|^2$? Please cite the source of the Sobolev kernel. I cannot find the description of the Sobolev kernel in the references e.g., [20]. 
Line 160: "problem 2" should be "problem (2)".
Line 162: "the error of reconstruction 2" should be "the error of reconstruction in (2)".
Line 162: $w_{i,j}^*$ should be $w_{i,j}$ (i.e., no asterisk).
Line 196: "on a Git in a near future" should be "on Github in the near future".
Line 212: "both experiment(s)".
Line 218: "on 2" should be "in Figure 2".
Table 1 caption: "record" should be "recorded".
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This work proposes a Riemannian metric learning applied to mixed-effect models which enables a balance between flexibility and interpretability in disease progression modeling. The proposed approach allows us to disentangle time and space variability while learning the inter-patient variability and the average trajectory from the data. Main innovation is the way of avoiding the selection of Riemannian metric through empirical arguments by proposing a semi-parametric method using a Reproducing Kernel Hilbert Space (RKHS).  
main_review: Originality:
The work is part of a larger effort to predict the disease progress. It offers  a promising and a flexible method and especially the Geodesic Bending (authors' name) is a clear originality of this work.

Quality: The quality is good. There is a nice balance between theoretical grounding and experiments. Supplementary material helped a lot. 
Clarity: The paper is pretty clear. I was struggling to understand that you will also use DCM in the results which creates confusion.  Please clarify in the introductory section (it is there but it is not clear) what you will compare with what.

Sgnificance: It is a nice paper that improves the way that the Riemannian  metric is chosen with an elegant way. 
limitations_and_societal_impact: 
1)Please put brief limitations and explanations why (in your opinion) what you propose works better than something probably more established.  
Typically I kindly suggest to add paragraph before Section 4 which explains some technical aspects about the reason why you find these Results (page 9). For example, what makes your approach to outperform on ADAS - Cog 13 and not on VENTS. Is it by chance?Is it the dataset and the distributions behind?
Please mention also the methods with whom you compare.  

2)
There is a nice bibliography that guides well the reader. However, I did not see any mention in the Multitasking learning approaches for disease progression prediction which have been highly popular nowadays and fairly less "complex" (in my opinion) for the multidisciplinary world. Indicatively you can initially check the works  of: 

Zhou et al 2011 A Multi-Task Learning Formulation for Predicting

Zhou et al 2012 Modeling disease progression via fused sparse group lasso,” 

Cao et al 2017 “Sparse shared structure based multi-task learning for MRI based cognitive performance prediction of Alzheimer’s disease
 or 
Suk e al 2016 Deep sparse multi-task learning for feature selection in Alzheimer’s disease diagnosis

It would be nice to (at least) discuss a little bit the pros and cons.

Probably some of these approaches are already included in the TADPOLE challenge. It is good to know moe details about who wins where and the level of winning (extreme outperformance or minor?) etc.


No Negative Societal Impact. I would rather say that there is only positive impact with his work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper presents a generalization of the linear mixed effect model to a Riemannian manifold. Instead of using a heuristically designed Riemannian metric, the paper proposes to learn the metric directly from data. This is achieved by learning a push forward function that maps the flat space to curved space. The method is validated on the ADNI data in metrics from the TADPOLE challenge. 
main_review: The Riemannian formulation of the problem, although falling to a relatively narrow research field, seems to be an interesting extension of prior established theory. Using an RKHS function to parameterize the diffeomorphic push forward mapping is a standard reasonable construction. There are several points that I would like the authors to further clarify.

1. I wonder how such a Riemannian concept/setting is essential for modeling this kind of problem. How do authors view the relationship between Riemannian modeling vs. directly using more flexible parameterization in flat space to learn trajectories (is it feasible to direct use RKHS in the context of Euclidean Linear mixed effect model)? The proposed method relies on geodesics on the manifold, i.e. linear modeling on a non-linear space. What are the pros and cons of that practice compared to non-linear modeling in flat space?

2. In line with the above comment, the Riemannian setup makes it an ‘isolated' approach that can only be compared to one baseline GB. I encourage the authors to discuss their proposal in a broader context and really try to see whether is proposal approach compares favorably to many other branches of methods in progression modeling. 

3. Can the authors confirm that training/testing subjects used herein are the same as the TADPOLE challenge. I'm confused by "To know whether our results are statistically significant in the absence of other challengers’ errors sample, we derive confidence intervals of..."  Is MMSE also provided as a training feature in the TADPOLE challenge? I remember TADPOLE predicts three outcomes, but only two are given in this paper (ADAS, VENTS).

minor: I don't understand why an isometric $\phi$ can induce curvature preservation.
limitations_and_societal_impact: Lack of connection to other disease progression modeling works.
review_title: null

Meta-review: This work proposes a Riemannian metric learning applied to mixed-effect models which enables a balance between flexibility and interpretability in disease progression modeling. This work is original in predicting the disease progress. The theoretical grounding and experiments are well provided. Overall, the paper is well written. There are some suggestions, such as related works and details of experimental setting, as well as some typos in notations. Please carefully consider all suggestions from reviewers in the final version. 
Decision: Accept (Poster)