Conditional Diffusion with Ordinal Regression: Longitudinal Data Generation for Neurodegenerative Disease Studies

Authors: Hyuna Cho, Ziquan Wei, Seungjoo Lee, Tingting Dan, Guorong Wu, Won Hwa Kim
Primary area: applications to neuroscience & cognitive science
Keywords: neurodegenerative disease, conditional diffusion model, longitudinal data analysis

Abstract:
Modeling the progression of neurodegenerative diseases such as Alzheimer’s disease (AD) is crucial for early detection and prevention given their irreversible nature. However, the scarcity of longitudinal data and complex disease dynamics make the analysis highly challenging. Moreover, longitudinal samples often contain irregular and large intervals between subject visits, which underscore the necessity for advanced data generation techniques that can accurately simulate disease progression over time. In this regime, we propose a novel conditional generative model for synthesizing longitudinal sequences and present its application to neurodegenerative disease data generation conditioned on multiple time-dependent ordinal factors, such as age and disease severity. Our method sequentially generates continuous data by bridging gaps between sparse data points with a diffusion model, ensuring a realistic representation of disease progression. The synthetic data are curated to integrate both cohort-level and individual-specific characteristics, where the cohort-level representations are modeled with an ordinal regression to capture longitudinally monotonic behavior. Extensive experiments on four AD biomarkers validate the superiority of our method over nine baseline approaches, highlighting its potential to be applied to a variety of longitudinal data generation.

Decision: accept (spotlight)
Reviewer scores: 8, 6, 8

Reviews and discussion:
<thread>
creation date:
1730551692637

thread_type:
official_review

<thread_object>
creator: reviewer_jkdk

<content>
summary:
The paper presents ConDOR, a novel conditional diffusion model for generating longitudinal neurodegeneration data with ordinal disease progression factors. The model's architecture integrates both cohort-level and subject-level characteristics through a dual-component approach. At the cohort level, it employs Bayes' Theorem combining an ordinal regression model (capturing disease stage relationships) with a kernel-based conditional distribution. For data generation, ConDOR utilizes two diffusion models: a Regional Diffusion Model (RDM) for generating baseline measurements across brain regions, and a Temporal Diffusion Model (TDM) for generating subsequent longitudinal data. The model also incorporates a domain conditioning mechanism to integrate data from multiple sources. The authors evaluate ConDOR on multiple biomarkers (Amyloid, Cortical Thickness, and Fluorodeoxyglucose) from two prominent neurodegenerative disease datasets (ADNI and OASIS), comparing against nine baseline methods including GANs, VAEs, and other diffusion-based approaches.

strengths:
1. The proposed model captures both spatial and temporal features through a combination of a Regional Diffusion Model and a Temporal Diffusion Model.
2. This new generative model addresses challenges associated with sparse, irregular, and widely spaced intervals in medical data.
3. The model strikes a balance between cohort-level and individual-level fitting, capturing generalized population trends while accommodating individual variability.
4. It introduces a novel integration of ordinal regression with diffusion models.
5. The experiments are comprehensive, with comparisons to nine baseline models, including GANs, VAEs, and other diffusion-based models, evaluated across three metrics. Implementation time is also compared.
6. The model is extended to a multi-domain setting, enhancing its generalizability and applicability to different data sources.

weaknesses:
1. The ordinal regression model might oversimplify the disease progression process. Additionally, the temporal diffusion relies on linear interpolation for temporal transitions, which may not accurately capture realistic disease dynamics.
2. There is a lack of comparison with traditional longitudinal baseline models commonly used in medical literature.
3. The model evaluation has not been clearly described. Did the authors split subjects into training and test sets, keeping all observations from each subject together, or did they split individual observations, potentially placing different time points from the same subject in both training and test sets?
4. The reproducibility of this work is not guaranteed, as the code has not yet been made available.

questions:
1. The model evaluation lacks clarity regarding whether the authors performed a subject-level or observation-level split. Specifically, did they keep all observations from each subject together, or did they split individual observations, potentially including different time points from the same subject in both training and test sets? It would be valuable to see how well the model predicts follow-up scans based on data from earlier time points, given that the Temporal Diffusion Model is a novel component. Additionally, for baseline models like DDPM that lack a temporal component, it would be interesting to understand how the authors utilize these models to generate follow-up scans over time.
2. The Temporal Diffusion Model uses linear interpolation to model progression in age and labels, which may not be ideal, as transitions between disease states are often abrupt or follow complex patterns. Furthermore, it would be beneficial to see theoretical proof that such linear interpolation preserves the properties of diffusion models.
3. Including some directions for future work in the conclusion would be beneficial for the research community.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730676153078

thread_type:
official_review

<thread_object>
creator: reviewer_xa3q

<content>
summary:
This paper proposes a conditional generative model for synthesizing longitudinal sequences. 
It first uses ordinal regression and kernel density estimation to model the conditional PDF and then interpolate gaps between consecutive observations. 
Two diffusion models are then trained to model baseline data and changes in follow-up samples. 
These diffusion models generate disease progression data by sequentially sampling.

strengths:
The design of the proposed model's structure design is innovative, partsuch as the combination of two diffusion models and the use of both cohort and subject-level interpolation for training. 

This paper provides clear comparisons to baseline models across multiple metrics and good visualizations.

weaknesses:
Some abnormal results are not well discussed or explained. 
For example, in Table 1, the variance in DDPM's performance is very high compared to other methods. 
Since DDPM is one of only two diffusion-based comparison methods, it would be helpful to provide an explanation for this abnormal performance.


The authors do not mention how to deploy the model. For example, how to use the trained model to generate new data. 
If my understanding of the generative process is correct, we only need to use these two diffusion models with random noise as the input when generating new data.
There is no option to allow the two diffusion models to generate samples conditioned on specific ages and disease severity, 
e.g., we can't use age as an input for these diffusion models when generating, 
although the authors claim that this model can generate data conditioned on these factors.

Some important technical details are also missing. Please refer to the questions below

questions:
RDM is designed to geneate the baseline sample $x_1$. However, during training, the authors use all the samples $x_t$ (t = 1, .., t) as independent cross-sectional data.
The justification for using such setting is not presented in the paper. 
For example, why not use only $x_1$ to train the RDM? 
Will this setting cause the RDM to be biased as some longitudinal samples have longer records or are recorded more frequently

Acoording to Table 3, the choice of hyperparameter $\lambda$ significantly impacts the performance of the proposed method. 
Howerver, how to select $\lambda$ is unclear, for example, which dataset and what metric do the author use to choose $\lambda$.
The author used 80% of the whole data for training and the rest 20% for testing for all experiments,
and it seems there is no validation dataset to optimize $\lambda$.
Therefore, the results shown in Table 3 is less convincing to me since they are propbably derived from either training or test set, 
and the results in Table 1 and 2 are also less convincing since we may not able to get the best $\lambda$ in practice.


The training strategies for the generative model are somewhat unclear to me. For example do the authors train RDM and TDM seperatedly or jointly?
$D$ seems to be another important hyperparameter, but how the authors chose $D$ is also unclear.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730706234193

thread_type:
official_review

<thread_object>
creator: reviewer_ruxz

<content>
summary:
This paper introduces a novel conditional generative model for synthesizing longitudinal sequences to study neurodegenerative diseases such as Alzheimer’s disease. The method uses ordinal regression and a diffusion model to generate realistic disease progression imaging data.

strengths:
1. Combing the cohort-level trend and subject-level trend for longitudinal data generation. 
2. Extensive validation on four Alzheimer's Disease biomarkers demonstrates the model's superiority over nine baseline approaches.

weaknesses:
1. The paper lacks a comprehensive theoretical justification for the proposed method. While the method is innovative, a deeper theoretical comparison with existing models could strengthen the argument for its necessity and effectiveness.
2. The description of the methodology, particularly the integration of cohort-level and subject-level samples, is somewhat convoluted. The paper could benefit from clearer explanations and more detailed algorithmic steps to enhance reproducibility.
3. The paper does not provide a thorough statistical analysis to support these claims. The lack of confidence intervals or significance testing weakens the robustness of the reported findings.
4. The discussion section is relatively weak in terms of interpreting the results and their implications. The paper does not adequately address the potential limitations of the proposed method or suggest directions for future research, which are crucial for a comprehensive understanding of the study’s impact.

questions:
1. How does the proposed method theoretically ensure the accurate representation of disease progression, especially considering the complex dynamics and irregular intervals in longitudinal data?
2. How does the proposed method theoretically improve upon existing generative models for longitudinal data? Are there any theoretical limitations or assumptions that need further clarification?
3. The paper introduces a dual-sampling approach combining cohort-level and subject-level samples. How does this method compare to other state-of-the-art techniques in terms of capturing individual-specific features and general trends? Are there any potential biases introduced by this approach?
4. The experiments are conducted on four AD biomarkers from MRI and PET images. How representative are these biomarkers and datasets of the broader neurodegenerative disease population? Are there any limitations in the experimental design that could affect the generalizability of the results?
5. The paper claims superiority over nine baseline approaches. How robust are these results across different metrics and datasets? Are there any specific scenarios or conditions under which the proposed method might underperform or fail?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732182450185

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General response to all reviewers

comment:
We sincerely thank all reviewers for their thoughtful and constructive feedback. We have addressed all concerns raised by the reviewers and kindly request that the reviewers consider our rebuttals. 

Also, we appreciate the recognition of our contributions, practical value, and experimental rigor. Notably, the reviewers highlighted:

$\textbf{(1) Technical Novelty}$:

$\bullet$ The design of the proposed model's structure design is innovative, e.g., the combination of two diffusion models and the use of both cohort and subject-level interpolation for training. ($\textbf{reviewer xa3Q}$)

$\bullet$ This new generative model addresses challenges associated with sparse, irregular, and widely spaced intervals in medical data. ($\textbf{reviewer jkdK}$)

$\bullet$ It introduces a novel integration of ordinal regression with diffusion models. ($\textbf{reviewer jkdK}$)

$\bullet$ This paper introduces a novel conditional generative model for synthesizing longitudinal sequences to study neurodegenerative diseases such as Alzheimer’s disease. ($\textbf{reviewer rUXZ}$)

$\textbf{(2)  Model Strengths and Capabilities}$:

$\bullet$ The proposed model captures both spatial and temporal features through a combination of a Regional Diffusion Model and a Temporal Diffusion Model. ($\textbf{reviewer jkdK}$)

$\bullet$ The proposed model combines the cohort-level trend and subject-level trend for longitudinal data generation. ($\textbf{reviewer rUXZ}$)

$\bullet$ The model strikes a balance between cohort-level and individual-level fitting, capturing generalized population trends while accommodating individual variability. ($\textbf{reviewer jkdK}$)

$\bullet$ The model is extended to a multi-domain setting, enhancing its generalizability and applicability to different data sources. ($\textbf{reviewer jkdK}$)

$\textbf{(3) Extensive Experiments}$: 

$\bullet$ Extensive validation on four Alzheimer's Disease biomarkers demonstrates the model's superiority over nine baseline approaches. ($\textbf{reviewer rUXZ}$)

$\bullet$ This paper provides clear comparisons to baseline models across multiple metrics and good visualizations. ($\textbf{reviewer xa3Q}$)

$\bullet$ The experiments are comprehensive, with comparisons to nine baseline models, including GANs, VAEs, and other diffusion-based models, evaluated across three metrics. Implementation time is also compared. ($\textbf{reviewer jkdK}$)
</content>
</thread_object>

</thread>
<thread>
creation date:
1742277625694

thread_type:
public_comment

<thread_object>
creator: ~hyuna_cho1

<content>
title:
Code Release

comment:
We share the code of ConDOR and all baseline models we used in our study, along with the pretrained models for all experimental settings, in this repository: https://github.com/Hannah37/ConDOR-ICLR25/tree/main
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes ConDOR, a novel conditional generative model for synthesizing longitudinal sequences and present its application to neurodegenerative disease data generation conditioned on multiple time-dependent ordinal factors, such as age and disease severity. The synthetic data are curated to integrate both cohort-level and individual-specific characteristics, where the cohort-level representations are modeled with an ordinal regression to capture longitudinally monotonic behavior. Extensive experiments are conducted. In sum: the algorithm is innovative, the problem comes from real-world challenges from the broader ML/Neuroscience community, and the experimental results are convincing. After the rebuttal stage, all the reviewers have unanimously supported this paper.

PC/SAC/AC and all reviewers will monitor this submission to ensure the reproductivity of this work - the authors promised to release codes of their methods and baseline methods along with their pre-trained models publicly. Raw data and/or descriptions of how to access the data in Appendix A are also required by the ICLR community.