Unsupervised Analysis of  Alzheimer’s Disease Signatures using 3D Deformable Autoencoders

Authors: anonymous  organization
Venue: MICCAI_org_2024_Workshop_MSB
Published: 2024-06-29 20:18:28

Abstract: With the increasing incidence of neurodegenerative diseases such as Alzheimer's Disease (AD), there is a need for further research that enhances detection and monitoring of the diseases. We present MORPHADE (Morphological Autoencoders for Alzheimer's Disease Detection), a novel unsupervised learning approach which uses deformations to allow the analysis of 3D T1-weighted brain images. To the best of our knowledge, this is the first use of deformations with deep unsupervised learning to not only detect, but also localize and assess the severity of structural changes in the brain due to AD. We obtain markedly higher anomaly scores in clinically important areas of the brain in subjects with AD compared to healthy controls, showcasing that our method is able to effectively locate AD-related atrophy. We additionally observe a visual correlation between the severity of atrophy highlighted in our anomaly maps and medial temporal lobe atrophy scores evaluated by a clinical expert. Finally, our method achieves an AUROC of 0.80 in detecting AD, out-performing several supervised and unsupervised baselines. We believe our framework shows promise as a tool towards improved understanding, monitoring and detection of AD. To support further research and application, we have made our code publicly available at https://anonymous.4open.science/r/MORPHADE-1925/.

Main paper: 
 introduction: 
 Due to the increased prevalence of neurodegenerative diseases and their effects on cognitive function, the study of such diseases is a highly active research field. As the leading cause of dementia [1], Alzheimer's disease (AD) is a particular focus of research advancements. However, the complex pathogenesis and progression mechanisms of AD remain only partially understood.Magnetic resonance imaging (MRI) has shown use in the non-invasive tracking of AD-associated brain changes, such as hippocampal and amygdala atrophy and ventricular dilation [16, 11] . Notably, several supervised machine learning methods utilizing MRI have been proposed which yield improvements in AD identification [24, 15, 23] . However, such methods are restricted by the need for large, annotated data sets. In contrast, unsupervised anomaly detection techniques [7, 2, 21, 25] offer a promising solution by modeling the distribution of healthy brain images to identify and localize anomalies without relying on labeled data.Nevertheless, unsupervised approaches face challenges in accurately analyzing structural abnormalities, particularly regions of atrophy, which are critical in AD research [5] . Classical techniques using multi atlas-based deformable registration [13] and morphometry methods [8, 3] have been proposed to analyze these structural changes. However, such methods allow analysis to be conducted only on a population-level, for instance as deviations from an atlas.In this work, we propose Morphological Autoencoders for Alzheimer's Disease Detection (MORPHADE ), a novel unsupervised anomaly detection framework based on deformable autoencoders (AEs) [4] which leverages deformation networks to generate patient-specific anomaly maps from 3D T1-weighted MRI brain scans. These anomaly maps allow not only AD detection, but also crucially reveal the location and degree of atrophy. Our main contributions are as follows:-We use deformation fields in an unsupervised framework to analyze ADrelated changes in the brain. To the best of our knowledge, this is the first use of such an approach using deep learning in the context of AD. -We extend deformable autoencoders to 3D, utilize adversarial training and propose a dual-deformation strategy to improve reconstruction fidelity and the localization of atrophy. -We accurately identify AD-affected brain regions, aligning our findings with clinical expectations. -We assess AD severity by correlating our findings with clinical medial temporal lobe atrophy scores, evaluated by a board-certified clinical expert. -Through comprehensive validation, we demonstrate superior performance in AD detection compared to unsupervised and even supervised baselines. 
background: 
 In unsupervised anomaly detection, reconstruction-based frameworks such as autoencoders (AEs) can be used to learn the distribution of healthy samples and subsequently identify samples that deviate from this norm as anomalous.The encoder E θ maps an input x to a lower-dimensional latent space and then the decoder D ϕ learns to reconstruct from this encoded representation. The parameters θ, ϕ of the AE are optimized given healthy input data χ = {x i , ..., x n } by minimizing the mean squared error (MSE) between the inputs and their reconstructions:M SE = min θ,ϕ N i=1 ||x i -D ϕ (E θ (x i ))|| 2 .(1)It is then assumed that during inference, the AE will generate a so-called pseudo-healthy reconstruction, in which only in-distribution healthy tissue can be successfully reconstructed and thus any reconstruction errors can be thought of as anomalies. A subject-specific map of anomalies can then be obtained by taking the residual between an input x and its reconstruction x recon = D ϕ (E θ (x)) as follows:EQUATIONDeformable Autoencoders (AEs) [4] were proposed as a method to alleviate false positives in the anomaly maps due to the limited reconstruction capabilities of traditional AEs. Since the top layers of the AE contain spatial information, deformable AEs use these layers to estimate a dense deformation field Φ that allows local adaptions of the pseudo-healthy reconstruction to the individual anatomy of the subject. The estimation of the deformation field is optimized using local normalized cross correlation (LNCC):EQUATION)where β is a weight that is kept relatively high to constrain the deformations to be smooth and local, allowing only small changes to the reconstructions. We therefore refer to this part of the network as the constrained deformer. The improved reconstruction, which we refer to as the morphed reconstruction, x morph , can then be obtained byx morph = x recon • Φ.The authors also propose to use perceptual loss (PL) [12] weighted by the hyperparameter α, in addition to the MSE when optimizing the AE parameters, to promote reconstructions that closely resemble the training distribution:EQUATION 
methods and materials: 
 We propose MORPHADE, shown in Fig. 1 , which builds upon deformable AEs. Firstly, we employ a 3D convolutional AE to enable the use of 3D images with the framework. Secondly, since PL uses 2D networks pre-trained on ImageNet, we employ an adversarial loss [9] to increase the realness of the reconstructions. We train a discriminator by minimizing this adversarial loss; therefore, the reconstruction loss becomes:EQUATIONwhere γ balances the production of realistic reconstructions while maintaining pixel-wise accuracy.Our major extension to the deformable AEs is the use of a dual-deformation strategy, in which we employ an unconstrained deformer in addition to the constrained deformer, with the aim of improving the localization of atrophic regions. As previously stated, the constrained deformer is trained with a high value of β to improve the generation of the pseudo-healthy reconstructions and thus reduce false positives in the anomaly maps. In contrast, the unconstrained deformer has the goal of reverting the pseudo-healthy reconstruction back to its original anomalous state. The deformer is trained with the same loss as in Eq. 3, but with Fig. 1 : Our approach, MORPHADE, integrates a dual-deformation strategy with a 3D autoencoder and adversarial training. The constrained deformer refines the reconstruction to generate a residual map with reduced false positives, while the unconstrained deformer is used to produce a folding map that highlights anomalies. The residual and folding maps together produce an anomaly map that allows the localization and assessment of the severity of atrophy. a low value of β, which allows the creation of unconstrained deformation fields. In such deformation fields, low values of deformation should occur in areas of healthy tissue. Conversely, in regions of atrophy, the deformation field exhibits foldings, or areas in which the mapping of the deformation from the pseudohealthy reconstruction to the original image is not one-to-one due to the loss of tissue volume. The determinant of the Jacobian of the deformation map, J Φ , can be used to determine local volume changes, with negative values indicating such foldings. Therefore, we highlight the anomalies by using the negative Jacobian values to generate a map of the foldings, m f oldings = max(0, -det(J Φ )).We finally multiply these foldings pixel-wise with the residual map from the constrained deformer to generate an anomaly map with reduced false positives and improved atrophy localization:EQUATIONImplementation. All networks were trained with Adam optimizer. The discriminator was trained with a learning rate of 1.0e -4 , otherwise 5.0e -4 was used. The framework was first trained with a high value of β = 10. We motivate this choice in Fig. 2a , where we show that using decreasing values of β during training results in blurrier reconstructions. Conversely, a high β value ensures that the AE does not overly rely on the deformations to achieve faithful reconstructions, but is instead forced to learn an accurate representation of the in-distribution data. After 200 epochs, the weights of these models were kept frozen while the deformation parameters were optimized for 100 epochs. At inference, we use a high value of β = 10 to obtain the residual maps and a low value of β = 0.01 to generate the folding maps. We demonstrate the need for lower β values to produce improved folding maps in Fig. 2b , where it can be seen that using low values accentuates the anomalous regions in the brain.Dataset and Preprocessing. Data used in the preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu) [19] .We used skull-stripped T1-weighted MPRAGE images of both male and female patients that are registered to the MNI brain template [17] . Our training set comprised 760 healthy control (HC) samples, with an additional 95 HC samples utilized for validation purposes. For the supervised baseline training, an additional 430 AD samples were used. The test set included 215 HC samples and 200 samples with AD. 
experiments and results: 
 Atrophy Localization. We first validate the effectiveness of our method in identifying atrophy in sub-cortical brain regions affected by AD. To achieve this, we used the FSL FIRST tool [18] to segment these regions and compute mean anomaly scores for each, shown in Fig. 3 . Our results indicate that AD patients exhibit notably higher anomaly scores in the hippocampus (left: 0.282 ± 0.495, right: 0.185 ± 0.382) and amygdala (left: 0.132 ± 0.207, right: 0.108 ± 0.208) compared to the hippocampus (left: 0.108 ± 0.193, right: 0.069 ± 0.125) and amygdala (left: 0.066 ± 0.115, right: 0.072 ± 0.111) for the healthy controls. These results are in line with the clinical expectation of these regions being sig-Fig. 3 : Anomaly scores for subcortical brain regions for Alzheimer's Disease (AD) and Healthy Control (HC) samples, showcasing markedly higher scores for AD samples in the hippocampus and amygdala, consistent with clinical literature. [6] Fig. 4 : Anomaly maps for AD patients alongside their corresponding medial temporal lobe atrophy (MTA) scores, demonstrating consistent alignment with ADrelated structural changes and clinical MTA assessments. nificantly affected by AD pathology [6] , indicating that MORPHADE is able to identify atrophy in clinically relevant brain regions.Atrophy Severity. We next evaluate the ability of our method to determine the severity of the localized anomalies by comparing our anomaly maps to medial temporal lobe atrophy (MTA) scores [20] that were assessed by a senior boardcertified neuroradiologist. These scores range from 0 to 4 and are assigned based on the degree of structural changes observed in the choroid fissure, the temporal horn of the lateral ventricle, and the hippocampus. Fig. 4 shows a visual correlation between the degree of atrophy highlighted in the anomaly map in these key regions and the MTA scores, demonstrating the utility of our method in determining the severity of detected anomalies. Pathology Detection. In this section, we assess the capability of MORPHADE in detecting AD at the patient level. Table 1 shows the Area Under the Receiver Operating Characteristic curve (AUROC) scores obtained when comparing our method to various baselines for identifying subjects with AD compared to healthy control (HC) subjects. Our model achieves an AUROC of 0.80, surpassing even the 3D supervised baselines ResNet [14] and DenseNet [10] , with AUROCs of 0.77 and 0.74, respectively. Furthermore, we obtain improved performance compared to methods proposed for unsupervised anomaly detection. These methods are only available in 2D, so were assessed slice-wise with the final anomaly scores obtained by averaging over the slices for each patient. f-AnoGAN [21] , Ganomaly [2] obtained AUROCs of 0.70 and 0.72, respectively. We also outperform Brainomaly [22] (AUROC 0.78), a method that is not strictly unsupervised since it requires pathological samples during training for improved performance.We also compare our results to a 3D adversarial AE to illustrate the benefit of utilizing the deformation fields with our method. Fig. 5 shows the reconstructions and residual maps obtained for both methods in representative AD and healthy controls (HC) subjects. Our method produces more refined reconstructions compared to the adversarial AE, shown by the improved MAE and SSIM scores. Moreover, the residual maps show fewer false positives for the healthy subject, while accentuating pathological areas for the AD subject. Using these improved residual maps alone for AD detection achieves a superior performance of AUROC 0.77 compared to 0.74 obtained by the adversarial AE.Finally, we demonstrate the utility of our dual-deformation approach, where AD identification was superior using our method compared to using only the residual maps from the constrained deformer (AUROC 0.77) or the folding maps from the unconstrained deformer (AUROC 0.79). Notably, the high performance of the folding maps underscores their effectiveness in detecting anomalies without relying on image differences between the input and reconstructions. The morphological adjustments facilitated by MORPHADE enhance reconstruction fidelity, yielding higher Structure Similarity Index (SSIM) values for our method's morphed reconstructions compared to those of the adversarial AE. The residual maps also demonstrate fewer reconstruction errors for the healthy subject, while highlighting atrophy for the subject with AD. 
discussion and conclusion: 
 In this work, we introduced MORPHADE, a novel framework leveraging 3D deformable AEs for unsupervised analysis of Alzheimer's Disease using T1weighted brain MRI. Our approach is unique in employing deformation fields within an unsupervised learning context to analyze, localize, and assess the severity of AD-related atrophy.Our results demonstrate that MORPHADE can effectively identify and localize atrophy in clinically relevant brain regions, such as the hippocampus and amygdala, which aligns with clinical expectations of AD pathology. Furthermore, the anomaly maps generated by our method show strong visual correspondence with MTA scores, underscoring the potential of our method in clinical assessments. Lastly, MORPHADE achieved an AUROC of 0.80 in detecting AD, outperforming several supervised and unsupervised baselines. This highlights the robustness of our method without requiring extensive labeled datasets, addressing a significant limitation in current diagnostic approaches.Future work could explore integrating MORPHADE's deformation metrics with established AD biomarkers, such as tau protein accumulation and amyloidbeta levels, to enhance understanding of disease progression. Additionally, expanding our framework to other neurodegenerative diseases could further validate its versatility and clinical utility.In conclusion, MORPHADE offers a promising tool for localization, and severity assessment of AD-related atrophy, contributing valuable insights into the progression and diagnosis of neurodegenerative diseases. Our findings suggest that this approach could significantly enhance the non-invasive monitoring and understanding of AD, paving the way for improved patient outcomes. 
 
 Appendix:  


Decision: The paper is a highly commendable piece of research, showcasing a novel and effective method for unsupervised Alzheimer's Disease detection using 3D brain MRI. The introduction of the dual-deformation strategy and deformable autoencoders represents a significant advancement in the field. The authors have comprehensively addressed all minor concerns the reviewers raised, demonstrating a commitment to clarity and robustness in their methodology. Given the strong experimental results, clinical relevance, and thorough responses to reviewer feedback, this paper is well-rounded and founded.

Reviews: review: main_review: main_review: My recommendation is based on several key factors:  Novelty: MORPHADE presents a unique approach combining 3D deformable autoencoders with a dual-deformation strategy for unsupervised AD analysis. This addresses limitations of both supervised approaches and previous unsupervised techniques. Strong Performance: The method achieves an AUROC of 0.80 in AD detection, outperforming both supervised and unsupervised baselines. This demonstrates its effectiveness and potential to advance the state-of-the-art. Clinical Relevance: The framework generates interpretable anomaly maps correlating with clinical MTA scores, enhancing potential for practical application in healthcare settings. Comprehensive Evaluation: The authors validate their method through multiple experiments, including atrophy localization, severity assessment, and AD detection, strengthening the credibility of the results. Unsupervised Approach: By not requiring labeled training data, MORPHADE addresses a significant challenge in medical image analysis, potentially enabling more widespread and efficient application of AI in AD research and diagnosis. Technical Soundness: The methodology is well-grounded in existing literature while introducing novel elements. The use of adversarial training and dual-deformation strategy are well-justified and effectively implemented.  While there are areas for improvement (e.g., more detailed ablation studies, longitudinal analysis, broader dataset validation), these do not significantly detract from the paper's overall contribution. Instead, they represent opportunities to further strengthen an already solid piece of research. The combination of novelty, strong performance, clinical relevance, and comprehensive evaluation makes this paper a valuable contribution to the field of AD research and medical image analysis. It not only advances the technical state-of-the-art but also has clear potential for real-world impact in improving AD detection and monitoring.
strength_weakness: strength_weakness: 1. Limited dataset: While the ADNI dataset is widely used, validating on additional datasets would strengthen the generalizability claims.
2. Computational requirements: The paper doesn't discuss the computational resources needed for training and inference, which could be important for practical implementation.
3. Limited comparison to other unsupervised methods: While some unsupervised baselines are included, a more comprehensive comparison to state-of-the-art unsupervised AD detection methods would be beneficial.
review_summary: review_summary: This paper introduces MORPHADE, a novel unsupervised framework for analyzing Alzheimer's Disease using 3D brain MRI. The method employs deformable autoencoders with a dual-deformation strategy to generate patient-specific anomaly maps without labeled data. MORPHADE accurately identifies AD-affected brain regions, correlates with clinical assessments, and outperforms both supervised and unsupervised baselines in AD detection. The approach offers a promising tool for improved AD understanding, monitoring, and detection, potentially enabling more efficient and accurate diagnoses.
main_review: main_review: Overall, paper is well written, with clear motivation, experiments and results. It would be a good contribution to the workshop.
strength_weakness: strength_weakness: * It is not clear how different values of $\beta$ can be used during inference. According to Eq. 3, $\beta$ is a part of the training loss function, and used during inference.  
* It is not clear how $m_{foldings}$ was normalized and used in Eq. 6.
* Maybe it would be a good idea to provide statistical significance test with folded validation.
review_summary: review_summary: The paper proposes to deformation with autoencoders for unsupervised learning to detect AD and localize atrophy in brain MRI.
main_review: main_review: Three main weaknesses identified in the paper (i.e., lack of significance tests, lack of diverse experiments, and discrepancy between reported results and prior work) can be addressed in the authors' rebuttal to raise the score but justify the current assignment.
strength_weakness: strength_weakness: 1. The experiments do not include any information about statistical significance. For example, in Table 1, both MORPHADE and MORPHADE with only folding maps are bold (which are described to indicate "best results"), but in the last paragraph of page 7, the authors mention that the results from Table 1 indicate superiority from the combined approach, which is contrary to the presentation. Can the authors thus explain if any statistical tests were performed or which statement is correct?
2. Discrepancies between results of competing methods and original publication, e.g., brainomaly performance in the original paper [1], are significantly lower than in this paper, although they seem to use the same benchmark. Can the authors explain this difference?
3. The paper would benefit from experiments with an openly available dataset to ensure the reproducibility of their evaluation and a broader comparison of their methodology.

Reference:
[1] Siddiquee, M. M., Shah, J., Wu, T., Chong, C., Schwedt, T. J., Dumkrieger, G., Nikolova, S., & Li, B. (2023). Brainomaly: Unsupervised Neurologic Disease Detection Utilizing Unannotated T1-weighted Brain MR Images.
review_summary: review_summary: This paper introduces MORPHADE, a framework that employs unsupervised learning techniques to detect and localize Alzheimer's Disease in MRI images. The main contribution is the use of 3D deformable autoencoders and a dual-deformation strategy to generate anomaly maps.
main_review: main_review: The idea is interesting, method is novel and well explained, both quantitative and qualitative results are shown. The paper qualifies a workshop publication.
strength_weakness: strength_weakness: More baseline methods can be compared. Improvements from baselines are marginal.
review_summary: review_summary: This paper uses deformation fields in unsupervised AD changes detection. This method is in 3D and adversarial training is introduced in the training process.

Comments: title: Rebuttal by Authors
Rebuttal by Authors
Rebuttal by Authors
Rebuttal by Authors
comment: We thank the reviewer for the thoughtful and constructive feedback and your recognition of novelty and clarity of our work.

Our primary focus and contribution lie in the accurate localization and severity assessment of Alzheimer’s Disease. Furthermore, we have compared our approach with several supervised and unsupervised methods in terms of classification performance. However, we may extend our experiments in future work to include more state-of-the-art methods, appreciating any suggested comparisons from the reviewer. 

Our detection results demonstrate improvements even over supervised methods; however, our primary focus extends beyond mere classification accuracy. We aim to localize and assess the severity of structural brain changes due to Alzheimer's Disease through an innovative use of deformations. Our approach generates detailed anomaly maps, providing valuable insights into affected brain regions. It is also important to note that we do not train the networks for classification directly which makes them less susceptible to shortcut learning (as opposed to supervised approaches) and might therefore show improved detection. However, we are eager to explore further aspects of performance, interpretability, early detection, and the clinical impact on patients in future work.
We thank the reviewer for the detailed and constructive feedback and appreciate the positive comments regarding the novelty of our work, and the positive correlation with clinical assessments.
1. By presenting both “MORPHADE” and “MORPHADE with only folding maps” in bold in Table 1, we aimed to emphasize the fact that while MORPHADE itself (combining residual maps and folding maps together) achieves the best performance, using only the folding maps alone also demonstrated strong results. This highlights the effectiveness of using deformations for detecting anomalies, without relying on differences between input and reconstruction. While we did not perform statistical significance tests, we will improve the clarity of our results, in particular using a different way to emphasize the results of "Only with folding maps".
2. Discrepancies between results: We tried our best to replicate the methods from their original implementations. We hypothesize that variations in the subset and sample sizes may result in performance differences.
3. More experiments: Our experiments aimed to demonstrate the efficacy of our method in localizing brain atrophy associated with Alzheimer’s Disease. We analyzed highlighted regions that aligned well with clinical expectations. Additionally, we compared our approach with clinically established MTA scores, evaluated by an expert neuroradiologist, underscoring our method's potential for precise localization and disease severity assessment. Furthermore, we showcased our approach surpassing several supervised and unsupervised baseline methods in terms of classification performance. We welcome suggestions for additional experiments to further extend our findings in future research. Your input would be greatly appreciated.

The ADNI database is already publicly accessible and can be utilized for reproducibility and further research. We have also released a complete csv file with the different subject and scan ids used in our study. We plan to extend our experiments to include more datasets and different pathologies in future work.

We appreciate the suggestion to revise our statement regarding robustness. We agree with the reviewer and will revise the statement in the final version.
We would like to thank the reviewer for the positive evaluation and the constructive feedback.

We acknowledge the need for further clarity regarding the use of different $\beta$ values. In our final version, we will revise the explanation in the last paragraph of page 4 and will change  "$\beta_{train}$" to  "$\beta_{constrained}$", and "$\beta_{inference}$" to "$\beta_{unconstrained}$" where necessary. Briefly, $\beta$ is used only during training in two phases:

**1. High β Phase:** We train the autoencoder (AE) and constrained deformer with a high $\beta$ to reduce blurriness in the AE reconstructions. 

**2. Low β Phase:** We then freeze the AE weights and only train a second deformer with a smaller $\beta$, which we call ‘unconstrained’. The unconstrained deformer generates folding maps during inference.

**3. Inference:** Finally, we perform inference with both the constrained (for obtaining residual maps) and unconstrained (for obtaining the folding maps) deformers.

Regarding the folding maps, $m_{foldings}$, in Equation 6: We use $m_{foldings}$ directly without normalization, however, we use gaussian filtering to smooth them. We will incorporate this into the revised version.

Additionally, we greatly appreciate the reviewer’s suggestions for future work. Adding statistical tests and experimenting with local pathology, such as MS lesions, will be a valuable extension to demonstrate the effectiveness and broader applicability of our method. Furthermore, exploring the use of hypernetworks to control the effect of hyperparameters like $\beta$ is an excellent idea that we plan to explore in future studies.
We thank the reviewer for the thorough and insightful feedback and for acknowledging the novelty, strong performance, clinical relevance and comprehensive evaluation of our work.

We conducted training and inference using an NVIDIA RTX A6000 GPU with 48GB of memory. The inference time for a single 3D volume using a plain autoencoder is ~10 ms. Our method is computationally efficient, with an inference time of ~60 ms; the additional use of deformation enables the analysis of atrophy without significantly increasing computational burden. We will include this information in the final version, if space permits.

We showcased our method's effectiveness on the ADNI dataset, which is widely recognized and utilized for Alzheimer’s Disease research. The MICCAI guidelines prevent further experiments, however, we plan to extend our experiments to include more open-source datasets and different pathologies in future work.

We are also grateful for the recommendations for future work. Our primary focus and contribution lie in the accurate localization and severity assessment of Alzheimer’s Disease and we have compared our approach with several supervised and unsupervised methods in terms of classification performance. However, we may extend our experiments in the future to include more state-of-the-art methods, appreciating any suggested comparisons from the reviewer. We will also consider expanding the ablation studies and conducting further analysis longitudinally, as well as with subjects with mild cognitive impairment in future work.