Title: Hierarchical Information Aggregation for Incomplete Multimodal Alzheimer's Disease Diagnosis
Venue: NeurIPS 2025
Accepted: yes
Keywords: Incomplete Multimodal Learning

State Space Model

Alzheimer's Disease Diagnosis
Abstract: Alzheimer's Disease (AD) poses a significant health threat to the aging population, underscoring the critical need for early diagnosis to delay disease progression and improve patient quality of life. Recent advances in heterogeneous multimodal artificial intelligence (AI) have facilitated comprehensive joint diagnosis, yet practical clinical scenarios frequently encounter incomplete modalities due to factors like high acquisition costs or radiation risks. Moreover, traditional convolution-based architecture face inherent limitations in capturing long-range dependencies and handling heterogeneous medical data efficiently. To address these challenges, in our proposed heterogeneous multimodal diagnostic framework (HAD), we develop a multi-view Hilbert curve-based Mamba block and a hierarchical spatial feature extraction module to simultaneously capture local spatial features and global dependencies, effectively alleviating spatial discontinuities introduced by voxel serialization. Furthermore, to balance semantic consistency and modal specificity, we build a unified mutual information learning objective in the heterogeneous multimodal embedding space, which maintains effective learning of modality-specific information to avoid modality collapse caused by model preference. Extensive experiments demonstrate that our HAD significantly outperforms state-of-the-art methods in various modality-missing scenarios, providing an efficient and reliable solution for early-stage AD diagnosis.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of heterogeneous multimodal data integration for ADNI diagnosis, specifically combining 3D medical images and tabular data. The authors introduce a novel fusion framework that leverages Mamba and Hilbert curves to capture both spatial locality and long-range dependencies in 3D imaging. To improve multimodal representation learning, they formulate a mutual information maximization objective that jointly promotes label discriminability and semantic consistency. Importantly, the method incorporates a reversed weighting scheme to mitigate modality dominance caused by imbalanced convergence. The proposed model achieves state-of-the-art performance across varying levels of modality missingness, and the visualizations effectively illustrate its ability to retain label-discriminative features.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is well presented with a clear research scope and motivation, making it accessible and easy to follow. \n2. The proposed method is innovative and demonstrates strong performance on the selected benchmarks, highlighting its potential contribution to the field.\n\nWeaknesses:\n1. The paper does not sufficiently justify or analyze the efficacy of the loss function used for enforcing modality consistency (second term in the mutual information loss function), which is critical given the multimodal nature of the task. \n2. The reasoning and analysis in modality imbalance are comparatively weak as the figure can not sufficiently demonstrate the issue. More evidence should be provided on the statement \"Our hybrid late-fusion strategy resolves this imbalance by allowing synchronized optimization across all modalities.\"\n3. The paper lacks proper citation and discussion of a recently published paper on using 3D Hilbert curve to model 3D medical images: \n\"Grela, Jacek, et al. \"Using space-filling curves and fractals to reveal spatial and temporal patterns in neuroimaging data.\" Journal of Neural Engineering 22.1 (2025): 016016.\"\n4. The paper does not include a comparison of model complexity or computational cost, which is essential for evaluating the method’s practicality and efficiency relative to existing approaches.\n\nQUESTIONS:\nPlease refer to the \"Weaknesses\" part in \"Strengths And Weaknesses*.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a heterogeneous multimodal framework, termed HAD, for the early diagnosis of Alzheimer’s disease (AD), targeting two primary challenges: modality incompleteness and long-range dependency modeling. The framework offers two key technical contributions. First, it integrates a Hilbert-curve-ordered Mamba module with a hierarchical feature extraction strategy to simultaneously capture both local and global information. Second, it introduces a unified mutual information learning objective that explicitly balances semantic consistency and modality specificity within a shared multimodal embedding space. The authors validate the effectiveness of their method under varying modality-missing scenarios, demonstrating its potential in early AD diagnosis.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This study targets the joint diagnosis of Alzheimer’s disease (AD) using multimodal data and explicitly addresses the issue of modality incompleteness in real-world clinical settings, highlighting its practical relevance.\n2. The method incorporates a Mamba module with Hilbert-curve-based scanning. Ablation studies demonstrate that this approach outperforms conventional alternatives on the utilized dataset.\n3. The proposed model accounts for both semantic consistency across modalities and modality-specific characteristics during the multimodal representation learning phase, enhancing representational quality.\n\nWeaknesses:\n1. Although the paper claims to address the challenge of missing modalities, the method section lacks dedicated mechanisms explicitly designed for this purpose. This discrepancy undermines the stated objective.\n2. The authors should clearly differentiate between the original contributions and the adopted components in the multimodal semantic representation learning module, and provide appropriate citations. Furthermore, the mechanism for preserving modality-specific information is insufficiently explained and lacks a solid theoretical foundation—more elaboration is needed.\n3. The experimental results do not convincingly demonstrate the superiority of the proposed fusion strategy. The method does not outperform baselines under a low missing rate (10%), and only shows advantages at a high missing rate (50%), raising concerns about its stability and robustness. A clear explanation for this performance variance is required.\n4. The current empirical evaluation is not fully convincing. To strengthen the evidence for the model’s effectiveness, additional experiments on other public AD diagnosis datasets are strongly recommended. Furthermore, comparisons with recent state-of-the-art methods such as MMTFN [1], MFASN [2], ShareGAN [3], and DSIP [4] should be included.\n5. The provided t-SNE visualizations do not effectively support the claims. Intermediate representations under certain modalities lack class separability, and thus cannot directly imply negligible contributions to classification performance. Moreover, the feature distributions across scenarios are not sufficiently distinct to substantiate the claim of a “strong modality preference.” The interpretability of the visualization needs improvement.\n6. The manuscript suffers from several clarity issues that hinder readability. For instance, the abstract mentions a \"spatial discontinuity caused by voxel serialization\" without offering adequate background context. Terminological inconsistencies are also present—for example, the architecture diagram shows the hierarchical feature extraction (HSFE) module as encompassing the HMamba module, while the main text describes them as separate, parallel modules. This inconsistency creates confusion about the actual structure of the model.\n\n[1] Miao S, Xu Q, Li W, et al. MMTFN: Multi‐modal multi‐scale transformer fusion network for Alzheimer's disease diagnosis[J]. International Journal of Imaging Systems and Technology, 2024, 34(1): e22970.\n\n[2] Jiao C N, Gao Y L, Ge D H, et al. Multi-modal imaging genetics data fusion by deep auto-encoder and self-representation network for Alzheimer's disease diagnosis and biomarkers extraction[J]. Engineering Applications of Artificial Intelligence, 2024, 130: 107782.\n\n[3] Wang C, Piao S, Huang Z, et al. Joint learning framework of cross-modal synthesis and diagnosis for Alzheimer’s disease by mining underlying shared modality information[J]. Medical Image Analysis, 2024, 91: 103032.\n\n[4] Xu H, Wang J, Feng Q, et al. Domain-specific information preservation for Alzheimer’s disease diagnosis with incomplete multi-modality neuroimages[J]. Medical Image Analysis, 2025, 101: 103448.\n\nQUESTIONS:\n1. The paper lacks a comprehensive review of related work in the field of Alzheimer’s disease (AD) diagnosis. It is recommended that the authors survey recent advances in this domain and incorporate a summary of representative methods. Moreover, the experimental section should include comparisons with established AD diagnostic baselines to more accurately position the contributions of this study.\n2. Regarding the use of five-fold cross-validation, there is a potential risk of data leakage. To ensure rigorous evaluation, the authors are advised to implement redundancy removal between training and testing sets.\n3. The paper modifies the original architecture or prediction layers of some baseline methods during comparative experiments, raising concerns about the fairness of these comparisons. The authors should clearly justify the necessity of such modifications and explain how fair evaluation conditions were maintained across all compared methods. Additionally, detailed training configurations—including hyperparameters, optimizers, and training epochs—should be provided to ensure reproducibility and transparency.\n4. The current experimental results are not sufficiently convincing. To strengthen the empirical support for the proposed method, further validation on additional publicly available AD diagnosis datasets is strongly encouraged. For clarity, the authors are also advised to explicitly highlight the top-1 and top-2 performing methods in all result tables.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel hierarchical information aggregation framework (HAD) for incomplete multimodal diagnosis of Alzheimer's Disease (AD). This framework integrates heterogeneous data types, including MRI, PET, cerebrospinal fluid (CSF), and clinical assessment data (CAD)—even when some modalities are missing. Key innovations include a Hilbert curve-based Mamba block for efficient feature extraction, a hierarchical multi-scale feature module, and a mutual information-based objective to balance modality-specific and shared representations. Experiments on the ADNI dataset show that HAD outperforms recent baselines under various missing modality scenarios.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The proposed HAD framework combines multi-view Hilbert curve-based feature extraction, hierarchical modeling, and mutual information-based representation learning, which is technically sound and well-motivated.\n- The methodology is generally well described, and the paper is well organized logically.\n- The hybrid late-fusion and spatial continuity preservation ideas are innovative.\n\nWeaknesses\n- The paper does not include single-modality baselines, making it difficult to evaluate the true benefit of multimodal fusion over strong unimodal predictors.\n- The procedure for random masking of modalities relative to the cross-validation split is not clearly described, raising concerns about experimental fairness and reproducibility.\n- Many baselines are modified to use a ResNet-50 backbone, but there is little detail on their training protocols or alignment with the proposed method, which may affect the fairness of the comparisons.\n- The impact of key architectural choices—such as the number of hierarchical levels in HSFE and the scanning strategy in Mamba—is not systematically explored through controlled experiments.\n\nQUESTIONS:\n- Q1: Can the authors include single-modality baseline results (i.e., models trained on each modality independently) to provide a clearer reference for the value of multimodal fusion?\n- Q2: Is the random masking of modalities performed before or after the cross-validation split? Are multiple random seeds or runs used to ensure robust and reproducible results?\n- Q3: Can the authors provide more details on the training and tuning procedures for the modified ResNet-50 baselines, and clarify whether these protocols are aligned with those of the proposed method?\n- Q4: Can the authors provide additional experiments or analysis on how the number of HSFE hierarchical levels and the choice of scanning strategies in Mamba affect the model’s performance?\n- Q5: In Tables 3–5, please consider highlighting the best results to improve the readability and transparency of the reported comparisons.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenges of incomplete and heterogeneous multimodal data in Alzheimer’s disease diagnosis by proposing a unified diagnostic framework that can handle missing modalities and diverse data structures. The framework combines a multi-view HMamba module for efficient 3D image modeling, a hierarchical spatial feature extractor, and a mutual information–based semantic learning strategy to robustly integrate heterogeneous modalities.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths.**\n\nThis paper presents a clear and well-defined contribution by proposing a novel framework to address the challenges of incomplete and heterogeneous multimodal data in Alzheimer’s disease diagnosis. The proposed method is supported by solid mathematical formulations with each component clearly described and logically integrated. Furthermore, the paper includes extensive experiments to demonstrate the effectiveness of the approach and a well-organized appendix that provides additional implementation details and analyses to enhance clarity.\n\n**Weaknesses.**\n\nWhile the proposed HMamba module, which incorporates Hilbert curve–based sequences, is conceptually well-founded and architecturally coherent, there remain several aspects that warrant further clarification and refinement.\n\n(1) First, although the use of Hilbert curves is conceptually motivated, the paper does not provide dedicated experiments or ablation studies to validate the effectiveness of this design choice, making it difficult to assess its contribution. \n\n(2) Second, while the paper states that a reverse indexing operation is applied to map the serialized outputs $o^l$ back to the 3D voxel space, the mechanism is not clearly described through equations or architectural diagrams. As a result, it is unclear how the spatial alignment between $o^l$ and the voxel-wise weights $a^l$ is ensured, making the fusion process less transparent and potentially hindering reproducibility.\n\n(3) Third, some of the figures and tables suffer from limited visual clarity, which may hinder reader comprehension. For example, in Figure 1, the directional flow of information is not clearly indicated due to ambiguous arrow styles. In Figure 3, the color distinction between 30% and 50% missing rates is not clearly distinguishable. It makes difficult to distinguish between conditions at a glance. Additionally, including SEN and SPE comparisons alongside ACC, AUC and F1 in Figure 3 would provide a more comprehensive evaluation. It would also improve readability if the best-performing models in Tables 3 to 5 were highlighted in bold.\n\nThis does not undermine the core idea of the paper, but suggest areas where clarity and reproducibility could be improved.\n\nQUESTIONS:\nQ1. In this paper, the baseline models were adapted to fit the proposed tasks. Most of them appear to use ResNet-50 as the backbone. Could you elaborate on the rationale behind choosing ResNet-50 for these modifications?\n\nQ2. The classification experiments are conducted across three binary tasks (e.g., AD vs. CN). Is there a specific reason for not including a 3-class classification setting (AD vs. MCI vs. CN), which could provide a more unified evaluation of the model’s overall capability?\n\nQ3. In Figure 2, multiple multi-view sequences are shown. It is unclear whether all views are treated equally or if different weights are applied based on their importance. Does the model include any attention mechanism across views, or could this be a potential direction for improvement?\n\nQ4. The proposed framework includes both the Hierarchical Spatial Feature Extractor (HSFE) and the HMamba module. However, no ablation studies are isolating the contribution of each module. If available, could you share any results that quantify the individual impact of these components on overall performance?\n\nLIMITATIONS:\nThe authors appropriately address the main limitations of the study. No additional limitations were identified during my review.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}