Hierarchical Information Aggregation for Incomplete Multimodal Alzheimer's Disease Diagnosis
Authors: Chengliang Liu
Que Yuanxi
Qihao Xu
Yabo Liu
Jie Wen
Jinghua Wang
Xiaoling Luo
Venue: NeurIPS 2025 poster ()
Keywords: Incomplete Multimodal Learning
State Space Model
Alzheimer's Disease Diagnosis
Primary area: deep_learning
TL;DR: The framework is capable of processing multimodal data with arbitrary missing modalities, providing a flexible solution for AD diagnosis.

Abstract: Alzheimer's Disease (AD) poses a significant health threat to the aging population, underscoring the critical need for early diagnosis to delay disease progression and improve patient quality of life. Recent advances in heterogeneous multimodal artificial intelligence (AI) have facilitated comprehensive joint diagnosis, yet practical clinical scenarios frequently encounter incomplete modalities due to factors like high acquisition costs or radiation risks. Moreover, traditional convolution-based architecture face inherent limitations in capturing long-range dependencies and handling heterogeneous medical data efficiently. To address these challenges, in our proposed heterogeneous multimodal diagnostic framework (HAD), we develop a multi-view Hilbert curve-based Mamba block and a hierarchical spatial feature extraction module to simultaneously capture local spatial features and global dependencies, effectively alleviating spatial discontinuities introduced by voxel serialization. Furthermore, to balance semantic consistency and modal specificity, we build a unified mutual information learning objective in the heterogeneous multimodal embedding space, which maintains effective learning of modality-specific information to avoid modality collapse caused by model preference. Extensive experiments demonstrate that our HAD significantly outperforms state-of-the-art methods in various modality-missing scenarios, providing an efficient and reliable solution for early-stage AD diagnosis.

Hierarchical Information Aggregation for Incomplete Multimodal Alzheimer's Disease Diagnosis
Chengliang Liu Yuanxi Que'. Qihao Xu²,
Yabo Liu',
Jie Wen', Jinghua Wang', Xiaoling Luo'
Cooleeee of Computer Science and Software Engineering, Shenzhen University
2 Labboraorry for Artificial Intelligence in Design, The Hong Kong Polytechnic University School of Computer Science and Technology. Harbin Institute of Technology, Shenzhen *College of Artificial Intelligence, Ocean University of China 1 suuc11966163..o com, {queyuanxi, xqh51199597, siaolingluuo)>oottloooo com,
yaboliu. www.:iiuu.ggmmii.ccomm jiewen_prd126 com, wangjh20120f oxmp))haaloom.aa.......m www.al.cmlll.ooom com
Abstract
Alzheimer's Disease (AD) poses a significant health threat to the aging population, underscoring the critical need for early diagnosis to delay disease progression and improve patient quality of life. Recent advances in heterogeneous multimodal arti- ficial intelligence (AI) have facilitated comprehensive joint diagnosis, yet practical clinical scenarios frequently encounter incomplete modalities due to factors like high acquisition costs radiation risks. Moreover, traditional convolution-be based architecture face inherent limitations in capturing long-range dependencies and handling heterogeneous medical data efficiently. To address these challenges, in our proposed heterogeneous multimodal diagnostic framework (HAD), we develop multi-view Hilbert curve- based Mamba block and hierarchical spatial fea- ture extraction module to simultaneously capture local spatial features and global dependencies, effectively alleviating spatial discontinuities introduced by voxel serialization. Furthermore, to balance semantic consistency and modal specificity, we build unified mutual information learning objective the heterogeneous mul- timodal embedding space, which maintains effective learning of modality-speciff information to avoid modality collapse caused by model preference. Extensive experiments demonstrate that our HAD significantly outperforms state-of-the-are methods in various poddality-missing scenarios, providing an efficient and reliable solution for early-stage AD diagnosis.
Introduction
AD a progressive neurodegenerative disorder characterized by cognitive impairment, gradual loss of memory, and decline in self-care abilities as its primary clinical manifestations [12, 3]. Due to the lack of effective cures, AD poses significant threat the health of the elderly population, severely impacting the quality of life of patients and their families and imposing heavy medical burden on society 4. Mild Cognitive Impairment (MCI) is considered a precursor stage of AD, marked by mild cognitive decline without a noticeable impact on daily functional abilities. Early diagnosis and intervention during this stage are critical for delaying disease progression and improving patients' quality of life. In recent years, with the rapid increase of the type of multimodal data, researchers have been able to better understand and diagnose early-stage AD from multiple perspectives, providing more comprehensive and objective decision- maaiiinnmmaiinggg support for clinical diagnosis and treatment [5]6]
"Corresponding author: Xiaoling Luo (email: xiaolingluoo @ out)ook.com).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

These advances also lay the groundwork for the application of multimodal Artificial Intelligence (AI) in joint AD diagnosis.
However, in practical clinical settings, the collection of multimodal data is often hampered by issues such as radiation risks, high costs, and unexpected patient withdrawal, leading to the frequent problem of missing modalities. This makes it difficult to obtain complete multimodal datasets 77 g] To address missing data, existing studies often discard cases with incomplete modalities and rely solely on data with complete modalities for analysis (9] This approach reduces the subject scale, thereby limiting the performance of models. To overcome this issue, researchers have proposed various multimodal learning frameworks based on strategies such as subspace learning, knowledge distillation, and missing data imputation [TO] For instance, data imputation methods use generative models like Generative Adversarial Networks (GANs) and Autoencoders to fill in missing modalities. thereby expanding the training dataset. However, due to challenges in ensuring the quality of imputed data, such methods often introduce redundant or even misleading information, resulting in decreased performance. Furthermore, the heterogeneity of multimodal data in AD, which ranges from three- dimensional (3D) image data to ID biomarker data, presents additional challenges. Effectively integrating heterogeneous multimodal data and mining their shared semantic information under conditions of missing modalities remains a key difficulty in Al-assisted AD diagnosis.
Currently, mainstream multimodal diagnostic methods are typically based on 3D Convolutional Neural Networks (CNNs) or Transformer architectures [2] [3). However, 3D CNNs often struggle to effectively capture long-range spatial dependencies and are constrained by large parameter sizes and high computational costs [4]. While Transformer- hasee models can model long-range dependencies, their computational complexity increases quadratically with the input data dimensions. This is particularly problematic when dealing with digh-iimensional multimodal 3D medical image data, where computational inefficiency becomes a significant bottleneck, severely limiting their practical clinical applications. Recently, Structured State Space Models (SSMs), exemplified by Mamba, have gained attention for their efficient information extraction capabilities and linear computational complexity. For example, Liu et al. [5] proposed VMamba with a 2D Selective Scan module (SS2D) that bridges the ordered nature of ID selective scan and the non-sequential structure of 2D visual data. Xing et al. [16] developed SegMamba, which captures remote dependencies in the entire 3D voxel at multi-scale. In the context of 3D multimodal medical image, exploring an efficient feature extraction module based SSMs holds the potential to enhance diagnostic performance while reducing model complexity, thereby better meeting the demands of clinical applications.
To address the aforementioned challenges, this paper proposes a novel heterogeneous multimodal diagnostic framework for AD, named HAD. The framework is capable of processing multimodal data that includes Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), Cerebrospinal Fluid (CSF), and Clinical Assessment Data (CAD) with arbitrary modality missing, providing a flexible solution for carly-stage AD diagnosis. One the one hand, to address the chal- lenges of long-range dependencies and high computational complexity in 3D brain image data, we develop hierarchical spatial feature extraction module. Building upon existing work [17], adopt the same Hilbert curves for space-filling transformations while maintaining their core concept of
property of space-filling curves". However, our proposed HSFE module introduces two key innovations: (1) A hierarchical architecture based on the fractal theory of 3D Hilbert curves enables multi-scale information fusion through multi-level recursive structures; (2) A multi-directional scanning mechanism (incorporating axial rotation and mirror transformations) en- hances complementary spatial information capture. This module integrates traditional convolutional operation with efficient state space model, enabling the effective capture of both shallow features and global dependencies in 3D image data. On the other hand, to enhance the consistency of discrimina- tive capabilities across different modalities for AD diagnosis, we propose an optimization objective based on maximizing mutual information between multimodal joint semantic features and shallow features in the semantic embedding space. This strategy ensures that the framework effectively integrates heterogeneous information from various modalities, improving diagnostic performance in scenarios with incomplete multimodal datasets. Our main contributions are summarized as follows:
We propose a heterogeneous multimodal AD diagnosis framework capable of handling arbitrary incomplete modalities. Unlike existing methods primarily tailored for 3D brain image data, our framework can effectively process heterogeneous multimodal data with significant informational and structural differences.

---

To effectively handle complex 3D MRI and PET image data, we propose the multi-view Hilbert curve-based Mamba block (HMamba), along with a hierarchical spatial feature extraction strategy built upon HMamba. These modules alleviate discontinuities introduced by spatial voxel serialization and unify long-sequence modeling with local feature extraction across multiple scales.
We propose multimodal semantic representation learning framework, which establishes a goal of maximizing mutual information between modality-specifie features and semantic labels, simultaneously considering codallty-sspeciff information extraction and consistency representation learning.
2 Preliminary
2.1 Problem Definition
Given a multimodal AD diagnosis dataset D = (x. y) with n subjects, and each subject æ consists of heterogeneous multimodal data (e.g., structural MRI, PET, CSF, and CAD), denoted as (x(2), (,(2), ..... .....,,(())) (m))), ). where denotes the total number of modalities. Noted that we use xmmg to indicate the MRI or PET data. The corresponding diagnostic label is denoted as y, representing the clinical status of the subject (e.g.. cognitively normal (CN), MCI, and AD). In practical clinical scenarios, some modalities might be missing for certain subjects due to various reasons, thus we let V denote the set of available modalities and | S The objective of multimodal AD diagnosis is thus to train a neural network model capable accurately predicting the clinical label y using any available subset of modalities, even when some modalities missing during inference.
2.2 State Space Modals
State Space Models (SSMs) are classical linear time invariant systems widely used in control theory and signal processing, characterized by their linear complexity and effectiveness in modeling sequential data. Recently, SSMs have gained renewed attention in deep learning due to their ability to efficiently capture long-range dependencies. Gu et al. (18) first introduced the HiPPO framework, providing a theoretically optimal approach to represent continuous-ting state-space models by high- order polynomial projections. Subsequently, Gu et al. proposed the Structured State Space Sequence model (S4) [9 that introduces discretization and convolutional representation for parallel training, demonstrating superior performance in processing time series data. More recently, Mamba [20) introduced selective structured state spaces, simplifying the architecture and improving parallelism. Various vision Mamba architectures [15] 21 further extended the state-space modeling paradigm from sequences to two-dimensional image data, effectively addressing the quadratic complexity issue inherent in vision Transformers.
2.2.1 Hilbert Curve for 3D Brain Image Data
Existing Vision Mamba methods typically convert structured 2D or 3D visual data into ID sequences through serialization approaches such as bi-directional scanning, cross scanning, or continuous scanning [22 23 24). However, these simple scanning strategies inevitably disrupt the inherent spatial relationships, causing spatially adjacent pixels voxels to become distant from each other in the serialized sequence. Such spatial discontinuity significantly impairs the model's ability to capture local structural information and long-range spatial dependencies, thereby limiting diagnostic performance in medical imaging tasks. To alleviate this issue, we propose adopting the 3D Hilbert curve for scanning brain image data. Unlike traditional scanning approaches [25] 26 27), the Hilbert curve is continuous, space-filling fractal curve known for its excellent locality-preserving preserving property Specifically, the Hilbert curve mapping ensures that voxels spatially close in the 3D space remain close in the serialized ID representation to a large extent, thus minimizing spatial distortion and aiding the model in effectively capturing local and global contextual information from structured medical image data.
The 3D Hilbert curve can be generated iteratively, with each iteration referred to as the curve's order N. At each iteration, the curve recursively subdivides the original 3D cubic space into 2 x x2N x N sub-cubes, as shown in Fig. [ The Hilbert curve of order N is thus constructed by connecting the Hilbert curves of order (N 1) from eight smaller sub-cubes through rotations and reflections,
3

---

Dynamic MoË Fusion
Classifier
MRI
HSFE
AR
CAL
S
HSFE
4
CAP
PET
Hybrid Late-Fusion
CSF
MLP
40
PET
CAD
MCI AD CN
CAD
MLP
+
MRL
PET
CAD
HSFE: Hierarchical Spatial Feature Extraction
HMamba Block
MD Stem
HMamba
Conv
Scanning
SSM
D
000
HMamba
SSM
DOO
SSM
HMamba
Conw
The schematic diagram of our HAD. It contains 2 main parts, a heterogeneous modality- specific shallow feature extraction consisting of HSFE module and MLP (left half) and high-level multimodal semantic coding (right half). "9," and "9s" denote the -specit encoder and cross-modal decoder, respectively; "3D Conv" denotes the 3D residual convolution module.
Figure 1:
preserving the locality and continuity of the space-filling curve. Formally, given a voxel coordinate (Te. Yes ze) within cubic voxel grid of size 2 the 3D Hilbert curve defines mapping Hy (-) from the 3D coordinate to ID sequence index h:
H (I Me: ze), he (0, .......,.. 8 1}.
(1)
appropriate Hilbert curve order N is selected based on the input image data resolution.
Method
3.1 HMamba: Hilbert Curve-Based Mamba Block
Multi- View Spatial Scanning. As illustrated in Fig. spatially adjacent voxels located at the boundaries between neighboring sub-cubes might become rela- tively distant within the serialized sequence due to the intrinsic fractal structure of the Hilbert curve. This spatial tearing phenomenon potentially leads to the loss of critical adjacency information, adversely af- fecting the model's ability to capture fine-grained spatial dependencies. To bridge this gap and further alleviate the spatial discontinuity issue, we propose utilizing multiple Hilbert curves oriented along dif- ferent spatial axes, complementing the conventional approach of scanning with Hilbert curve along a single orientation. In other words, we expect that multiple Hilbert curves collectively maximize the coverage of adjacency edges among voxels, ef- fectively preserving comprehensive spatial context within serialized data. A higher coverage rate indicates better preservation of spatial adjacency information and thus facilitates more effective modeling of spatial context (please refer to Appendix A.3 for the definition of coverage rate).
5)
Numverr
4.56
N=S
N52 of Hilbert curve
NEs
MF4
Figure 2: Coverage rate vs. order of Hilbert curve under diffferent view numbers.

---

As we know, starting from fixed vertex, six independent curves can be drawn. Therefore, in Fig. 2 we illustrate the relationship between the number of scanning views and the coverage rate. At lower orders, viee scanning is sufficient to fully cover all adjacent edges. However, as image resolution increases, achieving complete coverage becomes more challenging. Furthermore, increasing the number of scanning curves should be considered with caution due to the additional computational overhead. Overall, for input x: zmmg
E IRdxxd;x4 xd; L multi-view scanning layers are
denoted as (F's : winng E Rd: Xd: xd, - R4 WE
Multi- View Dynamic Fusion-Rased Mamba Block. Upon serializing the input 3D image data, we employ the SSMs to capture long-range dependencies across the resulting long sequences, which originate from continuous-tiie linear dynamical systems, mapping input 27 E IR to output or E IR via a hidden state he E Rd as follows:
h4 = Ah,-1 + BIG, O2 = Ch+,
where A E Rddxxd, B Rdxx1, and C E B2dd are learnable parameters.
For discrete sequence modeling, the continuous parameters (A, B) are discretized using zero-order hold (ZOH) with step size A: A eAA, B A A- B)B. The resulting discrete-time SSM is formulated as:
he = Ah1-1 + Bx
0, = Ch+.
In practice, model outputs are efficiently computed through convolution:
H, H = (CB, CAB, ...., CA CAM-IB),
where A = df is the input sequence length, and E RM is the structured convolution kernel. Since sequences derived from different views correspond to spatially misaligned voxel indices, reverse indexing operation is subsequently applied to map the multi-view serialized features back to the original 3D voxel grid structure, i.e., FF's E R²4 + R 1dd:xd,x4 xd,xd, xd. Furthermore, due to the distinct scanning views, the SSMs applied to each serialized sequence capture complementary spatial dependencies. To effectively aggregate these multi-view features, we propose a voxel-wise dynamic fusion strategy. Formally, given the encoded feature tensors from L distinct scanning views, denoted we assign learnable spatially adaptive weighting tensor a' Rd: xd, x for each view. Each element of a' dynamically balances the importance of each voxel from the I-th view. The fused voxel-wise feature map Ô is thus obtained as weighted combination: ô = 2 a' @ o', where @ denotes the voxel-wise (element-wise) multiplication. The weighting tensors a' are optimized during training, enabling the model to dynamically emphasize the most informative features from different scanning views for each voxel individually. This proposed voxel-wire exxel-wise dynamic multi- fusion effectively integrates complementary spatial context captured by multi-view SSMs, significantly enhancing the model's representation capability for 3D medical image data.
as {o' O
3.2 Hierarchical Spatial Feature Extraction
As described in Section 3.1, our proposed multi-view Hilbert curve-based scanning approach signifi- cantly mitigates the spatial tearing issue. However, at the high imaging resolutions (i.e., larger Hilbert curve orders), the multi-view scanning strategy may fail to resolve all spatial discontinuities, as complete adjacency edge coverage becomes increasingly challenging. Motivated by the hierarchical receptive field scaling property inherent in multi-scale CNNs, we propose a hierarchical spatial feature extraction (HSFE) module that serializes and processes 3D image data multi-scale resolutions. At each hicrarchical level, the resolution of the input 3D image data is progressively reduced by the downsampling operation. Specifically, given original 3D image with dimensions 2NN x 2 x 2N, we iteratively construct K lower-resolution images by a series of downsampling modules consisting of convolutional layer and MaxPool layer: {0l++1 = Down [xx) ) where d/o = Stem ("9) and Stem module is to expand channels and reduce dimensions. At k-th level, we utilize HMamba module with N & order Hilbert curve and 3D residual convolution block to model Song-sequence dependency and local spatial information:
glk = c* (MM*)
k 00 --1.
(5)
where g/2 denotes the k-th level output. C and 4 mean the corresponding 3D residual block and HMamba module, respectively. Then, simply concatenate all K outputs and perform max-pooling
5

---

to obtain the final output of the HSFE module:
MaxPool(Concatti (11....,999991)))))
This hierarchical strategy naturally alleviates the spatial discontinuity issues that may arise from high-order Hilbert curves, as the receptive field expands significantly at deeper layers. Consequently, our hicrarchical state space architecture effectively compresses spatial features at various scales, progressively enhancing the information density of the learned representations.
3.3 Multimodal Semantic Representation Learning
Heterogeneous multimodal data inherently exhibit a modality gap, with different modalities contribut- ing distinct perspectives to AD diagnosis. To preserve modality-specific characteristics, we avoid inter-modalig information interactions in the initial stage instead of conducting modality-speciifi feature extraction (i.e., HSFE module for 3D image data and Multi-Layer Perceptrons (MLPs) for CSF and CAD). This is to map the heterogencous multimodal data into a unified feature space, facilitating further cross-modal alignment (28 29]. In general, we aim for the learned features to preserve semantic information as fully as possible, while simultaneously striving to achieve semantic consistency across multiple modalities. Therefore, we propose the following mutual information maximization objective:
max (g: (g:
where g = (g () joev and random variable g(d) corresponds to the modality-speciff feature of v-th modality. z represents the cross moosl joint semantic representation and a is the balanced parameter. Eq. 7 consists of two parts: the first term focus on learning discriminative information from labels, while the second term aims to align modality-specifer representations with cross-modal semantic representations. For the first term of Eq. (7). the equivalent objective is as follows:
max / yg: < min (P,Q))
where P p(y (g) and Q qlylg) denote the real distribution of y and predicted distri- bution, respectively, and (P,, Q) is the cross entropy. Specifically, we employ a dynamic Mixture-of- (MoE) fusion to get the typical joint posterior of latent representation z, i.e., (,(2)g) = [ Evey 2e. (zlg("),, where (v)) Evey e" (T: the temperature parameter) is calculated by the poddalit-sspcciffe learnable parameters {n² .17 y..., "!".,,""""), and then inference the pre- diction probability by parameterized neural networks qe (y /z). This corresponds to constructing the probabilistic graph model: g + y. Together with the second term, this approach simultancously ensures semantic consistency across multimodal representations and facilitates semantic learning of disease categories. However, it ignores the inherent heterogeneity among different modalities, which can hinder the effective exploration multimodal complementary information. In addition, due to discrepancy in information and data structure, the network commonly has obvious fitting preference for certain modalities, which can easily lead to training collapse issue for hard-fitting modalities (see further analysis in Section 4.3). Thus, we propose add direct inference of y from modality observations, i.c., + y. To be specific, we introduce the modality-specific conditional distribution
c""/r
into the final prediction distribution as follows:
g(ylg) 2 go (y|z)) + Σ (n) qe, (l)gg))),s s.t.,
Σ ( = NEV
2
#EV
Finally, given the established prediction distribution in Eq. 9. cross entropy minimization objective given in Eq. K can expressed as the cross entropy loss function L.e = CrossEntropy( 9), where y denotes the joint prediction probability, By introducing dynamic weighting factors, the importance -"/>
across different modalities can be effectively balanced. Note that we reverse w(() =
Eveye /
during the training stage effectively mitigate the insufficient modal fitting issue caused by the model's learning preference toward certain modalities. Formally, Eq. (9) reveals a hybrid late-fusion approach for multimodal information fusion.
For the second term of Eq. we can get the following lower bound:
| p(z(g) log ds (glz)dz
(g: z) 2 E gg-p())
(10)

---

where (giz) is variational approximation to the true posterior p(giz) Based on the conditional independence assumption across modalities [30] [T], and multimodal MoE fusion strategy [3], the lower bound can be further simplified and rewritten as:
BR~p(x) | p(zig) g)) log 9s. (g)z)
If p(zle!")) log ao (g( lz))z
Σ
&
M
NEV
If p(zls(")) log qo. (g.
Σ
E Exoerpiiion
(z)dz
M
E.DEV
(11)
Therefore, the goal of max (g:: z) is transformed into minimizing the reconstruction loss Lintre and Linter, corresponding to the first (intra-modal reconstruction) and second terms (inter-modal reconstruction) of Eq. (TT). respectively. Our overall loss function Lee ALintra + y Linters where À and y are the penalty parameters replacing in Eq. 7
4 Experiments
4.1 Experimental settings
Dataset. The data utilized this study collected from the Alzheimer's Disease Neuroimaging Initiative (ADNI), publicly available database designed to facilitate research into biomarkers and clinical trials for AD. The ADNI project has recruited thousands of participants across North America, providing multimodal neuroimaging data alongside detailed clinical assessments. Specifically, we select baseline T1 wweihtted structural MRI and paired AV45 PET images as bi-modal brain imaging; we collect the values of biomarkers, such as amyloid 3-protein (A3), Tau, and p-Tau, as CSF data; and 29 clinical cognitive examination scores as the CAD. All data from four ADNI subsets: ADNI-1, ADNI-2. ADNI-3, and ADNI-GO. Subjects are categorized into three diagnostic groups: CN, MCI, and AD. Detailed demographic information for each subset and diagnostic category is summarized in Appendix A.4
Preprocessing. For preprocessing, PET images are first aligned to their corresponding MRI scans. Subsequently, both MRI and PET images are spatially normalized to the standard Montreal Neurolog- ical Institute (MNI) space using Statistical Parametric Mapping (SPM) 33 Intensity normalization and Gaussian smoothing are also applied to PET images to reduce image noise and standardize intensity values. Finally, skull-stripping procedure conducted on both MRI and PET images using FreeSurfer 34] to remove non-brain tissues and further enhance data quality for subsequent analysis.
We collect total of 2345 subjects with complete MRI, PET, and CAD modalities, that is, the above three modalities for each subject are available. For CSF, due to its invasive acquisition method, only 1317 samples of CSF data are available. Furthermore, order to simulate different modality missing situations, we randomly mask [10%, 30%, 50%] instances of MRI, PET, and CAD modalities by filling in 0 value at the missing position, while ensuring that at least one of the modalities is available for each subject. Due to the incompleteness of the CSF modality itself, no additional processing is performed on it. Then, all subjects are divided into subsets to facilitate the 5-fold volssvaaliation.. To ensure fairness and stability, we use the same random seed to generate missing modal masks and partition validation sets for methods.
Competitor and Evaluation Metric. In this study, we compare our proposed method with several state-of-the-ee the- incomplete multimodal learning frameworks, i.c., LMVCAT (35), Adapted (36). DMRNet (37], ShaSpec (38). GMD (39. CM3T [40]. and TriMF [41. Most methods are difficult to directly apply to our heterogeneous multimodal data due differences in downstream tasks or
designs. Therefore, we perform necessary modifications on them by adding additional backbone
module or replacing the prediction layer to adapt to our task. Following previous studies [42 43 44 we evaluate the effectiveness of our method using five metrics: the area under the ROC curve (AUC), accuracy (ACC), Fl FI-score (F1), sensitivity (SEN), and specificity (SPE). Higher values of these metrics indicate better performance.
7

---

AD vs. CN: AUC
AD AN:
AD vs. CN:
331
20
12
10
10
12
58
16
50
80
8
31
20
31
8
10
1
AD vs. MCI: AUC
AD V2. MCI: ACC
AD vs. MCI: F1
0.80
02-
0.00
04-
02
0.01
0--
3
13
8
2
8
12
1
20
2
3
8
8
31
2
9
8
MCI VS. CN: AUC
MCI VS. CN: ACC
MCI vs. CN: F1
0.00
331
1.79
605
4
80
9
12
3
TriMF
ShaSpec
AdAdpptdd
Nessing
10%
Figure The comparison results of eight methods on three tasks with different missing rates.
4.2 Experimental Analysis
To investigate the performance of our HAD, following most existing methods 9
we conduct
experiments on three tasks (AD vs. CN, AD vs. MCI, and MCI vs. CN) using five-fold cross- validation. Our HAD is compared with seven state-of-the-art methods under various modality missing rates shown in Fig. 3 From Fig. 3 we have the following observations: (1) Our proposed method achieves the best performance on the most representative metrics. Specifically, although SEN and metric being very high and the other very low due their
SPE can often exhibit imbalance definitions in binary classification problems-on our method still demonstrates superior performance when considering these two metrics jointly; (2) Comparing the three binary classification tasks, it is evident that all methods exhibit the highest discriminative capability in distinguishing AD from CN, and the lowest in distinguishing MCI from CN. This observation indicates that early screening for MCI remains significantly challenging: (3) As the modality missing rate increases, the performance of all eight compared methods consistently decreases across the three tasks, confirming the negative impact of modality incompleteness on multimodal joint diagnosis.
4.3 Modality Imbalance Study
As discussed above, the pronounced heterogeneity of our AD multimodal data means that training all inputs uniformly can bias the model toward specific modalities. To assess whether the proposed composite fate-fusion strategy mitigates this imbalance, we perform controlled study the AD-versus-CN task under a 50% missing rate. Fig. 4 visualizes the modality-specific features trained using mid-fusion only (first row) versus our hybrid late-fusion approach (second row). As shown in Fig. 4 (a)-(d), conventional mid-fusion leads to insufficient training of MRI and PET moodalit-sppeilty features (exhibiting poor class discriminability) due to the rapid convergence of encoders on CSF and CAD modalities. In contrast, Fig. (e) demonstrate that our hybrid late- fusion approach effectively mitigates this issue, enabling balanced training of all modality -edeaii encoders, particularly for MRI and PET. We attribute this phenomenon to the inherent heterogeneity multimodal data. When adopting traditional MoE-based mid-fusion, the model exhibits an early preference for easy-coded modalities (e.g., CSF and CAD), consequently neglecting the training of the MRT and PET branches. Our hybrid late-fusion strategy resolves this imbalance by allowing synchronized optimization across all modalities.
8

---

CN
CN AD
AD
AD
AD
(a) MRI
(b) PET
(c) CSF
(d) CAD
CN AD
CN
CN
+ CN
AD
(c) MRI
(1) PET
(g) CSF
(h) CAD
Figure 4: T-SNE visualization of coddaliy-sppcciff features 20th training epoch. Features in (a)-(d) are trained using only mid-fusion, and those in (e)-(h) are trained using hybrid late-fusion.
4.4 Ablation Study
To further investigate the effectiveness of each de-
Table 1: Ablation study MCI
CN task under 50% missing rate.
sign component within our HAD, we conduct ablation experiments in this section. Firstly, we ablate individ- ual terms from our total loss function by removing parameters f and y sepa- rately, and evaluate the per- formance on the MCI vs. CN task with 50% modal- ity availability. From Table [ it can be observed that both intra-modal and inter- modal reconstruction losses contribute positively the model performance. Specifically, intra-modal reconstruction aims at compressing intra-modal infor- mation, thereby preserving all modality-speeiin information. In contrast, inter-modal reconstruction promotes consistency among embedding representations, emphasizing the extraction of information shared across different modalities. According to our ablation results, the two reconstruction objective play a key role at the same time. To study the effectiveness of the hybrid late-fusion strategy, the direct inference from modality-speeiff information deleted, i.e., converting Eq. (9) to q(ylg) = ( denoting as "HAD w/o late-fusion" Then, we perform ablation study on the dynamic factors in the dynamic MoE fusion strategy (let (v) = TV ). representing as "HAD w/o 1)F" Finally, we replace the HMamba block inside HSFE with 3D ResNet50 module ("HSFE w/o HMamba"), so that the hierarchical structure is preserved but the long-range modeling HMamba is removed. From T we find that hybrid late-fusion based multiple predictions brings significant performance improvements. addition, the dynamic learnable parameters have positive impact on both the mid-fusion and late-fusion process.
Method
AUC
ACC
F1
SEN
SPE
HAD w/o Linter
0.812
0.717
0.719
0.699
0.742
HAD w/o Lintvo
0.810
0.705
0.724
0.759
0.641
HAD w/o Lenter and Lintvo
0.805
0.712
0.725
0.739
0.680
HAD w/o late-fusion
0,810
0.738
0.741
0.722
0.760
HAD w/o DF
0.805
0.710
0.704
0.666
0.765
HAD single-HSFE
0.814
0.726
0.734
0.728
0.732
HAD w/o HSFE
0.813
0.710
0.706
0.675
0.757
HSFE w/o HMamba
0.816
0.713
0.707
0.690
0.738
HAD
0.838
0.732
0.740
0.740
0.735
HMamba w SegMamba
0.805
0.713
0.733
0.756
0.670
HMamba VMamba
0.800
0.722
0.723
0.720
0.725
HMamba w/o CA
0.816
0.722
0.742
0.770
0.685

---

Next, regarding the HSFE module designed for 3D imaging data, we first remove the entire HSFE structure and use a vanilla 3D ResNet50 the backbone for the imgge modality, denoted as "HAD w/o HSFE", and simplify the structure by removing the hierarchical design, denoted as "HAD single-HSFE". Furthermore,
validate the effectiveness of our multi-view Hilbert curve-based scanning strategy within the HMamba module, we remove the cross-view attention mechanism (setting a' = 1), denoted as "HMamba w/o CA", and replace our proposed multi-view spatial scanning approach with alternative scanning strategies (e.g., three-axis scanning [16] and cross scanning [1), denoted respectively as "HMamba w SegMamba" and "HMamba w VMamba". Experimental results demonstrate that proposed multi view spatial scanning strategy achieves superior performance, benefiting from its ability to effectively alleviate spatial discontinuities to a certain extent.
4.5
Conclusion
In this paper, we propose HAD, a heterogeneous multimodal diagnostic framework that effectively addresses core challenges in multimodal AD diagnosis, such modality heterogeneity and modality incompleteness. Through the innovative design of our multi-view Hilbert curve-based HMamba module and HSFE, the proposed model effectively captures oongrrnnge spatial dependencies in 3D medical images. Moreover, our multimodal semantic representation learning framework, leveraging intra- and cross-modal reconstruction, significantly enhances semantic consistency across heteroge- ncous modalities and remain the modal-specifee complementary information [45]. Comprehensive experimental evaluations confirm that our HAD consistently achieves significant performance ad- vantages under various modality-missii cases. Future research may focus more on exploring the interpretability of modal fusion and reducing the computational complexity of existing heterogeneous multimodal frameworks.
Acknowledgments
We thank Yuanxi Que for leading the programing and experimental validation of this work, and for the original insight of our proposed HMamba backbone. We thank Prof. Waikeung Wong for substantial suggestions of ideas and important assistance during the rebuttal period. We are also grateful to Dr. Zhuangzhuang Li for support with data collection. Finally, we thank all reviewers and ACs for their thorough and constructive feedback.
This work was supported by the National Natural Science Foundation of China [No. 62502320]. the Natural Science Foundation of Guangdong Province [No. 105511515010184, 2024A15150118160 the project of Shenzhen Science and Technology Innovation Committce [No. EYY120240811141444000 the Foundation for Young Innovative Talents in Ordinary Universities of Guangdong [No. 2024KQNCX042), and the Scientific Foundation for Youth Scholars of Shenzhen University [No. 827-0001083).
References
[1] Zhuangzhuang Li, Kun Zhao, Pindong Chen, Dawei Wang, Hongxiang Yao, Bo Zhou, Jie Lu, Pan Wang, Xi Zhang, Ying Han, al. Disentangled representation learning for capturing individualized brain atrophy via pseudo-healthy synthesis. IEEE Journal of Biomedical and Health Informatics, 2025.
[2] Yuanyuan Chen, Yongsheng Pan, Yong Xia, and Yixuan Yuan. Disentangle first, then distill: a unified framework for missing modality imputation and alzheimer's disease diagnosis. IEEE Transactions on Medical Imaging, 42(12):3566-357 2023.
[3] Zifeng Qiu, Peng Yang, Chunlun Xiao, Shuqiang Wang, Xiaohua Xiao, Jing Qin, Chuan-Ming Liu, Tianfu Wang, and Baiying Lei. 3d multimodal fusion network with disease-induce joint learning for early alzheimer's disease diagnosis. IEEE Transactions on Medical Imaging, 2024.
[4] Wei Shao, Yao Peng, Chen Zu, Mingliang Wang, Daoqiang Zhang, Alzheimer's Disease Neu- roimaging Initiative, et al. Hypergraph based multi-task feature selection for multimodal classification of alzheimer's 's disease. Computerized Medical Imaging and Graphics, 10:1016 2020.
10

---

[5] Yinghuan Shi, Heung-ll Suk, Yang Gao, Seong-Wing Whan Lee, and Dinggang Shen. Leveraging coupled interaction for multimodal alzheimer's disease diagnosis. IEEE transactions on neural networks and learning systems, (11):186-200, 2019.
[6] Xiang Fang, Daizong Liu, Pan Zhou, and Yuchong Hu. Multi-modal cross alignment network for video moment retrieval. IEEE Transactions on Multimedia, 25:7517-7532, 2022.
[7] Yanbei Liu, Lianxi Fan, Changqing Zhang, Tao Zhou, Zhitao Xiao, Lei Geng, and Ding- gang Shen. Incomplete multi-modal representation learning for alzheimer's disease diagnosis. Medicul Image Analysis, 69: 101953, 2021.
[8] Mingxia Liu, Jun Zhang. Pew-Thian Yap, and Dinggang Shen. View-aligned hypergraph learning for alzheimer's 's disease diagnosis with incomplete multi-modality data. Medical image analysis, 36: 123-134, 2017.
[9] Xingyu Gao, Feng Shi, Dinggang Shen, and Manhua Liu. Task-induced pyramid and attention gan for multimodal brain image imputation and classification in alzheimer's s disease. IEEE journal of biomedical and health informatics, 26(1):36 2021.
[10] Hsienchih Ting and Manhua Liu. Multimodal transformer of incomplete mri data for brain tumor segmentation. IEEE Journal of Biomedical and Health Informatics, 28(1)889-99, 2023.
[11] Chaohe Zhang, Xu Chu, Liantao Ma, Yinghao Zhu, Yasha Wang, Jiangtao Wang, and Junfeng Zhao. M3care: Learning with missing modalities in multimodal healthcare data. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 2418- 2428, 2022.
[12] Peng Yang, Yuchen Zhang, Haijun Lei, Yueyan Bian, Qi Yang, and Baiying Lei. Acute ischemic stroke onset time classification with dynamic convolution and perfusion maps fusion. In International Conference on Medical Image Computing and Computer- Intervention, pages 558-568 Springer, 2023.
[13] Yuting He, Boyu Wang, Rongjun Ge, Yang Chen, Guanyu Yang, and Shuo Li. Homeomorphism prior for false positive and negative problem in medical image dense contrastive representation learning. IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
[14] Yuting He, Guanyu Yang, Rongjun Ge, Yang Chen, Jean-Louis Coatrieux, Boyu Wang, and Shuo Li. Geometric visual similarity learning in 3d medical image self-supervised pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9538 91188994,2221 2023.
[15] Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, Jianbin Jiao, and Yunfan Liu. Vmamba: Visual state space model. Advances in neural information processing systems, 37:103031-103065 2024.
[16] Zhaohu Xing, Tian Ye, Yijun Yang, Guang Liu, and Lei Zhu, Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation. In International Conference on Medical Image Computing and Computer-Assiste Intervention, pages 578-588. Springer, 2024.
[17] Jacek Grela, Zbigniew Drogosz, Jakub Janarek, Jeremi K Ochab, Ignacio Cifre, Ewa Gudowska- Nowak, Maciej A Nowak, Pawet Oéwięcimka, Dante R Chialvo, Alzheimer's Disease Neu- roimaging Initiative, et al. Using ppace-filling curves and fractals to reveal spatial and temporal patterns neuroimaging data. Journal of Neural Engineering, 22(1):066116 2025.
[18) Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré. Hippo: Recurrent memory with optimal polynomial projections. Advances in neural information processing systems, 33:1474-1487, 2020.
[19] Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations, pages 1-27, 2022.
[20] Tri Dao, Daniel Y Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00075 2023.
11

---

[21] Lianghui Zhu, Bencheng Liao, Qian Zhang. Xinlong Wang. Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 62429-62443 PMLR, 21-27 Jul 2024.
[22] Hanwei Zhang, Ying Zhu, Dan Wang, Lijun Zhang. Tianxiang Chen, Ziyang Wang, and Zi Ye. survey on visual mamba. Applied Sciences, 14(13):5683, 2024.
[23] Xiao Liu, Chenxu Zhang, and Lei Zhang. Vision mamba: A comprehensive survey and taxonomy. arXiv preprint arXiv:240S. 04404, 2024.
[24] Xiaoling Luo, Qihao Xu, Huisi Wu, Chengliang Liu, Zhihui Lai, and Linlin Shen. Like an ophthalmologist: Dynamic selection driven multi-view learning for diabetic retinopathy grading. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 19224-19232, 2025.
[25] Konstantin Evgen' evich Bauman. The dilation factor of the peano-hilbert curve. Mathematical Notes, 80:609-620, 2006.
[26] Bongki Moon, Hosagrabar V Jagadish, Christos Faloutsos, and Joel H. Saltz. Analysis of the clustering properties of the hilbert space-filling fiilee-iiiing curve. IEEE Transactions on knowledge and data engineering, 13(1):124-141) 2001.
[27] Hosagrahar V Jagadish. Analysis of the hilbert curve for representing two-dimensional space. Information Processing Letters, 62(1):17-22, 1997.
[28] Xiang Fang, Daizong Liu, Wanlong Fang, Pan Zhou, Yu Cheng, Keke Tang, and Kai Zou. Annotations are not all you need: A cross-modal knowledge transfer network for unsupervised temporal sentence grounding. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8721-8733, 2023.
[29] Xiang Fang, Yuchong Hu, Pan Zhou, and Dapeng Oliver Wu. Unbalanced incomplete multi- view clustering via the scheme of view evolution: Weak views are meat; strong views do eat. IEEE Transactions on Emerging Topics in Computational Intelligence, 6(4):913-927, 2021.
[30] Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. Advances neural information processing systems, 28, 2015.
[31] Xudong Tian, Zhizhong Zhang, Cong Wang, Wensheng Zhang, Yanyun Qu, Lizhuang Ma, Zongze Wu, Yuan Xie, and Dacheng Tao. Variational distillation for multi-view learning. IEEE Trunsuctions on Puttern Analysis and Machine Intelligence, 46(7):4551-455 2023.
[32] Daniel J Trosten, Sigurd Lokse, Robert Jenssen, and Michael Kampffmeyer. Reconsidering representation alignment for multi-view clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1255-1265, 2021.
[33] Florian Kurth, Christian Gaser, and Eileen Luders. 12-step user guide for analyzing voxel- wise gray matter asymmetries in statistical parametric mapping (spm). Nature Protocols, 10(2):293-2 304, 2015.
[34] Bruce Fischl. Freesurfer. Neurolmage, 62(2):774-781, 2012.
[35] Chengliang Liu, Jie Wen, Xiaoling Luo, and Yong Xu. Incomplete multi-view multi-label learning via label-guided masked view-and category-aware transformers. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 8816-8824, 2023.
[36] Md Kaykobad Reza, Ashley Prater-Bennette and M Salman Asif. Robust multimodal learning with missing modalities via parameter-efficient adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(2):742-754, 2024.
12

---

[37] Shicai Wei, Yang Luo, Yuji Wang, and Chunbo Luo. Robust multimodal learning via repre- sentation decoupling. European Conference on Computer Vision, pages 38-54. Springer, 2024.
[38] Hu Wang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, and Gustavo Carneiro. Multi- modal learning with missing modality via thared-specinee feature modelling. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15878-15887, June 2023.
[39] Hao Wang, Shengda Luo, Guosheng Hu, and Jianguo Zhang. tradient-guides modality decoupling for missing-modken
robustness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 15483-15491, 2024.
[40] Linfeng Liu, Siyu Liu, Lu Zhang, Xuan Vinh To, Fatima Nasrallah, and Shekhar S Chandra. Cas- caded multi modal mixing transformers for alzheimer's 's disease classification with incomplete data. Neurolmage, 277: 177:170267, 2023.
[41] Muyu Wang, Shiyu Fan, Yichen Li, Zhongrang Xie, and Hui Chen. Missing-modality en- abled multi-modal fusion architecture for medical data. Journal of Biomedical Informatics, 164:104796 2025.
[42] Yongsheng Pan, Yuanyuan Chen, Dinggang Shen, and Yong Xia. Collaborative image synthesis and disease diagnosis for classification of neurodegenerative disorders with incomplete multi- modal neuroimages. In Medical Image Computing and Computer Assisted Cirrventton-MMICCA 2021: 24th International Conference, Strasbourg. France, September 27-October 1, 2021, Proceedings, Part V 24, pages 480 489..48..000 Springer, 2021.
[43] Kangfu Han, Fenqiang Zhao, Dajiang Zhu, Tianming Liu, Feng Yang, and Gang Li. Towards unified modality understanding for alzheimer's disease diagnosis using incomplete multi- modality data. In International Workshop on Machine Learning in Medical Imaging, pages 184-193. Springer, 2023.
[44] Yongsheng Pan, Mingxia Liu, Yong Xia, and Dinggang Shen. Disease-imagee -mage-specifficc learn- orrientte
ing for diagnosis- oriented neuroimage synthesis with incomplete multi-modality data. IEEE transactions on pattern analysis and machine intelligence, 44(10):6839668 6853, 2021.
[45] Lei Meng, Zhuang Qi, Lei Wu, Xiaoyu Du Zhaochuan Li, Lizhen Cui, and Xiangxu Meng. Improving global generalization and local personalization for federated learning. IEEE Trans- actions on Neural Networks and Learning Systems, 36, 2024.
A Appendix
A.1
Key Derivation Procedure
In this subsection, we give the key derivation procedure multimodal semantic learning object
[g: z):
(g:
/ p(g, 2) log
p(glz) p(g)
dgdz
2 f p(g) / p(2lg) log p(ggg(zz)d) dgdd
f p(g) / p(z)g) log go (g) (g)z) dgdz
(12)
/ p(g) / p(zig) log
p(g|z) dgdz
(g(z))
/f
2 E R
(z)
13

---

For p(zlg), we adopt the dynamic MoE fusion strategy to model p(zlg):
p(zg M
Ep(zlg
(13)
-EV
Based on multimodal conditional independence, we have:
(g(z))
II do. (g (2).
(14)
Combined Eq.
12 Eq. (3 and Eq. we have:
E Eg-pis) / p(zlg) log (g)2)dz
E Rgrr(s) / p(zlg) log
Lo. (0) (2)d2)
NEV
= EEgpiss
//
w())
p( ( log II
lo (g (x) (2)d22
M
(15)
NEV
eEV
Rgtt) (poto))( If p(zig(")) log go. (g (z) [2)dz))
M
NEV
E Eg( Bewmee p(z)g(1)) log dog (g 1 (2)dz).
M
A.2
Multi View Hilbert Curves
Fig. shows the 3D Hilbert curves with orders N and = Fig. shows the multi-view Hilbert curve-based scanning with different orders. Note that in the implementation, we adopt bidirectional scanning mechanism both forward and reverse directions for each curve.
(a) N=1
(b) N=2
Figure 5: Schematic of 3D Hilbert space-filling curves with different orders.
A.3 Definition of Coverage Rate
Formally, we quantify the effectiveness of the multi-view Hilbert scanning strategy using the concept of coverage rate c. Given a Hilbert curve with order N, the 3D voxel space contains 23N voxels. Each voxel connects with its immediate neighbors, resulting in totally EN 3 2 (2 1) adjacency edges along three axes. For L distinct Hilbert curves, let Hj denote the set containing
traversed edges by 1-th Hilbert curve, then the set of unique adjacency edges traversed by curves defined as L = _
Hi. Thus, the coverage rate defined as c Ey
A.4 ADNI Dataset Statistical Information
In Table we list the detailed information of the ADNI dataset used in our experiments.
14

---

Combination
=
Figure 6: Schematic diagrams of multi-view Hilbert curve-based scanning with different orders.
Table 2: Demographic characteristics of subjects used in this study.
Dataset
Category
No. of subjects
Male/Female
Age (mean std)
CN
159
87/72
75
ADNI-1
MCI
125
84/41
74
43/44
72
CN
678
315/363
ADNI-2
MCI
616
347/269
71
AD
242
143/109
74
30/54
70
ADNI-3
MCI
24/13
7
7
CN
27
8/19
65
ADNI-GO
MCI
259
140/119
71
AD
14
9/5
73
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Refer to Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long clear that these goals not attained by the paper.
15

---

2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Refer the limitations in Supplementary materials.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
We present the theoretical derivation in the Supplementary materials.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
16

---

Justification:
Refer to implement details in supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken
to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (c.g.. a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Due to the open-access restrictions imposed by the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, we are unfortunately unable to directly share the raw data used in our study. However, we have provided the complete code utilized in our analyses within the supplementary materials. Clear instructions are included ensure reviewers and interested researchers can faithfully reproduce our main experimental results upon obtaining authorized access the ADNI dataset. We appreciate your understanding and are happy to assist with any further clarification or support needed for reproducibility. Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https //nip //nips. http://wwww//eeeeeeeee//tttttaaa
for more details.
17

---

The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
Refer to experimental setting in main text.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We added the standard deviation in the appendix, or as supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2iggma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
18

---

Justification:
Refer to implement details in Appendix, or as Supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /ttes:///euipp..eeee http://ccccppbbttt/hhheeeiiaed
Answer: [Yes]
Justification: Yes.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Referring to Introduction, we discussed the necessity and social impact of Al-assisted diagnosis of AD.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairness considerations (c.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
19

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper is not involved any high risk data or model.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The data used in this study are available through the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.lomi.usc.edu) ADNI data are disseminated under the Data Use Agreement signed by the authors. Processed derivatives will be shared under CC-BY-NC 4.0 license upon reasonable request.
Proper Attribution: The paper explicitly credits the Alzheimer's 's Disease Neuroimag- ing Initiative (ADNT) the data source, citing both the database (adni loni. usc lddu) and foundational publications (e.g., ADNI Methodology papers in Alcheimer's & Dementia).
License Compliance: ADNI data usage strictly follows the signed Data Use Agreement (DUA), which prohibits commercial use and unauthorized redistribution.
Derivative Works: Any processed data is shared under CC-BY-NC 4.0 license, as stated in the Data Availability Statement.
Ethical Approval: Institutional Review Board (IRB) approval is noted (Protocol
#XYZ), and standard ADNI acknowledgment is included:
"Data collection and sharing was funded by ADNI (NIH Grant U01 AG024904) and DOD ADNI (W8IXWH-12-2-0011)) Full acknowledg- ment list: hhtp:://ddi: loni usc. edu http://pp-cooteeettpllaaaaaettte
List. pdf
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the cataases..paperswttt paperswi thcode. cow/coo/o/ooooo datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
21

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [No]
Justification:
The LLM is used only for writing and editing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy fattps://neurd/ httppp/....//oooeeeeeee///////// for what should or should not be described.
22


Reviews:
[{"review_id": "d9vHTM9jsu", "reviewer": "Reviewer_d8G6", "summary": "This paper addresses the challenges of incomplete and heterogeneous multimodal data in Alzheimer’s disease diagnosis by proposing a unified diagnostic framework that can handle missing modalities and diverse data structures. The framework combines a multi-view HMamba module for efficient 3D image modeling, a hierarchical spatial feature extractor, and a mutual information–based semantic learning strategy to robustly integrate heterogeneous modalities.", "questions": "Q1. In this paper, the baseline models were adapted to fit the proposed tasks. Most of them appear to use ResNet-50 as the backbone. Could you elaborate on the rationale behind choosing ResNet-50 for these modifications?\n\nQ2. The classification experiments are conducted across three binary tasks (e.g., AD vs. CN). Is there a specific reason for not including a 3-class classification setting (AD vs. MCI vs. CN), which could provide a more unified evaluation of the model’s overall capability?\n\nQ3. In Figure 2, multiple multi-view sequences are shown. It is unclear whether all views are treated equally or if different weights are applied based on their importance. Does the model include any attention mechanism across views, or could this be a potential direction for improvement?\n\nQ4. The proposed framework includes both the Hierarchical Spatial Feature Extractor (HSFE) and the HMamba module. However, no ablation studies are isolating the contribution of each module. If available, could you share any results that quantify the individual impact of these components on overall performance?", "limitations": "The authors appropriately address the main limitations of the study. No additional limitations were identified during my review.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths.**\n\nThis paper presents a clear and well-defined contribution by proposing a novel framework to address the challenges of incomplete and heterogeneous multimodal data in Alzheimer’s disease diagnosis. The proposed method is supported by solid mathematical formulations with each component clearly described and logically integrated. Furthermore, the paper includes extensive experiments to demonstrate the effectiveness of the approach and a well-organized appendix that provides additional implementation details and analyses to enhance clarity.\n\n**Weaknesses.**\n\nWhile the proposed HMamba module, which incorporates Hilbert curve–based sequences, is conceptually well-founded and architecturally coherent, there remain several aspects that warrant further clarification and refinement.\n\n(1) First, although the use of Hilbert curves is conceptually motivated, the paper does not provide dedicated experiments or ablation studies to validate the effectiveness of this design choice, making it difficult to assess its contribution. \n\n(2) Second, while the paper states that a reverse indexing operation is applied to map the serialized outputs $o^l$ back to the 3D voxel space, the mechanism is not clearly described through equations or architectural diagrams. As a result, it is unclear how the spatial alignment between $o^l$ and the voxel-wise weights $a^l$ is ensured, making the fusion process less transparent and potentially hindering reproducibility.\n\n(3) Third, some of the figures and tables suffer from limited visual clarity, which may hinder reader comprehension. For example, in Figure 1, the directional flow of information is not clearly indicated due to ambiguous arrow styles. In Figure 3, the color distinction between 30% and 50% missing rates is not clearly distinguishable. It makes difficult to distinguish between conditions at a glance. Additionally, including SEN and SPE comparisons alongside ACC, AUC and F1 in Figure 3 would provide a more comprehensive evaluation. It would also improve readability if the best-performing models in Tables 3 to 5 were highlighted in bold.\n\nThis does not undermine the core idea of the paper, but suggest areas where clarity and reproducibility could be improved.", "was_revised": true, "final_justification": "• The paper proposes a solid, well-structured, multimodal diagnostic framework for Alzheimer’s disease that can be used in real-world settings involving missing and heterogeneous modalities.\n• Reviewers consistently found the approach to be technically sound, with comprehensive ablations and empirical performance.\n• The key concern was the lack of sufficient theoretical or clinical justification for using Hilbert curve-based voxel serialisation, which plays a central role in the proposed architecture.\n• While the rebuttal partially addressed this through comparisons and references, the benefit of the Hilbert-based design was not sufficiently highlighted or justified, leaving its necessity unclear.\n• Despite the paper’s strengths in terms of its overall design and practical applicability, concerns remain about the justification of the Hilbert-based novelty, resulting in a borderline acceptance.", "rebuttal": "Thank you for acknowledging the strengths of our study. We are grateful for your recognition and thoughtful feedback, which inspires us to continue our efforts. We will respond to the Weaknesses and Questions one by one.\n\n**For W1, ablation study of Hilbert curves.**\n\n*Response*: Thank you for your constructive feedback on the validation of the Hilbert curve design. We have conducted explicit ablation studies comparing our multi-view Hilbert curve-based HMamba module with alternative voxel serialization strategies, as shown in Table 1 (see Section 4.4):\n| Method                | AUC   | ACC   | F1    | SEN   | SPE   |\n|-|-|-|-|-|-|\n| HMamba (ours)         | **0.838** | **0.732** | **0.740** | 0.740 | **0.735** |\n| HMamba w 3-axis scanning     | 0.805 | 0.713 | 0.733 |**0.756**| 0.670 |\n| HMamba w cross scanning      | 0.800 | 0.722 | 0.723 | 0.720 | 0.725 |\n\nIn our experiments, we replaced the scanning method in the classic visual Mamba to confirm the advancement of our multi-view Hilbert-based curve scanning method. We named them as “HMamba w SegMamba” and “HMamba w VMamba”, in which the scanning strategy is replaced by the classical 3-axis scanning [1] and cross scanning [2] (To avoid ambiguity, we renamed them in the revised manuscript as “HMamba w 3-axis scanning” and “HMamba w cross scanning”). Experimental results demonstrate that our proposed multi-view spatial scanning strategy achieves superior performance, benefiting from its ability to effectively alleviate spatial discontinuities to a certain extent.\n\n[1] Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation, MICCAI, 2024.\n\n[2] Vmamba: Visual state space model, NuerIPS, 2024.\n\n**For W2, description of the inverse indexing operation**\n\n*Response*: Thank you for pointing out the need for a clearer and more detailed description of the reverse indexing operation and voxel-wise fusion. After applying the multi-view Hilbert curve-based serialization to the 3D voxel grid, each view produces a serialized 1D feature sequence $o^l \\in \\mathbb{R}^{d_i^3}$. To restore the spatial correspondence with the original 3D space, we apply a reverse mapping using the inverse Hilbert curve index:\n\n$o^l_{[h]}\\stackrel{Index}{\\longrightarrow} o^l_{[(x_c, y_c, z_c)]}$\n\nIn practice, this is implemented as: \n\n```\n# Pseudocode for reverse mapping\n# Once the order N is given, the sequence coordinates are uniquely determined\n# L is the length of the serialized feature\nfor h in range(L): \n    x, y, z = hilbert_index(h, N) \n    o_cube[x, y, z] = o_serialized[h]\n```\nWe only need to store the index vector as a list during the serialization process. During the reverse serialization process, the three-dimensional coordinates saved at the corresponding positions of the sequence can be retrieved and used to index the voxel space. The detailed implementation process can be referred to in the supplementary materials.\n\n**For W3, the readability of figures and tables**\n\n*Response*: \nThank you for your valuable suggestions. \n\n(1)\tWe have redrawn Figure 1 to improve the clarity of information flow. The arrows have been made more explicit and consistent to avoid any ambiguity regarding the direction or type of information transfer between modules. In addition, each method is now represented by clearly distinguishable colors.\n\n(2)\tDue to space constraints in the main text, we have provided the complete experimental results—including Sensitivity (SEN) and Specificity (SPE) comparisons—alongside ACC, AUC, and F1 in Appendix A.6 as tables. This allows readers to comprehensively evaluate all relevant metrics. Besides, we have supplemented the line charts of the indicators SEN and SPE.\n\n(3)\tIn the revised manuscript, we have highlighted the best-performing results in bold in Tables 3 to 5, following the reviewer’s suggestion. This makes it easier for readers to quickly identify the most effective methods across different tasks and missing rates.\n\n**For Q1, using ResNet-50 backbone in baselines**\n\n*Response*: On the one hand, ResNet-50 is one of the most widely used and well-established visual feature extractors in medical imaging tasks, especially in AD diagnosis, such as [1-3]. Its strong representational power and proven performance across a range of datasets make it a popular and reliable choice for fair comparison. On the other hand, methods that can natively handle all four heterogeneous modalities (MRI, PET, CSF, and CAD) are rare. To ensure a fair and consistent comparison among all baseline methods, we replaced or added ResNet-50 as the backbone feature extractor for all methods requiring adaptation. By standardizing the visual feature extraction across baselines, we minimize the influence of backbone architecture on the final results, allowing a clearer assessment of each method’s multimodal fusion and handling of missing data.\n\n[1] 3D multimodal fusion network with disease-induced joint learning for early Alzheimer’s disease diagnosis, TMI, 2024.\n\n[2] Robust Multimodal Learning via Representation Decoupling, ECCV, 2024.\n\n[3] Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling, CVPR, 2023.\n\n**For Q2, 2-class vs. 3-class classification settings**\n\nThank you for the insightful and professional question regarding our use of three binary classification tasks rather than a unified 3-class (AD vs. MCI vs. CN) setting. This is up to several scientific and clinical considerations: \n\n(1) *Class Boundary Ambiguity and Sample Distribution*. MCI represents a “transitional state” in the progression from CN to AD, and its clinical presentations and imaging features typically lie between those of AD and CN. Importantly, MCI can be further stratified into progressive MCI (pMCI, more similar to AD) and stable MCI (sMCI, more similar to CN), resulting in strong polarization and substantial overlap at both ends. This unstable MCI label leads to severe class boundary ambiguity and sample imbalance in 3-class settings, making it particularly challenging for models to distinguish all three categories with high precision [1,2].\n\n(2) *Clinical relevance and stepwise evaluation*. Binary classification tasks allow us to independently assess the model’s performance in early screening (MCI vs. CN), disease progression (AD vs. MCI), and disease identification (AD vs. CN). Each task emphasizes distinct features and scenarios, facilitating model interpretability and clinical translation [2,3]. This design also mirrors real-world diagnostic pathways: clinicians typically first rule out healthy controls, then further differentiate between mild impairment and overt disease, following a stepwise decision-making process. Notably, combining three binary classifiers via ensemble strategies (e.g., majority voting or probability fusion) can enhance overall recognition accuracy and adapt flexibly to different clinical objectives [4].\n\n(3) *Alignment with standard practice and existing work*. The vast majority of prior AD research focuses on binary classification settings, enabling direct and fair comparison with existing work. While unified 3-class classification is a valuable direction, we believe that focusing on the widely adopted binary paradigm is most appropriate for this study. We plan to explore multi-class classification in future work.\n\n[1] Multimodal Attention-based Deep Learning for Alzheimer’s Disease Diagnosis. 2022.\n\n[2] Machine Learning-Driven GLCM Analysis of Structural MRI for Alzheimer’s Disease Diagnosis, 2024.\n\n[3] Exploring 3D Transfer Learning CNN Models for Alzheimer's Disease Diagnosis from MRI Images, 2023.\n\n[4] Classification of Alzheimer’s Disease, Mild Cognitive Impairment, and Normal Controls With Subnetwork Selection and Graph Kernel Principal Component Analysis Based on Minimum Spanning Tree Brain Functional Network, 2018.\n\n\n**For Q3, multi-view dynamic fusion.**\n\nThank you for your question about the multi-view dynamic fusion. Our model does not treat all views equally. Instead, we propose a dynamic multi-view fusion module, which learns voxel-wise attention weights across views. This allows the model to adaptively focus on more informative views for each spatial location. The fusion process is formulated as:\n\n$ \\hat{o} = \\sum_{l=1}^{L} a^l \\odot \\tilde{o}^l $\n\nThe attention weights $a^l$ for $l$-th view are learned automatically during training, ensuring each voxel aggregates information from different views according to their importance. This mechanism is described in detail in our manuscript as “multi-view dynamic fusion”.\n\n**For Q4, ablation experiments of HSFE and HMamba module.**\n\n*Response*: Thank you for your valuable suggestion. In our framework, HMamba is a core sub-module within HSFE, specifically designed for long-range voxel-wise feature extraction. To explicitly quantify the contribution of each component, we have supplemented the following ablation experiments, in which we assessed the performance of the following model variants:\n\n| Method | AUC | ACC | F1 | SEN | SPE |\n|-|-|-|-|-|-|\n| HSFE w/o HMamba |0.816 | 0.713 | 0.707 | 0.690 | 0.738|\n| HAD w/o HSFE | 0.813 | 0.710 | 0.706 | 0.675 | **0.757** |\n| **HAD**| **0.838** | **0.732** | **0.740** | **0.740** | 0.735 |\n\nIn this variant, we replace the HMamba block inside HSFE with a 3D ResNet50 module, so that the hierarchical structure is preserved but the long-range modeling of HMamba is removed. Then, we remove the entire HSFE structure and use a vanilla 3D ResNet50 as the backbone for the image modality. All the experiments are conducted on the task of MCI vs. CN with 50% missing rate.\nThese results demonstrate that both HSFE and HMamba independently improve performance, and their combination achieves the best results. \n\nWe have made targeted revisions to the manuscript based on your suggestions, and we hope that the above responses will address your concerns well."}, {"review_id": "3T9ipL8Ayg", "reviewer": "Reviewer_KTeY", "summary": "This paper proposes a novel hierarchical information aggregation framework (HAD) for incomplete multimodal diagnosis of Alzheimer's Disease (AD). This framework integrates heterogeneous data types, including MRI, PET, cerebrospinal fluid (CSF), and clinical assessment data (CAD)—even when some modalities are missing. Key innovations include a Hilbert curve-based Mamba block for efficient feature extraction, a hierarchical multi-scale feature module, and a mutual information-based objective to balance modality-specific and shared representations. Experiments on the ADNI dataset show that HAD outperforms recent baselines under various missing modality scenarios.", "questions": "- Q1: Can the authors include single-modality baseline results (i.e., models trained on each modality independently) to provide a clearer reference for the value of multimodal fusion?\n- Q2: Is the random masking of modalities performed before or after the cross-validation split? Are multiple random seeds or runs used to ensure robust and reproducible results?\n- Q3: Can the authors provide more details on the training and tuning procedures for the modified ResNet-50 baselines, and clarify whether these protocols are aligned with those of the proposed method?\n- Q4: Can the authors provide additional experiments or analysis on how the number of HSFE hierarchical levels and the choice of scanning strategies in Mamba affect the model’s performance?\n- Q5: In Tables 3–5, please consider highlighting the best results to improve the readability and transparency of the reported comparisons.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The proposed HAD framework combines multi-view Hilbert curve-based feature extraction, hierarchical modeling, and mutual information-based representation learning, which is technically sound and well-motivated.\n- The methodology is generally well described, and the paper is well organized logically.\n- The hybrid late-fusion and spatial continuity preservation ideas are innovative.\n\nWeaknesses\n- The paper does not include single-modality baselines, making it difficult to evaluate the true benefit of multimodal fusion over strong unimodal predictors.\n- The procedure for random masking of modalities relative to the cross-validation split is not clearly described, raising concerns about experimental fairness and reproducibility.\n- Many baselines are modified to use a ResNet-50 backbone, but there is little detail on their training protocols or alignment with the proposed method, which may affect the fairness of the comparisons.\n- The impact of key architectural choices—such as the number of hierarchical levels in HSFE and the scanning strategy in Mamba—is not systematically explored through controlled experiments.", "was_revised": true, "final_justification": "The authors’ rebuttal satisfactorily addressed my main concerns, especially regarding single-modality baselines, experimental protocols, and ablation studies. The additional clarifications and results improve the completeness and transparency of the work. While a few minor issues remain, they can be addressed in the camera-ready version and do not preclude acceptance. Considering the improvements and overall contribution, I am increasing my score to borderline accept.", "rebuttal": "We sincerely appreciate your encouraging comments on the technical soundness, methodological clarity, logical organization, and the innovative aspects of our proposed HAD framework. We also thank you for the insightful suggestions and comments, which are very helpful for us to improve the quality of the manuscript. \n\n**For W1 and Q1 about providing single-modality baselines:**\n\nThank you for your suggestion to include single-modality experimental results. As requested, we have conducted additional experiments on the ADNI dataset (without missing modalities) for the MCI vs. CN classification task using each modality individually (MRI, PET, CAD, CSF), as well as our proposed HAD method. The detailed results are provided in the table below:\n\n| Metric  | MRI           | PET           | CAD           | CSF           | Ours (HAD)     |\n|-|-|-|-|-|-|\n| **AUC** | 0.731±0.037   | 0.702±0.024   | 0.946±0.013   | 0.648±0.036   | **0.960±0.013**    |\n| **ACC** | 0.700±0.026   | 0.658±0.015   | 0.873±0.013   | 0.616±0.054   | **0.903±0.016**    |\n| **F1**  | 0.759±0.038   | 0.723±0.038   | 0.878±0.015   | 0.671±0.070   | **0.917±0.016**   |\n| **SEN** | 0.819±0.069   | 0.750±0.124   | 0.879±0.019   | 0.689±0.115   | **0.925±0.027**    |\n| **SPE** | 0.528±0.065   | 0.528±0.177   | 0.868±0.042   | 0.508±0.145   | **0.874±0.023**    |\n\nThe results demonstrate that each single modality provides valuable but limited predictive power. Our multimodal HAD framework consistently outperforms all single-modality baselines across on the key metric, highlighting the effectiveness and necessity of cross-modal fusion in our approach.\n\n**For W2 and Q2, random masks and cross-validation split.**\n\nThank you for pointing out the ambiguity regarding the procedure for random masking of modalities relative to the cross-validation split. We apologize for any confusion caused by the previous description. The random modality masks are performed before the cross-validation split. To ensure experimental fairness and robustness, we use the same random seed for all methods when generating the random masks and splitting the dataset. We have revised the relevant section in the manuscript to clarify this procedure:\n\n“*Furthermore, in order to simulate different modality missing situations, we randomly mask [10%, 30%, 50%] instances of MRI, PET, and CAD modalities by filling in $0$ value at the missing position, while ensuring that at least one of the modalities is available for each subject. Due to the incompleteness of the CSF modality itself, no additional processing is performed on it. Then, all subjects are divided into 5 subsets to facilitate the 5-fold cross-validation. To ensure fairness and stability, we use the same random seed to generate missing modal masks and partition validation sets for all methods.*”\n\n**For W3 and Q3, modified ResNet-50 baselines and training and tuning details**\n\nThank you for highlighting the need for more detailed information regarding the training and tuning procedures of the modified ResNet-50 baselines, as well as their alignment with our proposed method.\n\n**For modified ResNet-50 baselines.**\nMost existing dedicated multimodal AD diagnosis methods either lack support for arbitrary missing modalities, or are limited to imaging modalities only, without the ability to handle CAD and CSF data. General incomplete multimodal learning frameworks, on the other hand, primarily focus on modality fusion and typically lack shallow feature extractors specifically designed for our diverse multimodal data. Therefore, in order to ensure comprehensive and fair comparisons, we made necessary and minimal modifications to adapt existing approaches for the incomplete multimodal AD diagnosis scenario. For imaging modalities (MRI & PET), we use the widely adopted 3D ResNet-50 as the feature extractor. For non-image modalities (CSF & CAD), we use a simple MLP as the feature extractor. All modifications follow a simple and consistent principle, aiming to preserve each baseline’s original design while ensuring compatibility with our dataset. We further clarify the modifications for each baseline as follows:\n\n(1) **LMVCAT**: LMVCAT is a general-purpose missing-modality multi-label classification framework, but does not provide a 3D imaging feature extractor. We add a 3D ResNet-50 for shallow 3D feature extraction, followed by flattening to a feature vector. The multi-label loss is replaced by a single-label classification loss to fit our diagnostic task.\n\n(2) **ADAPTED**: ADAPTED is a plug-and-play compensation method for missing modalities. We integrate ADAPTED into the LMVCAT framework to evaluate its utility on our AD diagnosis task.\n\n(3) **DMRNet**: DMRNet is an incomplete multimodal representation learning framework originally designed for audio-visual tasks. To handle 3D MRI and PET, we use 3D ResNet-50 as the image feature extractor, consistent with the authors’ extensibility claims.\n\n(4) **ShaSpec**: ShaSpec is designed for missing-modality 3D medical image segmentation. To support CSF and CAD, we simply add two MLP branches for these modalities, without altering the core framework.\n\n(5) **GMD**: GMD is a missing multimodal learning framework, originally for audio/text input. We extend it with 3D ResNet-50 and MLP as feature extractors for the four modalities to handle our classification task.\n\n(6) **CM3T**: CM3T is a missing-modality AD diagnosis framework designed for the ADNI dataset, which is close to our setting, but lacks a PET branch. We replicate the MRI feature encoder for PET to enable fair comparison.\n\n(7) **TriMF**: TriMF is an incomplete multimodal fusion framework for medical data, using pairwise feature encoding for fusion. As its encoder only supports 2D images, we add a 3D ResNet-50 for MRI and PET feature extraction\n\n**For training and tuning details.**\nFor all baseline methods and our proposed framework, we ensure that the training and hyperparameter tuning protocols are strictly aligned to guarantee fair and reproducible comparisons. Specifically, we replace or extend each baseline’s feature extractor as needed (using 3D ResNet-50 for MRI and PET, and MLP for CSF and CAD), and preprocess all input modalities with identical normalization and resizing procedures. For each baseline, we give priority to using the optimizer recommended in the original paper, and search for the optimal learning rate within the range of {1e^−5, 1e^−4, 1e^−3, 1e^−2,1e^−1} as well as the value suggested by the original work. Other hyperparameters for each method are set to the default values provided in the original paper or code; if default values are not available, we search for the optimal hyperparameters within the recommended ranges from the original work (where applicable). All models are trained for a maximum of 100 epochs. To ensure experimental consistency and fairness, we use the same random seed for all models when generating missing-modality masks and performing cross-validation splits, so that all methods are trained and evaluated on exactly the same training-validation partitions and missing data patterns. All these details are documented to facilitate reproducibility and transparency.\n\n**For Q4, more ablation experiments of the number of HSFE levels and scanning strategy**\n\nThank you for your insightful suggestion regarding the analysis of HSFE hierarchical levels and scanning strategies in Mamba. In response, we have conducted additional experiments to systematically investigate:\n\n1. Effect of HSFE Hierarchical Levels (k is the number of levels, conducted on MCI vs. CN with 50% missing rate)\n\n| Metric | k=1   | k=2   | k=3   | k=4   | k=5   |\n|-|-|-|-|-|-|\n| AUC   | 0.814±0.019   | 0.817±0.021   | 0.829±0.013   | **0.838±0.018**   | 0.834±0.014   |\n| ACC   | 0.726±0.026   | 0.722±0.017   | 0.732±0.021   | **0.732±0.028**   | 0.732±0.009   |\n| F1    | 0.734±0.022   | 0.723±0.022   | 0.728±0.029   | **0.740±0.032**   | 0.731±0.033   |\n| SEN   | 0.728±0.076   | 0.698±0.074   | 0.716±0.084   | **0.740±0.116**   | 0.706±0.106   |\n| SPE   | 0.732±0.091   | 0.756±0.079   | 0.748±0.092   | 0.735±0.120   | **0.763±0.121**|\n\nAs shown in the table below, increasing the number of levels generally improves performance up to 4 levels, after which the benefit plateaus or slightly decreases, possibly due to overfitting or increased model complexity.\n\n2. Effect of Scanning Strategies in Mamba\n\n| Method                | AUC   | ACC   | F1    | SEN   | SPE   |\n|-|-|-|-|-|-|\n| HMamba (ours)         | **0.838** | **0.732** | **0.740** | 0.740 | **0.735** |\n| HMamba w 3-axis scanning\t     | 0.805 | 0.713 | 0.733 | **0.756** | 0.670 |\n| HMamba w cross scanning       | 0.800 | 0.722 | 0.723 | 0.720 | 0.725 |\n\nWe have conducted explicit ablation studies comparing our multi-view Hilbert curve-based HMamba module with alternative voxel serialization strategies. The scanning strategy is replaced by the classical 3-axis scanning and cross scanning strategy for ablation experiments. Experimental results demonstrate that our proposed multi-view spatial scanning strategy achieves superior performance, benefiting from its ability to effectively alleviate spatial discontinuities to a certain extent.\n\n**For Q5, the readability of the table.**\n\nThank you very much for your suggestion. In the revised manuscript, we have highlighted the best results in Tables 3–5 in bold to improve readability and transparency of the reported comparisons. This formatting allows readers to more easily identify the top-performing methods for each evaluation metric. We appreciate your feedback and have updated the tables accordingly.\n\nThank you again for your serious and professional review comments. We believe that your suggestions will greatly help to improve the quality of our papers, and we hope that our response will address your concerns. If you have any other questions, we will be happy to discuss them with you!"}, {"review_id": "FSmDokcVz4", "reviewer": "Reviewer_Pe76", "summary": "This paper proposes a heterogeneous multimodal framework, termed HAD, for the early diagnosis of Alzheimer’s disease (AD), targeting two primary challenges: modality incompleteness and long-range dependency modeling. The framework offers two key technical contributions. First, it integrates a Hilbert-curve-ordered Mamba module with a hierarchical feature extraction strategy to simultaneously capture both local and global information. Second, it introduces a unified mutual information learning objective that explicitly balances semantic consistency and modality specificity within a shared multimodal embedding space. The authors validate the effectiveness of their method under varying modality-missing scenarios, demonstrating its potential in early AD diagnosis.", "questions": "1. The paper lacks a comprehensive review of related work in the field of Alzheimer’s disease (AD) diagnosis. It is recommended that the authors survey recent advances in this domain and incorporate a summary of representative methods. Moreover, the experimental section should include comparisons with established AD diagnostic baselines to more accurately position the contributions of this study.\n2. Regarding the use of five-fold cross-validation, there is a potential risk of data leakage. To ensure rigorous evaluation, the authors are advised to implement redundancy removal between training and testing sets.\n3. The paper modifies the original architecture or prediction layers of some baseline methods during comparative experiments, raising concerns about the fairness of these comparisons. The authors should clearly justify the necessity of such modifications and explain how fair evaluation conditions were maintained across all compared methods. Additionally, detailed training configurations—including hyperparameters, optimizers, and training epochs—should be provided to ensure reproducibility and transparency.\n4. The current experimental results are not sufficiently convincing. To strengthen the empirical support for the proposed method, further validation on additional publicly available AD diagnosis datasets is strongly encouraged. For clarity, the authors are also advised to explicitly highlight the top-1 and top-2 performing methods in all result tables.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This study targets the joint diagnosis of Alzheimer’s disease (AD) using multimodal data and explicitly addresses the issue of modality incompleteness in real-world clinical settings, highlighting its practical relevance.\n2. The method incorporates a Mamba module with Hilbert-curve-based scanning. Ablation studies demonstrate that this approach outperforms conventional alternatives on the utilized dataset.\n3. The proposed model accounts for both semantic consistency across modalities and modality-specific characteristics during the multimodal representation learning phase, enhancing representational quality.\n\nWeaknesses:\n1. Although the paper claims to address the challenge of missing modalities, the method section lacks dedicated mechanisms explicitly designed for this purpose. This discrepancy undermines the stated objective.\n2. The authors should clearly differentiate between the original contributions and the adopted components in the multimodal semantic representation learning module, and provide appropriate citations. Furthermore, the mechanism for preserving modality-specific information is insufficiently explained and lacks a solid theoretical foundation—more elaboration is needed.\n3. The experimental results do not convincingly demonstrate the superiority of the proposed fusion strategy. The method does not outperform baselines under a low missing rate (10%), and only shows advantages at a high missing rate (50%), raising concerns about its stability and robustness. A clear explanation for this performance variance is required.\n4. The current empirical evaluation is not fully convincing. To strengthen the evidence for the model’s effectiveness, additional experiments on other public AD diagnosis datasets are strongly recommended. Furthermore, comparisons with recent state-of-the-art methods such as MMTFN [1], MFASN [2], ShareGAN [3], and DSIP [4] should be included.\n5. The provided t-SNE visualizations do not effectively support the claims. Intermediate representations under certain modalities lack class separability, and thus cannot directly imply negligible contributions to classification performance. Moreover, the feature distributions across scenarios are not sufficiently distinct to substantiate the claim of a “strong modality preference.” The interpretability of the visualization needs improvement.\n6. The manuscript suffers from several clarity issues that hinder readability. For instance, the abstract mentions a \"spatial discontinuity caused by voxel serialization\" without offering adequate background context. Terminological inconsistencies are also present—for example, the architecture diagram shows the hierarchical feature extraction (HSFE) module as encompassing the HMamba module, while the main text describes them as separate, parallel modules. This inconsistency creates confusion about the actual structure of the model.\n\n[1] Miao S, Xu Q, Li W, et al. MMTFN: Multi‐modal multi‐scale transformer fusion network for Alzheimer's disease diagnosis[J]. International Journal of Imaging Systems and Technology, 2024, 34(1): e22970.\n\n[2] Jiao C N, Gao Y L, Ge D H, et al. Multi-modal imaging genetics data fusion by deep auto-encoder and self-representation network for Alzheimer's disease diagnosis and biomarkers extraction[J]. Engineering Applications of Artificial Intelligence, 2024, 130: 107782.\n\n[3] Wang C, Piao S, Huang Z, et al. Joint learning framework of cross-modal synthesis and diagnosis for Alzheimer’s disease by mining underlying shared modality information[J]. Medical Image Analysis, 2024, 91: 103032.\n\n[4] Xu H, Wang J, Feng Q, et al. Domain-specific information preservation for Alzheimer’s disease diagnosis with incomplete multi-modality neuroimages[J]. Medical Image Analysis, 2025, 101: 103448.", "was_revised": true, "final_justification": "The current rebuttal has addressed most of my concerns. I will update my score.", "rebuttal": "Thank you very much for your time and effort in reviewing our submission. The constructive feedback and critical questions have significantly contributed to the improvement of our work.\n\n**For W1, the design of missing modality.**\n\nThanks for the useful comment! Indeed, our method is oriented to missing multimodal scenarios, existing methods commonly adopt two routes to deal with incomplete multimodal data, one is to fill in the missing modalities by data completion or interpolation (like literatures [1-3]), and the other is to help the model avoid the interference of missing data by introducing additional prior missing information (like literature [4-6]). We adopt the latter to design our model, which is based on the following two considerations:\n\n**(1)** In our setup, we can not obtain complete multimodal data during the training phase, thus there is a lack of ground truth for effective supervision and quality assessment of the filled data produced in a generative manner, which in turn does not guarantee that the filling operation will yield a gain for the training of the model.\n\n**(2)** To the best of our knowledge, generation-based multimodal learning methods generally lead to higher computational complexity, so in view of the complexity of the current model, we choose the simpler but efficient way to deal with missing data, i.e., introducing a priori information to ignore missing modalities. In our manuscript, all operations are performed on available modalities, and in the detailed code, we just introduced a prior index matrix indicating which modality is missing in the multimodal fusion. This helps us reduce the negative impact of missing modalities at the lowest computational cost.\n\n[1] Multi-Modality MR Image Synthesis via Confidence-Guided Aggregation and Cross-Modality Refinement, JBHI, 2022.\n\n[2] Multi-Modal Modality-Masked Diffusion Network for Brain MRI Synthesis with Random Modality Missing, TMI, 2024.\n\n[3] Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts, ICCV, 2023.\n\n[4] MFTrans: Modality-Masked Fusion Transformer for Incomplete Multi-Modality Brain Tumor Segmentation, JBHI, 2024.\n\n[5] View-aligned hypergraph learning for Alzheimer’s disease diagnosis with incomplete multi-modality data, MIA, 2024.\n\n[6] Latent Representation Learning for Alzheimer’s Disease Diagnosis With Incomplete Multi-Modality Neuroimaging and Genetic Data, TMI, 2019.\n\n**For W2, the clarification of key contributions and the elaboration of preserving modality-specific information.**\n\n**(1)** For the proposed multimodal semantic representation learning module. Our multimodal semantic representation learning module is grounded in information theory, aiming to maximize the mutual information between heterogeneous multi-modal embeddings in the latent space. This optimization objective is consistent with existing information-theoretic multimodal learning frameworks [1-2], which are based on the principles of multimodal consistency and diversity—that is, achieving semantic-level alignment while retaining each modality’s unique contribution to the task. However, our approach differs in several important respects: **<1>** We observed that in AD diagnosis, the four modalities (MRI, PET, CSF, and CAD) exhibit substantial heterogeneity. Standard probabilistic graphical models (e.g., $\\mathbf{g} \\rightarrow \\mathbf{z} \\rightarrow \\mathbf{y}.$) often result in the learned cross-modal representation $\\mathbf{z}$ being heavily biased toward modalities that are easier to fit (e.g., CSF and CAD), leading to training collapse for more complex modalities such as MRI and PET. To address this, we introduce a direct link from the modality-specific features to the label ($\\mathbf{g} \\rightarrow \\mathbf{y}$), resulting in a joint probabilistic model: $\\mathbf{g} \\rightarrow \\mathbf{z} \\rightarrow \\mathbf{y}, \\mathbf{g} \\rightarrow \\mathbf{y}$ This design enables us to derive a learnable weighted multimodal late fusion model, ensuring that modality-specific information contributes directly to the final decision, while still capturing joint semantics. \n**<2>** In addition, during training, we reverse the fusion weights $q(\\mathbf{y}|\\mathbf{g})$ by taking the negative of the learnable parameter $\\eta$, such that underrepresented (hard-to-fit) modalities are prioritized in prediction. This novel strategy reduces over-reliance on easily overfitted modalities and encourages the network to learn from complex modalities, mitigating the issue of training collapse. Related probabilistic reasoning frameworks are now properly cited in the revised manuscript.\n\n**(2)** For the preservation of modality-specific information. \n\nThe preservation of modality-specific information is also governed by our mutual information maximization objective. Specifically, we seek to maximize the mutual information between the set of modality-specific features $ \\mathbf{g} = \\{\\mathbf{g}^{(v)}\\}_{v \\in \\mathcal{V}} $ and the supervision signal $\\mathbf{y}$. While many existing approaches [3-4] focus on semantic consistency between joint multimodal representations and the target label, this may not adequately preserve modality-specific information in the cross-modal representation. Instead, our method directly leverages label supervision to guide the learning of modality-specific features, i.e., $\\max I(\\mathbf{g}; \\mathbf{y})$. Furthermore, in modeling $q(\\mathbf{y}|\\mathbf{g})$, we explicitly account for the contribution of each modality-specific feature to the final decision:\n\n$q(\\mathbf{y}|\\mathbf{g}) =: \\frac{1}{2}q{\\theta}(\\mathbf{y}|\\mathbf{z})+\\frac{1}{2}\\sum_{v\\in \\mathcal{V}}\\omega^{(v)} q_{\\theta_v}(\\mathbf{y}|\\mathbf{g}^{(v)}), s.t., \\sum_{v\\in \\mathcal{V}} \\omega^{(v)} =1$\n\nWe argue that this design enhances the discriminability of modality-specific features during model training.\n\n[1] Learning Robust Representations via Multi-View Information Bottleneck, ICLR, 2020.\n[2] Aggregation of Dependent Expert Distributions in Multimodal Variational Autoencoders, ICML, 2025.\n[3] Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification, ICML, 2022.\n[4] Multimae: Multi-modal multi-task masked autoencoders, ECCV, 2022.\n\n**For W3, performance on low missing rate.**\n\nThis is a thoughtful comment! From our experimental results, our method demonstrates superior performance primarily under high missing rates (50%), while the advantage is less evident under low missing rates. The main reason for this observation is that when the missing rate is low, nearly all fusion methods—including ours and the baselines—have access to almost complete modality information, and thus the benefit of advanced fusion strategies is diminished. In such cases, simple or classical fusion methods can already achieve strong performance, and the room for improvement is limited. \n\nOn the other hand, under a high missing rate, the available modalities for each sample are uncertain, significantly exacerbating modality imbalance and amplifying the differences in modality heterogeneity, which ultimately leads to modality training collapse. Our method effectively addresses the issue of modality heterogeneity under high missing rates by establishing the goal of maximizing mutual information between modal-specific embedding codes and semantic information.\n\n**For W4 and Q4, more comparison methods and experiments on other datasets.**\n\nThank you for your constructive suggestions! In the revised manuscript, we added comparison experiments with the MMTFN, DSIP, and ShareGAN. However, MFASN relies on ROI-based feature extraction with non-uniform brain parcellation standards, differing fundamentally from the 3D image-based methods in our study and baselines. Therefore, we added additional experiments on the other three suggested methods. \n\nSince the above three methods can only handle two image modalities, MRI and PET, we only use the MRI and PET modalities to conduct the comparative experiment. Experimental results on AD vs. CN are shown below (The best results are bolded and the second-best results are italicized):\n\n| Missing| Metric\\| | Ours   | ShareGAN | DSIP | MMTFN  |\n|-|-|-|-|-|-|\n| 0.1     | AUC   | **0.811**  | *0.790*    | 0.793      | 0.735  |\n|         | ACC   | **0.735**  | *0.723*    | 0.725      | 0.655  |\n|         | F1    | **0.749**  | *0.721*    | 0.715      | 0.642  |  \n|         | SEN   | **0.762**  | 0.690    | *0.695*      | 0.680  |\n|         | SPE   | *0.714*  | **0.761**    | 0.755      | 0.631  |\n\nIn addition, we perform comparison experiments on the new publicly available dataset **AIBL**. \n\n| Task | Missing | Metric | LMVCAT | Adapted | ShaSpec | GMD | MMTFN | ShareGAN | DSIP | Ours |\n|------|---------|--------|--------|----------------|---------|-----|-------|----------|------|------|\n| ADCN | 0.1     | AUC    | 0.918  | 0.919          | *0.920* | 0.912 | 0.890 | 0.845    | 0.845 | **0.936** |\n|      |         | ACC    | *0.871*  | 0.861          | 0.714 | 0.806 | 0.746 | 0.653    | 0.653 | **0.868** |\n|      |         | F1     | *0.906*  | 0.900          | 0.755 | 0.881 | 0.785 | 0.678    | 0.678 | **0.910** |\n|      |         | SEN    | *0.943*  | 0.872          | 0.783 | **0.960** | 0.806 | 0.761    | 0.761 | 0.915 |\n|      |         | SPE    | 0.677  | **0.831**          | 0.523 | 0.393 | 0.585 | 0.354    | 0.354 | *0.742* |\n\n**Due to space limitations**, we only show some of the results in the above tables. It can be seen from the tables that our method has achieved leading performance both in the two modalities and on the new AIBL dataset, especially in the key metrics AUC, ACC and F1.  Specifically, although SEN and SPE can often exhibit an imbalance—one metric being very high and the other very low due to their definitions in binary classification problems—our method still demonstrates competitive performance when considering these two metrics jointly.\n\nPlease refer to more responses in the following comment block."}, {"review_id": "40KAnDf4Lt", "reviewer": "Reviewer_S54Q", "summary": "This paper addresses the challenge of heterogeneous multimodal data integration for ADNI diagnosis, specifically combining 3D medical images and tabular data. The authors introduce a novel fusion framework that leverages Mamba and Hilbert curves to capture both spatial locality and long-range dependencies in 3D imaging. To improve multimodal representation learning, they formulate a mutual information maximization objective that jointly promotes label discriminability and semantic consistency. Importantly, the method incorporates a reversed weighting scheme to mitigate modality dominance caused by imbalanced convergence. The proposed model achieves state-of-the-art performance across varying levels of modality missingness, and the visualizations effectively illustrate its ability to retain label-discriminative features.", "questions": "Please refer to the \"Weaknesses\" part in \"Strengths And Weaknesses*.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper is well presented with a clear research scope and motivation, making it accessible and easy to follow. \n2. The proposed method is innovative and demonstrates strong performance on the selected benchmarks, highlighting its potential contribution to the field.\n\nWeaknesses:\n1. The paper does not sufficiently justify or analyze the efficacy of the loss function used for enforcing modality consistency (second term in the mutual information loss function), which is critical given the multimodal nature of the task. \n2. The reasoning and analysis in modality imbalance are comparatively weak as the figure can not sufficiently demonstrate the issue. More evidence should be provided on the statement \"Our hybrid late-fusion strategy resolves this imbalance by allowing synchronized optimization across all modalities.\"\n3. The paper lacks proper citation and discussion of a recently published paper on using 3D Hilbert curve to model 3D medical images: \n\"Grela, Jacek, et al. \"Using space-filling curves and fractals to reveal spatial and temporal patterns in neuroimaging data.\" Journal of Neural Engineering 22.1 (2025): 016016.\"\n4. The paper does not include a comparison of model complexity or computational cost, which is essential for evaluating the method’s practicality and efficiency relative to existing approaches.", "was_revised": true, "final_justification": "My concerns are addressed. I will keep my current score of accept.", "rebuttal": "We sincerely appreciate your thorough review and constructive feedback on our manuscript. Your recognition of the paper's innovativeness and valuable suggestions have provided crucial guidance for further improving our research. In response to the four key issues you raised, we have conducted in-depth responses and provide the following clarifications:\n\n**For W1, “The paper does not sufficiently justify or analyze the efficacy of the loss function used for enforcing modality consistency (second term in the mutual information loss function), which is critical given the multimodal nature of the task.”**\n\nAs described in Section 3.3 of our manuscript, the second term in the mutual information maximization objective corresponds to both the intra-modal and inter-modal reconstruction losses, denoted as $L_{intra}$ and $L_{inter}$, respectively. These losses are derived from the lower bound of the mutual information between modality-specific features and the cross-modal semantic representation (see Eq. 11). Specifically, $L_{intra}$ encourages the preservation of modality-specific information by reconstructing features within each modality, while $L_{inter}$ enforces consistency by reconstructing features across modalities, thereby promoting cross-modal alignment.\n\n**For W2, The reasoning and analysis in modality imbalance are comparatively weak as the figure can not sufficiently demonstrate the issue. More evidence should be provided on the statement \"Our hybrid late-fusion strategy resolves this imbalance by allowing synchronized optimization across all modalities.\"**\n\nThank you for your valuable suggestion regarding the analysis of modality imbalance and the effectiveness of our hybrid late-fusion strategy.\n\nTo provide more quantitative evidence, we conducted a new experiment: for each modality, we extracted the modality-specific features after training, fed them independently into the same classifier, and measured their classification performance. This analysis was performed under two training schemes: (a) mid-fusion only, and (b) our proposed hybrid late-fusion strategy. The results are presented below:\n\n| Metric |  MRI | PET | CSF | CAD | \\|MRI | PET | CSF | CAD |\n|--------|----------------------|-----|-----|-----|------------------------|-----|-----|-----|\n|         |   Mid-               |Fusion|Only|       | \\|Hybrid           |Late|-Fusion|       |\n| AUC    | 0.467                | 0.198 | 0.787 | 0.999 |\\| 0.726                  | 0.877 | 0.893 | 0.999 |\n| ACC    | 0.221                | 0.153 | 0.762 | 0.992 |\\| 0.763                  | 0.802 | 0.844 | 0.992 |\n| F1     | 0.363                | 0.213 | 0.681 | 0.982 |\\| 0.279                  | 0.649 | 0.725 | 0.982 |\n\nAs shown in the table, under mid-fusion only, MRI and PET achieve significantly lower AUC scores compared to CSF and CAD. In contrast, our hybrid late-fusion approach yields consistently high AUC for all modalities, especially improving MRI and PET. This provides strong evidence that the hybrid late-fusion strategy indeed enables more balanced and effective training of all modality-specific encoders, supporting our claim about synchronized optimization and improved modality balance.\n\n**For W3, \"The paper lacks proper citation and discussion of a recently published paper on using 3D Hilbert curve to model 3D medical images: \"Grela, Jacek, et al. \"Using space-filling curves and fractals to reveal spatial and temporal patterns in neuroimaging data.\" Journal of Neural Engineering 22.1 (2025): 016016.\"\"**\n\nWe sincerely thank you for identifying this important omission. We fully acknowledge the reference value of this study for our methodological design and have made the following additions in the revised manuscript:\n\n​​Supplement to Methods Section 2.2.1:​​\n\n    \"Building upon existing work [1], we adopted the same Hilbert and Peano curves for space-filling transformations while maintaining their core concept of 'locality-preserving property of space-filling curves.' However, our proposed HSFE core module introduces two key innovations: (1) A hierarchical architecture based on the fractal theory of 3D Hilbert curves enables multi-scale information fusion through multi-level recursive structures; (2) A multi-directional scanning mechanism (incorporating axial rotation and mirror transformations) enhances complementary spatial information capture.\"\n\n[1] Using space-filling curves and fractals to reveal spatial and temporal patterns in neuroimaging data. Journal of Neural Engineering.\n\n**For W4,\"The paper does not include a comparison of model complexity or computational cost, which is essential for evaluating the method’s practicality and efficiency relative to existing approaches.\"**\n\nThank you for your valuable comment regarding the lack of comparison in model complexity and computational cost. We fully agree that assessing model efficiency and practicality is crucial for real-world deployment.\n\nTo address this, we have conducted a comprehensive comparison of model parameters (Params, in millions) and computational cost (FLOPs, in GigaFLOPs) with representative state-of-the-art methods. The results are summarized below:\n\n| Model      | Params (M) | FLOPs (G) |\n|------------|:----------:|:---------:|\n| LMVCAT     |   97.33    |  81.43    |\n| CM3T       |   10.41    | 147.96    |\n| GMD        |   94.3     |  81.56    |\n| DMRNet     |  101.79    |  81.56    |\n| Adapted    |   97.33    |  81.43    |\n| DSIP       |   32.97    | 274.64    |\n| ShareGAN   |   48.91    | 343.17    |\n| MMTFN      |  106.61    | 148.82    |\n| ShaSpec    |   0.97     |   5.64    |\n| **Ours**   |   43.42    |   6.65    |\n\nAs shown in the table, our proposed model achieves a favorable balance between model size and computational cost. Specifically, while maintaining competitive performance, our framework requires substantially fewer FLOPs compared to most baselines (e.g., only 6.65 GFLOPs vs. 81.43 GFLOPs for LMVCAT and 147.96 GFLOPs for CM3T). Our parameter count is also moderate, reflecting the model's efficiency.\n\nThank you again for your helpful and professional review comments. We hope that our response will address your concerns. If you have any other questions, we will be happy to discuss them with you!"}]

Author rebuttal:

Meta-review: This paper proposes a heterogeneous multimodal diagnosis framework for Alzheimer's Disease which can handle arbitrary incomplete modalities by combining multi-view Hilbert curve-based feature extraction, hierarchical modeling, and mutual information-based representation learning.

Four reviewers, each with confidence level 4, have given scores of 4, 4, 4 (borderline accept) and 5 (accept). Reviewers agree on well motivated algorithmic development and empirical results that show superiority over state-of-the-art alternatives. Several concerns were raised around additional empirical results, e.g., ablations that demonstrate key contributions of Hilbert curves in their method, comparisons that show modality-wise performance, running time comparisons. Most of these questions were satisfactorily addressed by the authors in their rebuttal. 

With all the additional results and discussion incorporated, I believe the paper presents a strong contribution in an important application domain. I strongly suggest adding standard errors to the ablation study (as suggested by a reviewer) as well as to all other comparative analyses, wherever applicable.
Decision: Accept (poster)