Democratizing Clinical Risk Prediction with Cross-Cohort Cross-Modal Knowledge Transfer
Authors: Qiannan Zhang
Manqi Zhou
Zilong Bai
Chang Su
Fei Wang
Venue: NeurIPS 2025 poster ()
Keywords: Risk Prediction
Cross-Cohort
Multimodal Learning
Primary area: machine_learning_for_sciences

Abstract: Clinical risk prediction plays a crucial role in early disease detection and personalized intervention. While recent models increasingly incorporate multimodal data, their development typically assumes access to large-scale, multimodal datasets and substantial computational resources. In practice, however, most clinical sites operate under resource constraints, with access limited to  EHR data alone and insufficient capacity to train complicated models. This gap highlights the urgent need to democratize clinical risk prediction by enabling effective deployment in data- and resource-limited local clinical settings. In this work, we propose a cross-cohort cross-modal knowledge transfer framework that leverages the multimodal model trained on a nationwide cohort and adapts it to local cohorts with only EHR data. We focus on EHR and genetic data as representative multimodal inputs and address two key challenges. First, to mitigate the influence of noisy or less informative biological signals, we propose a novel mixture-of-aggregations design to enhance the modeling of informative and relevant genetic features. Second,  to support rapid model adaptation in low-resource sites, we develop a lightweight graph-guided fine-tuning method that adapts pretrained phenotypical EHR representations to target cohorts using limited patient data. 
Extensive experiments on real-world clinical data validate the effectiveness of our proposed model.

Democratizing Clinical Risk Prediction with Cross-Cohort Cross-Modal Knowledge Transfer
Qiannan Zhang, Manqi Zhou, Zilong Bai, Chang Su, Fei Wang"
Weill Cornell Medicine, Cornell University
ma124005,,az4110,zzz,,, , corneee cornell...........
Abstract
Clinical risk prediction plays crucial role in early disease detection and personal- ized intervention. While recent models increasingly incorporate multimodal data, their development typically assumes access to large-scale, multimodal datasets and substantial computational resources. In practice, however, most clinical sites operate under resource constraints, with access limited to EHR data alone and in- sufficient capacity to train complicated models. This gap highlights the urgent need to democratize clinical risk prediction by enabling effective deployment in data- and shource-limited local clinical settings. this work, we propose a cross-cohort cross-modal knowledge transfer framework that leverages the multimodal model trained nationwide cohort and adapts it to local cohorts with only EHR data. We focus on EHR and genetic data as representative multimodal inputs and address two key challenges. First, mitigate the influence of noisy less informative biological signals, we propose novel mixture-of- aggregggtts design to enhance the modeling of informative and relevant genetic features. Second, to support rapid model adaptation in low-resource sites, we develop lightweight graph-guided fine-tuning method that adapts pretrained phenotypical EHR representations local cohorts using limited patient data. Extensive experiments on real-world clinical data validate the effectiveness of our proposed model.
Introduction
Predictive diagnosis and risk prediction play a pivotal role in clinical practice, particularly for chronic conditions such Alzheimer's disease and related dementias, which stem from complex interplay of genetic and environmental factors and impose significant socioeconomie burdens and public health challenges. As current treatments may slow progression yet cannot reverse the pathological process, early risk prediction before clinical diagnosis aims for personalized monitoring, proactive management and preventive care advance slow disease progression and improve health outcomes.
Recent studies on disease risk prediction primarily rely on electronic health record (EHR) data to cap- ture weal-world patient phenotypical information [21 27 50 43] including diagnoses, prescriptions, etc. A growing number of research has explored introducing additional biological modalities, such as medical imaging, genomics and proteomics, boost predictive performance 22 40 10
56 59].
However, most existing approaches are developed for a single cohort, under the assumption that large patient populations, modalities, and ample computational resources available for training. In reality, this assumption often fails to hold, as many regional health providers may contend with insufficiency of patient samples, costly modalities, and computing infrastructures train predictive models locally. Thus, democratizing clinical risk prediction capability to local cohorts is an important step toward enabling accessible and equitable healtheare solutions across diverse clinical scenarios.
Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Inspired by the growing development of nationwide biobanks such as the
Source Site
Target Sites
All of Us Research Program and UK Biobank [6 [35, which collect high-volume multimodal data besides EHRs to advance biomedical research, and driven by the goal to democra- tize predictive modeling for local clin- ical sites constrained to EHRs alone, this study investigates the following research question: Can we develop a predictive model using multimodal data from nationwide cohort and transfer it to resource-limited local cohorts with only EHR data, enabling accurate risk prediction without sharing sensitive patient information?
Knowledge
Transfer
Training
Fine turing
Cross-Cohore
Nationwide Cohort
Local Clinical Cohorts
Multimodal Data
Unimodal Data
Cross-Modal
High Computing Capacity
Low Computing Capacity
Figure I: Conceptual diagram of cross-cohort cross-modal knowledge transfer.
Despite its appeal, developing such a model is non-trivial due to two challenges. First, multimodal data is inherently noisy or imprecise due measurement errors and multifactorial biological processes, which can complicate the correlation between multimodal features and disease outcomes, degrading predictive accuracies. Second, cross-man moos--mdaal learning from nationwide multimodal cohorts to local unimodal cohorts inevitably faces distribution shifts due to different population characteristics, data collection protocols, model to adapt to the target cohort. However, the limited data and computational resources at local cohorts make it difficult perform effective fine-tuning. Insufficient data increases the risk of overfitting, while constrained computational capacity limits the use of powerful models that typically require substantial computational overhead.
healthcare practices. These shifts necessitate fine-tuning of the transferred
Motivated by these challenges, this paper proposes a cross-cohort, cross-modal knowledge transfer framework named C to support the democratization of clinical risk prediction at resource-limited local clinical sites. In this work, we focus on utilizing EHR and genetic modalities on the source biobank due to the wide availability of EHRs across cohorts and the essential information that genetic data offers to capture individual-level biological mechanisms associated with disease risk [1). To address the first challenge of noisy and imprecise genetic data, we design strategy for gene encoding with mixture-of coatuur--o--gggegaaait off-aggregttios.. This approach employs multiple aggregation tokens to jointly identify and aggregate meaningful genetic features from different views through attention modulation and feature reconstruction, while mitigating the influence of noisy or less informative signals. To address the second challenge involving limited data and computational resources that hinder model fine-tuning, we draw inspiration from the increasing use of foundation models. Specifically, we adopt an EHR foundation model as the EHR encoder, which enables rapid and efficient phenotypical EHR representation generation, even when local cohorts have only a small number of patients. However, fine-tuning a foundation model remains extremely difficult in snouucc-coossrrinee clinical sites due to the substantial computational demands. To overcome this limitation, we propose a lightweight fine-tuning method that utilizes limited patient data by constructing a bipartite graph between patients and the medical concepts recorded in their EHRs. A graph neural network (GNN) is then applied to this bipartite graph to adjust the foundation model model-initialized phenotypical embeddings based on their associated medical concepts, achieving effective adaptation without modifying the foundation model itself. Finally, with the obtained phenotypical EHR and genetic embeddings and the goal of transferring model to resource-limiter local cohort with unimodal data, we leverage teacher- student distillation to achieve cross moddl knowledge transfer. The teacher prediction model is distilled into student prediction model, which is then transferred along with the GNN to the local cohort for further une-tuning and downstream risk prediction.
We study the crucial and practical problem of democratizing clinical risk prediction capabilities in resource-limited local cohorts by leveraging knowledge from nationwide multimodal cohorts; 2) We propose a novel miiture-of--agregeee design to enhance the modeling of informative and relevant genetic features while mitigating the influence of noisy or less informative signals; 3) We develop lightweight graph-guided fine-tuning method that adapts pretrained phenotypical EHR representations to target cohorts using limited patient data, without requiring updates to the foundation model; 4) We conduct extensive experiments on real-world clinical data to demonstrate the effectiveness and generalizability of our model.
Our contributions can be summarized as follows: 1)
2

---

2 Related Work
Cross-modal learning. Conventional multimodal learning methods aim to capture complementary information across modalities by multimodal fusion B 59, typically assuming complete modality availability. However, modality missingness, common in practice, presents a widespread challenge to their applicability [54]. Therefore, research has focused on learning cross-modal knowledge to compensate for missing information, falling in two main directions. One line of work relies on modality completion that reconstructs the missed modalities from observed ones through generative
53 29] contrast, modality-robust learning seeks to build representations that remain effective under incomplete observations without explicit reconstruction.
networks [44 68 7 25
16
51
19
34
Common strategies explore adaptive integration [28] 57.. invariant representations [2 61] or
to leverage the modality interactions. Besides, knowledge distillation is also employed where the teacher model with multimodal data supervises modality-
graph-based relations [62 58
63
64
55
incomplete students to transfer cross-modal knowledge 36 37
49
20,
48) These methods are
primarily designed to handle random or block-wise missingness, under assumptions of modality availability test time, distributional consistency across modalities, or in-domain training and deployment. They often struggle to generalize when certain modalities are consistently absent at deployment, and become less applicable in our setting, where modality availability and data distribution differ across cohorts.
Cross-cohort knowledge transfer. The healthcare community increasingly advocates for cross- cohort model deployment to facilitate the transfer of beneficial knowledge, drawing inspiration from cross-domain transfer learning across disciplines 124, 66 600 65]. Despite its promise to enhance predictive diagnosis and risk prediction through the use of multi-instiutional data, the question of how to achieve effective multimodal learning across cohorts remains largely open and underexplored due to practical constraints on data sharing and computational resources [8, 18]. In fields such as computer vision and natural language processing, multimodal cross-domain adaptation addresses domain shifts to facilitate knowledge transfer with multiple modalities. Classical approaches primarily focus on reducing domain discrepancies by aligning representations through domain adversarial learning or by extracting domain-invariant features (3 [6] 30, [3] These methods typically assume consistent modality availability, shared samples across domains, and simultaneous access to both source and target domains during training. A few works consider more realistic scenarios, where the target domains lack certain modalities or have non-overlapping modalities compared the source [67]. while they still necessitate joint training with target domains. In contrast, our setting assumes no access to the target domain during training and involves consistent modality missingness at deployment across all target samples. This poses more challenging generalization problem, requiring source- source-trained trained models exhibit stronger cross-cohort transferability.
3 Preliminaries
Problem Definition. We consider the task of risk prediction in a cross-cohort cross-modal setting, where the source cohort comprises a nationwide patient population with rich multimodal data, while the target cohort consists of local patient population with only EHR data available. In this paper, we focus on two modalities: electronic health records (EHRs) and genetic profiles. Specifically, in the source cohort = {(r.. 9h No, where Ng denotes the total number of patients, each patient i is
associated with (1) a sequence of longitudinal EHR events T; = {ee, e? ...., c!!rl) extracted from their medical history, (2) binary genetic profile 9, E {0, 11.5 indicating the absence or presence of genetic mutations for the N genes under consideration, and (3) a disease label y: E {0. 1) indicating the occurrence of the outcome. EHR events r; capture the patient's 's phenotypical information, where each event e, corresponds to a coded medical concept, e.g., a diagnosis or a prescribed drug. Here we denote the set of medical concepts as C. To enable early prediction, only EHR events occurring prior to the prediction time are utilized. A prediction model F(r., g.) + 9; E [0, 1] is trained on this cohort to estimate the disease risk using both EHR and genetic data. In the target cohort T (ry, yソ) ))))% >y consisting of Nr patients, only EHR data is available. This reflects practical scenario in local cohorts, where biological data such as genetic profiles are typically unavailable. The target cohort may differ from the source cohort demographically and clinically.
In this paper, we aim to develop a multimodal prediction model using the source cohort S that can be effectively transferred and adapted to the unimodal, ciriibttionsshiftee target cohort T, under the
3

---

Graph-guided Phenotypical Representation Fine-tuning
Cross-Modal Knowledge Transfer
P
C1
Student Model
EHR Foundation
Pa
C2
Phenotypical
Representation
Model
Teacher Model
Patients
Medical Concepts
Representotion
...
Apgregation Token y(M)
Aggregation Token (R)
Updated Aggregation Token
Transformer Encoder
hi
Gene Decode D
Attention Modulation
Gene Tokens
Modulated Gene Takens
Gene Feature Reconstruction
Gene Encoding with AfigeatnsfAsgrrggatoon
Gene encoding with mixture-of- aaggureeof-aggregaatonn identifies the most relevant and informative genes to learn genetic representation; 2) Graph-guided phenotypical representation fine-tuning adapts pre-trained EHR embeddings through a lightweight design; 3) Cross-modal knowledge transfer facilitates model transfer to target cohorts for gene-free deployment, by distilling knowledge from both EHR and genetic modalities.
Figure 2:
An overview of the C framework:
constraint that data from S and T cannot be jointly accessed due to restrictions. To facilitate adaptation, assume access to small labeled subset Tiabelod C. T, which can be used to fine-tune the model using only EHR data available the target cohort.
non-sharing
MOTOR. MOTOR (Many Outcome Time-Oriented Representations) 42 pretrained foundation model designed to encode structured EHRs into patient-level embeddings. Given a longitudinal where each event et may include clinical codes (e.g.. diagnoses.
EHR record {el e2, ..... e r),, medications, lab tests, etc.) and corresponding timestamps, MOTOR transforms this sequence into dicensiamnni embedding vector that captures the temporal dynamics and clinical context of the patient's 's history. The model is pretrained on large-scale EHRs and claims data using self-supervised time-to-event prediction objective, thus learning informative representations for survival and risk modeling tasks.
4
The Proposed Method
In this section, we present our proposed model, C which comprises three key components: 1) gene encoding with nixture-of-aggregatton 2) graph-guided EHR representation fine-tuning, and 3) cross-modal knowledge transfer with distillation. The overall framework is illustrated Figure
4.1 Gene Encoding with Mixture-of-Aggregattin
In the source cohort, for patients with available genetic profiles, we first aim to encode the genetic data into representations. With each gene represented with binary indicator 9² E {0, 1} to denote if any mutation occurs or not, these indicators collectively form an individual-level input vector g E {0, 1).N. where N is the total number of genes. To embed these discrete features into a continuous space, we treat each gene token and assign it a learnable embedding, yielding a sequence of gene token embeddings (g* ,g......""""
NN E I for each patient, with d denotes dimensionality.
Gene-level binary vectors are typically sparse and noisy, as not all genetic variants are equally associated with and contributory disease prediction. To enable the model to adaptively focus on the most relevant subset of genes, we introduce learnable aggregation token for each patient that attends all gene token embeddings, aggregates information across genetic features and implicitly identifies those most predictive of the health conditions. This allows the model to dynamically prioritize informative genetic signals on per-patient basis. Formally, we denote the learnable aggregation token by E IRd. Then, multi-head attention is applied with as the query and the gene token embeddings as keys and values, producing soft attention distribution over gene tokens using the standard self-attention 45). It results in set of non nog-nngative mask weights
Aggregation Token.

---

{m1,.... myy }, where each my E (0. and Σ m, = 1, as determined by a softmax function. These weights quantify the relative importance of each gene in the context of the specific patient and are utilized to softly modulate the gene token embeddings. Specifically, each gene token is blended with the aggregation token following:
g (1- m.) m, .
(I)
so that highly attended genes are more strongly influenced by the aggregation token. The resulting
sequence [v. g. ................................ gg is further fed into a Transformer encoder. Ultimately, its output at the first position, corresponding to the updated aggregation token, is employed as the patient-level genetic representation hG, which integrates information from all gene token embeddings support risk prediction. The remaining contextualized outputs {hi,... h.. }or from the encoder are utilized to reconstruct the original gene binary features. If features corresponding to important genes can be accurately reconstructed, it suggests that the aggregation token effectively captures and integrates individual genetic information.
Gene Feature Reconstruction. In addition to producing personalized genetic representations, the model is trained to reconstruct the original genetic features from the contextualized genetic outputs (hi,... ..., hy }. This reconstruction encourages the encoder to preserve information relevant to the input genetic features, and guides the aggregation attention to concentrate on meaningful genes by penalizing reconstruction errors at attended positions. Each contextualized gene token embedding h; is transformed to a binary prediction via a shared decoder as follows:
softmax (Wh; + b):
where W and b are learnable parameters of the decoder. We calculate a weighted reconstruction loss using the aggregation attention scores {mi J.V attended positions, as defined below:
thereby emphasizing reconstruction at highly
N Cree
CB(g', g"),,
where CE(-) denotes the cross-entropy loss. It promotes effective aggregation by prioritizing infor- mative genes and minimizing the influence of less relevant ones.
Although single aggregation token can highlight important genes, its representational capacity can be constrained when modeling complex, multifactorial biological signals. A single token is required to condense relevant gene-level information into a fixed-size vector, which might overlook the input's 's functional complexity, particularly for polygenic diseases where risk is distributed across multiple genetic components. To overcome this limitation and better capture the compositional patterns of genetic data, we extend the single-token design to tixture--of-agregaall framework. Specifically, we introduce M independent aggregation tokens, each functioning specialized expert that attends to the gene token embeddings from a distinct perspective and produces its own representation of the genetic feature. Formally, we learn set of expert-specific aggregation tokens (v(1), ..... v (M), v(K) E J. For each expert k. we compute soft attention weights over the
Bistureeof-AAgregattin
gene token embeddings, yielding a set of mask scores {m (k:) Ji=1+ with which expert-specific gene token embeddings are modulated:
(k) (1)) (M) = (1 m (k)) g' + m
The sequence (v(k), ,g' (k)
[ is passed through the shared Transformer encoder, producing an expert-specific genetic representation h and reconstruction logits g²h for each gene using Eq. To combine outputs from all experts, we adopt averaging to obtain he, g" and m, as follows:
M Σ
M
M
hg =
M
1
G
g
M
9
(k)
m.
M
1
Σ
m
(5)
where ÿ and m, are utilized in Eq. 3 This allows the reconstruction objective Crer to benefit from multi-expert knowledge. The extension encourages diverse specialization among experts and allows the model to capture multiple, potentially disentangled genetic functionalities.
5

---

4.2 Graph-Guided Phenotypical Representation Fine- Line-Tuning
Beyond genetic information, EHR data as critical resource provides rich phenotypical information for disease risk prediction. Inspired by the recent success of foundation models, we leverage the pretrained EHR foundation model MOTOR [42] to encode each patient's 's longitudinal medical history into fixed-diment -iceediinnsional embedding. Specifically, given an EHR sequence from patient, we obtain its embedding he = Motor(r). However, since both the source and target cohorts may differ from the data used in MOTOR pretraining, in terms of population characteristics and record conventions, the resulting embeddings generally suffer from distribution shift. Nevertheless, fine-tuning the foundation model inevitably demands substantial computational resources and labeled data, which are often unavailable in real-world local cohorts. To address this challenge, we propose lightweight graph-guided fine-tuning strategy that adapts the pretrained embeddings novel cohorts without updating the parameters of MOTOR itself. Instead, we leverage graph capturing cohort-specific patterns between patients and EHR concepts conduct fine-tuning tailored to cohort characteristics and thus optimize predictive capability.
Graph Construction. To facilitate the adaptation, we construct a bipartite graph 9 = =V,E), where the node set V includes both patients and EHR concepts. Specifically, we define 1 U V.. where represents the patients in the cohort S, and V. = (C,, C21,02.....0 ..K the set of distinct medical concepts observed in the cohort. An undirected edge (v,, E & is added if patient j has at least one occurrence of concept C* in their EHR record rj prior to the prediction time. The resulting graph captures co-occurrence relationships between patients and clinical events and serves as structural guidance for message passing during the adaptation process.
Vp = {Ui, ..,,...; U:g.]
denotes
(x)
Graph-guided Embedding Adaptation. Given the bipartite graph G = EV,E), we apply graph neural network (GNN) to refine patient embeddings using local structural context. Let {hg giji denote the initial MOTOR- beseed embeddings for the patients in Vp. and let C fc& 1 denote learnable embeddings for concept nodes in V. We perform L layers of message passing as:
Ns
(1+1) g
=
Σ W(x;)) (LEN()))
w(O
(0)) c
(e+1) ((t1)= Ck
=
Σ
1 [(cx)]
W
bg
(Ox)(())
where W. and W. are learnable parameters, and (6.) is nonlinear activation function, and / denotes node neighborhood. At each layer, patient nodes aggregate information from connected concept nodes, and vice versa. The resulting hg at layer L denotes phenotypical representations for patients based on EHR information, accounting for the cohort characteristics.
4.3 Cross-Modal Knowledge Transfer with Distillation
To support prediction in local cohorts without genetic data, we adopt a sather- distillation framework that transfers knowledge from gene-aware model (teacher) to gene-free model (student), enabling generalization target cohorts that lack genetic information.
The teacher model is trained on the source cohort with genetic and EHR data available. It takes as input the learned genetic representation h& from the gene encoder, along with phenotypical embedding hg after graph-guided adaptation. These inputs are concatenated and fed into the teacher model produce output logits, as supervision for teacher training, defined Le:
y1= frechhe ([hg; hg]), Lueschrr CE(y,,
In contrast, the student model operates without access to genetic data. relies solely on the EHR information hy to generate predictions y, = Sanutent (hr), where teacher and student models are implemented as multi-layer perceptrons.
Knowledge transfer is achieved through logit-level distillation, where the student is trained with hybrid objective to integrate the supervised signal and distillation loss, which aligns its output y, with the teacher's prediction The total training loss is defined as:
Lstudent (1 AKD) CE v) + ARD Lxo(y..
(8)

---

where LKD is the KL divergence between student and teacher logits, CE denotes the cross-entrope loss, and ARD is a trade-off hyperparameter. This setup allows the student to inherit decision boundaries shaped by gene-level information, thereby improving performance in gene-missing scenarios while maintaining deployment feasibility gene-free environments.
4.4 Optimization
During training on the source cohort, each iteration begins by fine-tuning the representations obtained from MOTOR to produce adapted phenotypical embeddings and encoding genetic features to generate genetic embeddings. The teacher model is then trained using both the supervised loss in Eq. 7 and the reconstruction loss in Eq. (3) following f'asscher CE((,, + 3Lree- The output logits from the
teacher are subsequently used as supervision to train the student model via Eq. (8). with all other components kept fixed. Once training converges, the model is transferred to the target cohort for
adaptation. We fine-tune the patient representation and the student model using a limited number of target patients following L'andent = CE(y,: a> y), applying the same graph-guided fine-tuning procedure. The fine-tuned student model is then evaluated for risk prediction on the target cohort.
5
World Experiments
5.1 Experimental Setup
Datasets. We evaluate the proposed model C using real-world healthcare data. In this work, we focus the Alzheimer's Disease and Related Dementias (ADRD) prediction prior to disease onset, considering its pervasive influence in the elderly population and practical data availability, We leverage the national All of Us Research Platform [3] the source cohort, and three target cohorts respectively from one local EHR data warehouse and two sub-networks (denoted as INSIGHT-A and INSIGHT-B) from the INSIGHT Clinical Research Network []] to simulate setting. We identified ADRD cases and controls in these repositories and conducted preprocessing following common practices in ADRD predictive modeling [2 More dataset details can be found in Appendix A.
Baselines. As our study focuses on the realistic scenario for risk prediction, existing work fails to directly apply. Related works may target random missingness in single domain, requiring adaptation to fit our setting, or rely on access to cross- scenario. To show the effectiveness of C we select two categories of methods from general domains: (1) modality imputation methods, including CMAE 34, MVAE (53), GAN 7 and SMIL 29:: (2) modality-robus learning models, consisting of MUSE (55). MoMKE (57) DrFuse [61] and CMKD [69]. For evaluation, we adapt these methods to align with our problem setup to ensure a fair comparison. Further baseline details are provided in Appendix A.2.
data during training. which is not feasible in
Experimental Details. We evaluate model performance using AUROC, score, sensitivity, and positive predictive values (PPV). While AUROC and F1 are standard classification metrics, we additionally report sensitivity and PPV both the 95% specificity, which are clinically relevant: the former reflects the model's ability to detect true cases under strict false positive constraint, and the latter indicates the trustworthiness of positive predictions in clinical decision making.. We provide more detailed experimental settings in Appendix A.3.
5.2 Experimental Results
Peformance Comparison. Table presents the performance of the C'M and baseline models one source cohort and three target cohorts. 'Tee best results are highlighted in bold, while the top baseline scores are underlined. Modality imputation-baat based baselines, including CMAE, MVAE, GAN, and SMIL, exhibit subpar performance, likely due to their reliance on generative models to infer genetic features from EHR representations. Given the high dimensionality, noise, and sparsity in genetic data, the imputed genetic features are often inaccurate and degrade overall prediction performance. Modality robust learning models such as CMKD, MoMKE, DrFuse, and MUSE employ various strategies to integrate multimodal information, but still yield suboptimal performance. MoMKE assumes a unified encoder design across modalities; however, our setting, EHR and genetic data exist in fundamentally different semantic spaces, making it ineffective to use a genetic encoder for processing EHR representation. CMKD attempts to align EHR and genetic embeddings, but this alignment can distort the EHR representation, reducing its informativeness and harming performance.
7

---

Table I: Performance comparison across source and target cohorts. Metrics are in percentage (%) Sens @95 and PPV @95 represent sensitivity and PPV values at the 95% specificity, respectively.
All of Us (Source)
Lo (Target)
Method
AUROC
F1
Sens @95
AUROC
F1
Sens@95
PPV@95
CMAE
34
13.09
13.36
30.26-1.74
38.66 28.66... 22.30
61.22.00.6
20.72
1:216
16.75
26. 10:00
MVAE
53
72.35
21.79
39.03 28.07
2422
62.06=a.10
21.04
16.57+047
15.93
GAN 7
+2.42+0.u 30.33
30.48+1.37
38 38.39
62.67
22.00
16.79+12
25.81+146
SMIT.
29
13.591004
31.54
31.79
19.24
63.17
22.400
17.46.11z
26.21:128
1:28
DrFuse
61
70.95
16.02+296
28.07
36.46_0066
61.47
14.00=m.0
13.85+105
24.97
MUSE
55
66.81+1.2
66.
20.97
19.50+1.96
28 1..98
59.23+2.10
13.1342.8
10.52+3.72
17.934339
CMKD
69
68.341094 1094
21.55
26.39
=2.30
59.47 1.03
13.67
4.0.0
14.39
1154
23.81
MoMKE
57
10.951022
18.444005
28.95
38.19
61.89
11.58
1136
25.65
C
+034
36.95
40.79 #0.47 45.93
71.82
31.78
+7.9 36.66
INSIGHT-A (Target)
INSIGHT-B (Target)
Method
AUROC
F1
Sens@95
PPV@95
AUROC
F1
Sens@95
PPV@9
CMAE 53
64.89
22.37+1.1
16.01 +1.11
24.46
68.11
24.91 +2,09
31.75 19.89
+0.40
MVAR
53
64.96.029
23.11
16.63
25.18
67.55...14
23.18
20.39
32.27
GAN 7
65.25
22.10+276
16.53
25. 25.06=0.99
68.41
25.71 20.24.055
32.06+a77
SMIL 229
65.94
22.57:1988
17.24-0044
25.62.=1.04
69.01
26.28+144
20.80-061
32.4
FuFuse
61
64.77+065
12.0842.00
15.64=0.44
24.15
67.69
16.38+30s
19.92+075
31.704am
MUSE
5SS
59.31+342
13.6443.6
10.50+3.29
47.44
62.25
21.38
11.43
20.50+437
CMKD
169
62.57
15.29
12.38-1.43
18.57
64.89+1146
23.70:2m
12.92+120
23.61 1093
MoMKE
57
64.77
12.92+306
16.28
24.75=1.21
67.38
12.44
19.56
31.36
C
72.44
1-228
23.61
26.63#0.5
34.75
75.39 +0.23 36.04
29.68 +.48
41.03
Table 2:
Ablation study across source and target cohorts.
All of Us (Souree)
LocalEHR (Target)
Method
AUROC
F1
Sens@95
PPV@95
AUROC
F1
Sens@95
PPV@95
M w/o Aggre
78.69 20.2 35.98 10ot 38. 1..59 43.61
70,2640.0
29.94
23.89
33.09
M w/o Graph
72.27+0.:
30.09
23.03
33.65+134
60.9340a8
18.49
10,44
17.88 +0.98
C'M M w/o MOTOR
70.88
30,65
25.00 37.62+1.1
62.44.042
21.04
12.26
20.43
M w/o Gene
77.70
31.56
35.5
42.52:1114
70.70
31.33
24.60
34.58
C
79.94
36.95.051
40.79.0.17
45.93
71822.01
31.78
27.96.1.04
36.66.1.05
DrFuse seeks shared patterns between EHR and genetic representations, yet the inherent noise in genetic data may lead to imprecise shared representations that negatively impact downstream predictions. MUSE constructs a patient-modality graph by representing each modality as a node, which may introduce overly dense and noisy connections, further limiting its effectiveness. The proposed C achieves the best performance among all methods, highlighting the effectiveness of our approach and its robustness in handling phenotypical and genetic information practically.
Ablation Study. To assess the effectiveness of each component in our model, we conduct an ablation study on source and target cohorts, with results shown in Table 4 We evaluate four variants: (1) without the -oxture-of-agggegaale
(w/o Aggre), (2) without the bipartite patient-concept graph for fine-tuning (w/o Graph), (3) without pre-trained EHR representations, using randomly initialized EHR representations (w/o MOTOR), and (4) without genetic data (w/o Gene). From Table 4 we observe that all ablated variants underperform the full model, highlighting the importance of each component. In particular, removing the pre-trained phenotypical EHR representation leads to notable performance drop, demonstrating that the foundation model provides informative EHR representation and alleviates the need to train complex EHR encoder from scratch, especially beneficial in resource- resource-limited target sites where fine-tuning such models is challenging. The variant without the patient-concept graph shows degraded performance, suggesting that distributional shifts exist between cohorts and the pre-trained embeddings alone are insufficient to fully represent the phenotypical status of patients in new cohorts. The performance decline of the variant without the mixture--ffaggere aggrreggats validates the effectiveness of our tixturr-of--mak-tooknn. design, indicating that indiscriminately using all genes introduces noise and negatively impacts prediction. Finally, the variant without genetic features confirms the necessity of incorporating genetic information, as it contributes complementary signals that enhance phenotypical representation and risk prediction.
8

---

0.85
0.45
Effect of the Number of Aggregation Tokens. To explore the influence of the number of aggregation tokens, we present experimental results in Figure 3 comparing performance when varying the number of aggregation tokens from 1 to 4. In our design, ag- gregation tokens are used to identify informative and relevant genetic features, subsequently compressing meaningful genetic information through reconstruc- tion. The caaturr-of-aggregaation the same set of genes. From the results, we observe that models with 2 to 4 aggregation tokens outperform the variant using only one, likely because single aggregation token tends to focus on a narrow subset of genes and overlook other important features. However, performance slightly degrades with 4 aggregation tokens compared 2 or 3, possibly due the introduction of redundant or noisy expert views that dilute the model's ' focus. Notably, models with or 3 aggregation tokens achieve comparable performance across various metrics, indicating that moderate number of tokens achieves a balance between diversity and relevance in capturing genetic information.
0.30
0.40
AUROC
FI Score
0.35
0.75
0.30
0.70
0.25
0.65
Figure 3: Effect of #aggnegation tokens.
strategy leverages diverse perspectives from multiple experts on
Effect of the Number of Transformer Layers. To investigate the impact of the number of Transformer layers, conduct experiments with layer counts ranging from to 4. As shown in Figure 4 the vari- ant with only one Transformer layer yields inferior performance, likely due to its limited capacity to cap- ture complex gene feature interactions and long-range dependencies. Variants with 3 and 4 layers perform better but still fall short compared to the 2-layer variant, possibly because the training data size is insufficient to support effective learning in deeper models and may lead to overfitting or optimization difficulties. Notably, the 2-layer Transformer achieves the best overall performance, suggesting that provides an optimal balance between model capacity and generalization under the current setting.
0.40
0.50
AUROC
FI Score
0.75
Alleth
0.70
0.30
Figure 4: Effect of #transformer layers.
On Alternative Graph Constructions. In model. we construct a patient-concept bipartite graph. An alternative approach, denoted as GCombined, inte- grates gene information directly into the graph by treating each gene concept node. To assess the impact of these two graph construction strategies, we compare their performance. As shown in Figure 5 C consistently outperforms GCombined across all metrics. This result suggests that incorporating genetic information directly during graph fine-tuning degrades performance, likely because the inherent noise and imprecision of genetic data negatively influence the learning of phenotypic representations derived from EHRs.
0.8
C'M GCombined
C
GCombined
0.4
0.2
AUROC
SenugR5PPVigg5
AUROC
Seasig95 PP1//RE
Figure 5: Performance comparison on graph construction strategies.
Further Analysis. We provide more experimental results and further analyses in Appendix B.
Conclusion
We presented a cross-cohort, cross-modal knowledge transfer framework to address the practical challenges of deploying the clinical risk prediction model in resource-limiter settings. Our method enables the knowledge transfer from multimodal model to local cohorts with only EHR data, without requiring access to genetic information during deployment. By introducing gene encoder with mixture-of-are -fgaaggeeaatiinnss to handle noisy genetic inputs and a lightweight graph-guides strategy for efficient adaptation, our framework achieved consistent and strong performance across diverse real-world cohorts. A key limitation of our current work is its focus only on Alzheimer's disease and related dementias. In future work, we plan to extend the framework to more types of chronic diseases, e.g., Parkinson's 'ss disease, to further assess its generalizability across disease domains.
fine-tuning

---

Acknowledgments and Disclosure of Funding
The authors would like to acknowledge the support from NSF 2212175, NIH RFI RFIAG072449, RFIAG084178, ROIAG080991, R01AG080624, R0176474448, ROI AG0762233 and ROINS 140142.
References
[1] Insight clinical research network. Inttps://insightrrmn org/
Accessed: 2024.
[2] Mohammad Al Olaimat, Jared Martinez, Fahad Saeed, Serdar Bozdag, and Alzheimer's Dis- ease Neuroimaging Initiative. Ppad: deep learning architecture to predict progression of alzheimer's disease. Bioinformatics, 2023.
[3] Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learn- ing: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence, 2018.
[4] Esther E Bron, Stefan Klein, Janne M Papma, Lize C Jiskoot, Vikram Venkatraghavan, Jara Linders, Pauline Aalten, Peter Paul De Deyn, Geert Jan Biessels, Jurgen AHR Claassen, et al. Cross-cohort generalizability of deep and conventional machine learning for mri-based diagnosis and prediction of alzheimer's 's disease. Neurolmage: Clinical, 2021.
[5] Annalisa Buniello, Jacqueline A L. MacArthur, Maria Cerezo, Laura W Harris, James Hayhurst, Cinzia Malangone, Aoife McMahon, Joannella Morales, Edward Mountjoy, Elliot Sollis, et al. The nhgri-ebi gwas catalog of published gedome-wide association studies, targeted arrays and summary statistics 2019. Nucleic acids research, 2019.
[6] Clare Bycroft, Colin Freeman, Desislava Petkova, Gavin Band. Lloyd T Elliott, Kevin Sharp, Allan Motyer, Damjan Vukcevic, Olivier Delaneau, Jared C'Cnnelll et al. The uk biobank resource with deep phenotyping and genomic data. Nature, 2018.
[7] Lei Cai, Zhengyang Wang. Hongyang Gao, Dinggang Shen, and Shuiwang Ji. Deep adversarial learning for multi-modality missing data completion. In KDD, 2018.
[8] Eunsuk Chang and Javed Mostafa. The use of snomed ct, 2013-2020: a literature review. Journal of the American Medical Informatics Association, 2021.
[9] Cheng Chen, Qi Dou, Hao Chen, Jing Qin, and Pheng-Ann Heng. Synergistic image and feature adaptation: Towards cross-modality domain adaptation for medical image segmentation. In AAAI, 2019.
[10] Qiuhui Chen and Yi Hong. Alifuse: Aligning and fusing multimodal medical data for computer- aided diagnosis. In 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2024.
[11] Xue Chen and Yanjun Peng. Cmcd-net: Unsupervised domain adaptation with contrastive learn- ing for cross-modality and cross-disease brain lesion segmentation. In 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2024.
[12] Carolyn Clancy and Francis Collins. Patient-centered outcomes research institute: the intersection of science and health care. Science Translational Medicine, 2010.
[13] Yulan Dai, Beiji Zou, Xiaoyan Kui, Qinsong Li. Wei Zhao, Jun Liu, and Miguel Bordallo López. Strong multimodal representation learner through cross domsin distillation for alzheimer's dis- ease classification. 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2024.
[14] Christopher B Forrest, Kathleen McTigue, Adrian Hernandez, Lauren W Cohen, Henry Cruz, Kevin Haynes, Rainu Kaushal, Abel N Kho, Keith A Marsolo, Vinit P Nair, et al. Pcorett 2020: current state, accomplishments, and future directions. Journal of Clinical Epidemiology. 2021.
10

---

[15] C Scott Gallagher, Geoffrey S Ginsburg, and Anjené Musick. Biobanking with genetics shapes precision medicine and global health. Nature Reviews Generies, 2025.
[16] Judy Hoffman, Saurabh Gupta, and Trevor Darrell. Learning with side information through modality hallucination. In CVPR, 2016.
[17] Yi Hong and Marcia Lei Zeng. International classification of diseases (icd). KO Knowledge Organization, 2023.
[18] Dani Kiyasseh, Tingting Zhu, and David Clifton. A clinical deep learning framework for continually learning from cardiac signals across diseases, time, modalities, and institutions. Nature Communications, 2021.
[19] Jian Lang, Zhangtao Cheng, Ting Zhong, and Fan Zhou. Retrieval-uugmented dynamic prompt tuning for incomplete multimodal learning. arXiv preprint arXiv:2501.01120, 2025.
[20] Mingcheng Li, Dingkang Yang. Xiao Zhao, Shuaibing Wang, Yan Wang, Kun Yang. Mingyang Sun, Dongliang Kou, Ziyun Qian, and Lihua Zhang. Correlation-decoupled knowledge distilla- tion for multimodal sentiment analysis with incomplete modalities. In CVPR, 2024.
[21] Qian Li, Xi Yang, Jie Xu, Yi Guo, Xing He, Hui Hu, Tianchen Lyu, David Marra, Amber Miller, Glenn Smith, et al. Early prediction of alzheimer's 's disease and related dementias using real-world electronic health records. Alzheimer's is & Dementia, 2023.
[22] Wei Li, Yifei Zhao, Xi Chen, Yang Xiao, and Yuanyuan Qin. Detecting alzheimer's disease on small dataset: knowledge transfer perspective. IEEE journal of biomedical and health informatics, 2018
[23] Xinyao Li, Yuke Li, Zhekai Du, Fengling Li, Ke Lu, and Jingjing Li. Split to merge: Unitying separated modalities for unsupervised domain adaptation. In CVPR, 2024.
[24] Yiying Li, Yongxin Yang. Wei Zhou, and Timothy Hospedales. Feature-critic networks for heterogeneous domain generalization. In ICML, pages 3915-3924, PMLR, 2019.
[25] Hong Liu, Dong Wei, Donghuan Lu, Jinghan Sun, Liansheng Wang, and Yefeng Zheng. M3ae: Multimodal representation learning for brain tumor segmentation with missing modalities. In AAAI, 2023.
[26] Ye Liu, Lingfeng Qiao, Changchong Lu. Di Yin, Chen Lin, Haoyuan Peng, and Bo Ren. Osan: A one-stage alignment network unify multimodal alignment and unsupervised domain adaptation. In CVPR, 2023.
[27] CG Lyketsos, SB Roberts, Elaine K Swift, A Quina, G Moon, Kremer, P Tariot, H Fillit, DE Bovenkamp, PP Zandi, et al. Standardizing electronic health record data on ad/adrd to accelerate health equity in prevention, detection, and treatment. The journal of prevention of Alzheimer's 's disease, 2022.
[28] Mengmeng Ma. Jian Ren, Long Zhao, Davide Testuggine, and Xi Peng. Are multimodal transformers robust to missing modality? In CVPR, 2022.
[29] Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng. Smil: Multimodal learning with severely missing modality. In AAAI, 2021.
[30] Xinhong Ma, Tianzhu Zhang, and Changsheng Xu. Deep multi-modality adversarial networks for unsupervised domain adaptation. IEEE Transactions on Multimedia, 2019.
[31] Arjun Masurkar, Karyn Marsh, Brianna Morgan, Dominique Leitner, and Thomas Wisniewski. Factors affecting resilience and prevention of alzheimer's 's discase and related dementias. Annals of Neurology, 2024.
[32] Stuart Nelson, Kelly Zeng, John Kilbourne, Tammy Powell, and Robin Moore. Normalized names for clinical drugs: Rxnorm years. Journal of the American Medical Informatics Association, 2011.
11

---

[33] Sarah M Neuner, TCW Julia, and Alison M Goate. Genetic architecture of alzheimer's disease. Neurobiology of disease, 2020.
[34] Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y Ng. Multimodal deep learning. In ICML,, 2011.
[35] All of Us Research Program Investigators. The "all of us" research program. New England Journal of Medicine, 2019.
[36] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In ICCV, 2019.
[37] Fan Qi, Xiaoshan Yang, and Changsheng Xu. A unified framework for multimodal domain adaptation. In Proceedings of the 26th ACM international conference on Multimedia, 2018.
[38] Suraj Rajendran, Weishen Pan, Mert R Sabuncu, Yong Chen, Jiayu Zhou, and Fei Wang. Learning across diverse biomedical data modalities and cohorts: Challenges and opportunities for innovation. Patterns, 2024.
[39] Andrea H Ramirez, Lina Sulieman, David J Schlueter, Alese Halvorson, Jun Qian, Francis Ratsimbazafy, Roxana Loperena, Kelsey Mayo, Melissa Basford, Nicole Deflaux, al. The all of us research program: data quality, utility, and diversity. Patterns, 2022.
[40] Mayur Rele, Anitha Julian, Dipti Patil, and Udaya Krishnan. Multimodal data fusion integrating text and medical imaging data in electronic health records. In International Conference Innovations and Advances in Cognitive Systems. Springer, 2024.
[41] Patrick Schober and Thomas Vetter. Propensity score matching in observational research. Anesthesia & Analgesia, 2020.
[42] Ethan Steinberg, Jason Alan Fries, Yizhe Xu, and Nigam Shah. Motor: time-to-event foundation model for structured medical records. In ICLR, 2024.
[43] Alice S Tang, Katherine P Rankin, Gabriel Cerono, Silvia Miramontes, Hunter Mills, Jacquelyn Roger, Billy Zeng, Charlotte Nelson, Karthik Soman, Sarah Woldemariam, al. Leveraging electronic health records and knowledge networks for alzheimer's 's disease prediction and sex- specific biological insights. Nature Aging, 2024.
[44] Luan Tran, Xiaoming Liu, Jiayu Zhou, and Rong Jin. Missing modalities imputation via cascaded residual autoencoder. In CVPR, 2017.
[45] A Vaswani. Attention all you need. In NeurlPS, 2017.
[46] Erica A Voss, Rupa Makadia, Amy Matcho, Qianli Ma, Chris Knoll, Martijn Schuemie, Frank J DeFalco, Ajit Londhe, Vivienne Zhu, and Patrick Ryan. Feasibility and utility of applications of the common data model multiple, disparate observational health databases. Journal of the American Medical Informatics Association, 2015.
[47] Hu Wang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, and Gustavo Carneiro. Multi-modal learning with missing modality via shared-specific feature modelling. In CVPR, 2023.
[48] Hu Wang, Salma Hassan, Yuyuan Liu, Congbo Ma, Yuanhong Chen, Yutong Xie, Mostafa Salem, Yu Tian, Jodie Avery, Louise Hull, et al. Meta-learned moodality-weigheee knowledge distillation for robust multi-modal learning with missing data. arXiv preprint 240.0155 2024.
[49] Hu Wang, Congbo Ma, Jianpeng Zhang, Yuan Zhang, Jodie Avery, Louise Hull, and Gustavo Carneiro. Learnable cross-modal knowledge distillation for multi-modal learning with missing modality. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2023.
12

---

[50] Jiankun Wang, Sumyeong Ahn, Taykhoom Dalal, Xiaodan Zhang, Weishen Pan, Qiannan Zhang, Bin Chen, Hiroko H Dodge, Fei Wang, and Jiayu Zhou. Augmented risk prediction for the onset of alzheimer's s disease from electronic health records with large language models. arXiv preprint arXiv:2405. 16413, 2024.
[51] Yuanzhi Wang, Zhen Cui, and Yong Li. Distribution-comsistenn modal recovering for incomplete multimodal learning. In ICCV, 2023.
[52] Shicai Wei, Yang Luo, Yuji Wang, and Chunbo Luo. Robust multimodal learning via represen- tation decoupling. In ECCV, 2024.
[53] Mike Wu and Noah Goodman. Multimodal generative models for scalable weakly-supervised learning. In NeurlPS, 2018.
[54] Renjie Wu, Hu Wang, Hsiang-Ting Chen, and Gustavo Carneiro. Deep multimodal learning with missing modality: A survey. arXiv preprint arXiv:2409.07825, 2024.
[55] Zhenbang Wu, Anant Dadu, Nicholas Tustison, Brian Avants, Mike Nalls, Jimeng Sun, and Faraz Faghri. Multimodal patient representation learning with missing modalities and labels. In ICLR, 2024.
[56] Muhao Xu, Zhenfeng Zhu, Youru Li, Shuai Zheng, Yawei Zhao, Kunlun He, and Yao Zhao. Flexcare: Leveraging cross-task synergy for flexible multimodal healtheare prediction. In KDD, pages 3610-3620, 2024.
[57] Wenxin Xu, Hexin Jiang, and Xuefeng Liang. Leveraging knowledge of modality experts for incomplete multimodal learning. In Proceedings of the 32nd ACM International Conference on Multimedia, 2024.
[58] Xian Xu, Xiao Xu, Xiang Li, and Guotong Xie. Grmi: Graph representation learning of multimodal data with incompleteness. In International Conference on Database Systems for Advanced Applications. 2023.
[59] Zhen Xu, David R So, and Andrew M Dai. Mufasa: Multimodal fusion architecture search for electronic health records. In AAAI, 2021.
[60] Shuwen Yang, Guojie Song, Yilun Jin, and Lun Du. Domain adaptive classification on hetero- geneous information networks. In IJCAI, 2021.
[61] Wenfang Yao, Kejing Yin, William K Cheung, Jia Liu, and Jing Qin. Drfuse: Learning disentangled representation for clinical multi-modal fusion with missing modality and modal inconsistency. In AAAI, 2024.
[62] Jiaxuan You, Xiaobai Ma. Yi Ding, Mykel J Kochenderfer, and Jure Leskovec. Handling missing data with graph representation learning. In NeurIPS, 2020.
[63] Chaohe Zhang, Xu Chu, Liantao Ma, Yinghao Zhu, Yasha Wang, Jiangtao Wang, and Junfeng Zhao. M3care: Learning with missing modalities in multimodal healthcare data. In KDD, 2022.
[64] Qiannan Zhang, Weishen Pan, Zilong Bai, Chang Su, and Fei Wang. Unified insights: Harness- ing multi-modal data for phenotype imputation via view decoupling. In NewrIPS, 2024.
[65] Qiannan Zhang, Shichao Pei, Qiang Yang, Chuxu Zhang, Nitesh V Chawla, and Xiangliang Zhang. Cross-domain few-shot graph classification with a reinforced task coordinator, In AAAI, 2023.
[66] Qiannan Zhang, Xiaodong Wu, Qiang Yang. Chuxu Zhang, and Xiangliang Zhang. Few-shot heterogeneous graph learning via cross-domain knowledge transfer. In KDD, 2022.
[67] Weichen Zhang, Dong Xu, Jing Zhang, and Wanli Ouyang. Progressive modality cooperation for multi-modality domain adaptation. IEEE Transactions on Image Processing, 2021.
[68] Jinming Zhao, Ruichen Li, and Qin Jin. Missing modality imagination network for emotion recognition with uncertain missing modalities. In ACL, 2021.
13

---

[69] Long Zhao, Xi Peng, Yuxiao Chen, Mubbasir Kapadia, and Dimitris N Metaxas. Knowledge as priors: Cross-modat knowledge generalization for datasets without superior knowledge. In CVPR, 2020.
A More Details on Experimental Setup
A.1 Datasets
We evaluate the proposed method and baseline approaches with one nationwide source cohort and three local target cohorts. The nationwide All of Us Research Program is primarily utilized to build the source cohort of ADRD with EHR and genetic modalities. Three cohorts from LocalEHR, INSIGHT-A, and INSIGHT-B, are leveraged simulate realistic, resource-limited clinical scenarios in our study.
A.1.1 All of Us Research Program
The source cohort for ADRD prediction is assembled from the All of Us Research Program 39) launched in 2018 as a nationwide U.S. initiative designed to advance biomedical research. It collects de-identified historical health information from distributed health assessment centers standardized with the OMOP common data model 46). while retaining biological samples for detailed biological analyses. We use the All of Us dataset for its extensive, diverse cohort, which enables robust ADRD prediction across populations and heterogeneous clinical records. The cohort construction process on All of Us is illustrated as follows.
We first identify ADRD cases using clinically established criteria 21 The onset time for each case is defined as the earlier of their first ADRD diagnosis or first anti-dementia prescription. Control samples are similarly aged individuals with no record of dementia-related diagnoses and no exposure to anti-dementia medications. To mitigate confounders and enhance clinical relevance, we construct the case-control matched cohort with a matching ratio of 1:10 following standard practices in observational health research [41] by age and clinical status. And both groups are required to be 50 years or older at onset. As neurodegenerative diseases often progress gradually, making early detection essential for timely intervention, disease onset, with the observation window spanning from the start of each patient's EHR records to the prediction time. Accordingly, all EHR records documented after the prediction time are excluded from model input. As a result, cases and controls are required to have at least one year of EHR history in the observation window.
define the prediction time one year prior to the
Meanwhile, genetic factors have been widely implicated in the biological mechanisms underlying ADRD. These include processes such as amyloid-beta production and clearance, tau phosphorylation. lipid metabolism, and neuro-inflammatory responses [33 31 Additionally, genomic data Consider- ing the accessiability of genetic data in the program, accordingly. we leverage genomic information in the source cohort, given its biological relevance to ADRD and its availability at scale, and focus on the genomic variant data, preserving alleles exceeding a oopulation-specine frequency threshold (>1%) or allele count (>100) for a set of ADRD-related genes based on GWAS Catalog [5]. Due to partial biospecimen collection and processing constraints, genetic information is unavailable for 3,027 individuals in the All of Us cohort, resulting in a random missingness rate of 26.7%. For each individual, gene is assigned value of in of any mutation; otherwise, it is assigned O. This yields binary gene features reflecting the presence of associated genetic variants, serving as the individual's genetic profile.
A.1.2 Local Clinical Repository
The local hospital's 's EHR repository (LocalEHR) contains patient-level clinical information, where EHR data are standardized under the PCORnet Common Data Model [14). Cases and controls for ADRD are defined using the same criteria as those applied in the All of Us cohort. We construct a case-control matched cohort using propensity score matching with the same ratio as the All of Us cohort based on clinical factors. Consistent criteria as All of Us for cohort construction and prediction timing are subsequently enforced for this repository.
14

---

Table 3: Dataset Statistics
Dataset
#Patients
#Genes
Missing Rates
All of Us
11,356
217
26.7%
LocalEHR
6,548
100%
INSIGHT-A
19,062
100%
INSIGHT-B
17,424
100%
A.1.3 INSIGHT Clinical Research Network
Supported by the Research Network (CRN) [I] aims to improve health outcomes and care delivery by analyzing healthcare utilization across major clinical and academic centers in the New York City. We use two sub-networks from the network, referred to as INSIGHT-A and INSIGHT-B, each containing longitudinal electronic health records (EHRs) spanning over ten years from diverse patient populations in the metropolitan area. All EHR data are standardized using the PCORnet Common Data Model 14. For each sub-network, we apply identical criteria as All of Us and construct the cohort, respectively. As different clinical sites often adopt distinct data models and record protocols, the constructed cohorts at both the source and target sites are processed accordingly. In this study, we leverage demographics, diagnoses, and medications due to the major role they play in clinical decision-making. Each type information is encoded using standard medical concept vocabularies 17 12, and the corresponding timestamps represented as discrete events, forming temporally ordered sequence for each patient as input to the foundation model. Data statistics are summarized in Table 3.
Outcomes Research Institute (PCORT) 12 the INSIGHT Clinical
A.2 Baselines
To show the effectiveness of C we compare C eight baselines and adapt these methods to align with our problem setup to ensure a fair comparison. The baseline methods compared in our experiments include:
CMAE (34 employs a cross-modal autoencoder, which is first learned on a subset of patients with complete modalities by reconstructing purposely masked-out modalities. Once trained, the model is applied impute missing modalities for all patients. Imputation of genetic modality is performed at test time on the source cohort, and throughout on the target cohorts.
MVAE 53 builds multimodal variational autoencoder that learns joint latent distribution robust to missing data. Leveraging a product-of-expert. architecture to enable parameter sharing across modalities, MVAE takes input data with different modality combinations. We conduct MVAE for EHR and genetic modalities, and naturally skip the genetic network during inference.
GAN proposes a generative adversarial network for Alzheimer's 's disease prediction with incomplete imaging modalities. The generator reconstructs the missing modality from the available one, while the discriminator evaluates their coherence and performs classification. Inference performed by imputing the genetic modality, followed by disease prediction using the discriminator.
SMIL [29] integrates Bayesian meta- meta-learning jointly learn a reconstruction network, which estimates the missing modality as a weighted combination of -speciffi priors inferred from modality-complett samples, and regularization network that conducts uncertainty guided feature regularization. During inference, SMIL takes samples with EHR modality as input and reconstructs the genetic modality for further joint prediction.
MUSE [55] represents modient-modaltty relationships as a bipartite graph, where patients and fine-ge tne-grained EHR modalities are nodes, and modality features define the edges. A Siamese GNN is trained on original and augmented graphs to enhance robustness to missing modalities. Patient nodes are initialized with (LoToR-dervee embeddings, and binary genetic features are encoded on edges connecting patients to genetic modality nodes. To ensure consistent evaluation under modality missingness, two fine-grained EHR modality nodes (diagnosis and medication) are used, with concept-level codes embedded as edge attributes. During inference, EHR-alone patients only maintain edges with fine- fine-grained modality nodes for EHRs.
15

---

Algorithm 1 The Training Procedure of C on the Source Cohort
1: Input: Multi-modal inputs {Tr,, M.), EHR foundation model Motor, bipartite graph G;
2: for each iteration do
2: Obtain the phenotypical representation hy for each patient from Motor using ri
2: Conduct graph-guided phenotypical representation fine-tuning and obtain hg using Eq,
2: Encode genetic profile 9t. and obtain updated aggregation token as patient-level genetic representation he. as well as contextualized genetic outputs using (4) and (5)
2: Compute reconstruction loss using (2)-
2: Using phenotypical embedding h (L) and genetic embedding he optimize teacher model ftencher based on the loss Lteacher including the reconstruction loss and classification loss;
2: Optimize the student model fsuueent using (8)
3: end for
3: Evaluation using the student model fstueent and trained GNN.
adopts two-stage learning process, consisting of unimodal expert training and experts mixing training, where a gating mechanism assigns weights based on modality availability and quality. Missing modalities are handled via zero-imputation during inference.
MoMKE 57
DrFuse [61] tackles modality missingness by learning disentangled latent representations for EHR and imaging modalities. A disease-aware attention mechanism applied to fuse shared and modality-speciie representations, while selectively disregarding the network branch of modality when it is absent. During inference, the network for genetic modality remains inactivated.
CMKD 669 transfers knowledge from stronger modality (teacher) to weaker modality (student) with paired inputs. The student model is trained to match the intermediate representations from the teacher model and is ultimately applied with weaker modality alone during inference.
A.3 Experimental Settings
To conduct graph-guided finetuning obtain phenotypical representations, a two-layer GCN is adopted with 16 hidden units, along with a 16-dimensional embedding layer to represent 2634 medical concept nodes, and one transformation layer that transforms the foundation model output to initialize patient nodes. In addition, the transformer encoder consists of two layers with two heads, and we determine the expert number via search in (1,2,3,4), while the gene decoder is an MLP with one hidden layer. Attention modulation is achieved using multi-head attention mechanism with two heads. Both the teacher and student models are implemented as multi-layer perceptrons. The trade-off parameter 3 for gene feature reconstruction is selected via grid search over [0.01, 0.05, 0.1, 0.5, 1) and set as 1. The trade-off parameter ARD of knowledge distillation for the student model is set as 0.01 by grid search over { (0.001. 0.005, 0.01, 0.05, 0.1, 0.5, 1.0]. The learning rate of C and bascline models is selected from (0.01. 0.001, 0.0005).
The proposed model is trained with the Adam optimizer employing early stopping based on validation performance. Model performance on the testing set of All of Us is reported to assess within- domain model generalizability. To simulate low-resource clinical settings, we randomly sample a small subset of each target cohort to fine-tune the model transferred from All of Us. In our main experimental results, for three target cohorts, we randomly sample 500 patients for fine-tuning, respectively. Besides, fine-tuning and evaluation on the target cohorts are both performed in CPU- only environments to assess practical deployment feasibility. Similarly, baseline approaches are trained on All of Us, and afterwards finetuned with the same small sample set on each target cohort before evaluation. Because these methods are not tailored to scenario, we adapt their public implementations to our setting and determine hyperparameters via parameter search.
For datasets, the All of Us cohort is randomly split into training/validationnessig at a 6:2:2 ratio. The training set includes both EHR data and genetic information, with a random missingness rate of 26.7% for the genetic modality. The trained model is first evaluated on the testing set of the source cohort using EHR data alone, and further fine-tuned with a limited set of EHR-alone samples on each target cohort to assess cross-cohort generalizability. Furthermore, for fair comparison, all baseline approaches are trained and evaluated using the same data splits and are provided with representations from the foundation model as patient features. We report the average performance as well as the
16

---

Table 4: Impact of the transferred model under varying numbers of fine-tuning patients in INSIGHT-B "Wo T" indicates training from scratch (without using the transferred model), while "W/ refers to fine-tuning with the transferred model.
#Patients
100
200
300
400
500
W/o
W/T T
W/o T
W/T
W/o
W/T
W/o T
W/T
W/o
W/ T
AUROC
0.6620
0.7308
0.7186
0.7502
0.7241
0.7517
0.7298
0.7524
0.7421
0.7539
F1
0.2299
0.3522
0.2759
0.3276
0.2866
0.3541
0.2921
0.3579
0.3077
0.3604
Sens@95
0.1194
0.2803
0,2161
0.2950
0.2472
0.2961
0,2597
0.2965
0.2760
0.2968
PPV@95
0.2180
0.3955
0.3377
0.4088
0.3543
0.4080
0.3744
0.4074
0.3928
0.4103
standard deviation of three random runs All of Us and the target cohorts. The training procedure of C is shown in Algorithm 1.
B Further Analysis and Discussion
B.1 More Experimental Results
Effect of the Tradeoff' Parameter 3. The trade- off parameter 3 controls the balance between the supervised training objective and the gene fea- ture reconstruction loss. We evaluate the model under different values of B and present the re- sults in Figure 6. small leads to a weak re-
0.8
Leatiik
AUROC
F1 Score
0,30
construction signal, causing suboptimal gene en-
coding and imperfect genetic embeddings. Con- versely, a large B shifts the optimization focus excessively toward reconstructing gene features,
Figure 6: Effect of trade-off weight
which impair the Icarning of task-relevant supervised representations and ultimately degrade overall performance. The best performance is achieved with a moderate value of 3, indicating the importance of balancing these objectives. Overall, the impact of the tradeoff parameter is not large, and | usually slightly affects the performance.
Effect of the Number of Patients in the Target Cohorts for Fine-tuning.
investigate the impact of the fine- tuning dataset size, we conduct additional experiments by varying the number of labeled patients from the target cohorts used for fine-tuning. As shown in Figure 7, the model achieves quite good performance even when fine- tuned with very small set of labeled patients. As the size of the fine-tuning set increases, the performance contin- to improve, but with diminishing returns. This result highlights the robustness and practicality of our proposed framework, particularly in resource-limee patients are available for fine-tuning.
Lenttiltt
AUROC
0.8
INSIGITS
0.7
100
200
300
400
500 1000 1500 2000
Figure 7:
Effect of number of patients.
clinical settings where only small number of labeled
Effect of the Transferred Model. To evaluate the effectiveness of the transferred model, we compare model performance on the target cohort (INSIGHT-B) between models trained from scratch and those fine-tuned from the transferred model's parameters. As shown in Table 4. when the number of available patients in the target cohort is limited, using the transferred model offers a significant advantage by providing near-optimal starting point for optimization. These results underscore the necessity of studying cross-cohort knowledge transfer to enable effective model adaptation in low-resource settings.
Time Complexity Analysis. our setting, target sites typically have limited computational resources, which constrain their ability to develop models independently. This motivates the need for cross- cohort knowledge transfer. To ensure our approach aligns with such constraints, we analyze the time complexity target sites. First, the time complexity of tnn-ttnnig the transferred model, which
17

---

consists of a GNN and student MLPs, is linear to the number of patient samples, because the time complexity of GNN is (V [E))) and the time complexity of MLP is O N₂) where is the hidden dimension, [V] and E. denote the number of nodes and edges in the graph, and N, is the number of patients target cohort. Second, since the transferred model comes with pre-trained parameters that are already near the optimal solution for the target cohort, only minimal fine-tuning and a small amount target cohort data are required. These show that our method effectively overcomes the limitations posed by resource-limited target sites.
C Broader impacts
This work aims to democratize clinical risk prediction by enabling the transfer predictive models trained multimodal nationwide data to local clinical settings with limited resources and unimodal EHR data. If widely adopted, the proposed C framework could help early risk detection, such as for Alzheimer's Disease and Related Dementias, in underserved populations and healthcare systems with constrained access to genetic testing and computational infrastructure. By reducing reliance on costly or inaccessible modalities, this approach promotes more equitable access precision medicine tools. However, we also note the importance of local validation, continuous monitoring, and inclusive data practices in model transfer across cohorts to ensure robust and responsible model usage.
D More Preliminaries
Self-attention is a key mechanism for modeling dependencies within sequence by computing contextualized representations for each token through interactions with all others. Given an input sequence of tokens X {x1, #2,....... ...,.... Jn J. self-attention computes the representation of each token as a weighted sum of tokens in the sequence. This is achieved through the computation:
Self-Attention.
QK
Attention Q, K, V) = softmax
V,
(9)
vdz
where Q. K, and V are the query, key, and value matrices projected from the input X, and de the dimensionality of the key vectors used for scaling. The attention weights capture pairwise relevance between tokens, allowing the model to dynamically focus on different parts of the sequence. Multi-head self- self-attention extends this mechanism by running multiple attention operations in parallel, enabling the model to capture diverse semantic relationships.
18

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly summarize the paper's contributions, framework, and main findings. These sections consistent with the detailed content presented the subsequent sections.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have considered the limitations of the proposed model, which are discussed in the Conclusion section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
19

---

Answer:
[NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide detailed descriptions of the experimental settings in Appendix, and release the source code to facilitate reproduction.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
20

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We release the code in https://gihuubcoom////aahhhhhhhh
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https
/ "pipsscc/puullc//uui /ttpe//gguddeeeeeeouuuuumbbbba ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The training and the test details can found in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Experimental results in Table and Table show the standard deviation.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
21

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Appendix, we provide comprehensive overview of the software environ- ment and hardware setup in the experimental setting section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
We affirm that this research fully adheres the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: In Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Our model does not have high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have obtained authorization from the All of Us platform and INSIGHT Clinical Research Network to use their data.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the batasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new asset is proposed.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification:
The paper obtained IRB based on the requirements of the institution.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification:
This research does not incorporate LLMs in the development of core methods or experiments. Any usage of LLMs was limited to writing assistance, which does not impact the scientific rigor originality of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy lattps://neurip:: http///rrrp.ccc/oooeeeerre////// for what should or should not be described.
25


Reviews:
[{"review_id": "YbdOQmhiqh", "reviewer": "Reviewer_aRk9", "summary": "This paper focuses on a specific setting for clinical risk prediction: transferring predictive models across cohorts that differ in data availability, specifically from a national cohort with access to both genetic and electronic health record (EHR) data to resource-limited cohorts with only EHR data. To tackle this, the authors propose a cross-cohort, cross-modal knowledge transfer framework, $\\mathbf{C^3M}$, which integrates a mixture-of-aggregations design with a lightweight, graph-guided fine-tuning strategy. The proposed $\\mathbf{C^3M}$ is evaluated on real-world clinical datasets to assess its effectiveness.", "questions": "Please refer to points W1 through W7 above for detailed concerns and specific recommendations related to the problem setting, technical novelty, and experimental evaluation, among other aspects of the paper.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**S1.** The problem of clinical risk prediction, especially in cross-cohort and cross-modality scenarios, is of clear clinical and scientific importance. Addressing this challenge requires the development of technically sound and transferable methodologies to improve predictive performance across heterogeneous data sources.\n\n**S2.** The proposed $\\mathbf{C^3M}$ framework consists of three main components: Gene Encoding with Mixture-of-Aggregations, Graph-Guided Phenotypical Representation Fine-Tuning, and Cross-Modal Knowledge Transfer via Distillation. The overall design is reasonable from a technical standpoint.\n\n**S3.** The experimental evaluation includes comparisons with relevant baselines, ablation studies to assess the contribution of each component, sensitivity analyses on key hyperparameters, and computational efficiency assessments.\n\n**S4.** The paper is clearly written and well-structured, which supports readability and facilitates understanding of the technical contributions.\n\n**W1.** The problem setting investigated in this work is relatively narrow, as it assumes the availability of both genetic and EHR modalities in a global (source) cohort, and the need to transfer the learned clinical risk prediction model to target cohorts with only EHR data. While this setting is technically well-defined, its practical relevance and general applicability are not sufficiently justified. Moreover, in the related work discussion, the authors note that existing methods are not directly comparable due to differences in problem settings. This further underscores the need for a more detailed explanation of the real-world motivation behind the proposed setting. Specifically, the authors had better elaborate on why this transfer setting is more realistic, e.g., how often such scenarios arise in practice.\n\n**W2.** A notable concern pertains to the limited technical novelty of the proposed framework. While $\\mathbf{C^3M}$ integrates several components—such as Mixture-of-Aggregations, graph-guided fine-tuning, and knowledge distillation—these are primarily based on off-the-shelf techniques, and the paper does not clearly articulate the specific technical challenges encountered or addressed in combining them. Furthermore, the two key challenges identified in the introduction as motivations for $\\mathbf{C^3M}$ have been studied in prior work. As a result, the contributions of the paper may appear incremental, and the originality of the framework is not convincingly established.\n\n**W3.** The experimental evaluation exhibits inconsistency in dataset selection across different analyses. For example, the ablation study is conducted on All of Us and OHSU. Figure 3 reports results on All of Us and INSIGHT MS, while Figure 4 involves All of Us and INSIGHT Columbia. This inconsistent usage of datasets without a clear justification weakens the coherence and comparability of the experimental findings. To enhance the credibility of the evaluation, it would be important either to standardize the dataset choices across experiments or to provide a clear rationale for the selection of specific dataset pairs in each analysis.\n\n**W4.** The paper would benefit from the inclusion of case studies or qualitative analyses that provide interpretable insights into the learned representations. For example, in the Mixture-of-Aggregations component, it would be informative to demonstrate how the model encourages diverse specialization among experts through the proposed extension, potentially capturing distinct and disentangled aspects of genetic functionality. Such analyses would not only enhance interpretability but also help concretely illustrate the clinical relevance and advantages of $\\mathbf{C^3M}$ in supporting clinical risk prediction.\n\n**W5.** Given the focus on clinical risk prediction—a domain with significant clinical implications—a medical validation would be valuable to assess the practical utility of the proposed $\\mathbf{C^3M}$ framework in real-world medical decision-making.\n\n**W6.** According to the dataset statistics in Table 1 (appendix), the global cohort includes more patients than the local cohorts. This is somewhat counterintuitive and would benefit from clarification regarding cohort definitions and the rationale behind this setup.\n\n**W7.** In the computation analysis, it would be helpful to include corresponding results for the baseline methods to better contextualize how the proposed framework balances predictive performance and computational efficiency relative to existing approaches.", "was_revised": true, "final_justification": "I appreciate the authors' comprehensive rebuttal, which addresses most of my earlier concerns—particularly the newly added discussions on practical relevance and general applicability, the expanded ablation experiments, and the efficiency analyses. In light of these, I am pleased to increase my rating.", "rebuttal": "Dear Reviewer aRk9,\n\nThank you for the review and comments. We respond to your questions below.\n\n**W1. The practical relevance and general applicability of the setting are not sufficiently justified**\n\nWe respectfully argue that our work utilizes a source cohort from the All of Us Program, a large-scale biomedical initiative launched in 2015 by the National Institutes of Health. As one of the most comprehensive and demographically diverse biobanks, it represents a nationwide effort and remains openly accessible to researchers worldwide, enabling studies on various health conditions [1]. Our problem setting reflects a growing trend in translational biomedical AI [2], and aligns with current priorities of the HHS Plan 2025 [3], which calls for democratizing AI to reduce the “digital divide” for underserved populations and under-resourced healthcare systems, where models must bridge the gap between rich-resource training environments and real-world clinical deployment. \n\nHowever, compared with well-funded biobanks, most clinical institutions (e.g., community hospitals, regional health systems) only retain structured EHRs in routine practice, lacking genetic testing due to cost, infrastructure, or policy constraints. This results in a practical distribution shift between well-resourced source cohorts with both EHR and genetic data, and target scenarios with EHRs only.\n\nOur goal is to capitalize on biomedical signals available in multimodal nationwide cohorts (e.g., All of Us) to improve clinical risk prediction in resource-limited settings. This transfer setting is particularly realistic in ongoing nationwide efforts to democratize precision medicine [3], where insights derived from well-characterized research cohorts should generalize to more typical clinical settings. The target real-world clinical repositories,  e.g., OHSU and INSIGHT, only contain EHRs without accompanying genetic data. Our work explicitly addresses this critical yet underexplored setting, which we believe will grow increasingly relevant as genomics-informed models gain traction, but data gaps remain across clinical institutions.\n\n[1] All of Us Research Program Investigators. The 'All of Us' research program. New England Journal of Medicine. 2019.\n\n[2] Biobanking with genetics shapes precision medicine and global health, Nature Reviews Genetics, 2024.\n\n[3] U.S. Department of Health and Human Services. Strategic Plan for the Use of Artificial Intelligence in Health, Human Services, and Public Health. 2025.\n\n**W2. Limited technical novelty**\n\nWe respectfully clarify that our paper addresses a critical yet underexplored task, cross-cohort cross-modal knowledge transfer, and proposes a framework specifically designed for this setting, which, to our knowledge, has not been studied before. While our method incorporates concepts such as graph learning and knowledge distillation, these techniques are not simply and directly applied in our setting. Instead, we introduce novel designs specifically tailored to the unique challenges of this problem. We emphasize that our goal is not to combine existing concepts simply, but to design a cohesive system where each element is purposefully constructed to serve the overarching objective, rather than being used as standalone techniques.\n\nIn particular, our mixture-of-aggregation module for genetic data is fundamentally different from the standard mixture-of-experts approach, which routes input to relevant neural networks for training. In our design, we introduce the aggregation token to capture patient-level genetic information, accounting for the noise and context dependency in genetic expression; meanwhile, we leverage multiple tokens to model the cumulative effects across different genetic pathways, enabling fine-grained fusion on modality-specific signals. Likewise, the graph-guided fine-tuning strategy also diverges from conventional graph learning or traditional fine-tuning. Rather than updating large pretrained models, we adopt a lightweight, graph-based mechanism to adapt patient representations by leveraging cohort-specific healthcare patterns. This enhances adaptability to population, coding convention, and disease pattern shifts across sites, while preserving efficiency in low-resource settings. Importantly, knowledge transfer is a necessary part of our solution, serving the ultimate goal of transferring knowledge from rich-resourced cohorts to the target ones. To our best knowledge, both the mixture-of-aggregation and the graph-guided phenotypical representation fine-tuning components are novel contributions, thoughtfully developed to meet the particular demands of cross-cohort cross-modal clinical risk prediction. \n\nWe would also like to note that, rather than being purely motivated by methodological challenges, our work is grounded in a real-world problem: the limited availability of high-cost data modalities and limited computational resources in many regional healthcare systems. While issues such as noisy or imprecise multimodal data have been studied in domains like CV and NLP, they remain underexplored, particularly in the context of genetic and EHR data fusion. Canonical multimodal methods are not capable of handling our setting, as discussed in related work. Moreover, the challenge of limited computation at local cohorts is a practical constraint to our cross-cohort setting. This particular need and task setup remain unaddressed, underscoring the novelty and significance of our contribution. \n\n**W3. Difference in used datasets**\n\nWe would like to clarify that the dataset for the ablation study was randomly chosen, considering space limit and figure readability, and not guided by any specific selection criteria. We provide additional experimental results on the OHSU cohort below to augment the analyses presented in Figures 3 and 4.\n\n|Figure 3 |1|2|3|4 |\n|--------|--------|--------|--------|--------|\n|AUROC |70.35|71.82|71.76|70.53|\n|F1|30.24|31.78|31.59|30.64|\n\n| Figure 4 |1|2|3|4|\n|--------|--------|--------|--------|--------|\n| AUROC| 68.71| 71.82|70.68|70.36|\n| F1| 27.82|31.78|29.41|29.27|\n\n\nFor Figures 3 and 4, the additional results from the OHSU dataset show a consistent pattern with those observed in the All of Us, INSIGHT-MS cohorts, and INSIGHT-Columbia. We will standardize the dataset choices in revision. \n\n**W4. Case studies or qualitative analyses**\n\nThank you for the suggestion. We will include t-SNE visualizations in the revision as case studies to illustrate how gene representations are distributed under different aggregation tokens before being passed to the Transformer encoder. While t-SNE provides an intuitive view of representational differences, we are unable to include additional figures during the rebuttal phase. As a quantitative alternative, we compute the Earth Mover’s Distance (EMD) between the two clusters of gene embeddings when using two aggregation tokens. EMD measures the minimal effort required to transform one distribution into another, making it suitable for comparing embedding distributions. In our case, the EMD is approximately 5.6, versus 0 for identical distributions. This non-zero value indicates that the representations learned from the two aggregation tokens are indeed distinct, supporting the hypothesis that our design promotes diverse and potentially disentangled genetic representations.\n\n**W5. Medical validation**\n\nWe respectfully note that our validation on the OHSU and INSIGHT cohorts can be viewed as a form of practical medical validation. These target EHR datasets are derived from real-world hospitals actively serving patients, containing routinely collected records from genuine clinical encounters in Oregon and New York City. Our cohort construction adheres closely to domain expertise and established medical research criteria. As such, our target cohort results demonstrate the model’s practical utility and robustness in realistic clinical environments. We also fully agree that direct medical validation is an important next step and plan to collaborate with our clinical partners to assess the alignment between model outputs and clinical decision-making processes. \n\n\n**W6. Dataset statistics**\n\nThough we utilized the “source cohort” term in the paper, it has no implication on the patient number but refers to a nationwide population and biological modality availability. The All of Us program recruits participants nationwide, forming one of the most diverse health research datasets to date. In contrast, both INSIGHT-MS and INSIGHT-Columbia represent patient populations across the NYC area, which enables larger patient coverage due to the region's high population density.\n\nThe patients in each cohort are identified through a strict procedure with clinically relevant definitions and filtering criteria; the final cohort size depends only on the covered population scale and the cohort construction procedure, for which we kindly refer the reviewer to the cohort descriptions in Appendix 1.1.\n\nImportantly, the work focuses not on absolute cohort sizes, but rather on addressing real-world disparities between cohorts, from a national biobank with diverse, representative populations to regional populations and systems; from multimodal data (EHR + genetics) to unimodal settings (EHR only); and from high-resource computing environments to resource-constrained clinical sites. This setup simulates the practical challenges in translating risk prediction models into more typical real-world healthcare settings.\n\n**W7. Computational efficiency of baselines**\n\nWe provide a comparison between our model and two representative baselines, SMIL and MoMKE. The fine-tuning time on the OHSU is approximately 300s for our method (Appendix 2.1 (Computation Analysis)). In contrast, SMIL and MoMKE require approximately 2× and 4.3× longer, respectively, under the same computing setup. Our method shows significantly lower computational overhead and better predictive performance."}, {"review_id": "Ks2CckjIiI", "reviewer": "Reviewer_GdmP", "summary": "This paper presents an innovative framework, C3M (Cross-Cohort Cross-Modal), designed to democratize clinical risk prediction in resource-constrained settings. The framework addresses the challenges of predicting disease risks using EHR and genetic data across different cohorts, particularly beneficial for chronic conditions like Alzheimer’s disease.", "questions": "1. How can the C3M framework be extended to other chronic diseases beyond ADRD, such as Parkinson’s disease or cardiovascular conditions?\n2. What are the potential societal impacts and ethical considerations of deploying this framework in clinical settings? Are there any risks associated with its use that should be addressed?\n\nBy addressing these questions, the authors could further strengthen their paper and provide a more comprehensive discussion of the framework's applicability and implications.", "limitations": "n/a", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "There are several strong aspects about this paper\n1. The paper clearly identifies a significant challenge in healthcare—predicting disease risks accurately without relying on extensive computational resources or shared data. To solve this the proposed approach (C3M) is a novel method to handle noisy genetic data and adapt EHR embeddings efficiently, making it suitable for local clinical use. The framework is well-structured with three key components, each addressing specific challenges in data handling and model adaptation.\n\n2. Extensive experiments validate the model’s effectiveness across diverse cohorts, demonstrating its practicality and superior performance compared to existing methods.\n\n3. The paper provides detailed descriptions of each component, enhancing understanding of how C3M operates. Further, while not explicitly detailed, the inclusion of code and experimental details supports reproducibility, which is crucial for research integrity.\n\nGiven all of these aforementioned key strengths, the paper lacks in a few aspects:\n1. The framework is validated primarily on Alzheimer’s disease and related dementias (ADRD), limiting its generalizability to other chronic diseases.\n2. The potential societal impacts and broader implications of the framework are not thoroughly explored.", "was_revised": true, "final_justification": "Thanks to the authors for their detailed response. Ultimately the usage scenario for this model currently is established in a narrow scope - however given the possible impact I have updated my scores", "rebuttal": "Dear Reviewer GdmP,\n\nThank you for the review and comments. We appreciate your recognition of the \"significant challenge in healthcare\", \"novel method\", and \"The framework is well-structured\". And we respond to your questions below.\n\n\n**1. The framework is validated primarily on Alzheimer’s disease and related dementias (ADRD)**\n\nOur focus on Alzheimer’s disease and related dementias (ADRD), including Alzheimer’s disease, vascular dementia, Lewy-body dementia, frontotemporal dementia, and mixed dementia, is motivated by both public health significance and technical modeling considerations. ADRD represents one of the most pressing global health challenges, currently affecting over 55 million people worldwide, with projections to triple by 2050. It is the leading cause of disability among older adults and consistently ranks among the top causes of death across countries.\n\nUnlike some chronic conditions that may present abruptly or lack well-defined preclinical stages, ADRD is characterized by a long prodromal phase, during which subtle cognitive, behavioral, and clinical changes gradually accumulate, yet often go unrecognized in routine care. This lack of definitive early biomarkers in structured EHR data further elevates the importance of computational prediction approaches.\n\nIn contrast to conditions such as diabetes or hypertension, where early diagnosis is routine and evidence-based treatment guidelines are well established, ADRD remains underdiagnosed in its early stages and lacks curative treatments. This makes early detection and risk stratification not only more urgent but also more impactful for guiding clinical trials, patient monitoring, and intervention planning.\nIn addition, ADRD is comparatively well represented in large-scale EHR datasets, offering sufficient cases for the development and validation of machine learning models, more so than some other chronic diseases with lower prevalence or more ambiguous diagnostic labeling.\n\nLastly, due to the time-consuming nature of data preprocessing, such as defining disease cases and controls with clinical expertise, constructing appropriate cohorts, and harmonizing real-world EHRs across the four sites, we have focused our current efforts on ADRD. As noted in our discussion of future work, we plan to extend the proposed framework to other chronic neurodegenerative conditions, such as Parkinson’s disease. To apply C3M to Parkinson’s disease, we will follow the same data preparation procedure described in Appendix 1.1, while incorporating domain expertise to define Parkinson’s disease cases and controls based on clinical criteria. Once the data is prepared, the C3M model can be readily applied to the Parkinson’s disease cohort without model modification. \n\n\n**2. Potential societal impacts and broader implications**\n\nThis work aims to democratize clinical risk prediction by enabling the transfer of predictive models trained on multimodal, nationwide datasets to local clinical settings with limited resources and unimodal EHR data. If widely adopted, the proposed C3M framework could improve early detection and risk stratification of chronic neurodegenerative diseases, such as Alzheimer’s Disease and Related Dementias, especially in underserved populations and healthcare systems with limited access to genetic testing and computational infrastructure. By reducing reliance on costly or inaccessible modalities, this approach promotes more equitable access to precision medicine. However, careful attention is needed as performance disparities may emerge across cohorts, which is not inherent flaws of the model but practical challenges due to variations in healthcare practices. This highlights the importance of local validation, continuous monitoring, and inclusive data practices."}, {"review_id": "urXjtkt5xo", "reviewer": "Reviewer_jjL6", "summary": "This paper addresses a practical challenge in developing clinical multimodal models: training on a large-scale, nationwide cohort and adapting the model to a local cohort with partially missing modalities. To tackle this, the authors propose C³M, a method designed to handle two key issues—noisy data and limited computational resources during local deployment. The approach introduces two main contributions: (1) a mixture of aggregation strategies to extract informative genetic features, and (2) a lightweight fine-tuning mechanism that adapts the model using limited patient data. Experimental results show that C³M achieves strong performance on both the source and target cohorts.", "questions": "1. Could the authors provide more details on missing data patterns across datasets and how these are handled during training and evaluation?\n2. The results suggest that increasing the number of aggregation tokens does not yield performance gains. Is there an explanation for this saturation, and could it be task- or data-dependent?\n3. Are temporal dynamics in the EHR sequences modeled or considered in any way? If not, how might this affect the method’s applicability to longitudinal clinical data?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper is clearly written and well-structured, which facilitates understanding of both the motivation and technical contributions.\n2. Extensive experiments are conducted on real-world datasets, with comparisons to a wide range of baselines. The inclusion of thorough sensitivity analyses further supports the model’s effectiveness.\n3. The proposed approach is well-motivated for real-world deployment, addressing practical challenges such as missing modalities and resource constraints in local clinical settings.\n\nWeaknesses:\n1. The concept of cross-modal knowledge transfer via distillation is not clearly explained, and its specific contribution to performance remains unclear.\n2. The paper lacks detail on the input features—both genetic and EHR—and does not provide statistics on the patient cohorts, making it difficult to assess the dataset characteristics and generalizability.\n3. It is not clear how the method handles potential temporal patterns in EHR data, which are often crucial for clinical prediction tasks.\n4. The code is not provided, which hinders reproducibility.", "was_revised": true, "final_justification": "My concerns are addressed. I will keep my current score of accept.", "rebuttal": "Dear Reviewer jjL6,\n\nThank you for the review and comments. We appreciate your recognition of the \"well-motivated approach\" and \"clearly written and well-structured paper\". And we respond to your questions below.\n\n**1. (W1) The concept of cross-modal knowledge transfer via distillation**\n\nWe would like to clarify that knowledge transfer via distillation is a necessary component of our model. The teacher model, trained on the source cohort, requires both EHR and genetic data as input. Specifically, it takes phenotypical and genetic representations. However, the target cohorts contain only EHR data and lack access to genetic information, making it infeasible to directly apply the teacher model in target cohorts. To address this, we design a distillation strategy that transfers knowledge from the teacher model to a student model, which operates solely on the phenotypical representation derived from EHR data. By leveraging this distillation framework, we enable effective knowledge transfer from a gene-aware model (teacher) to a gene-free model (student), allowing the model to generalize to real-world target cohorts without genetic input. Therefore, distillation serves as a foundational component of our model; without it, the model cannot be effectively adapted to the target cohorts.\n\n**2. (W2) Details on the input features**\n\nWe provide a brief introduction below, with full details in Appendix 1.1.\n- EHR features include structured, timestamped records of patient healthcare data such as diagnoses, medications, as well as demographics, standardized across sites using SNOMED and RxNorm coding vocabularies.\n- Genetic features are derived from the All of Us ACAF callset and filtered based on established SNPs recorded in the GWAS catalog associated with ADRD.\n- We describe the cohort construction, case-control matching strategy, and index date assignment in detail following the literature across all four datasets (All of Us, OHSU, INSIGHT MS, and INSIGHT Columbia), and provide the statistics on the four patient cohorts in Table 1 of the Appendix. \n\n\n**3. (W3 and Q3) How the method handles potential temporal patterns in EHR data**\n\nIn our paper, we adopt the EHR foundation model MOTOR to capture temporal patterns by taking EHR sequences as input and learn patient-level embeddings, analogous to how BERT-series models are used in natural language processing to learn contextualized representations.\n\n\n**4. (W4) Reproducibility**\n\nWe would like to kindly clarify that the GitHub link is included in Section 5 of the attached checklist. As the rebuttal guidelines prohibit including links in the response text, we respectfully refer the reviewer to Section 5 for access.\n\n**5. (Q1) Provide more details on missing data patterns across datasets and how these are handled during training and evaluation**\n\nThanks for the question. We focused on a practical scenario, where the target cohorts from local healthcare providers only collect routine EHR data, while the All of Us program, as the source cohort, has measured genetic data through nationwide efforts. Therefore, the genetic data is completely missing on target datasets, namely OHSU, INSIGHT MS, and INSIGHT Columbia. Meanwhile, random missingness of genetic data also occurs on All of Us at a rate of 26.7%. Our model addresses missingness in the All of Us through the teacher–student framework, where the teacher remains fixed and the student model learns solely from EHR data when genetic information is absent. \n\n\n**6. (Q2) Explanation for the number of aggregation tokens**\n\nAs shown in Figure 3 and discussed in Section 5.2, we observe that increasing the number of aggregation tokens from 1 to 2–3 improves performance, because a single token tends to focus narrowly on a subset of genetic features, potentially missing important signals. The use of multiple tokens enables a mixture-of-aggregation strategy, capturing complementary biological patterns from different perspectives.\n\nHowever, performance saturates or slightly declines with 4 tokens, which we attribute to the introduction of redundancy or noise. Multiple aggregation tokens may begin to extract overlapping or irrelevant information, thereby diluting the model’s discriminative capacity. This phenomenon is likely task-dependent, particularly influenced by the complexity and signal strength of the disease under study. For the task involving subtle or sparse genetic associations, increasing the number of aggregation tokens beyond three may introduce more noise than meaningful diversity, thereby limiting potential performance gains."}, {"review_id": "nszgHBSe1H", "reviewer": "Reviewer_BXf7", "summary": "This paper proposes a novel cross-cohort, cross-modal knowledge transfer framework to enable effective clinical risk prediction in resource-constrained settings where only unimodal data is available. The model is trained on a multimodal source cohort with EHR and genetic data, and adapted to local target cohorts via a lightweight graph-guided fine-tuning mechanism. The framework includes a mixture-of-aggregations module to robustly encode noisy genetic features and uses teacher-student knowledge distillation for cross-modal transfer. Experiments on real-world Alzheimer’s disease datasets demonstrate improved generalization across target institutions.", "questions": "Please check the weakness part.", "limitations": "The authors mentioned the limitations of the work and proposed future directions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper is well-written, clearly presenting the motivation and structure of the proposed method. The key contributions of the paper include:  a strong real-world motivation and practical focus on resource-limited deployment scenarios;  novel architectural components such as the mixture-of-aggregations and graph-guided tuning;  and good reproducibility support with public code and implementation details.\n\n\n\nWeaknesses:\n\nThe main weakness of the paper is the evaluation is narrowly focused on Alzheimer’s and related dementias. The authors propose a general framework but lack an evaluation of cross-disease generalization performance. It would strengthen the paper to include other chronic diseases to demonstrate broader applicability. Is it possible to quickly test C3M's performance on other chronic diseases?\n\nAnother major concern is that, although the paper primarily claims that the method is Cross-Cohort Cross-Modal Knowledge Transfer, the model without genetic data (w/o Gene) shows suboptimal performance in OHSU, with no significant differences from the original method on some metrics (e.g., AUROC and F1 score). Can a single target dataset effectively assess the impact of cross-modal data in knowledge transfer? In other target datasets, does the performance of the model without genetic data (w/o Gene) show a significant difference compared to the original method?", "was_revised": true, "final_justification": "I think it is a useful work to address a practical problem.\n\nBut its novelty is limited.", "rebuttal": "Dear Reviewer BXf7,\n\nThank you for the review and comments. We appreciate your recognition of the \"strong real-world motivation\" and the \"novel architectural components\". We respond to your questions below.\n\n**1. Evaluation on Alzheimer’s and related dementias.**\n\nOur focus on Alzheimer’s disease and related dementias (ADRD), including Alzheimer’s disease, vascular dementia, dementia with Lewy bodies, frontotemporal dementia, and mixed dementia, is motivated by both public health significance and technical modeling considerations. ADRD represents one of the most pressing global health challenges, currently affecting over 55 million people worldwide, with projections to triple by 2050. It is the leading cause of disability among older adults and consistently ranks among the top causes of death across countries. \n\nUnlike some chronic conditions that may present abruptly or lack well-defined preclinical stages, ADRD is characterized by a long prodromal phase, during which subtle cognitive, behavioral, and clinical changes gradually accumulate, yet often go unrecognized in routine care. This lack of definitive early biomarkers in structured EHR data further elevates the importance of computational prediction approaches. \n\nIn contrast to conditions such as diabetes or hypertension, where early diagnosis is routine and evidence-based treatment guidelines are well established, ADRD remains underdiagnosed in its early stages and lacks curative treatments. This makes early detection and risk stratification not only more urgent but also more impactful for guiding clinical trials, patient monitoring, and intervention planning. In addition, ADRD is comparatively well represented in large-scale EHR datasets, offering sufficient cases for the development and validation of machine learning models, more so than some other chronic diseases with lower prevalence or more ambiguous diagnostic labeling.\n\nLastly, due to the time-consuming nature of data preprocessing, such as defining disease cases and controls with clinical expertise, constructing appropriate cohorts, and harmonizing real-world EHRs across the four sites, we have focused our current efforts on ADRD. As noted in our discussion of future work, we plan to extend the proposed framework to other chronic neurodegenerative conditions, such as predicting Parkinson’s disease. To apply C3M to Parkinson’s disease, we will follow the same data preparation procedure described in Appendix 1.1, while incorporating domain expertise to define Parkinson’s disease cases and controls based on clinical criteria. Once the data is prepared, the C3M model can be readily applied to the Parkinson’s disease cohort without model modification. \n\n\n**2. Ablation study for genetic data**\n\nWe presented ablation studies with randomly selected datasets due to space constraints in the submission. Here, we provide additional results on our model and its gene-free variant across two other target datasets: INSIGHT MS and INSIGHT Columbia in the Tables below. The results show notable differences between C3M and its gene-free variant. Besides, although the differences in AUROC and F1 metrics for OHSU are not particularly large, substantial improvements in Sensitivity@95 and PPV@95 indicate that the model performs considerably better on high-confidence predictions. This demonstrates its enhanced ability to accurately identify true positives while satisfying the stringent 95% specificity constraint, which is especially valuable in clinical decision-making contexts.\n\n| INSIGHT MS          | AUROC       | F1           | Sens@95      |   PPV@95   |\n|----------------|-------------|--------------|--------------|--------------|\n| C3M            | 72.44±0.28  | 23.61±1.27   | 26.63±0.51   | 34.75±0.62   |\n| C3M w/o Gene   | 70.26±0.34  | 21.47±1.40   | 22.17±0.47   | 31.96±0.58   |\n\n| INSIGHT Columbia          | AUROC       | F1           | Sens@95      | PPV@95     |\n|----------------|-------------|--------------|--------------|--------------|\n| C3M            | 75.39±0.23  | 36.04±1.10   | 29.68±0.48   | 41.03±0.63   |\n| C3M w/o Gene   | 73.18±0.34  | 33.86±1.32   | 24.75±0.72   | 36.52±0.59   |"}]

Author rebuttal:

Meta-review: After carefully evaluating the paper and considering the reviewers' feedback, we recommend acceptance of this work. The paper presents a well-motivated and technically sound framework, C3M, for cross-cohort, cross-modal clinical risk prediction, addressing an important challenge in healthcare AI: transferring models from resource-rich to resource-limited settings. Despite minor limitations, this work makes a valuable contribution to clinical AI by enabling robust cross-modal knowledge transfer in resource-constrained settings. The methodological rigor, clear presentation, and practical relevance justify acceptance. We encourage the authors to address the reviewers' concerns—particularly regarding generalizability and dataset consistency—in the final version.
Decision: Accept (poster)