An Investigation of Memorization Risk in Healthcare Foundation Models
Authors: Sana Tonekaboni
Lena Stempfle
Adibvafa Fallahpour
Walter Gerych
Marzyeh Ghassemi
Venue: NeurIPS 2025 poster ()
Keywords: Healthcare
evaluation
foundation model
memorization
Primary area: machine_learning_for_sciences
TL;DR: We propose black-box tests to detect harmful memorization in foundation models trained on structured EHR data. Validated on a public model, our toolkit supports privacy audits by distinguishing generalization from privacy-compromising memorization.

Abstract: Foundation models trained on large-scale de-identified electronic health records (EHRs) hold promise for clinical applications. However, their capacity to memorize patient information raises important privacy concerns. In this work, we introduce a suite of black-box evaluation tests to assess privacy-related memorization risks in foundation models trained on structured EHR data. Our framework includes methods for probing memorization at both the embedding and generative levels, and aims to distinguish between model generalization and harmful memorization in clinically relevant settings. We contextualize memorization in terms of its potential to compromise patient privacy, particularly for vulnerable subgroups. We validate our approach on a publicly available EHR foundation model and release an open-source toolkit to facilitate reproducible and collaborative privacy assessments in healthcare AI.

An Investigation of Memorization Risk in Healthcare Foundation Models
Sana Tonekaboni*
Lena Stempfle" MIT
MIT
Broad Institute of MIT and Harvard
Chalmers University of Technology
Vector Institute stonekabemit.edd
University of Gothenburg stempfle@mis edu
Adibvafa Fallahpour'
Walter Gerych Worcester Polytechnic Institute Computer Science Department wgerych@wpi.co edu edu
University of Toronto
Vector Institute
University Health Network (UHN)
adibvafa, wwwlaaaoorrraaaaaiaaaaa utoronto. ca
Marzyeh Ghassemi MIT mghassem@mit..c edu
Abstract
Foundation models trained on large-scale de-identified electronic health records (EHRs) hold promise for clinical applications. However, their capacity to memorize patient information raises important privacy concerns. In this work, we introduce suite of black-box evaluation tests assess privacy-related memorization risks in foundation models trained on structured EHR data. Our framework includes methods for probing memorization at both the embedding and generative levels, and aims to distinguish between model generalization and harmful memorization clinically relevant settings. We contextualize memorization in terms of its potential to compromise patient privacy, particularly for vulnerable subgroups. We validate approach publicly available EHR foundation model and release open- source toolkit to facilitate reproducible and collaborative privacy assessments in healthcare AI.
Introduction
Foundation models trained on Electronic Health Records (EHR-FM) have been proposed as promising direction for advancing healthcare [38]. However, there are concerns that large models trained on sensitive patient data may memorize and expose private health information [36), even
under legal de-identification. Previous work on memorization in Large Language Models (LLMs) found that targeted prompts can reveal private data from training data sets EHR-FM, the risk is comparable, but the sensitivity of clinical data makes it even more critical to proactively assess and mitigate memorization threats.
50 30 For
Model memorization is often studied through the lens of adversarial privacy attacks, such as mem- bership inference, attribute inference, and training data reconstruction, that aim extract sensitive
"Equal contribution.
"Code available
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

information from models [3 40 45 19 39, While these approaches provide important lower bounds on leakage, they often ignore the deployment context of the model. Recent work has introduced the notion of contextual integrity [37]
defining privacy breaches as violations of context-specific norms governing information flow. This distinction is particularly critical in clinical settings, where beneficial generalization of clinical knowledge may be similar to harmful memorization of patient- level information. We want a model to infer tuberculosis for a 36-year-old who presents with weight loss and cough. We do not want to infer tuberculosis for a 36-year-old who presents only with hypertension and knee pain, merely because that same patient had TB in the training data. In this work, we propose contextual, risk- based evaluation framework for evaluating memorization in EHR-FM, with a focus on identifying practical privacy threats.
MMssur how targeted prompts can trigger the disclosure of sensitive patient information through Generative and Embedding memorization.
Objective E:
Objective II: Distinguish between patient lovel memorization and populstion-evel generalization assess risk patient privacy.
Memorization
Embedding Memorization
Privacy risk assessment
T1 Reconstruction lest
T3 - Probing lest
T5 - Perturbalinn test
EHR-FM reconstruct training
embeddings reveal
memorized sample revealing information about an individual or a general al concition?
semplee with prompt informatio?
senstivee information?
T2- Sensitive altribule lest EHR-FM sensilive information about training samples?
T4 - Membereship Inference lest EHR-FF embeddinge mombership training cahort?
T6 Patient--evel and sub- sub-population leet
Does memorization digrrpporionattly impact Individals or subgroups?
Patient Embecting space
7x
Gemented Poltent Trjjctoriee
Predicted oode
Pred sted code
Precideed took e WESTERTONNY
Phocesol oode?
Gote1
Gate "HANSFEL TO//D
Code L4G/720001
A/NGGO
ACC/53
cracttory
ensepessse spoid
Palling
Takket
treolerrddi depermmen
with
Taaler with Very
Klter 7hame
gatroer Fertits
Figure 1: Our proposed tests (T1-T6) to evaluate memorization in EHR-FM, grouped by two core objectives. T1-T4 measure the extent to which models reveal training data, targeting both the embedding and the generated data domain. T5-T6 perform patient-level evaluations to quantify individual privacy risks from memorization or data leakage.
We explore memorization and its risk patient privacy in EHR-FM trained on private data sequences of structured medical codes, and released as black-boxes with prompt-only access. EHR-FMs are distinct from clinical LLMs as they are not trained with free text such as clinical notes, where memorization risks have been explored [29]. We introduce a set of pragmatic EHR-FM tests with two main objectives (Figure D] (1) to quantify different types of memorization that can be triggered via prompting, and (2) to assess the associated privacy risks by distinguishing harmful memorization at the patient level from benign population-level generalization. Overall, these tests evaluate the privacy risk of data leakage as a function of information an adversary can access. Our tests can be used to identify vulnerable samples and subcategories of individuals for further safeguarding prior to public
release of model using post-training safety layers 31 red-teaming 42 or retraining (28 25
We first introduce a practical formalization of memorization tailored EHR-FMs. Next, we design and evaluate two generative memorization tests (T1-T2) on a publicly available EHR-FM [14]
to assess its ability to reconstruct training data. Our results show that minimal individual-level knowledge (c.g., age) does not reveal sensitive attributes; however, we identify the additional contextual information that can trigger leakage. Our proposed privacy risk assessment tests (T5-T6) quantify patient-level memorization risk and assist in distinguishing between patient-level memorization and generalization. We also employ two embedding memorization tests (T3-T4) motivated by prior concerns that deep embeddings can leak private information [49]. These tests examine both membership inference and leakage of sensitive medical attributes from embeddings (overview in Figure 7 in the appendix). These tests address an important yet understudied aspect of EHR-FMs and offer a practical framework for systematically evaluating memorization and privacy risks as these models continue to scale and see broader clinical use.

---

2
Related work
EHR Foundation Models EHR-FMs are large models trained on structural EHR data using self- supervision at scale that can be adapted to wide range of downstream tasks [6]. These models process sequences of standardized medical codes (diagnosis ICD, procedures, medications, lab tests) alongside temporal tokens indicating time intervals, creating patient trajectories (example shown in Figure D Two main category of EHR-FM includes Encoder models, like BEHRT [32], CLMBR- T [52]. and CEHR-BEER BERT [41] for representation learning and Decoder models like ESGPT [35]. ETHOS 44,, EHRMamba 14 that are trained to generate patient records. Initial work in EHR-FM has demonstrated significant improvements over task-specific architectures in limited data settings [52]. domain adaptation, and robustness to distribution shifts [17] However, study of memorization in these models, or even a definition of what memorization is in this setting is still an open problem.
Memorization in Privacy and Security Memorization has been extensively studied in privacy and security research 3
40 45 19]. Categories of attacks include but not limited to membership
property inference (49] [6], and model extraction (27), each targeting different aspects of models and assuming varying levels of transparency. Memorization in these settings is typically defined as being able to extract any training data from the model [1] 40). However, in EHR settings, it often more practical assess the sensitivity level of the extracted data focusing on the
inference 48 12
deployment context [5]].
Memorization in Foundation Models The study of memorization of foundation models mainly focuses on large language models (LLMs) [ [T0 40]. Memorization is described as model's capacity to store and reproduce specific training data patterns which can result in data leakage. While high-capacity models differ from standard ML models in their capacity to generalize without
overfitting I 71, they are still vulnerable to data leakage [TO]
54 34 24 by membership
inference 48 153,, property inference [16). and model inversion attacks [S]
Memorization in Healthcare Foundation Models Memorization in domain-specific foundation models, particularly those trained on medical data, lacks comprehensive study. In particular, their implications for healthcare domains and patient privacy remain unclear. The closest work to ours investigates extraction of personal health identifiers from an LLM trained on medical data (29). finding basic probes ineffective while noting advanced attacks' potential. We focus on models trained on structural EHR data investigate how such risks could manifest in rich temporal medical records.
3 Testing for Memorization
In EHR-FM systems, we expect models to learn from every patient they encounter and internalize that knowledge to make better predictions. Generalization is desirable, but model should not reveal memorized examples if such disclosures compromise patient privacy. To address this, we introduce set of practical evaluation tests (summarized in Figure []] designed to quantify memorization, and assess whether the memorized content poses privacy risks.
3.1 EHR-FM Testing Framework
As most EHR-FM give black-box and prompt-only access users, designed our tests to evaluate whether prompts alone can trigger the disclosure of sensitive patient information. We first create four tests (T1-T4) designed to measure memorization under two domains of information extraction: Generative memorization and Embedding memorization (Figure l]. Generative memorization tests assess a model's s ability recreate data from the training cohort based on the following definition:
Definition 3.1 (Generative memorization) Given prompt p. consisting of a series of codes, model f can generate a time series of EHR records s utilizing greedy decoding, where the concatena- tion of [p||s] is part of sample x.
Embedding memorization tests focus on encoder-based EHR-FMs [32], and measure how much a model memorize and reveal about training sample through the embeddings as formalized below:
Definition 3.2 (Embedding memorization) Given a prompt Ps consisting of a series of codes, model f can generate an embedding z that can be utilized recover information associated with sample in the training dataset.
3

---

Finally, we argue that the degree of memorization does not always align with the degree of risk to patient privacy. Therefore, additional tests (T5-T6) are presented to assess individual- risk and to distinguish between patient-level memorization and population-level generalization.
3.2 Testing Setup
All proposed tests measure memorization function of: (I) the amount of information an adversary uses to reveal information, and (2) the risk associated with the revealed information. Throughout the tests, we use the following setups reflect the amount of information an attacker has access to:
Random: No information is provided to the models and it generates embeddings sequences only from its prior. This reflects the internalized knowledge
model.
Static: The model is prompted with any demographic attributes it has been trained on, such age or biological sex. These represent easily accessible and public information about individuals. N-codes: The model is prompted with the first N codes of patient record. Such prompts require more information about an individual patient.
We can also curate more sophisticated categories prompts. For instance, prompting the model with information about an individual's medication history. Different prompt setups will assess memorization based on what information can be accessed by an adversary and would give insight into what information has higher likelihood of revealing private information.
To reflect the risk associated with revealed information, we select a number of codes, such as ICD- 10 medical codes, related to diagnoses with social stigma as representatives of high-risk sensitive attributes. This information is classified under infectious diseases, substance abuse, and mental health conditions (see Table i the appendix). We have selected these categories, they are highly protected under regulations such as HIPAA and GDPR [[]. often involving conditions that are poorly understood and associated with social taboos. The choice of sensitive attributes can be different depending on the setting or datasets, and we provide this categorization as an example.
3.3 Benchmark Implementation
We demonstrate testing framework on EHRMamba2 [14.
We select this model representative benchmark because it one of the few existing models designed for both patient embedding learning and EHR generation (forecasting). Additionally, its publicly available architecture details enable reproducible training. Also, EHRMamba2 is trained on the public MIMIC-IV dataset (23), which enables direct testing of memorization known training samples, unlike other released models that are trained on private and inaccessible datasets.
Objective I: Measuring Memorization in EHR-FM
We evaluate EHR-FM generative memorization and embedding memorization with tests designed to measure the amount of training information extractable from the model and the risk associated with the information.
4.1 T1 Trajectory memorization test
TI examines how similar a sequence of generated codes is to patient trajectory from the training data. Quantifying EHR sequence similarity is a nontrivial task, with two key nuances: (1) EHR code similarity may depend their clinical context, for instance, two medications treating the same condition should have a higher similarity score compared to unrelated codes. (2) EHR data is inherently time-series-based therefore, identical codes appearing at different time points should be penalized when measuring similarity between sequences.
We develop similarity metric to quantify EHR sequence similarity. We first employ MedBERT 43 which adapts the BERT framework for EHR data, to map EHR codes embeddings h(.)) that capture clinical semantics (Figure 7 in the Appendix). This allows our distance measure to incorporate clinically meaningful semantics through the embeddings. Our distance score dEMD (Fquation I then uses time- weighted Wasserstein Earth Mover's Distance (EMD) [46] to measure the distance

---

less frequent codes, as between 2 sequences of embeddings, where d(h(ss;), h(s2,)) indicates the pairwise cosine distance between codes generated by MedBERT.
We incorporate the time difference between event tokens as penalty for temporal alignment. The weight T.,; measures the visit time difference between token in one sequence and token j in the other. EHR-FMs encode temporal information with time tokens, indicating the gap between each recorded code. The time penalty determined based on how granular the EHR-FM record time is and how sensitive we want our score to be to time alignment. In our benchmark model, only time gaps larger than hour are recorded, and we penalize every hour of misalignment with a weight of 1.
[s]] 122
dEMD = inf TET =1
T Tidd(M((1))
We validate the functionality and sensitivity of our proposed metric in Appendix A usingg synthetic pa- tient trajectories, analyzing how the measured distance changes across variations in patient trajectories. related and unrelated medications, and diagnoses.
Analysis: We compare the average distance (measured by dEMD) of generated codes to the true trajectories for different prompt setups (Random, Statie, 10, 20, and 50 codes), on our benchmark model. Figure 2a shows the distance over 100 prediction codes (|s) = 100) for 3K individuals in the pretraining cohort. Following the strategy of language models (9. for every prompt, hundreds of trajectories are sampled, and the distribution is used to quantify memorization.
We observe that prompting the model with more information about individual results in better sequence predictions (lower D) This shows that the more information an attacker provides about individual, the likelier it is for the model recreate a patient EHR trajectory. However, distance measures are relative and may not easily translate to risk. Samples performing worse than random indicate that the prediction no better than random guess by the model. The high variance around the values indicates that forecast performance varies significantly between individuals, which highlights the importance of individual-level investigation
persoomefo samples.
Distance between real and generated trajectaries
Low Distance Predi cted Codes
1.5
Mou CAAFE 22300
Distance
1.0
0.5
0.0
Rardom
Static
20 codes
Una wer (NN))
natet foar (WN)
fae dea (ery LNN wnt
Do- (viry Low)
R000 tair Lad
HINH (vbry Lund
odte (homal)
anten tao (Uo))
Guzan Laed
seeatii (N00)
acaum
(a)
Fsteatum
(b)
Model predictions in trajectory memorization. (a) Average distance between generated and ground truth EHR sequences of 100 codes for different prompting setups. (b) Predicted tokens and their frequencies in the best-performing trajectory prediction. All tokens are related common lab measures, and most often normal values.
Figure 2:
While revealing any information about individual's health record is undesirable, the risk associated with the memorized information can significantly vary. For example, noting a low white blood cell (WBC) count poses far less risk than revealing highly sensitive details, such patient's 's HIV status. Our letie investigation of high-performing samples reveals that trajectories with a smaller distance do not necessarily imply a higher risk. In fact, these predictions correspond short trajectories with common codes such as routine lab measures with normal values.
Figure 2b shows the top predicted codes for the best-performing prompts, prominently featuring com- mon lab codes and time-related tokens. Our results further show that the model closely approximates the true token frequency distribution (Figure Ba However, it
5

---

evidenced by the deviation from the diagonal in the low-frequency range. This shift indicates that the model favors more frequent codes, revealing bias against generating rare codes.
Measuring similarity offers an intuitive and quantitative signal for memorization, but does not directly capture risk. To address this, we follow up T1 with T2, which explicitly targets codes whose leakage could pose harm to patients-thus quantifying memorization risk more concretely.
4.2
T2 Sensitivity test
Leaked information poses different risks, depending the sensitivity of that information. The objective of T2 is to evaluate the likelihood of model memorizing and revealing high-risk information about individual. This information is represented sensitive attributes defined in Section 3.2 We adopt a similar setup to TI but explicitly remove any codes directly related to the sensitive condition from the prompt. T2 then measures the probability that the model reveals the sensitive attribute in its generated trajectory. The test passes the model does not leak the sensitive attribute. Conversely, the generated sequence contains the attribute, it indicates memorization event that must be further validated to assess the associated privacy risk.
Table I: Performance of the EHR-FM benchmark in predicting sensitive diagnoses. Results are reported for three categories of sensitive attributes (infectious disease, substance abuse, and mental health) under different prompt conditions. Higher prompt lengths increase the likelihood of revealing sensitive codes, highlighting privacy risks associated with memorization.
Sensitive
Patient
Prompt
AUROC AUPRC
Precision
Recall
Positive
attribute
prevalence
prediction count
Infectious
0.0476
Statics
0.548
0,053
0.000
0.000
0
disease
0.0476
10 codes
0.672
0.153
0.353
0.004
0.0478
20 codes
0.697
0.177
0.444
0.006
8
0.0473
50 codes
0.742
0.219
0.411
0.018
23
Substance
0.0672
Statics 0.622
0.090
0.000
0.000
abuse
0.0665
10 codes
0.706
0.171
0.467
0.023
49
0.0661
20 codes
0.719
0.191
0.594
0.024
41
0.0665
50 codes
0.751
0.267
0.577
0.084
162
Mental health
0.0611
Staties 0.604
0.081
0.000
0.000
0.0605
10 codes
0.669
0.138
0.889
0.004
8
0.0608
codes
0.681
0.170
0.733
0.007
11
0.0604
50 codes
0.724
0.286
0.721
0.114
202
Analysis: We evaluate T2 on the benchmark model for different levels of prompting, and the findings are summarized in Table With short prompts (Static baseline) the predictive performance of the model is near random. Showing that the benchmark doesn't 't reveal sensitive information about patient, only given the individual's age or demographics. However, the model's ability to reveal sensitive attributes increases with an increasing amount of prompt information. For an individual-level analysis, we select conservative threshold of 30% and select all positive predictive prompts that generate the sensitive attribute in more than 30% of their trajectories (right column of T). These identify potential problematic samples, but the risk memorization depends on the context provided to the model. In fact, not all positively predicted samples reflect memorization. well-performing model, like skilled practitioner, should make informed predictions based on meaningful prompts, not mere memorization.
sample fails the sensitivity test when the model generates a sensitive attribute even though was excluded from the prompt. These are the most problematic cases where the model predicts condition based on irrelevant input, indicating that may have memorized spurious correlations or, more concerningly, details about specific individual. For instance, one of our positive samples included a lab result showing a very low Absolute CD8 Count, model predicted an HIV diagnosis in over 30% of sampled trajectories. While this does not necessarily indicate memorization, it reflects meaningful reasoning. Low CD8 counts are commonly observed in HTV patients and, although not sufficient for full diagnosis, they increase the likelihood of HIV being suspected.

---

In contrast, we identify prompts that lack clear indicators of disease, yet the model still predicts sensitive diagnosis. For example, one of our prompts was 48- 48-year-old individual transferred to the ED with history of falls and discharged shortly after, representing minimal clinical detail. The model predicts alcohol abuse 31% % of the estimated trajectories. Despite the absence of relevant cues, the model reveals a sensitive diagnosis, causing this sample to fail the sensitivity test. We flag these prompts for further investigation in future test to further assess their risk to patient privacy,
Token Frequency Distribution by Type (both freq >= 25)
(b)
10a
Type
Distribution of Detection Scores (AUC = 0.568)
LAB
INF ISDN
MEDICATION
False True
10a
Generated Frequeniy
800
30*
Count
600
309
400
10'
-8 Detection
10"
10a
101
10a
100
Original Frequency
Figure Code generation and membership inference. (a) Frequency of EHR code randomly generated by EHRMamba2 versus the original distribution of codes in the training dataset. (b) Distribution of detection scores from membership inference attack based on model confidence. The x-axis shows scores, the y-axis shows counts, with colors indicating member (True) non-member (False) samples.
4.3 T3- Probing test
T3 employs a probing test 2 to evaluate memorization by examining information stored in embeddings. We train a probing model g(.) to predict sensitive information from embeddings, using either external data or a subset of the training dataset. This setup reflects the information available to an adversary, and T3 measures memorization risk a function of how much knowledge the probe requires to recover the sensitive content. If probes successfully extract sensitive information from embeddings, developers should apply additional post-processing to remove such information before releasing the model. We validate our proposed T3 test on a synthetic temporal dataset (as shown in Appendix B) to demonstrate it successfully identifies memorized samples controlled setting with positive control cases.
We apply the probing test to our benchmark model by prompting with the first N codes from a patient record. Embeddings are extracted up to the first 10, 20, or 50 tokens and remain frozen during probing. We then train a classifier on the embeddings to predict sensitive diagnoses ( (s))), defined in Section 3.2 where f(x) denotes the foundation model embeddings and ý the predicted diagnosis. If the classifier successfully infers the label from the embedding, we consider the model to have memorized sensitive information. The highest-risk scenario arises when the probing model, trained on a separate test cohort, can still successfully extract sensitive codes. This is particularly concerning, as it suggests that adversary equipped with an external dataset and the trained model could recover sensitive information about individuals in the training cohort.
Analysis:
We train the probing model with embeddings of a separate test cohort as well as varying fractions of training data. High accuracy with minimal data suggests strong memorization rather than generaliza- tion. Table in the appendix reports probing attack performance on sensitive diagnoses, measured across different prompt lengths (10, 20, 50 tokens) and training. Extracting sensitive information only from the embeddings is difficult, even if adversary can access portion of the training data.

---

AUROC values across all sensitive attributes remain around 0.5, indicating no clear memorization signal and suggesting random performance. Notably, AUPRC and Fl generally decline with increased training data, hinting at inconsistent memorization patterns. These trends may be partially influenced by dataset size variations: 113,579 for the test set, 102,222 for 0. 1,, and 90,864 for 20%. If probing performance does not improve even with access to training data, as we see with our benchmark, then the model passes T3. Although, the sensitive signal may be encoded only within a specific subgroup, prompting T6, which targets subgroup-level probing to uncover such hidden memorization.
4.4 T4- Membership inference
T4 evaluates Membership Inference in EHR-FM; specifically, an adversarial attack that reveals whether specific sample was included in a model's 's training cohort [12]. Many MI attacks succeed using only model outputs such as logits 48 1 33 underscoring that risk extends even to EHR-FMs released as black-box models. This type of risk differs from direct patient data exposure as it requires an attacker to possess prior knowledge about an individual and then confirm their presence in the training data. When such inference succeeds, it can reveal sensitive details about the individual, including the institution or location of their clinical care or other training cohort-related information.
Analysis: We prompt the model with part of the patient EHR trajectory and extract the estimated
logits for each token. Following the strategy introduced in [47). we estimate the membership score as the expected value of the log probabilities of the least probable tokens. The detection scores are shown in Figure 3b for 10K samples from the train and test cohort. The slight distribution separation suggests marginally higher detection score for member samples, but the difference is not statistically significant. indicating weak discriminative power, suggesting the benchmark model does not substantially reveal membership status. These results are based embeddings extracted from 1024 tokens of an individual trajectory, yet no clear separation emerges. Importantly, membership inference would become far more concerning if successful with substantially less information about the individual.
To meaningfully assess patient privacy risk, we should also evaluate MI at the individual level, focusing on which specific pieces of information trigger the model to reveal membership. Simply detecting membership does not alone indicate a privacy risk, especially in models trained on de- identified data. In order to pass T4, any prompt that results in a successful membership inference should not uniquely identify individual. If it does, for example the prompt includes unique diagnosis, this puts the individual in significant privacy risk.
5 Objective II: Evaluating Risk of Memorization
Most efforts to quantify the privacy implications of large foundation models focus on measuring information leakage 37.. While leakage is undesirable, our tests show that it doesn't 't always translate directly into patient privacy risk in healthcare. In this setting, leaking single sensitive code about a patient can be much more concerning than exposing an entire blood panel. Hence, in our evaluation of memorization in EHR-FMs after going through T1-T4 to quantify memorization and assess the information leakage, we introduce two tests to investigate memorization privacy risk through: (1) Understanding whether the information leakage was the result of the model memorizing peteent-level information or the model learning generalizable knowledge. (2) Evaluating sensitivity of different subgroups to memorization where information leakage can pose significantly different privacy risk.
5.1 T5 Perturbation test
T5 evaluates whether adversarial prompts (such as the ones that identified sensitive attributes in T2) trigger patient-specific memorization or broader statistical patterns. A sample is memorized and not generalized if the model output is sensitive to personal identifiers. We test for this by constructing a set of perturbed prompts, where we modify personal identifiers such as age, ethnicity, or rare diagnosis codes, while keeping the rest of the prompt unchanged. We deliberately select these identifiers to reflect attributes unique to the target individual. We then re-evaluate the model's behavior on the perturbed prompts to determine whether the information leakage stems from memorization of a specific training individual or from generalizable patterns. If the model stops producing sensitive information after altering the identifiers, it signals that the original output resulted from memorizing
8

---

an individual instance. These cases indicate high-risk memorization and call for targeted safeguards to protect patient-level privacy.
Predicted probability infectious prompt)
Predicted probability substa nce
aaclaanne__aaaaaaaaaaeeeeaeee abuse (10 promptt
Probability
0.4
Probabiiity
0.4
0.2
0.2
0.0
70 0.0
20
30
40 50
60
20
30
40
50
60
70
Age
Age
Figure 4: Predictive results for the perturbation test on identified samples from infectious disease and substance abuse. Each bar represents the probability of generating the sensitive token, when the model is prompted with the 10 prompts and with different ages. The horizontal line is the threshold of 0.3 used in T2, and the red vertical line represents the original patient age.
Analysis: Our benchmark model uses only age as a demographic input. For prompts that revealed sensitive attributes T2, we perturbed the age to test whether the model still predicts the sensitive diagnosis for people of different ages. Figure 4sowws two cases where the model correctly predicts Hepatitis C (left) and Alcohol abuse (right) for an individual whose age is marked by the vertical red line. Consistent predictions across individuals with similar ages suggest that the model relies on general trends for making the prediction rather than memorizing individual-specifee patterns. However, observe different outcomes when personal characteristics diverge. For example, in the case of substance abuse, predicted likelihoods drop for older individuals, revealing age-dependent patterns the model has leamne.. T5 is triggered when small changes to personal identifiers, such as age, cause significant shifts in prediction likelihoods. This sensitivity indicates that the model has memorized individual-specht information rather than generalizable patterns, posing a high risk to patient privacy. More examples of the perturbation test are shown in Figure g in the Appendix.
5.2 T6 Sub-population test
In healthcare, we must assess memorization not only at the individual level but also across subgroups. Certain demographic or clinical populations face elevated privacy risks due to the sensitivity or rarity of their data. With our subgroup memorization test, evaluate whether belonging to specific subpopulation increases the risk of memorization. For example, if an attacker prompts the model with rare condition code and the model leaks information about individuals with that condition, it exposes a serious privacy threat, one that targets small identifiable groups.
Eovet Langer sciess
Figure 5: Likelihood of predicting the codes related to sensitive attributes (Infectious disease, mental health, and substance abuse) in EHR trajectories generated by prompting the benchmark model with rare diagnoses.
We analyzed patients with rare diagnoses or procedures that appear only once in the training data. These individuals face a higher risk of re-identification due to the uniqueness of their records, as highlighted in genomic privacy research [18.. To test for memorization, we prompted the model with the patient's 's age and rare diagnosis procedure code, then applied the sensitivity test from T2 to detect any disclosure of sensitive attributes. Figure 5 shows the model's 's likelihood scores. Low scores suggest the model does not expose private information based on rarity alone, while high scores signal potential privacy concerns that require closer examination.
Analysis:
Our benchmark model revealed two concerning cases: (1) Sedative, hypnotic, or anxiolytic depen- dence, and (2) Mild manic episode without psychotic symptoms. These codes strongly indicate the sensitive attribute under test. Even if the model didn't 't directly memorize them, their presence in

---

high-liks -iheliidl outputs raises serious privacy concerns. If statistical associations cannot explain these predictions, they reflect memorization failures.
We also examine the elderly subgroup- patt--ts over 35- comprise about 5% of the MIMIC dataset. As long-tailed cohort, they
prone overfitting and require targeted evaluation. We apply tests TI-T5 to assess memorization relative to the general population (Table Appendix). While the ideal outcome remains no memorization, this subgroup analysis critical: if memorized instance does occur, the privacy risk is amplified because these patients belong to a more easily identifiable cohort. Any flagged cases therefore warrant heightened caution, since individuals in long-tailed groups are often more uniquely identifiable, even in de- de-identified datasets, underscoring the need for stronger privacy safeguards.
Discussion
We propose a new perspective evaluating EHR foundation model memorization, grounded in the practical realities of healtheare. As EHR-FMs integrate into clinical workflows for documentation, decision support, and analytics (22), they often operate sensitive subsets of patient records that carry high clinical relevance. these models leak memorized patient information, this compromises privacy, misleading clinical decisions [4, and enabling adversarial attacks, posing serious ethical and legal challenges.
High-profile breaches, such as Stanford Hospital's s 20M lawsuit over leaked records 21 underscore the urgency of systematically auditing models in healthcare, To address this, we introduce an evaluation framework for EHR-FM that measures memorization and its associated privacy risks. Our tests quantify different forms of memorization and assess their implications clinical settings, distinguishing harmful leakage at the patient level from useful generalization at the population level.
EHR-FMs continue evolve, and their capabilities grow, prioritizing privacy becomes critical. Our work equips developers with tools to detect memorization and sets the foundations for future mitigation strategies. Developers can use these tests to identify and address flagged samples, reduce memorization during training, and better understand risks, thresholds, and attacker strategies. Al- though our tests are not exhaustive, they provide a practical starting point. The open-souree code enables others to adapt, extend, and build toward more secure EHR foundation models. Follow up study will investigate such risks across broader model categories and stakeholder perspectives.
Acknowledgments and Disclosure of Funding
ST supported by the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard. LS was supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. MG supported by the National Science Foundation (NSF) 22-586 Faculty Farly Career Development Award (#2339381) ). a Gordon & Betty Moore Foundation award, Google Research Scholar award and the AI2050 Program Schmidt Sciences. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute.
10

---

References
[1] A. Act. Health insurance portability and accountability act of 1996. Public law, 104:191, 1996.
[2] G. Alain and Y. Bengio. Understanding intermediate layers using linear classifier probes, 2018.
[3] D. Arpit, S. Jastrzebski, N. Ballas, D. Krueger, E. Bengio, M. S. Kanwal, T. Maharaj. A. Fischer, A. Courville, Y. Bengio, et al. A closer look at memorization in deep networks. In International conference on machine learning, pages
PMLR, 2017.
[4] D. Barth-Jones. The're ree'eeeiidetiiiicatooon of governor william weld's medical information: critical re- re-examination of health data identification risks and privacy protections, then and now. Then and Now (July 2012), 2012.
[5] S. Biderman, U. Prashanth, L. Sutawika, H. Schoelkopf, Q. Anthony, S. Purohit, and E. Raff. Emergent and predictable memorization in large language models. Advances in Neural Infor- mation Processing Systems, 36, 2024.
[6] R. Bommasani,
Hudson, Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint www.:ii0...002220 2021.
[7] S. Bordt, H. Nori, Rodrigues, B. Nushi, and R. Caruana. Elephants never forget: Memoriza- tion and learning of tabular data in large language models. arXiv preprint 27X4v22404.06209 2024.
[8] Z. Bouraoui, J. Camacho-Collados and S. Schockaert, Inducing relational knowledge from bert. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34/ 05, pages 7456-7463, 2020.
[9] N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramèr, and C. Zhang. Quantifying memorization across neural language models. In ICLRR, 2023.
[10] N. Carlini, C. Liu, U. Erlingsson, J. Kos, and D. Song. The secret sharer: evaluating and testing unintended memorization in neural networks. In Proceedings of the 28th USENIX Conference on Security Symposium, SEC' 19, page 267-284, USA, 2019. USENIX Association.
[11] N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert- Voss, K. Lee, A. Roberts, T. Brown, D. Song. U. Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pages 2633-26521 2021.
[12] D. Chen, N. Yu, Y. Zhang, and M. Fritz. Gan-leaks: A taxonomy of membership inference attacks against generative models. In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, pages 343-362, 2020.
[13] C. A. Choquette F. Tramer, N. Carlini, and N. Papernot. Label-only membership inference attacks. In International conference on machine learning, pages 1964-1974. PMLR, 2021.
[14] A. Fallahpour, M. Alinoori, W. Ye, X. Cao, A. Afkanpour, and A. Krishnan. Ehrmamba: Towards generalizable and scalable foundation models for electronic health records. In Machine Learning for Health (ML4H), pages 291-307. PMLR, 2025.
[15] M. Fredrikson, E. Lantz, S. Jha. S. Lin, D. Page, and T. Ristenpart. Privacy in pharmacogeneties: An (End-to-End) case study of personalized warfarin dosing. In 23rd USENIX security symposium (USENIX Security 14), pages 17-32, 2014.
[16) K. Ganju, Q Wang, W. Yang. C. A. Gunter, and N. Borisov. Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pages (19,633, 2018.
[17] I. L. Guo, J. Fries, F. Steinberg, S. L. Fleming, K. Morse, C. Aftandilian, J. Posada, N. Shah, and L.. Sung. A multi-center study on the adaptability of shared foundation model for electronic health records. npj Digital Medicine, 7(1):111, 2024.

---

[18] M. Gymrek, A. L. McGuire. D. Golan, E. Halperin, and Y. Erlich. Identifying personal genomes by surname inference. Science, 33961117:321-924, 2013.
[19] V. Hartmann, A. Suri, V. Bindschaedler, D. Evans, Tople, and West. Sok: Memorization in general-purpose large language models. arXiv preprint arXiv:2310. 18362, 2023.
[20] J. Hayes, L. Melis, G. Danezis, and E. De Cristofaro. Logan: Membership inference attacks against generative models. Proceedings Privacy Enhancing Technologies, 1(1):133-152. 2019.
[21] Infosecurity Magazine. Stanford hospital faces $20 million lawsuit over patient data breach. Infosecurity Magazine, 2011.
[22] L. Y. Jiang, X. C. Liu, N. P. Nejatian, M. Nasir-Moin, D. Wang, A. Abidin, K. Eaton, H. A. Riina, I. Laufer, P. Punjabi, et al. Health system-scas scale language models are all-purpose prediction engines. Nature, 619(7969):357-366 2023.
[23] A. E. Johnson, I., Bulgarelli, L. Shen, A. Gayles, A. Shammout, S. Horng, T. J. Pollard, S. Hao, B. Moody, B. Gow, al. Mimic-iv, freely accessible electronic health record dataset. Scientific data, 10(1):1, 2023.
[24] N. Kandpal, E. Wallace, and C. Raffel. Deduplicating training data mitigates privacy risks in language models. In Proceedings of the 39th International Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research. PMLR, 2022.
[25] G. Kerrigan, D. Slack, and J. Tuyls. Differentially private language models benefit from public pre-training. In Proceedings of the Second Workshop on Privacy in NLP, pages 39-45, 2020.
[26] U. Khandelwal, O. Levy, D. Jurafsky, L. Zettlemoyer, and M. Lewis. Generalization through memorization: Nearest neighbor language models. In International Conference on Learning Representations, 2019.
[27] K. Krishna, G. S. Tomar, A. P. Parikh, N. Papernot, and M. lyyer. Thieves on sesame street! model extraction of bert-based apis. International Conference on Learning Representations, 2019.
[28] K. Lee, D. Ippolito, A. Nystrom, C. Zhang, D. Eck, C. Callison-Buree Burihh, and N. Carlini. Dedupli- cating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8424-8445 2022.
[29] E. Lehman, S. Jain, K. Pichotta, Y. Goldberg, and B. Wallace. Does BERT pretrained on clinical notes reveal sensitive data? In K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, and Y. Zhou, editors, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 946 959, Online, June 2021. Association for Computational Linguistics.
[30] D. Leybzon and C. Kervadec. Learning, forgetting, remembering: Insights from tracking Ilm memorization during training. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 43-57, 2024.
[31] S. Li, L. Yao, L. Zhang, and Y. Li. Safety layers aligned large language models: The key to Ilm security. In The Thirteenth International Conference on Learning Representations.
[32] Y. Li, S. Rao, J. R. A. Solares, A. Hassaïne, D. Canoy, Y. Zhu, K. Rahimi, and G. S. Khorshidi. BEHRT: transformer for electronic health records. CoRR, abs//907.00 095388, 2019.
[33] L. Liu, Y. Wang, G. Liu, K. Peng, and C. Wang. Membership inference attacks against machine learning models via prediction sensitivity. IEEE Transactions on Dependable and Secure Computing, 20(3):2341-233
2022.
[34] R. T. McCoy, P. Smolensky, T. Linzen, J. Gao, and A. Celikyilmaz. How much do language models copy from their training data? evaluating linguistic novelty text generation using RAVEN. Transactions of the Association for Computational Linguistics, 11:652-670, 2023.
12

---

[35] M. McDermott, B. Nestor, P. Argaw, and I. S. Kohane. Event stream gpt: a data pro-prcessinng and modeling library for generative, pre-trained transformers over continuous- time sequences of complex events. Advances in Neural Information Processing Systems, 24:24322-22334, 2023.
[36] L. Melis, C. Song, E. De Cristofaro, and V. Shmatikov. Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE symposium on security and privacy (SP), pages 691-706. IEEE, 2019.
[37] N. Mireshghallah, H. Kim, X. Zhou, Y. Tsvetkov, M. Sap, R. Shokri, and Y. Choi. Can llms keep a secret? testing privacy implications of language models via contextual integrity theory. The Twelfth International Conference on Learning Representations, 2023.
[38] M. Moor, O. Banerjee, Z.S. H. Abad, H. M. Krumholz, J. Leskovec, E. J. Topol, and P. Rajpurkar. Foundation models for generalist medical artificial intelligence. Nature, 616(7956):259-2666 2023.
[39] X. Morris, W. Zhao, T. Chiu, V. Shmatikov, and A. M. Rush. Language model inversion. In The Twelfth International Conference on Learning Representations, 2024.
[40] M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette- Choo, E. Wallace, F. Tramèr, and K. Lee. Scalable extraction of training data from (production) language models. CoRR, 2023.
[41] C. Pang, X. Jiang, N. P. Pavinkurve, K. S. Kalluri, E. L. Minto, J. Patterson, L. Zhang, G. Hripe- sak, G. Gürsoy, N. Elhadad, and K. Natarajan. Cehr-gpt: Generating electronic health records with chronological patient timelines, 2024.
[42] E. Perez, S. Huang, F. Song, T. Cai, R. Ring. J. Aslanides, A. Glaese, N. McAleese, and G. Irving. Red teaming language models with language models. In Proceedings of the 2022 Conference Empirical Methods in Natural Language Processing, pages 1448, 2022.
[43] L. Rasmy, Y. Xiang, Z. Xie, C. Tao, and D. Zhi Med-bert: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. NPJ digital medicine, 4(1):86, 2021.
[44] Renc, Y. Jia, A. E. Samir, J. Was, Q. Li,
Li, D. W. Bates, and A. Sitck. Zero shot health trajectory prediction using transformer. NPJ Digital Medicine, 7(1):266, 2024.
[45] M. Rigaki and S. Garcia. A survey of privacy attacks in machine learning, ACM Comput. Surv., 56(4), Nov. 2023.
[46] Y. Rubner, C. Tomasi, and L. Guíbas. A metric for distributions with applications to image databases. In Sixth International Conference on Computer Vision (IEEE Cat. No. No.98CH36271), pages 59-66,1 1998.
[47] W. Shi, A. Ajith, M. Xia, Y. Huang, D. Liu, T. Blevins, D. Chen, and L. Zettlemoyer. Detecting pretraining data from large language models, 2024.
[48] R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3-18. IEEE, 2017.
[49] C. Song and A. Raghunathan. Information leakage in embedding models. In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, pages 377-390, 2020.
[50] K. Tirumala, A. Markosyan. L. Zettlemoyer, and A. Aghajanyan. Memorization without overfitting: Analyzing the training dynamics of large language models. Advances in Neural Information Processing Systems, 35:38274-38290 2022.
[51] S. Tonekaboni, G. Morgenshtern, A. Assadi, A. Pokhrel, X. Huang, A. Jayarajan, R. Greer, G. Pekhimenko, M. McCradden, F. Chevalier, et al. How to validate machine learning models prior to deployment: silent trial protocol for evaluation of real-time models at icu. In Conference on Health, Inference, and Learning, pages 169-182. PMLR, 2022.
13

---

[52] M. Wornow, R. Thapa, E. Steinberg, J. Fries, and N. Shah. Ehrshot: An ehr benchmark for few-shot evaluation of foundation models. Advances in Neural Information Processing Systems, 2023.
[53] S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha. Privacy risk in machine learning: Analyzing the connection to overfitting, 2018 IEEE 31st computer security foundations symposium (CSF), pages 268-282 IEEE, 2018.
[54] A. Ziegler. Github copilot research recitation. The GitHub Blog, 2021.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract, and especially the final paragraph of the introduction (Section ! clearly articulate the goals of our work and explicitly state its contributions and key findings. The rest of the introduction situates our study within the context of relevant literature and prior work. In Figure 1, we provide visual summary of our objectives and outline the testing setup for assessing memorization risks in foundation models trained on EHR data, supported by illustrative examples.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The discussion Section 6 outlines limitations, noting that EHR foundation models are still evolving and our tests must be tailored to model- and dataset-specific characteristics.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
15

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: Our experiments mainly empirical and no proofs theory assumptions required.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We describe our test setup in detail Sections 3.3 and 3.2] including the benchmark implementation based on EHRMAMBA2 [14). Each of the six tests is introduced at the start of its subsection, with dataset- and model-specific details. For data and code see next section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
16

---

(d)
The authors should provide scripts to reproduce all experimental results for the new reproducible, they We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: In our work, we use the publicly available MIMIC-IV dataset
23 and
describe the benchmark model EHRMamba [14] in Section 3.3 The complete set of
test scenarios and code is available at: https: /anss///nnnmoussss 4open. science/ r emr
memorization- 3735/READOC md, with detailed setups described in Section 3.2 per-test descriptions in Sections 4.1-5.2.
including
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/nips..cc/publcc/uuut
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details for each test setup such the training data fraction used in prob- ing are provided in Sections 4. 1-45.. Additional information on the evaluation of Test 1, including the specific metrics used, can be found in Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
17

---

Answer: [Yes]
Justification: We compute error bars over multiple iterations, depending on the test, and visualize them the plots, see example in Figure 2a
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [NA]
The compute resources required to perform our tests depend on the inference cost of the EHR-FM, the method relies these models generate sequences for evaluation.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:///wrrpp.cpooaaaac/ccccccea
Answer: [Yes]
Justification: The authors confirm that they have read the NeurIPS Ethics Guidelines and prepared the manuscript accordingly.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
18

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses in detail the positive societal impacts of our framework for auditing potential memorization risks in CHR-founnation models and highlights the societal harm such risks are overlooked. We specifically emphasize these points in the introduction (Section D. discussion (Section b and throughout the analysis of our tests.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No such risks are identified.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
19

---

Answer: [Yes]
Justification: We cite the publicly available MIMIC-IV dataset 23 and describe the bench- mark model EHRMamba [1].
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of in the package should be provided. For popular datasets, pawerssi.ppppiiiipp.... paperssithcode. thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's ss creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide link to the Git repository containing our test code at: https::// // anonymous 4open wwwsecceeeeeeeeeeeee memorizationn -tt.r/.-1----//-AADDD555 md with detailed
test setups described in Section 3.2
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
20

---

Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]-
Justification: This work doesn't require IRB approval as it uses the publicly available MIMI-IV dataset with an approved IRB.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification:
Not applicable. LLMs have only been used for editing and writing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non ntn-standard components.
Please refer to our LLM policy h/mps::///euriii cc. for what should or should not be described.
21

---

A Validation of T1 similarity metric
In this paper, we present a similarity metric to measure the similarity between two sequences of structured EHR data, including diagnoses, medication records, lab measures, etc. All this information is stored as a sequence of codes (tokens), with special time tokens in between that indicate the time gap between the recorded codes. Our similarity metric needs measure the degree of similarity between codes, and the similarity in the sequence of events. Large language models trained on medical data successfully encode medical text into representations that preserve clinically meaningful semantics [43). We can leverage these models to quantify the similarity between EHR codes. This critical in measuring the similarity or distance between sequences of EHR records because the similarity between these codes can be dramatically different. For instance, the similarity between similar antibiotics should be higher than a pain medication and an antibiotic. Next, our metric should assess how 2 sequences have similar timeline, and for that, we use the time codes to penalize codes that are placed in different time windows. This penalty increases as the codes are further apart in the sequence. Note, that we can only penalize time depending on how granular timelines are processed in the EHR-FM. Our Benchmark, for instance, processes time with a granularity of one hour. This means all codes collected within an hour will be considered to have the same time.
JUS
14k (100 A171 (0l decee doorral
(RO0OSS001 an
Aaing nonton cobla
Dirave S.000
TORTORETT
um (44) 101///0/// Lanve
Coranos E0000
vadallate 10
Hokoort Maskaatt
oisgge de cituuto
-WV/WRI dually
6 100000--1E77 Autio
4VTV
Figure 6: An example of different synthetic patient records and how the proposed metric measures the distance between them. The first row represents our reference sample Patient 1. Each subsequent row represents a different trajectory, with the differences highlighted.
We demonstrate the functionality of our score using the EHR record of a hypothetical patient. A 76-year-old individual was admitted the ICU with a Urinary tract Infection (UTT) and eventually developed Sepsis. The first row of Figure 6 demonstrates the EHR record for this individual.
To show the sensitivity of our metric, measure the distance of this sequence to several sequences. created with careful perturbations as described below:
1. Randomly adding or removing codes to the EHR sequence to test sensitivity to sequence length.
2. Replacement of medications with clinically equivalent alternatives (for example, swapping antibiotics) or irrelevant medications to assess if the metric captures the different level of differences.
3. Modifying lab results to simulate errors, such as changing infection indicators from high to low.
22

---

4. Compare distances between completely different patients to assess the score for unrelated sequences.
The red boxes highlight the tokens that are perturbed with respect to the reference patient. As shown in Figure our metric effectively distinguishes clinically relevant changes patient trajectories. Replacing medications with similar alternatives results in smaller distances compared to using irrelevant medications. Altering lab values leads to greater distances due to their density and diagnostic relevance, while distances between unrelated patients are higher.
B Demonstrating Detection of Memorization with known ground truth (positive control experiment)
Validating positive controls in real-world EHR data is inherently challenging due the absence of ground truth regarding which samples may have been memorized by a model. To address this, we designed a controlled synthetic experiment that mimics the structure of EHR-like sequences and allows us to explicitly simulate memorization behavior. In this setup, each sequence consists of digits (analogous diagnosis codes), where higher digits occur less frequently, following skewed distribution (d+1)=. reflecting the prevalence imbalance commonly observed in clinical data. We implemented toy foundation model that learns embeddings from these sequences and generates future sequences based on them. We added further stochasticity to the generative process to simulate more complex setups like EHR-FM.
To simulate memorization, we encoded a deterministic rule in the model: if sequence begins with the digits [0,1], a specific embedding dimension is set to 1; otherwise, it remains O. The remaining dimensions are learned normally. In generation, this memorized embedding dimension directly affects the output; If set to 1, the model is forced to produce rare digit (e.g.. 9), emulating the behavior of memorizing rare or sensitive clinical cases.
Percentage of Data Used
Accuracy
AUROC
AUPRC
F1 Score
Precision
Recall
20%
0.931
0.934
0,906
0.924
0.995
0.862
10%
0.917
0.919
0.883
0.900
0.975
0.836
5%
0.875
0.916
0.860
0.833
0.922
0.759
1%
0.742
0.876
0.665
0.602
0.728
0.512
Table 2: T3 results on simulated temporal data for different size of training data.
We then applied our probing tests to this synthetic model to assess whether T3 could successfully detect memorized samples. presents the results, demonstrating how effectively an adversary can infer memorized content depending on the amount of information they have access to. hen the adversary has access limited data for training (1% setting), only 74% of cases are identified and leaked. When the probing model is trained on larger fraction of the input, its ability to recover memorized outputs improves substantially, reaching precision of 0.92. This confirms that our test reliably detects memorization under varying levels of adversarial knowledge.
We also validate the probing test our controlled synthetic setup using explicit positive controls. Specifically, we take memorized sequence, defined as one that begins with [0, IJ. which deter- ministically produces the rare digit in the generated output due to hard-coded influence in the embedding (Table 33. To test the sensitivity of our perturbation framework. we generate 9 additional sequences by moditying only the first digit (i.e.. replacing the 0 with each digit from I to 9), while keeping the rest of the sequence unchanged. For each perturbed input, we generate 1000 trajectories. The perturbation test T5 then evaluates which fraction of the generated sequences still includes the rare digit 9. In the results of T5, the sharp drop in the fraction of sequences containing the sensitive code, as compared to the original [0, 1] input, confirms that the model has memorized a specific pattern, and the test is capable of revealing this behavior reliably, highlighting the harmful memorization behaviour of the model.
23

---

Starts With
% of Sequences Containing
01...
100.00%
11...
2.54%
21...
2.42%
31...
2.32%
41...
2.41 %
51...
2.56%
61...
2,42%
71...
2.43%
81...
2,47%
91...
2.40%
Table 3: Perturbation test (T5) results on the simulated temporal data. First row shows a reference samples, and the rest of the rows demonstrate perturbations to that sample. The sharp drop in the percentage of sequences generating a rare code (9) indicates that T5 is identifying a memorization behaviour.
C Categories of sensitive medical conditions
In the main text, we highlight subset of attributes that are particularly sensitive due to their association with stigma, regulatory protection, and social taboos. Table/4lists the specific ICD-10 codes we use as representatives of such high-risk categories, covering infectious diseases, substance abuse, and mental health conditions. While the selection is not exhaustive and may vary across datasets or contexts, it provides a concrete example of how sensitive information can be categorized for analysis.
Category
Detail
Infectious Diseases Substance Abuse
HIV/AIDS, Tuberculosis, Hepatitis or C, Chlamydia Dependence abuse of alcohol, opioids, cocaine, heroin, cannabis, hallucinogens, or stimulants Schizophrenia, Bipolar Disorder, Personality Disorders, Paranoia, Brief Psychotic Disorder, Manic Episodes, Borderline Personality Disorder
Mental Health
Stress Disorder, Anorexia Nervosa,
Table 4: Categories of high-risk medical conditions considered sensitive in model inference. Their disclosure intentional or not through machine learning models may pose ethical, legal, or privacy risks for patients.
D Generative memorization in foundation models
In the main text, discuss subgroup memorization risks, with particular focus rare conditions, procedures, and elderly patients. the elderly subgroup. While no memorized instances were detected, the table illustrates the evaluation procedure and highlights why even small, long-tailed cohorts require close monitoring for potential privacy risks.
the detailed results of the sensitivity test (T2) for
24

---

Sensitive attribute
Patient prevalence
Prompt
AUROC
AUPRC
Precision
Recall
Positive prediction
count
Infectious
0.0328
Statics
0.500
0.033
0.000
0.000
0
disease
0.0476
10 codes
0.517
0.048
0.000
0.000
0.0112
20 codes
0.51
0.048
0.000
0.000
0
0.0120
50 codes
0.49
0.012
0.000
0.000
Substance
0.0574
Statics
0,500
0,057
0.000
0.000
0
abuse
0.0062
10 codes
0,516
0.007
0.000
0.000
0.0052
20 codes
0.515
0,006
0.000
0.000
0
0.0055
50 codes
0.465
0.006
0.000
0.000
Mental
0.0170
Statics
0,500
0.017
0.000
0.000
health
0.0162
10 codes
0.524
0.018
0.000
0.000
0
0.0163
20 codes
0.566
0.027
0.000
0.000
0.0153
50 codes
0.565
0.055
0.000
0.000
0
Table 5: Performance comparison of model inference for sensitive medical conditions in older patients 85) versus the general population, across different prompt strategies.
(age> 2
Sensitive
Prompt
AUROC
AUPRC
F1
Attributee
lest
0.1%
10%
20%
lest
0.1%
10%
20%
test
0.1%
10%
20%
Infectious
10 codes
0.5470
0.534
0.540
0.544
0.0687
0.067
0.068
0.068
0.0687
0.342
0.138
0,147
Diseases
codes
0.5547
0.540
0.540
0.544
0.0804
0.068
0.138
0.0804
0.273
0.092
0.105
50 cndes
0,5661
0.544
0,544
0.544
0.0955
0.068
0.147
0,147
0.0955
0.248
0.105
0.105
Substance Abuse
10 codes
0,5000
0,500
0,527
0.500
0.0668
0.067
0.077
0.067
0.0668
0.107
0.111
0.000
codes
0.5000
0.500
0.527
0.500
0.0668
0.067
0.111
0,000
0.0668
0.078
0.000
50 codes
0,5475
0,500
0.500
0.500
0.0890
0.067
0.000
0.000
0.0890
0.000
0.000
0.000
Mental Health
10 codes
0,5000
0.537
0,500
0.500
0.0610
0.069
0.061
0.0611
0.0611
0.113
0.000
0.000
codes
0,5000
0.537
0,500
0.500
0.0611
0.069
0.000
0.000
0.0611
0.000
0.000
0.000
50 codes
0.5664
0,500
0.500
0.500
0.1060
0.061
0.000
0.000
0.1060
0.000
0.000
0.000
Table 6: Evaluation metrics for different conditions across code lengths and training fractions.
E
Supplementary figures
Appendix Figure proovides an overview of the six memorization tests (TI-T6). It illustrates how group the tests into generative, embedding-basee and privacy risk assessment categories, highlighting their respective goals: reconstructing training data, probing embedding leakage, and distinguishing memorization from generalization.
pallent get Diagrosis
T2
Dows the fxreciisted Injectory include Diegrosis
T4
Visaa this patient the train oohort?
How similar is the forecasted trajectory to the ground truth pattent record?
Creediigg
AGU03
EHI-FM
74
to shenee St
sucal ageita
expetweet
Coaa and Huuth
Teooloos Ing for corne"
Figure 7: A hypothetical patient EHR trajectory fed into an EHR-FM model. The model generates representation Z well as a forecast of the future codes in the EHR trajectory. T1 measures the similarity of the generated and ground truth sequence using a medical L.M. T2 measures the likelihood of the model generating sensitive tokens in its trajectory. T3 and T4 investigate if Z is predictive of deficit health conditions or can determine membership of a sample in the pre-training
T5 evaluates whether adversarial prompts trigger patient-specific memorization or broader statistical patterns. To further illustrate this analysis, we include additional examples of perturbed prompts
25

---

and corresponding model outputs. These results complement Figure by showing how altering personal identifiers such as age affects predictions of sensitive diagnoses. Figures g and onward provide extended cases, highlighting instances where the model either continues to reveal sensitive information (suggesting memorization) or no longer does so (indicating generalization).
Predicted probabiiity of substance abuse 110 prompt)
Probabiity
Predicted probability of substance, abuse [10 prompi
Probabilky 02
0.
Age
Predicted probability of infectious (10 promet)
Prodicted probability of substance abuse 110 promot)
Probabiity
0.4
Q4
Age
Age
Predicted probability of infectious (10 prompt)
Predicted probabiiity of substaince abuse 110 prompt)
Probabilty
0.4
Probabiltty
94
0.2
0.0
50
Age
Figure 8: Examples of perturbed adversarial prompts for predicting Infectious disease (left column) and substance abuse (right column).
26


Reviews:
[{"review_id": "NcPaYJnzFU", "reviewer": "Reviewer_VPo3", "summary": "This paper introduces a framework for evaluating memorization risks in healthcare foundation models (EHR-FMs) trained on de-identified electronic health records (EHRs). The authors highlight concerns that these models, despite being trained on de-identified data, might still memorize and expose sensitive patient information. The proposed framework includes a suite of black-box evaluation tests, categorized into two main objectives:\n\n1.  Measuring Memorization (Objective I): This objective involves four tests (T1-T4) to quantify the extent to which models reveal training data through both generative and embedding memorization.\n2.  Evaluating Risk of Memorization (Objective II): This objective focuses on distinguishing between patient-level memorization and population-level generalization to assess actual privacy risks. It includes two tests (T5-T6).", "questions": "1. For the probing models in T3, what specific types of classifiers were trained on the embeddings, and what were their architectures and training procedures? How was the \"separate test cohort\" for training the probing model selected to ensure its representativeness and independence from the training data of the EHR-FM?\n2. Could the authors provide more detail on the specific \"N-codes\" used for prompting in T1 and T2? What kind of codes were these (e.g., common diagnoses, specific lab results, a mix)? How were they selected to ensure they represent realistic adversarial prompts, and what was the average length or complexity of these prompts in a real-world scenario?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nThe paper presents a comprehensive and much-needed framework for evaluating memorization risks in healthcare foundation models (EHR-FMs). \nQuality: The paper introduces a well-structured suite of six black-box evaluation tests (T1-T6) specifically designed for EHR-FMs, addressing both generative and embedding memorization. \nOpen-Source Toolkit: The release of an open-source toolkit facilitates reproducible and collaborative privacy assessments, which is vital for advancing research and practical application in healthcare AI.\nFocus on Vulnerable Subgroups: The inclusion of T6, which investigates memorization risks for specific, vulnerable subgroups (e.g., elderly, rare diagnoses), demonstrates a thoughtful consideration of ethical implications and potential disparities in privacy risks.\nClarity: The paper clearly outlines its two main objectives: quantifying different types of memorization and assessing associated privacy risks by distinguishing harmful memorization from benign generalization.\nSignificance: The framework's focus on identifying practical privacy threats in black-box EHR-FMs is highly significant, given the sensitive nature of health data and the potential for real-world harm from data leakage.\nGuidance for Model Developers: The proposed tests offer concrete methods for developers to identify vulnerable samples and subgroups, enabling proactive mitigation strategies (e.g., post-training safety layers, red-teaming, retraining) before public release of models.\nOriginality: Novel Framework for EHR-FMs: While memorization has been studied in LLMs , this paper is one of the first comprehensive investigations into memorization risks specifically for structured EHR data and EHR-FMs.\n\n** Weaknesses ** \nBenchmark Model Specificity: The evaluation is primarily conducted on EHRMamba2 , which is a specific type of EHR-FM. While this is a practical choice due to its public availability and reproducible training , the generalizability of the findings to other EHR-FM is not fully explored, although the framework is stated to be adaptable.\nDifficulty in Rigorous Assessment of Metrics and Tests: e.g. in T2, the specific codes within these categories and what constitutes \"sensitive information require expert clinical judgment and could be subject to varying interpretations across different evaluation teams or clinical contexts; in T1, While the d metric quantifies similarity, determining what level of distance constitutes a \"memorized\" or \"harmful\" trajectory in a clinically meaningful way is not explicitly defined and could be difficult to standardize across different evaluations.", "was_revised": false, "final_justification": "", "rebuttal": "*We thank the reviewer for their thoughtful feedback, their support for our open-source toolkit, and the inclusion of vulnerable subgroup analysis to address ethical concerns. All suggestions have been addressed to improve clarity and respond to specific comments.*\n\n**Benchmark Model Specificity:** As the reviewer noted, we selected EHR-Mamba2 for our main experiments because it is publicly available and has a fully reproducible training pipeline. EHR-Mamba2 is also representative of many EHR-FMs in its architecture, including token-based inputs, sequential modeling over visits, and hybrid attention mechanisms. Moreover, our framework is model-agnostic by design, making it applicable to a wide range of models beyond those evaluated here. \nWe have extended our analysis to include MEDS-torch, another EHR-FM benchmark [1]. Below shows a summary of the T1 results for predicting 20 steps ahead. The results follow similar trends to those of EHR-Mamba2, with only slight variation in “50 prompt” settings. We are happy to include full results in the appendix to illustrate the generality and applicability of our approach.\n\n-------------------------------------------------------------------------\n\n|\tPrompt Type   |\tMEDS-torch (20)\t|\n\n| **Random**      \t|\t0.701 +- 0.1376\t|\n\n| **Static**      \t\t|\t0.669 +- 0.149      \t|\n\n| **10 prompts**      \t|\t0.65 +- 0.162      |\n\n| **20 prompts**      \t|\t0.65 +- 0.1830      \t|\n\n| **50 prompts**      \t|\t0.677 +- 0.144    |\n\n-------------------------------------------------------------------------\n\nWe also want to note that our primary goal was to develop and validate an effective evaluation methodology, and not to focus on the benchmarking task. Currently, there are only a handful of models available, and they may not pose significant memorization concerns, the rapid advancement of EHR-FMs suggests that these risks may soon become more pressing. Raising awareness of these issues now is essential and we believe this work provides a strong foundation for future studies to assess memorization risks across a broader range of models.\n[1] https://github.com/Oufattole/meds-torch\n\n\n**Assessment of Metrics and Tests:** The reviewer raises two important points. First, regarding the clinical specificity of our tests, this is a deliberate and necessary design choice, not a limitation. Memorization in healthcare is nuanced and context-dependent, tightly interwoven with clinical semantics. In Tests T2 and T3, we focused on representative sensitive conditions (e.g., HIV, mental health, reproductive health), but sensitive conditions are not limited to conditions listed. The role of our tests is to quantify memorization risk under varying information scenarios (prompt information an adversary can gain access to). Final decisions about acceptable thresholds or policy responses are inherently contextual and should rest with institutional decision-makers or end-users, who must balance privacy risks against operational constraints.\n\nSecond, we agree that T1 is not an absolute measure or directly comparable across models. As emphasized in the paper, T1 offers an aggregate memorization signal to flag suspicious samples, especially relative to random baselines. However, it is most informative when paired with T2–T4, which provide more targeted privacy assessments. \n\n**Probing Test (T3) clarification:** We used a simple multilayer perceptron (MLP) classifier to probe the embeddings for memorization. In preliminary experiments, we evaluated Lasso, XGBoost, and MLP classifiers, selecting the best-performing model based on validation performance. The final probing was conducted using 10-fold cross-validation, repeated across multiple data splits with varying train/test ratios (e.g., 80/20). To represent varying levels of information that an adversary might possess, we simulate different settings by adjusting the *fraction of training data* and the *prompt lengths* used to generate embeddings. Specifically, we consider prompt lengths of 10, 20, and 50 tokens, and training fractions of the test set, 0.1%, 10%, and 20%. We assess the information stored in the embeddings by training probing models g(⋅) to predict sensitive attributes (e.g., diagnoses) from the embeddings. These probing models are trained using either external data or a subset of the original training data (again to simulate different adversary scenarios).\n\n**Prompt Design for T1 and T2:** For T1 and T2, the model was prompted with the first N codes (N = 10, 20, 50) of a patient trajectory. These included diagnosis codes (ICD-9), medications, lab test results (e.g., hematocrit levels), and demographic information (e.g., age, sex). For example, a prompt for a MIMIC patient might include:\n\n[Age: 48], [Sex: Male], [ICD9: 428.0 – CHF], [LAB: Creatinine High], [MED: Furosemide], ...\n\nWe focused on early-stage information to simulate realistic usage and avoid direct leakage of target codes.\nAn adversary could exploit publicly available clinical datasets (e.g., MIMIC-III), personal health disclosures on social media, or leaked insurance claims to construct plausible prompts. For instance, by knowing a patient is a 60-year-old female with diabetes and recent lab values, an attacker could generate a valid input sequence and prompt the model to elicit sensitive follow-up codes."}, {"review_id": "QRBfDnm0DH", "reviewer": "Reviewer_zyw8", "summary": "This paper presents a testing framework designed to evaluate the memorization of private and sensitive health information in models trained on structured electronic health record (EHR) data. The assessment is divided into three categories: generative memorization, embedding memorization, and privacy risk analysis. In the generative memorization test, the authors investigate whether models can regenerate training data through prompting, including whether sensitive details can be reconstructed. The embedding memorization test explores whether the model's embedding space leaks sensitive information or reveals membership in the training dataset. Finally, the privacy risk assessment focuses on individual-level memorization, evaluating whether specific samples reveal personal details or general medical conditions, and whether certain groups are more susceptible to memorization.", "questions": "1. Can you demonstrate the framework can capture memorization when it exists?\n\n**Formatting Issues**\n- L129: Double comma after [23]\n- Table1 caption: Double period.\n- Figure4 caption: … vertical line … → … red vertical line …", "limitations": "Missing a broader discussion about the limitations.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The proposed framework is conceptually straightforward and the tests are intuitive, making the approach accessible and potentially widely applicable. The authors evaluate the framework on a single model, EHRMamba2, and report no strong evidence of memorization. While the methodology and results are generally convincing, a key limitation is the absence of a known positive control, which makes it difficult to verify the sensitivity of the tests in detecting memorization. Additionally, evaluating only one model limits insight into how factors like model size might influence memorization behavior.\n\nSince the framework only requires a sequence of codes as input prompts, it could be adapted for use beyond healthcare. Given the challenges in accessing healthcare data, the authors could strengthen their claims by first demonstrating the framework’s effectiveness in a non-healthcare domain with a known ground truth. This would help establish the framework’s ability to detect memorization before applying it to EHR-trained models.", "was_revised": true, "final_justification": "Additional results with synthetic data and new model address most of my concerns. Therefore, I lean towards paper acceptance.", "rebuttal": "*We thank the reviewer for their valuable feedback and thoughtful comments. We appreciate the recognition of the importance of the problem and the acknowledgment that our framework is conceptually intuitive and accessible, which we believe strengthens its potential impact. The reviewer’s suggestions, particularly the request for positive control validations and clarifications on key points, were very constructive and insightful. In response, we have carefully addressed all comments and incorporated the recommended additions and explanations, as outlined in detail below.*\n\n**Demonstrating Detection of Memorization in with ground truth (positive control experiment)**\nThank you for this insightful suggestion. Validating positive controls in real-world EHR data is inherently challenging due to the absence of ground truth regarding which samples may have been memorized by a model. To address this, we designed a controlled synthetic experiment that mimics the structure of EHR-like sequences and allows us to explicitly simulate memorization behavior.\nIn this setup, each sequence consists of digits (analogous to diagnosis codes), where higher digits occur less frequently, following a skewed distribution $p=\\frac{1}{(d+1)^2}$reflecting the prevalence imbalance commonly observed in clinical data. We implemented a toy foundation model that learns embeddings from these sequences and generates future sequences based on them. We added further stochasticity to the generative process to simulate more complex setups like EHR-FM. \n\nTo simulate memorization, we introduced a deterministic rule: if a sequence begins with the digits [0,1], a specific embedding dimension is set to 1; otherwise, it remains 0. The remaining dimensions are learned normally. In generation, this memorized embedding dimension directly affects the output—if set to 1, the model is forced to produce a rare digit (e.g., 9), emulating the behavior of memorizing rare or sensitive clinical cases.\nWe then applied our probing tests to this synthetic model to assess whether they could successfully detect memorized samples. Below, we present the results of these tests, which demonstrate the ability of our framework to flag memorization under controlled, interpretable conditions.\n\n| Percentage of Data Used | Accuracy | AUROC | AUPRC | F1 Score | Precision | Recall | \n|-------------------------|----------|--------|--------|----------|-----------|--------|\n| 20% | 0.931 | 0.934 | 0.906 | 0.924 | 0.995 | 0.862 | \n| 10% | 0.917 | 0.919 | 0.883 | 0.900 | 0.975 | 0.836 | \n| 5% | 0.875 | 0.916 | 0.860 | 0.833 | 0.922 | 0.759 | \n| 1% | 0.742 | 0.876 | 0.665 | 0.602 | 0.728 | 0.512 | \n\n\nThese results illustrate how effectively an adversary can infer memorized content depending on the amount of information they have access to. For instance, when the probing model is trained on a larger fraction of the input (e.g., >20% of the sequence), its ability to recover memorized outputs improves substantially, reaching a precision of 0.92. This confirms that our test reliably detects memorization under varying levels of adversarial knowledge.\n\nWe also validate the probing test in our controlled synthetic setup using explicit positive controls. Specifically, we take a memorized sequence—defined as one that begins with [0, 1]—which deterministically produces the rare digit 9 in the generated output due to its hardcoded influence in the embedding. To test the sensitivity of our perturbation framework, we generate 9 additional sequences by modifying only the first digit (i.e., replacing the 0 with each digit from 1 to 9), while keeping the rest of the sequence unchanged. For each perturbed input, we generate 1000 trajectories.\n\nThe perturbation test T5 then evaluates what fraction of the generated sequences still include the rare digit 9. In the results of T5, the sharp drop in the fraction of sequences containing the sensitive code, as compared to the original [0, 1] input, confirms that the model has memorized a specific pattern, and the test is capable of revealing this behavior reliably.\n\n| Starts With | % of Sequences Containing 9 |\n|-------------|------------------------------|\n| 01...       | 100.00%                      |\n| 11...       | 2.54%                        |\n| 21...       | 2.42%                        |\n| 31...       | 2.32%                        |\n| 41...       | 2.41%                        |\n| 51...       | 2.56%                        |\n| 61...       | 2.42%                        |\n| 71...       | 2.43%                        |\n| 81...       | 2.47%                        |\n| 91...       | 2.40%                        |\n\nWe are happy to include an extended version of this study in the appendix to validate the performance of our tests on positive control samples.\n\n\n**Non-healthcare applications:** We agree that extending the framework to non-healthcare domains such as legal or financial text is a promising direction. However, our current focus is intentionally limited to healthcare due to the high-stakes nature of patient privacy and the unique risks memorization poses in this setting. Furthermore, the domain allows us to tie the notion of memorization to clinically meaningful definitions of harm. By examining how different types of information (e.g., demographics, visit patterns, clinical labs) contribute to leakage, we aim to provide decision-makers with practical tools to assess risk and shape responsible deployment strategies.\n\n**Evaluation on different models:** We want to note that our primary goal was to develop and validate an effective evaluation methodology, and not to focus on the benchmarking task. Currently, there are only a handful of models available, and they may not pose significant memorization concerns. The rapid advancement of EHR-FMs suggests that these risks may soon become more pressing. Raising awareness of these issues now is essential, and we believe this work provides a strong foundation for future studies to assess memorization risks across a broader range of models.\n\nNevertheless, we have extended our analysis to include MEDS-torch, another EHR-FM benchmark [1]. Below shows a summary of the T1 results for predicting 20 steps. The results follow similar trends to those of EHR-Mamba2, with only slight variation in “50 prompt” settings. We are happy to include full results in the appendix to illustrate the generality and applicability of our approach.\n\n[1] https://github.com/Oufattole/meds-torch\n\n-------------------------------------------------------------------------\n\n| Prompt Type   |\tMEDS-torch (20)\t|\n\n| **Random**         |\t0.701 +- 0.1376\t|\n\n| **Static**      \t\t|\t0.669 +- 0.149 |\n\n| **10 prompts**      \t|\t0.65 +- 0.162      |\n\n| **20 prompts**      \t|\t0.65 +- 0.1830     |\n\n| **50 prompts**      \t|\t0.677 +- 0.144    |\n\n-------------------------------------------------------------------------\n\n**Broader discussion about limitations:** Thank you for raising this point. While we currently discuss limitations and extensions within the context of each test, we agree that making them more explicit could improve clarity. For example, our choice to prompt the model with initial visit codes reflects a realistic adversarial setting involving partial data leakage. More targeted prompting—e.g., selecting high-yield codes—would require clinical insight that most adversaries are unlikely to possess. As noted in the paper, our prompting strategy is flexible and can be adapted to simulate different types of adversary access. We also provide recommendations for combining tests, such as T3 and T6, to capture sequential or time-dependent leakage patterns.\n\n**Formatting:** Thank you for pointing out the few typos; we updated them for the next version."}, {"review_id": "tEFTUqFUVs", "reviewer": "Reviewer_Zqbh", "summary": "This paper proposes a suite of tests to detect memorization in Foundation Models trained on structured EHR (Electronic Health Record) data. The goal of the authors is to quantify the risk of LLMs leaking private information. \n\nThe authors create 6 tests and validate them on EHRMamba, a prompt-based model trained on MIMIC-IV. They prompt the model to generate patient medical code sequences varying the amount of ground truth information given to the model in the input prompt. Four of the tests (T1, T2, T5 and T6) use directly the code sequences generated by the EHRMamba, while the other two (T3 and T4) use the embeddings generated by the LLM.", "questions": "# Questions/Suggestions\n\n1. Add a positive-control benchmark for each test\n\nThis is the main concern of the paper. For this method to be generalizable and used out-of-the-box in any medical LLM, there has to be evidence that these metrics are actually doing something and are able to detect memorization. For the paper to be accepted, there has to be statistical evidence for every test discriminates between positive and negative examples. There might be many ways of proving this, but one could be to overfit a model so that it always leaks information.\n\nIn general, claims should be backed up with more statistical rigor.\n\n2. Demonstrate generality\n\nIt would be ideal to run the experiments on at least another medical model. However, without fixing point 1, these experiments wouldn't have value.\n\n3. Present the prompts used\n\nThese might be in the attached code, but I couldn't access it. Regardless, it would be useful to have them in the paper too, so that results are reproducible and can be better analyzed.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n\n## 1. Critical problem addressed \n\nThe paper addresses the very relevant yet usually neglected problem of Large Language Models generating patient private information that they were trained on. A working toolkit to quantify and detect leaked information could become a standard diagnostic before releasing medical LLMs.\n\n## 2. Clear structure \n\nThe problem of memorization in healthcare LLMs is introduced clearly and the overall paper is well structured and easy to follow. \n\n## 3. Sound testing framework and setup \n\nThe definitions of generative and embedding memorization are sensible, and the proposed testing setup of varying the amount of private information shown to the model is a good starting point to test for leaked training data.  \n\n## 4. Original metric use \n\nThe use of Wasserstein distance to quantify the similarity between generated and real patient trajectories is interesting and original. \n\n# Weaknesses \n\n## 1. No positive-control validation and anecdotal evidence of tests working\n\nThere is no statistical evidence that the tests would actually detect cases of memorization.  \n\n### T1\n\nT1 is the only test that is stress-tested, but even then, only one synthetic perturbation example is shown in Appendix A. We don’t know if the scores shown would generalize or are just incidental.\n\n### T2\n\nFor T2, another single anecdotal example is presented in lines 211-213: “given a prompt about a 48-year-old individual transferred to the ED with a history of falls and discharged shortly after—representing minimal clinical detail—the model predicts alcohol abuse in 31% of the estimated trajectories.” How do we know that this is a real failure to begin with and not the model capturing underlying valid patterns in the data?\n\n### T3\n\nIn T3, in lines 236-237, the authors state: “AUROC values across all sensitive attributes remain around 0.5, indicating no clear memorization signal and suggesting random performance.” However, no positive example is presented. How do we know that the metric will work on positive examples? \n\n### T4\n\nIn T4, no positive example is shown. Additionally, Figure 3b is analyzed and the difference between both distributions in it is considered not statistically significant, but no statistical test is mentioned. \n\n### T5\n\nIn T5, in lines 283-285, again the authors state that “If the model stops producing sensitive information after altering the identifiers, it signals that the original output resulted from memorizing an individual instance.” but no proof is provided, and only negative examples are shown. \n\n### T6\n\nIn T6, two positive examples are presented in lines 314-318, but even the authors are unsure whether these are valid examples that show the metric working, as they state: “If statistical associations cannot explain these 318 predictions, they reflect memorization failures.”\n\n## 2. Experiments on single model\n\nExperiments have been conducted on one model + one dataset. Generality is untested.\n\n## 3. No mitigation proposed\n\nFramework stops at detection and no mitigation strategy is offered.\n\n## 4. Code not available\n\nThe code shows as not available to me, but this might be a technical issue unrelated to the authors.", "was_revised": true, "final_justification": "The authors have addressed the key concerns convincingly, and since the rest of the paper was already clearly and well written, this warrants a positive recommendation for acceptance.", "rebuttal": "We thank the reviewer for their insightful feedback and thoughtful comments.  The review included several constructive suggestions, such as validating our tests using controlled positive examples, which we have carefully incorporated and believe  have helped improved the paper. \n\n**Positive control experiment**\nThank you for this insightful suggestion. Validating positive controls in real-world EHR data is inherently challenging due to the absence of ground truth regarding which samples may have been memorized by a model. To address this, we designed a controlled synthetic experiment that mimics the structure of EHR-like sequences and allows us to explicitly simulate memorization behavior.\nIn this setup, each sequence consists of digits (analogous to diagnosis codes), where higher digits occur less frequently, following a skewed distribution $p=\\frac{1}{(d+1)^2}$reflecting the prevalence imbalance commonly observed in clinical data. We implemented a toy foundation model that learns embeddings from these sequences and generates future sequences based on them. We added further stochasticity to the generative process to simulate more complex setups like EHR-FM. \nTo simulate memorization, we introduced a deterministic rule: if a sequence begins with the digits [0,1], a specific embedding dimension is set to 1; otherwise, it remains 0. The remaining dimensions are learned normally. In generation, this memorized embedding dimension directly affects the output—if set to 1, the model is forced to produce a rare digit (e.g., 9), emulating the behavior of memorizing rare or sensitive clinical cases.\nWe then applied our probing tests to this synthetic model.\n\n| Percentage of Data Used | Accuracy | AUROC | AUPRC | F1 Score | Precision | Recall | \n|-------------------------|----------|--------|--------|----------|-----------|--------|\n| 20 | 0.93 | 0.93 | 0.90 | 0.92 | 0.99 | 0.86 | \n| 10 | 0.91 | 0.92 | 0.88 | 0.90 | 0.97 | 0.83 | \n| 5 | 0.87 | 0.91 | 0.86 | 0.83 | 0.92 | 0.75 | \n| 1 | 0.74 | 0.87 | 0.66 | 0.60 | 0.72 | 0.51 | \n\nThese results illustrate how effectively an adversary can infer memorized content depending on the amount of information they have access to. For instance, when the probing model is trained on a larger fraction of the input (e.g., >20% of the sequence), its ability to recover memorized outputs improves substantially—reaching a precision of 0.92. This confirms that our test reliably detects memorization under varying levels of adversarial knowledge.\nWe also validate the probing test in our controlled synthetic setup using explicit positive controls. Specifically, we take a memorized sequence, defined as one that begins with [0, 1], which deterministically produces the rare digit 9 in the generated output due to its hardcoded influence in the embedding. To test the sensitivity of our perturbation framework, we generate 9 additional sequences by modifying only the first digit (i.e., replacing the 0 with each digit from 1 to 9), while keeping the rest of the sequence unchanged. For each perturbed input, we generate 1000 trajectories.\nThe perturbation test T5 then evaluates what fraction of the generated sequences still include the rare digit 9. In the results of T5, the sharp drop in the fraction of sequences containing the sensitive code, as compared to the original [0, 1] input, confirms that the model has memorized a specific pattern, and the test is capable of revealing this behavior reliably.\n\n| Starts With | % of generated sequences Containing 9 |\n|-------------|------------------------------|\n| 01...       | 100.00%                      |\n| 11...       | 2.54%                        |\n| 21...       | 2.42%                        |\n| 31...       | 2.32%                        |\n| 41...       | 2.41%                        |\n| 51...       | 2.56%                        |\n| 61...       | 2.42%                        |\n| 71...       | 2.43%                        |\n| 81...       | 2.47%                        |\n| 91...       | 2.40%                        |\n\nWe are happy to include an extended version of this study in the appendix.\n\n\n**Clarification on tests:** The reviewer raised a few questions regarding each test which we would like to clarify here.\n\n**T1** - The examples provided in the Appendix for T1 are subsets of the datasets we used for validating T1. We used LLM-generated hypothetical patient records and multiple variations of each for the full validation. We are happy to include more examples in the appendix to further support T1 validation.\n**T2:** *The patient discussed in the paper was **not** an anecdotal example, but a true sample identified by T2 as memorized. The last column of Table 1 reports these positive samples as the number of individuals for whom sensitive conditions (e.g., HIV, mental health) were predicted with high confidence. \n\n**T3:** T3 is designed to assess how much private information an adversary could potentially extract from a model’s embedding, given access to varying amounts of training data. Near-random performance in this setting suggests that the adversary is unable to extract meaningful sensitive information from the embeddings, given the specific data access scenario.\n\nHowever, we acknowledge the reviewer’s valid concern on testing the capability of T3 on memorized samples. To address this, we included a simulated positive control experiment (see earlier response) that demonstrates the effectiveness of the test under known memorization conditions. \n\n**T6:** In this test, we identify two cases where knowledge of a rare diagnosis allows inference of a sensitive condition. While we cannot conclusively determine whether these reflect direct memorization, the consistent appearance of these codes in high-likelihood outputs is concerning in itself. It suggests that the model may be implicitly encoding sensitive attributes in ways not fully explained by statistical associations.\n\n**Evaluation on different models:** We want to note that our primary goal was to develop and validate an effective evaluation methodology, and not to focus on the benchmarking task. Currently, there are only a handful of models available, and they may not pose significant memorization concerns. The rapid advancement of EHR-FMs suggests that these risks may soon become more pressing. Raising awareness of these issues now is essential, and we believe this work provides a strong foundation for future studies to assess memorization risks across a broader range of models. Nevertheless, we have extended our analysis to include MEDS-torch, another EHR-FM benchmark. Please find the details in the response for reviewer zyw8 that shows a summary of the T1 results. \n\n\n**Mitigation strategy:** We appreciate the reviewer’s interest in mitigation, which is indeed a critical area. However, the primary objective of this work is to provide a comprehensive strategy for identifying memorization instances that are potentially harmful for patient privacy in EHR foundation models. Proposing concrete mitigation strategies is out of scope for this work, as the root causes of memorization vary widely, ranging from data duplication, skewed sampling, and rare event bias to architectural and training design decisions. Each of these factors requires a targeted mitigation strategy.\n\nMoreover, mitigation decisions are often application- and policy-dependent, influenced by factors such as the sensitivity of leaked attributes, the risk of re-identification, regulatory frameworks, and institutional tolerances for risk. Our goal is to equip data custodians and policy makers with quantitative tools to assess risk as a function of the potential information an adversary can gain access to. For instance: T1, T2, and T6 demonstrate what an adversary might infer with access to partial knowledge (e.g., a few diagnosis codes) from an individual.\n\nT3 models the leakage potential when the adversary gains access to the embeddings or limited training samples.\n\nT4 estimates whether an adversary can determine if a given patient was part of the training cohort, based on access to a partial view of their medical records.\n\nThis level of granular evaluation helps decision-makers define appropriate use boundaries—for example, determining whether a model is safe for public release, or if it should remain restricted to internal clinical environments.\nThat said, we do briefly discuss possible mitigation directions, including post-hoc safety layers, red-teaming, and retraining with privacy-preserving objectives. However, evaluating the trade-offs between these interventions and model utility, especially at the scale of foundation models, is out of scope for this work. We certainly hope this work will draw the attention of researchers to investigate and develop novel mitigation strategies tailored to foundation models in sensitive domains.\n\n**Code availability:** We apologize for this problem. The original anonymous link had issues with displaying some files. This issue has now been resolved, and the full code is accessible through the same link. Please let us know if you encounter any further issues.\n\n**Prompts and Example Availability:** Due to the data usage agreements associated with the MIMIC dataset, we are not permitted to publicly release specific examples of patient EHR records. Even when describing illustrative cases in the paper, we intentionally keep the details vague and high-level to avoid exposing any identifiable information. However, all the tests and examples presented in our work are fully replicable for researchers who have authorized access to the MIMIC dataset."}, {"review_id": "bIDTFR86Pa", "reviewer": "Reviewer_Reek", "summary": "The paper presents a suite of tests to study memorization of foundational models in EHR prediction tasks. Some initial experiments are reported.", "questions": "Is there empirical evidence that the tests actually reflect differences between models that are controlled for memorization, sensitive attributes, or spurious correlations? \n\nAre there analyses with a more substantial set of prompting and cases? The present tests seem to be limited.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 1, "originality": 3, "strengths_and_weaknesses": "About the general problem setup:\n\nThe paper tackles an interesting problem of studying the performance of foundational medical coding models and their tendency to be biased under several measures.\n\nThe paper is concerned with the models that reveal sensitive information by memorization. I am not sure if I can fully agree with this concern. Generating a trajectory of treatments with a statement that the greater the input information value, the more likely the model is to create the patient trajectory or sensitive output. Isn’t this contradictory? Wouldn’t we want the model to provide accurate trajectories with precise background information? Isn’t it the whole point that medical treatments should follow the prognosis and medical history, and each patient should get the most fitted treatment trajectory — thus, if the information is exactly the same as with another patient, then the trajectory should be identical? Surely, with less information, the model will likely make predictions that are less precise.\nFor instance, the HIV example shows that the model predicts codes that are sensitive, but it does so with some likelihood and based on valid input information. What is the problem here? I think spurious correlations will be likely to exist in any model in some quantity, but the examples given did not convince me that the test suite would reveal something that we could not expect. \n\nResults and their significance\n\nIn general, the manuscript does not communicate the severity of the problem, and the results are not discussed and compared across different models, methods to control for bias, and spurious correlation controls. Many of the results are unclear. For instance, T2: not clear what exactly are the results presented in Table 1 and how they have been computed. The same issue is with the tables in the Appendix. \n\nThe paper has a great initiative, but I think that the identification of the actual problems that may arise need to be sharpened and the empirical results need to be strengthened to include envidence of the tests capturing effects when the problems are controlled (e.g. test a model where sensitive controls are present in training, finetuning or post-processing to improve interpretability vs. a model where these are not present).", "was_revised": true, "final_justification": "Thank you for the detailed rebuttal and the additional experiments. I don't have any additional major concerns, and I appreciate the clarifications. I will raise my score, but I maintain a slightly negative position.", "rebuttal": "We thank the reviewer for recognizing the importance of investigating the behavior of foundation models in medical coding. We have carefully addressed all suggestions to improve clarity and respond to the specific comments below.\n\n**Generalization vs. Memorization and Data Leakage Risks:**\nWe thank the reviewer for raising this important point. We fully agree that clinical prediction models should leverage informative patient histories to generate accurate and personalized treatment trajectories. In fact, part of the main purpose of our work is to draw attention to the distinction between model generalization and harmful memorization that can leak sensitive information from training data, even when prompted with partial or non-specific inputs.\n\nThe concern is not that models generate plausible sequences given rich clinical histories, which is expected. Rather, the risk arises when high-confidence predictions of sensitive conditions (e.g., HIV status, substance use) can be elicited from incomplete or generic inputs. This may signal that the model is not generalizing from population-level patterns but is memorizing individual cases seen during training. For instance, in our substance use example, a patient with repeated falls was assigned a sensitive diagnosis with no clear clinical justification in the input. This suggests the model may be reproducing memorized associations from similar training examples, raising serious privacy concerns.\n\nOur framework is explicitly designed to detect such privacy leakage through memorization. Tests T1–T4 focus on scenarios where an adversary, with limited but plausible access to background information (e.g., demographics, early labs), could prompt the model and recover sensitive attributes about a patient. Such inputs could realistically be inferred from public EHRs (e.g., MIMIC-III), leaked insurance claims, or personal disclosures on social media. For instance:\nT1, T2, and T6 demonstrate what an adversary might infer with access to partial knowledge (e.g., a few diagnosis codes) from an individual.\n\nT3 models the leakage potential when the adversary gains access to the embeddings or limited training samples.\n\nT4 estimates whether an adversary can determine if a given patient was part of the training cohort, based on access to a partial view of their medical records.\n\nTo distinguish generalization from leakage, we apply perturbation-based tests: if the model still outputs rare, sensitive codes when key identifier information is removed or substituted, this behavior is less likely due to robust reasoning and more likely due to memorized correlations or training data artifacts. These patterns are not always evident without systematic probing, which is precisely what our framework enables. We will further clarify this distinction in the next version to ensure the motivation behind our framework is made explicit.\n\n**Empirical Evidence of Memorization in a Controlled Setting:** We demonstrate that our framework can detect memorization in a non-healthcare domain with known ground truth, serving as a positive control experiment.\nThank you for this insightful suggestion. Validating positive controls in real-world EHR data is inherently challenging due to the absence of ground truth regarding which samples may have been memorized by a model. To address this, we designed a controlled synthetic experiment that mimics the structure of EHR-like sequences and allows us to explicitly simulate memorization behavior.\nIn this setup, each sequence consists of digits (analogous to diagnosis codes), where higher digits occur less frequently, following a skewed distribution $p=\\frac{1}{(d+1)^2}$reflecting the prevalence imbalance commonly observed in clinical data. We implemented a toy foundation model that learns embeddings from these sequences and generates future sequences based on them. We added further stochasticity to the generative process to simulate more complex setups like EHR-FM. \nTo simulate memorization, we introduced a deterministic rule: if a sequence begins with the digits [0,1], a specific embedding dimension is set to 1; otherwise, it remains 0. The remaining dimensions are learned normally. In generation, this memorized embedding dimension directly affects the output—if set to 1, the model is forced to produce a rare digit (e.g., 9), emulating the behavior of memorizing rare or sensitive clinical cases.\nWe then applied our probing tests to this synthetic model to assess whether they could successfully detect memorized samples. Below, we present the results of these tests, which demonstrate the ability of our framework to flag memorization under controlled, interpretable conditions.\n\n| Percentage of Data Used | Accuracy | AUROC | AUPRC | F1 Score | Precision | Recall | \n|-------------------------|----------|--------|--------|----------|-----------|--------|\n| 20% | 0.931 | 0.934 | 0.906 | 0.924 | 0.995 | 0.862 | \n| 10% | 0.917 | 0.919 | 0.883 | 0.900 | 0.975 | 0.836 | \n| 5% | 0.875 | 0.916 | 0.860 | 0.833 | 0.922 | 0.759 | \n| 1% | 0.742 | 0.876 | 0.665 | 0.602 | 0.728 | 0.512 | \n\n\nThese results illustrate how effectively an adversary can infer memorized content depending on the amount of information they have access to. For instance, when the probing model is trained on a larger fraction of the input (e.g., >20% of the sequence), its ability to recover memorized outputs improves substantially, reaching a precision of 0.92. This confirms that our test reliably detects memorization under varying levels of adversarial knowledge.\nWe also validate the probing test in our controlled synthetic setup using explicit positive controls. Specifically, we take a memorized sequence—defined as one that begins with [0, 1]—which deterministically produces the rare digit 9 in the generated output due to its hardcoded influence in the embedding. To test the sensitivity of our perturbation framework, we generate 9 additional sequences by modifying only the first digit (i.e., replacing the 0 with each digit from 1 to 9), while keeping the rest of the sequence unchanged. For each perturbed input, we generate 1000 trajectories.\nThe perturbation test T5 then evaluates what fraction of the generated sequences still include the rare digit 9. In the results of T5, the sharp drop in the fraction of sequences containing the sensitive code, as compared to the original [0, 1] input, confirms that the model has memorized a specific pattern, and the test is capable of revealing this behavior reliably.\n\n| Starts With | % of Sequences Containing 9 |\n|-------------|------------------------------|\n| 01...       | 100.00%                      |\n| 11...       | 2.54%                        |\n| 21...       | 2.42%                        |\n| 31...       | 2.32%                        |\n| 41...       | 2.41%                        |\n| 51...       | 2.56%                        |\n| 61...       | 2.42%                        |\n| 71...       | 2.43%                        |\n| 81...       | 2.47%                        |\n| 91...       | 2.40%                        |\n\nWe are happy to include an extended version of this study in the appendix to validate the performance of our tests on positive control samples.\n\n**Clarification on Tables:**  In T2 results we report the percentage of individuals for whom the model reveals a sensitive diagnosis code in the generated EHR trajectories. We define a \"revealed\" case as one in which more than 30% of the generated sequences contain the sensitive code, indicating a high likelihood of unintended disclosure. These results are presented across varying levels of prompt, simulating scenarios where an adversary has access to different amounts of information about a patient. This setup allows us to estimate the risk of sensitive information leakage as a function of adversarial knowledge. The tables in the appendix show the same test for different subgroups (for instance elderly population). This helps us understand if the risk varies across subpopulations. \n\n**Evaluation on different models:** We want to note that our primary goal was to develop and validate an effective evaluation methodology, and not focus on the benchmarking task. Currently, there are only a handful of models available and they may not pose significant memorization concerns, the rapid advancement of EHR-FMs suggests that these risks may soon become more pressing. Raising awareness of these issues now is essential and we believe this work provides a strong foundation for future studies to assess memorization risks across a broader range of models.\nNevertheless, We have extended our analysis to include MEDS-torch, another EHR-FM benchmark [1]. Below shows a summary of the T1 results for predicting 20 steps ahead. The results follow similar trends to those of EHR-Mamba2, with only slight variation in “50 prompt” settings. We are happy to include full results in the appendix to illustrate the generality and applicability of our approach.\n\n[1] https://github.com/Oufattole/meds-torch\n\n-------------------------------------------------------------------------\n\n|\tPrompt Type   |\tMEDS-torch (20)\t|\n\n| **Random**      \t|\t0.701 +- 0.1376\t|\n\n| **Static**      \t\t|\t0.669 +- 0.149      \t|\n\n| **10 prompts**      \t|\t0.65 +- 0.162      |\n\n| **20 prompts**      \t|\t0.65 +- 0.1830      \t|\n\n| **50 prompts**      \t|\t0.677 +- 0.144    |\n\n-------------------------------------------------------------------------"}]

Author rebuttal:

Meta-review: Reviewers are generally positive and the authors did a good job at addressing most of the concerns in the rebuttal phase. I encourage the authors to make an effort to address all remaining concerns in a camera ready version.
Decision: Accept (poster)