Information-Theoretic Membership Inference for Granular Quantification of Memorization
Authors: Jiashu Tao
Reza Shokri
Venue: ICLR 2026 Poster (2026)
Keywords: membership inference attack
mia
privacy
llm privacy
memorization
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce a new state-of-the-art membership inference attack, InfoRMIA that dominates RMIA on all benchmarks. We also propose a token-level attack framework that has high power and can pinopint info leakage to token levels.

Abstract: Machine learning models are known to leak sensitive information, as they inevitably memorize (parts of) their training data. This risk is amplified for large language models (LLMs), which are trained on massive corpora and therefore create a more urgent need for privacy assessment prior to release. The standard method to quantify privacy is via membership inference attacks, where the state-of-the-art approach is the Robust Membership Inference Attack (RMIA). In this paper, we introduce \textbf{InfoRMIA}, a principled information-theoretic formulation of membership inference that consistently outperforms RMIA across benchmarks while improving computational efficiency. Moving beyond attack performance alone, we show that treating sequence-level membership inference as the gold standard obscures how memorization manifests in LLMs. To address this limitation, we propose a fine-grained memorization assessment framework based on token-level signals, with InfoRMIA serving as its algorithmic backbone. Our approach identifies which tokens within generated outputs are memorized, localizing privacy leakage from sequences to individual tokens. This framework enables more precise analysis of LLM memorization and potentially targeted mitigation strategies such as exact unlearning.

Published as a conference paper at ICLR 2026
INFORMATION-THERREE MEMBERSHIP INFERENCE FOR GRANULAR QUANTIFICATION OF MEMORIZATION
Jiashu Tao and Reza Shokrit
i National University of Singapore, Google Research (jiashut, reza @ezmp.comp.. nus edu. sg
ABSTRACT
Machine learning models are known leak sensitive information, they in- evitably memorize (parts of) their training data. This risk is amplified for large language models (LLMs), which are trained on massive corpora and therefore create a more urgent need for privacy assessment prior to release. The standard method to quantify privacy is via membership inference attacks, where the state- of-the-art art approach is the Robust Membership Inference Attack (RMIA). In this paper, we introduce InfoRMIA, a principled pntemmatiin-thhoore formulation of membership inference that consistently outperforms RMIA across benchmarks while improving computational efficiency. Moving beyond attack performance alone, we show that treating sequence-level membership inference the gold standard obscures how memorization manifests LLMs. To address this limi- tation, we propose fine- fine-grained memorization assessment framework based on token-level signals, with InfoRMIA serving as its algorithmic backbone. Our ap- proach identifies which tokens within generated outputs are memorized, localiz- ing privacy leakage from sequences to individual tokens. This framework enables more precise analysis of LLM memorization and potentially targeted mitigation strategies such as exact unlearning.
INTRODUCTION
In the past decade, researchers have shown that machine learning (ML) models inevitably memorize parts of their training data (Feldman) 2020 Feldman Zhang 2020). Memorized data, once identi- fied and extracted, can pose severe privacy risk. It is increasingly concerning as the contemporary, easily accessible large language models (LLMs) are trained on datasets so large that we are running out of training data (Villalobos et al.) 2024). These LLMs have seen nearly all data generated by humans. Even limited memorization by them can translate into significant privacy risks.
The current standard for quantifying privacy is membership inference attacks (MIAs) (Shokri al. 2017]. where the attacker or privacy auditor aims to determine if a given data sample was part of the target model's training set. A stronger attack means the attacker can more accurately distinguish members (training data) from non-members, implying greater information leakage from the model. This ability to separate members from non-members not only signals privacy risk but also raises the possibility of training data reconstruction. It is also closely linked to memorization, as it is the root cause of successful MIAs. Hence, MIAs are widely regarded as the backbone of ML privacy research. The srate-of-the-an (SOTA) MIA is the Robust Membership Inference Attack (RMIA) (Zarizzadeh al. 2024), but its reliance on separate population dataset, whose size scales linearly with the training set, could be a potential limitation, especially for LLMs.
In the first part of the paper, we thoroughly analyze RMIA, from its formulation to signal computa- tion, and propose more principled statistical test by casting RMIA's setup as composite hypoth- esis testing problem. Our approach can also be interpreted through information theory, where we quantify dominance over population data in bits rather than in sample counts. This transforms the attack signal from discrete to continuous, eliminating sensitivity to the population dataset size. We observe that our attack, InfoRMIA, consistently outperforms RMIA on tabular, image, and text datasets, while requiring far fewer population samples. Thus, InfoRMIA is a lower-cost, higher- power membership inference attack and establishes new SOTA.

---

Published as a conference paper at ICLR 2026
Average score (private tokens only)
0.5
345 (sampli 2200 axg-0-45), 2xg_
0.4
1982. With the Eeggeimeejely Sighirt com her communicationn icalion clarne sie he #2TAt//mmii ite ladgg of honer this vintual war the oor 784-5305 ahoays at ready stratagic ciecuratioe ielow determentiion Joning on this
0.2
challanges emoodying garning adventur
navigates through
Seq 368 (sample Indes 16422 avg 440, avg wgg.rnc-0 182)
"entry date": 2204911 200-:00000000 "entry time:: AM "lccarion" *Bs 458", "ochawiers" iMPoctieed cistress interance technigues" "Used inte prespprronal effectireness skiss "Rovlewed dary cards"), "roactions" empowered by disress sereesilleanne "1dontiic "Succesenuly interporoonal
-
Pearson 0.497
review ("ontry 8, "user_ ln:: otllnaguwuzte0s: "passport_ Jott "0"
0.0
0.1
0.2
0.3
0.4
T
(b) Token-level membership heatmaps obtained from our fine-grained framework reveal that sequences with the highest sequence-level scores primarily memorize non-private tokens.
Average score (all tokens)
(a) Sequence-level membership scores show weak correlation with the scores of private tokens.
Figure 1: Sequence-level membership inference can misestimate privacy leakage, sensitive in- formation is often localized to a small subset of private tokens. The two subfigures illustrate that sequence-level auditing results may diverge substantially from token-level privacy assessments en- abled by our framework. Results are obtained using InfoRMIA on GPT-2 models fine-tuned on the ai4privacy dataset (Appendix F.2) which provides privacy masks indicating private token locations.
Although MIAs the gold standard in quantifying privacy, they follow striet setup defined by the membership inference game (Yeom et al. [2018] Ye et al.] [002] Zarifzadeh et al., (024), which falls short in quantifying true information leakage (Tao & Shokri 2025), especially for LLMs. The cur- rent privacy quantification setup for LLMs largely inherit designs developed for MLPs and CNNs: perform MIA on a set of member and non-member sequences. However, transformers operate se- quentially, generating predictions token by token. Assigning a single membership label to an entire sequence compresses rich token-based information into single bit, losing granularity and analogous to lossy compression (Figure Tap.
To address this limitation, we propose a token-level MIA framework for fine fine-grained analysis of memorization and information leakage in LLMs. We motivate token-level analysis for three key reasons. First, each token completion corresponds to a single prediction step in autoregressive trans- formers, making token-level evaluation naturally aligned with model behavior and the definition of membership inference. Second, sequence-leve membership scores are inherently aggregated from token-level signals, making them less semantically meaningful for privacy assessment. For exam- ple, memorized facts may yield high sequence- based membership scores despite not leaking private information. Third, argue that private information in sentence is usually contained in few words/tokens. Measuring the average memorization of the entire sequence mainly with common words leads inaccurate privacy assessment (Figure lb). Token-level analysis can narrow the focus to truly sensitive components, enabling more accurate privacy quantification. By pinpointing the information leakage individual tokens and words, we can potentially protect privacy more effec- tively by performing targeted machine unlearning, which would prevent unlearning useful informa- tion from non-private texts, while surgically removing the memorization of private information.
In summary, this paper makes the following contributions: (1) we propose InfoRMIA, a principled and efficient improvement over RMIA that achieves new SOTA performance; and (2) we introduce a token-level privacy assessment framework, which offers finer-grained insights into memorization and leakage LLMs, while achieving strong membership inference capabilities.
RELATED WORK
2.1 MEMBERSHIP INFERENCE
Membership inference Shokri et al. 2017) is a class of inference attacks that aims to determine if any given point query part of the training set of machine learning model 6. Since its inception, there has been significant progress in the inference strategies. Shokni et al. (2017) trained shadow models to predict the membership label directly. Yeom et al. (2018) proposed simpler
2

---

Published as a conference paper at ICLR 2026
approach that uses the loss values as the membership signal. To achieve higher inference accuracy, researchers have proposed multiple reference model-based membership inference tests to calibrate the raw signal the target query. Ye et al. (2022) trained set of reference models on the population dataset, and counted the number of them with lower probability on the target x: |Callini et al. (2022) constructed reference models that train with or without the target point to simulate two distributions of model outputs on the target æ: the IN and OUT distributions. Assuming Gaussians, the attacker computes the likelihood ratios under the two distributions as the membership signal. The state-of- the-art attack, RMIA (Zriizaddh et al. (024), improves further upon reference model-based attacks by counting how many similar data points each test point dominates.
Membership inference techniques on CNNs and MLPs can be adapted to work on LLMs, where the goal to predict if any given text sequence part of the training dataset. Due to the high computation cost to train reference models, L.LM-spccfficc MIAs tend to be reference model-free. Carlini et al. (202T) used entropy, or more easily, zlib (Gailly & Adler 2004) compression, to calibrate sequence-based membership likelihood. Mattern et al. (2023) compared the perplexity gap between the target and neighboring sequences, while (Shi et al. 2024) looked at the tokens with the smallest probability. Duan et al.] (2024) published a benchmark, MIMIR, to evaluate LLM MIAS and found that all of these methods perform poorly on pretrained LLMs. (Zhang et al. (2024) and Zhhng al. (025) improved upon the methods in MIMIR by incorporating additionalx calibration.
2.2 MEMORIZATION
Memorization of machine learning models is defined in a leave-one-out fashion by Feldman 2020) Due to its prohibitively high computation cost on L.MMs, many alternative definitions have been proposed. So far, verbatim memorization (Carlimi et al.. 2021 2023), which means the output se- quence exactly matches one of the training sequences, is the most popular notion. If the sequence generated verbatim when conditioned on given prompt, the memorization term is called dis- coverable (Carlini et al.] 2023 Nasr et al., 2023) or extractable memorization (Nasr et al.) 2023), depending on whether the prompt is crafted by an adversary. |Hayes et al.) (2025b) introduced their probabilistic variations, considering the stochastic nature of LLMs. There are other memorization notions such as k-eidetic (Carlini et al. 2021) and counterfactual memorization (Zhang et al. 2023).
3 IMPROVING RMIA WITH AN EHTORMATION-THEOREEE INSPIRED TEST STATISTIC
In this section, we first briefly explain the problem statement of membership inference under the current SOTA attack, RMIA (Zarifzadeh et al., 2024). We then introduce an improved version of RMIA, which we call information-theoreti stronger than the original RMIA across all datasets and thus establishes a new -tate-of-the--at.
RMIA (InfoRMIA). This new attack is consistently
3.1 MEMBERSHIP INFERANCE IN RMIA'S SETTING
In membership inference, the adversary, or the attacker, aims to infer whether any given query is part of the training data the target model 0. The adversary only has access to the model output, which can be treated p(x[0). The adversary is also assumed to be able to train reference models θ on datasets drawn from the training data distribution of 6, and to have a population dataset Z, which is also drawn from the same underlying data distribution. The formal definition is commonly described as an inference game (Yeom et al. 2018 Ye et al., 2022] Carlini et al. 2022 Zarifzadeh
et al. 2024 Tao & Shokri 2025) (See Appendix
3.2
THE ORIGINAL ROBUST MEMBERSHIP INFERENCE ATTACKS (RMIA)
Carlini al. 2022 argue that the optimal way to tackle the membership inference problem is to
frame it as hypothesis testing problem and then apply a likelihood ratio test (LRT). In RMIA,
Zarifzadch et al. (2024) formulate the hypothesis testing setup as:
Ho : The target model 0 is trained with one of the data E Z,
(1)
H1 : The target model 0 is trained with the given T.
3

---

Published as a conference paper at ICLR 2026
The original test statistic of the LRT in RMIA can be written
Test Statistic = ((((2)) p(0)2) 27)
where 0 is the target model, 2 is the target query, 2 is drawn from population Z the same data distribution the training data, and y is hyperparameter that serves as threshold.
In simple terms, RMIA counts the proportion of "similar" data z that the target x dominates. In practice, the test statistic is written as
Test Statistic Ps p(xx(0) p(210) p(x) p(z)
27)
1 Z
Σ =EZ
p(z]0) p(x)
p(z10) p(z)
27).
where I(-) is the identity function and the first equality follows directly from Bayes' Theorem.
Note that the formulation in Eqn makes the membership score a discrete value whose granularity depends the size of Z. Intuitively, the more data RMIA uses, the finer the "bins" become, and the more distinguishing and precise the signal gets. Empirically, Zarifzadeh et al. (2024) also reported this relationship between the size of Z and the attack performance. The empirical insight was that using Z of about 10% of the training set size is sufficient. However, for LLMs, even 10% represents an astronomical number of samples.
3.3 INFO-THEORETE RMIA (INFORMIA)
Instead of counting how much population data the target point dominates, measure how many bits the target point saves in explaining the target model relative to the population data in expectation. That is, want to measure
Eg [ ((0((()))) - (- p(0lz)) = -logp(0)>>) - E, log p(0
(4)
By applying the same Bayesian decomposition in Zarifzadeh et al. (024) and some basic manipu- lations, we can obtain the following equivalent formulation of our new test statistic in Eqn 4
p(0
p(x10)p(=)
Test Statistic
=
Σ p(z) log
p(0|z)
=
Σ p(=) log
p(= ()p(x)
ppxx ())
p(2)
log
+ Σ p(z)logg
(5)
p(x)
p(=10)
ppxx(0) p(x)
log
+ DKL (p(z) p|
Note that the formulation is only valid when E. p(2) D Hence, for an empirical or approximated (in RMIA's case) p(z), we need to normalize p(=)0(()>000))))))) p(2)/ p(z). Similarly, for the last step to hold, require that E. p(=[0) 1. For simplicity, we use p(=) and p(z[0) in the rest of the paper to denote the normalized distribution of population data z. this new test statistic is inspired by information theory, we refer to it as InfoRMIA. Its pseudocode can be found in Appendix
Interpretation of the test statistic
It is interesting that the test statistic has two parts:
1. log p(zl0) which measures the amount of information gain in explaining x given a model m(a)
0. This can be seen as the memorization of & by model 0.
2. DKL (p(2))lp(2())) which captures the distributional differences between the base proba- bilistic distribution of z and that conditioned on model 8. This is reminiscent general- ization analysis, as it reflects the changes in the model's predictive performance on 2's.
Actually, when attacking one single fixed model with a fixed population dataset, the attack performance unchanged even if E, p(z) o 1. This is because the test statisties would be reduced to E, p(2) log p(0]x) = C' C'log((0(rr) pff where C >. p(2) > O is constant. Hence, the test statistic would preserve the same total order among all x's log likelihood values.
4

---

Published as a conference paper at ICLR 2026
3.4 WHY INFORMIA SUPERIOR
Both test statistics of the original and InfoRMIA are principled approaches to solve the same hy- pothesis testing problem (Equ I derived from the same membership inference game (Appendix (A]. The original RMIA (Equation 3 performs multiple pairwise tests between H1 and each null hypoth- esis. Each test requires a threshold y. The final score is the proportion of null hypotheses rejected in all the pairwise tests. As mentioned before, this score is inherently discrete, with granularity determined by | and increments of [
InfoRMIA does not perform multiple pairwise tests. Instead, it opts for a more systematic approach.
Similar to what TTo & Shokri (2025) observed, the scenario described by Equation [ is composite hypothesis testing problem. One of the principled solutions is to use Bayes Factor TToo & Shokrij 2025 Jeffreys ) where compute the expected log likelihood of the composite null hypothe- sis by Eg [log p [] Now it becomes clear that InfoRMIA's test statistic (Equation corresponds to the log of the likelihood ratio when using the Bayes Factor.
Apart from using a more accurate and established test, InfoRMIA also supersedes the original RMIA by using continuous test statistic (See Equation 4 5). This results in significantly higher precision in the membership score and also eliminates the need for the hyperparameter 7, Since the granularity of the score is no longer dictated by the size of Z. InfoRMIA is much less dependent on large Z, significantly reducing computational overhead when / is fixed and lowering complexity by a constant factor. Experiment results in Section 6 validate these improvements.
Factors affecting the gap The performance gap between InfoRMIA and RMIA mainly depends on the "niceness" of the distribution of population signals p( p(=)0)/p(z) in RMIA. As the population signal distribution gets more even, the loss of precision in the discretization step (computing the percentile) decreases, thereby narrowing its performance gap with InfoRMIA.
4 TOKEN-LEVEI, INFORMIA FOR ATTACKING LLMS
We have now justified why InfoRMIA surpasses the original RMIA and becomes the new SOTA attack. We now propose our token-level framework where we can pinpoint information leakage and more truthfully estimate privacy risks with token-level InfoRMIA.
4.1 FROM SEQUENCES TO TOKENS
So far, membership inference and privacy risks for LLMs have been defined on the sequence level, i.e, whether a given sequence is a member. The majority of the LLM MIAs aim to compute a score on each sequence (Carlini et al. | Mattern al. 2023) based on its perplexity. However, delving into the mechanisms of LLMs, we can quickly realize that sequence is not one single output, but an ordered list of outputs. For example, given a training sequence x = { (.. the LLM 0 optimizes the losses f(22, 0(x; )), ((x3, (21122)) [(xx, ((z) (( TA- is more than one training sample; it resembles a dataset containing & 1 training (subsequence, label) pairs. To properly adapt existing MIAs to LLMs, should treat each token generation step, which "labels" each "prefixal" subsequence (subsequences from the start), as one prediction and compute its membership likelihood. In this way, for any sequence of length k, the LLM goes through k 1 prediction steps, and we should obtain - 1 membership scores. In comparison, the existing framework only computes single membership signal for each sequence, which is highly compressed signal that loses rich information at each token position.
)). Each sequence
The token-level framework also provides more realistic privacy notion for LLMs. Many re- searchers have pointed out that the current privacy definition via membership inference is too strict and not comprehensive enough, especially for language data, as only considers exact matches as privacy concerns (Tao & Shokri 2025 Duan et al. 2024). We believe that the pri- vacy risk of a text sequence primarily resides the tokens carrying the sensitive information. From an information- PrivBits EmeVrri - log p(x) < Σ. - log p(x). where Vyrio is the set of all privacy concerning tokens in the data. From the inequality, it is obvious that the existing privacy notion is treating all tokens in the member sequence as private, leading to inflated membership scores in evaluation. In
point of view, the total private information in bits can be computed by
5

---

Published as a conference paper at ICLR 2026
this process, the true information leakage can be diluted overshadowed the sequence level (Figure B), especially in long texts and documents. This masks the signals from the truly private to- kens and leads to inaccurate auditing results (since are evaluating the upper bounds). Moreover, a sequence-based analysis framework also fails to pinpoint the source of true leakage. This affects downstream tasks like unlearning: we cannot make the model forget the sensitive information, but rather entire documents that may contain useful general semantic knowledge.
With token-level framework, users compute leakage via every token completion. They can then easily visualize what tokens are memorized outputs and check if they are sensitive (Figure lb). For auditors who know where the personally identifiable information (PII) is, they can also choose to directly check the model's 's memorization extent on the corresponding tokens. We build this interface and will explain in detail in Section We want to highlight that we assume that users of the interface know what sensitive tokens are. This interface is not for automatically quantifying the privacy risk of system on a data distribution, but rather an inspection tool for knowledgeable users, such as data owners and privacy auditors, to diagnose fine-grained information leakage. More importantly this flexible framework can be adapted to audit n-gram privacy Duan et al. (2024); Liu et al. (2025), by summing up the token-based membership scores in any chosen n-gram as its memorization score.
4.2 LOVEN INFORMIA
Our token-level framework relies on an MIA that can operate on the token level. We propose to conduct InfoRMIA (Equation [5) at each token generation step, treating all tokens labels for their respective prefixal substrings, and then compute token-based score. Additionally, we no longer have to curate separate population dataset Z. Instead, we treat all possible tokens in the vocabulary other than the ground-truth x as z. For example, if the ground-truti is 3, then = {a E V * 3). where V is the vocabulary of the tokenizer. The pseudocode can be found in Appendix
C In this way, we have a data-dependent Z that removes the high cost of curating and computing an independent population dataset. This makes the attack more feasible, especially for pretrained LLMs.
We want to emphasize that since p(x(0) + ExEz E p(=10) and Ezev p(2) Evey Avge,,,p(2 (0 Avg Evev p(z)0ret) = L, we can have equivalent formulation of the test statistic in Eqn 4 that does not require normalization (full derivation in Equation 10):
p(0|zz)
Test Statistic
Σ p(2)
(6)
((()2))
#EZ
p(<x(0p((2)
p(z10)p(x) <<=x00p(>>)
Σ p(=) log
+ p(x) log
((=10)p<x)
[p(x(0)
p(2)
Σ p(>) log
Σ p(=) log SEV
(())
p(=|0)
zEV
p(xx(0) p(x)
= log
+ DKL (p(z) p(210))
(9)
Equation serves as alternative form of our test statistic in Equation 5 that works with already normalized probabilities. where we can include x in our Z and compute the KL divergence on all possible token choices, without removing the ground truth token and gathering the remaining logits. But in our implementation, we reuse the equivalent form in the second last line of Equation
4.3 FROM TOKEN-LEVEL TO SEQUENCE-LEVEL MIAS
The reigning privacy auditing and evaluation framework is on the sequence level. And for attackers with no knowledge of what private tokens are, the sequence-level notion is still the only choice. Here, we describe how to use token-level MIAs to perform sequence-level MIAs. We do not claim that this is the optimal way to evaluate token-level MIAs; this is more like a proof of concept. But results in Section 6.3 prove that token-level MIA is useful, powerful and versatile. To obtain a sequence- based membership score, which is an aggregated notion as we argued, we inevitably need to aggregate token-based scores. For each given sequence {x142 ... vk), .kk). our token-

---

Published as a conference paper at ICLR 2026
beanola_indeesee
asg_privemnn)
Business Pan de commerce "Intrcducton" Le commerce electronicue est on constanto éeclution, et pour róusd Sans ca marché dynomioue, est essentiel
dlaveir use straiégle sclide el des objectile cleirs Notre business plan pour notre entreprise encommerre vise a delimi nos actions el nos objectils pour prosperer dans la socteur du commmrree on ligne. "Stratégless as oléshs 1." "Segmontation du Manchent, Nous utiliooions les informations de nos cllents pour diviser marzhe segments spécf
Seq 434 [sample_]i baamlle_indew-20000 200e, avgi- agg-0.333 0.330, org_pffv-nan)
Team colleborate Pietforms Enhanced Pediatric Care Dear Team, continuuus improve pediatric services, we ccordination
introduce ains technology effecien ents. Key Features
palient the everall quality provided
professionals
Two of the ten most memorized sequences by a finetuned GPT-2 model on ai4privacy dataset, identified by InfoRMIA, contain private tokens. These pose little privacy risk, yet sequence-based frameworks overestimate their risk. See also Figure
Figure 2:
based MIA produces k membership scores {s....., 8k-1), and the sequence- based score is Aggregate(s1.... 8k-1).
The simplest aggregation averaging. A stronger aggregation could depend on the model or under- lying data distribution. Such tailored aggregation typically needs to be optimized on additional hold- out data. However, such a model and dataset specific aggregator that requires additional knowledge and computation power is not always realistic. For practicality, we only evaluate generic aggregation methods, such as averaging and min-k, k, in this paper.
5 TOKEN-LEVEL PRIVACY ASSESSMENT INTERFACE
In this section, we first demonstrate that our token-level framework, using InfoRMIA as the MIA backbone, can be used to visualize memorization on the token level. With knowledge of the sensitive tokens, users can conduct more insightful analyses of the privacy risks of the target model for given sequences, which can reveal much more than AUCs. Otherwise, auditing according to the average (sequence-level) privacy notion is recommended (See Section
5.1 VISUALIZING INFORMATION LEAKAGE ON THE TOKEN LEVEL
With token-based membership scores, we build a simple HTML interface to visualize a heatmap of token-level memorization over input text (Figures lb 2 g
where the darkness of the highlight re- flects the degree of memorization. This fine-gt ene-grained view enables more accurate privacy assessment, as auditors can directly inspect leakage on actual private tokens.
We find empirical evidence supporting our intuition that sequence-level signals may not correspond true privacy risks. Specifically, we observe a low correlation between sequence-level and private- token scores (Figure [a) and discover that many of the most "memorized" sequences either contain disproportionately little private information (Figure 7 or no private information at (Figure 2) Conversely, also find evidence that signals from private tokens often diluted by the presence of many common tokens in long texts (Figure g).
This fine- mne-grained analysis is only possible with token-level framework and highlights the limita- tions of existing pruuence-based notions of privacy. We believe this tool can highly valuable for practitioners and auditors who need precise, interpretable privacy quantification.
5.2 TOKEN-LEVEL ANALYSIS REVEALS MORE THAN AUCS
Our token-level framework also reveals insights that aggregate metrics like AUC cannot capture. For AG News (Appendix F.I. we hypothesize that sensitive information typically appears in personally identifiable information (PII). We therefore use SpaCy (Honnibal & Montani 2017) to classify en-
tities. Overall, token-level scores roughly follow a normal distribution (Figure 5 Tokens labeled
PERSON and WCRK SF_FFFFAA_ ART have the highest average membership scores (Figure 3 Table 18), indicating that names of people and artworks top 1% of the highest-scoring tokens, we find that these two types of tokens also have the highest memorization rate (Table 18 reinforcing that PIT is disproportionately more memorized.
likely to be memorized. Examining the
7

---

Published as a conference paper at ICLR 2026
Mean score by entity type (top)
Mean token score
0.10
0.05
0.00
WORK OF ART
PERSON
ORG
QUANIITY
GPE
EVENT
NORP
NONE
FAC
TIME
Entity
Figure 3: Average token-level membership scores across the most frequent named entity types on AG News dataset. Tokens associated with names exhibit higher memorization signals, suggesting heterogeneous memorization behavior across semantic token types. The "None" category corre- sponds to tokens not identified as nouns. Details of the target model provided in Section 6.1
0.6
0.4
token score
2
0.0
-0.2
Non-private
Tokens
Only
(a) Non-private tokens exhibit larger variance and higher average token-level membership scores, ob- scuring signals from private tokens.
(b) Aggregating token-level scores into sequence- level reduces private token membership signals de- spite higher token-level maxima.
Figure 4: Token-level and sequence-lever membership score distributions for a fine-tuned GPT-2
model on the ai4privacy dataset. The results illustrate how sequence-level aggregation obscure privacy leakage signals that originate from a small subset of private tokens. Additional details are provided in Table [9 and Figure
For ai4privacy (Appendix (.2). each sequence includes "privacy mask" that marks synthetic per- sonal information. We can thus divide tokens into private and non-private sets. We find that the average membership score of private tokens is slightly lower than that of non-private tokens (Ta- ble 19 Figure 4a). This suggests that the high AUCs reported by sequence-level MIAs may largely reflect memorization of non-private content, which less relevant for privacy. Hence, AUC alone is a poor indicator of true privacy risk in LLMs.
EXPERIMENTS
In this section, we present the attack performances of InfoRMIA and token-level InfoRMIA. We will describe the setup in Section 6.1 then show that InfoRMIA dominates RMIA and LiRA in Section 6.2, We then demonstrate our token-level framework's ss competitive performance in auditing sequence-level privacy (described in Section 4.3) on fintuned and pretrained LLMs in Section 6.3
6.1 EXPERIMENTAL SETUP
Setup For easy benchmarking, we use the new ML Privacy Meter which is an open-source Python library that audits privacy based RMIA, released by the same lab behind the RMIA paper
https:: / thub. hotebb..oo/prii vacyt lab/m1
aaatmmlpriiaaaaaeeeeeeeeeeeeeeee wwwppaacyenneeeeeeeee meter paacy_meteerrrraaeaaee
InfoRMIA will be merged
into main branch via a pull request to replace RMIA as the core MIA engine.
'There are incorrect RMIA implementations online. For correctness, opt for the library from the same lab/authors. See also Appendix
8

---

Published as a conference paper at ICLR 2026
We evaluate on all three default benchmarks in the Privacy Meter: Purchase 100 (Shokri al. 2017 CIFAR-I0 10 (Krizhevsky et al. 2009) and AG News (Zhang et al. 2013). which cover tabular, image and text datasets. Note that the AG News dataset is used for text generation instead of classification. The details of how we used the tool are Appendix Meter takes in configuration file that specifies meta information and hyperparameters. It will then train a set of models of the given architecture for the specified epochs, each on a randomly selected half of the dataset's training split, identical to LiRA and RIMA. The target model will be chosen randomly from the set, rendering the rest reference models. We also manually split the dataset and train the models in the same way when experimenting on ai4privacy in Table [ We then conduct offline attacks on randomly sampled sets of target models' members and non-members, computing their membership scores under different attacks and evaluating the AUCs and TPR at small FPR
For those who are unfamiliar, the Privacy
levels.
Model architectures We use GPT-2s (Radford et al., 2019) for AG News and ai4privacy, NideRessNets28-22 (Zagoruyko & Komodakis 2016) for CIFARIO, 10, and two two-layer MLPs for Purchase 100. We use epoch on AG News, and epochs on the other two for Table and 4 epochs on bothe datasets for T The other training details are in Appendix B
6.2 INFORMIA DOMINATES THE ORIGINAL RMIA
In both Tables and InfoRMIA dominates RMIA and LiRA in all cases. Besides higher AUCs, InfoRMIA greatly improves the TPR at very small FPR levels, indicating stronger member identifi- cation power without making (many) mistakes. Cs(Caniin et al. (2022) argued, this metric is better indicator of the true membership inference power. More importantly, we notice that InfoRMIA less sensitive to the size of the population data Z. This supports our theoretical derivation earlier, and is extremely valuable in practice, as the attack can now be run more accurately without large pool of population data, reducing the computation overhead and real-life infeasibility.
Table 1: Comparison of AUC and TPR 1.....0..0 %FPR between RMIA, LiRA, and InfoRMIA, both with 4 reference models under different datasets and population sizes. For CIFAR- 10, the Privacy Meter does not include augmentations in training and attacking, hence the lower numbers compared to the RMIA paper. For Purchasel 00, the Privacy Meter uses much larger training set, better evaluation per Suri et al. 2024). Since LiRA does not use Z, we replicate its numbers across different [Z].
AG News
CIFAR-10
Purchase
IZ|
100
1000
1000
10000
1000
10000
AUC
0.8574
0.8766
0.8229
0.8327
0.5311
0.5432
RMIA
TPR B.R00.190 OPPR
0.00%
1.60%
0.00%
0.00%
0.00%
0.00%
AUC
0.8784
0.8784
0.8330
0.8330
0.5754
0.5754
InfoRMIA
TPR @2.1000%PPP
12.0 %
12.0% %
5.82%
5.82%
0.32% %
0.32%
AUC
0.8641
0.8641
0.8242
0.8242
0.5398
0.5398
LiRA
TPR @0. 1%FR
1.80%
1.80%
0.12%
0.12%
0.12%
0. 0.12%
6.3 SOLVING SEQUENCF-LEVHI MEMBERSHIP INFERENCE WITH TOKEN-LEVEI. MEMBERSHIP SIGNALS
We also demonstrate that token-level membership inference can be used to conduct sequence-based membership inference with competitive performance, despite not being designed for it. We show in Table 2 that token-level membership inference with simple averaging yields competitive performance in sequence-level membership inference evaluation.
Finetuned LLMs
WWe audit the privacy risk of individual target models in this table. This makes the second term in In- foRMIA's test statistic (Fquation 4 identical for all test queries z's, keeping the order of test statisties among queries, and ultimately the attack performance, unchanged by the varying /

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of AUC and TPR I 1&FP between the sequence-based and token-based InfoRMIA, and the original RMIA and LiRA when attacking finetuned LLMs. The epochs column refers to the finetuning epochs. We use one reference model throughout the evaluation.
Datasets
Epochs
RMTA
InfoRMTA
InfoRMIA (token)
LIRA
AUC
TRP@TPR
AUC
TRP@ FPR
AUC
TRP@IPR
AUC
TRP@TPR
AG News
0.839
0.00%
0.843
23.0%
0.836
20.2%
0.795
3.6%
0.945
0.00%
0.945
16.2%
0.942
20.6%
0.882
9.00%
0.643
6.6%
0.644
10.6%
0.620
9.0%
0.630
3.8%
ai4privacy
0.821
26.0%
0.822
27.2%
0.804
23.2%
0.782
10.4%
Besides finetuned models, we also evaluate our newly proposed method on pretrained models (Table 3 8) on the most popular benchmark, MIMIR (Duan et al. 2024). As Duan al. (2024) pointed out, reference-based MIAs do not perform well on MIMIR due to the lack of quality reference models. Ideally, the reference models should have the same model archi- tecture and be trained on the same data distribution as the target LLM, but with (partially) disjoint datasets (which effectively makes them OUT models in RMIA's terminology). However, for pre- trained L.Mss, it is not possible find such perfect reference models. In our experiments, we find that using an earlier snapshot of the LLM is more useful, so use the step-1 Pythia- 6CM checkpoint as our sole reference mode This a very practical solution, as this checkpoint can be easily trained with lower-end hardware within short period of time, even if it is not available. Be- cause it is very OUT, it gives better results than using later checkpoints as the reference models (See Appendix G.3). Since in-distributional population data be easily obtained,
Pretrained LLMs
only evaluate the token-level InfoRMIA in our experiments, which has a similar computational complexity as the Ref method (Carlini et al. 2021 in MIMIR. The explanation of each method in the table is in Appendix F.3.T
We observe that our token-level InfoRMIA, although not specifically designed for sequence-level membership inference, achieves one of the strongest membership inference performances (Table even when using only a single, less ideal reference model. In Tables 8 we report results obtained by using the step-1 checkpoint of the target model as the reference and observe mini- mal change in utility. We further show that our method is the strongest reference-based MIA for pretrained LLMs, outperforming prior reference-model approaches (Ref). We also find that sim- ple averaging outperforms the min-k aggregation when targeting high true-positive rates at low false-positive rates (TPR at small FPR). This is expected, min-k aggregation essentially acts non-member detector: non- members tend to contain more low cow-poobability tokens (Shi et al. 2024). Consequently, min-k is less suitable for nigh-precision member detection with minimal errors. How- ever, when evaluating using AUC (Table 5). the ordering reverses. This aligns with the argument by [Carlini et al.] (2022) that AUC can be misleading as privacy metric. Nonetheless, many re- cent works evaluating on MIMIR still report only AUC in their main text, which may encourage a suboptimal trend for future development of LLM MIAs.
CONCLUSION
In this paper, we propose a new enffommttion-thhorede formulation of the membership inference test. The resulting attack, InfoRMIA, consistently outperforms the prior sftte-oftthe-art RMIA across tabular, image, and text datasets, while eliminating RMIA's reliance on large population set. Its superior performance stems from a more principled statistical test and the use of a continuous score rather than a discrete one.
We then introduce a new perspective for analyzing privacy risks LLMs through a token-level analysis framework. It can reveal which tokens are memorized within each sequence, while achiev- ing higher membership inference power. By uncovering fine-grained memorization patterns, our token-level framework enables more precise privacy risk estimation, and opens the door to down- stream applications such as targeted machine unlearning and guided data reconstruction and extraction. We leave the systematic exploration of these applications to future work.
Wee use the non-deduped version it sees fewer unique training sequences and hence more OUT.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We have described the training details, including model architectures, dataset splits, and essential hyperparameters in the paper. We will release the code and submit pull requests to the benchmarks (ML-Privacy-Meter and MIMIR) to facilitate the reproduction of the results presented in the paper.
DISCLOSURE ON THE USE OF LLMS
L.Mss have only been used in polishing writing and non creative part of coding, such as refactoring, annotating the code, and standardizing the format.
REFERENCES
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pp. 2633-2650, 2021.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Mem- bership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1897-1914, 1914. IEEF, 2022.
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations, 2023.
Debeshee Das, Jie Zhang, and Florian Trantèr. Blind baselines beat membership inference attacks for foundation models. In 2025 IEEE Security and Privacy Workshops (SPW), pp. 118-125. IEEE, 2025.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? In First Conference on Language Modeling, 2024. URL
https://openreviiee net forumrrrrnioorr wow.mmmiiavv01110555 9pSkU
Vitaly Feldman. Does learning require memorization? a short tale about long tail. In Proceedings of the 52nd anmual ACM SIGACT symposium on theory of computing, pp. 954-959, 2020.
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33:2881- 2891, 2020.
Jean-loup Gailly and Mark Adler. Zlib compression library. 2004.
Lco Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima. Shawn Presser, and Connor Leahy. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.0003 2020.
Jamie Hayes, Ilia Shumailov, Christopher A Choquette-Choo, Matthew Jagielski, Georgios Kaissis, Milad Nasr, Meenatchi Sundaram Muthu Selva Annamalai, Niloofar Mireshghallah, Igor Shilov, Matthieu Meeus, al. Exploring the limits of strong membership inference attacks on large language models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025a.
Jamie Hayes, Marika Swanberg, Harsh Chaudhari, Itay Yona, Ilia Shumailov, Milad Nasr, Christo- pher Choquette- Choo. Katherine Lee, and A Feder Cooper. Measuring memorization in lan- guage models via probabilistic extraction. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pp. 9266 2025b.

---

Published as a conference paper at ICLR 2026
Matthew Honnibal and Ines Montani. spaCy 2: Natural language understanding with Bloom em- beddings, convolutional neural networks and incremental parsing. To appear, 2017.
Harold Jeffreys. Theory of probability. 1939.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ken Ziyu Liu, Christopher A Choquette-Choo, Matthew Jagielski, Peter Kairouz, Sanmi Koyejo, Percy Liang, and Nicolas Papernot. Language models may verbatim complete text they were not explicitly trained on. In International Conference on Machine Learning (ICML), 2025.
Pratyush Maini, Hengrui Jia, Nicolas Papernot, and Adam Dziedzic. Llm dataset inference: Did you train on my dataset? Advances in Neural Information Processing Systems, 011122069-124000 2024.
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, and Taylor Kerg-Kirkpatrick. Membership inference attacks against language models via neigh- bourhood comparison. In Findings of the Association for Computational Linguistics: ACL 2023,
pp. 11330-11343, 2023.
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ip- polito, Christopher A Choquette- -Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2311.17035,
2023.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9 2019.
Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. 20/7 IEEE symposium on security and privacy (SP), pp. 3-18. IEEE, 2017.
Anshuman Suri, Xiao Zhang, and David Evans. Do parameters reveal more than loss for membership inference? arXiv preprint arXiv:2406. 11544, 2024.
Jiashu Tao and Reza Shokri. Range membership inference attacks. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 346-361. IEEE, 2025.
Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn. Position: Will we run out of data? limits of IIm scaling based on juman-generated data. In Forty- first International Conference on Machine Learning. 2024.
Roy Xie, Junlin Wang, Ruomin Huang, Minxing Zhang, Rong Ge, Jian Pei, Neil Gong, and Bhuwan Dhingra. Recall: Membership inference via relative conditional log-likelihoods. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 867 8671-8689, 2024.
Jayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, and Reza Shokri. En- hanced membership inference attacks against machine learning models. In Proceedings of the 2022 ACM SIGSAC Conference Computer and Communications Security, pp. 3093-3106, 2022.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learn- ing: Analyzing the connection to overfitting. In 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pp. 268 282..82.16 IEEE, 2018.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference 2016. British Machine Vision Association, 2016.
12

---

Published as a conference paper at ICLR 2026
Sajjad Zarifzadeh, Philippe Liu, and Reza Shokri. Low-cost high-power membership inference attacks. In International Conference on Machine Learning. pp. 58244-2244540 -82822. PMLR, 2024.
Chiyuan Zhang, Daphne Ippolito, Katherine Lec, Matthew Jagielski, Florian Tramèr, and Nicholas Carlini. Counterfactual memorization in neural language models. Advances in Neural Information Processing Systems, 36:39321-39366 2023.
Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Frank Yang. and Hai Li. Min-k%++: Improved baseline for pre-training data detection from large language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https::/cpeereeeeeeeee net forum? Litee/oooommddd//ooooofaaaaaaa
Weichao Zhang, Ruqing Zhang, Jiafeng Guo, Maarten Rijke, Yixing Fan, and Xueqi Cheng. Pre- training data detection for large language models: divergence- based calibration method. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp- 5263-5274, 2024.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text clas- sification. Advances in neural information processing systems, 28, 2015.
A THE MEMBERSHIP INFERENCE GAME
Membership inference is often formulated inference game. According to the auditing modes (privacy of a fixed model/data record/training algorithm), the game formulation also varies eYe al, 2022 Here, provide the game formulation when auditing the privacy of fixed model.
Definition (Membership Inference Game YYom al. 2018 Ye et al. 2022 Carlini et al. 2022 Zarifzadeh et al. 2024 Tao & Shokri 2025 Let π be the data distribution, and let T be the training
algorithm.
The challenger samples a training dataset D <- T, and trains a model - T(D).
2. The challenger sumples data record zo <- π from the data distribution, and training data record 21 D.
3. The challenger slips fair coin to get the bit b E 10. and sends the target model 0 and data record 21 the adversary.
4. The adversary gets access to the data distribution and access to the target model, and outputs bit - A(0,
5. b b. output (success). Otherwise, output O.
13

---

Published as a conference paper at ICLR 2026
B DERIVATION OF INFORMIA SCORES
p(01z)
Test Statistic
p(z) log
p(0]=)
=E2
p(z(0)p((2
Σ p(2) log
((2]0)p(x))
#E7
p(<]10)p(2)
p(xx10)p(z)
Σ p(2)log log
+ p(x) log
p(=10)p(x)
p(x ppxx0)pp())
(10)
pp(x(0)p(z) p(=10)p(x))
Σ p(2) log
2EV
pp(x10)
p(=)
Σ SEV
p(z) log
Σ SEV
p(z) log
p(x:)
pp21())
p(xz(0) p(x)
log
DKL (p(z) [[(()))
C PSEUDOCODE OF RMIA AND INFORMIA
In this section, we outline the consolidated pseudocode (Alg I of RMIA, InfoRMIA, and token- level InfoRMIA for an easy comparison of the attacks. Lines 5 to 9 clearly show that the token-level InfoRMIA has the smallest computation overhead due the absence of the additional population dataset. Its p(z[0) terms are also directly obtainable from the calculation of P(z(0) since each prediction step computes the softmax scores over the entire vocabulary. Note that for LLMs, the hyperparameter is optimal at (Hayes et al.) (025a). When using token-level InfoRMIA, each æ is prefixal subsequence instead of full sequence (See Section 4.1
D DEFENDING INFORMIA
Similar to all other MIA techniques, InfoRMIA relies model's memorization for successful attacks. It does not use additional information or require higher levels access to the target models. Therefore, differentially private training algorithms that reduce memorization of training data are still effective in defending against InfoRMIA.
E IMPLEMENTATION DETAILS
E. RMIA REFERENCE MODEL TRAINING
We use the default hyperparameters in ML Privacy Meter to train target and reference models when comparing the original and InfoRMIA, except for the number of epochs. For each dataset, the hyperparameter choices can be found in https: /qithub..uuu./g com/ pwivacytrust.sseeecc.pp. lab/ m1 privacy cettee//teee/mssee/eeee
For CIFAR-10 and Purchase-100 100, we use 100 epochs, while for AG News, we use 1 epoch. We use the one-liner command python run_mia py -f configs/ /xx////xxxx/ yaml run the experiments, where the yaml files correspond to the respective default configs in the GitHub repo
For ai4privacy experiments, we train target and reference models, which are initiated from GPT-2 models, on randomly selected halves of the training set, identical to the setup of LiRA and RMIA.
E.2 SOFTWARE AND HARDWARE
For all transformer models and language datasets, the libraries from Huggingface. The training process also uses Huggingface's 's Trainer class. All computations are done on two NVIDIA RTX-3090 and two H100 GPUs.
14

---

Published as a conference paper at ICLR 2026
Algorithm MIA Score Computation with Offline RMIA, InfoRMIA Token-Level InfoRMIA, modified from Zarifzadeh et al. (2024).
Input:
Target model 8, set of k reference models target query z; hyperparameters 7, a. popu- lation dataset Z (only for RMIA and InfoRMIA)
Output: Membership score ScorevIA (z:; () of x given the target model 0
1: Compute p(x:0) and p(x:0,) for all 0, Q
1 Ee.ee
2: p(z)OUT
k
1
3: p(x)
2 (1)+))())) + a))
p(x10)
4: Ratio, p(x)
5: if Token-Level InfoRMIA then
6: Take all other tokens except the ground-truth
See Section 4.2
7: else
8: Take population data from the population dataset Z as 2
9: end if
10: for each z do
11:
Compute p(z10) and p(=10,) for all 0,, E θ
p(=) - 1: Eo.ee Pr(=10,)
Ratio, + p(z10)
13:
p(z)
14: end for
15: if RMIA then
Ratio Ratio= 27)
16:
ScoromIA (2:0M (x: - 7 Eeez
17: else
InfoRMIA and Token-Level InfoRMIA > See Equation
18:
ScoreMIA (2; + log Ratio - E. p(2) log Ratio,
end if
E.3
INFORMIA SIGNAL COMPUTATION
Several public implementations of RMIA differ in subtle but important details, which can lead to inconsistent results in practice. Since the core computation of InfoRMIA closely resembles RMIA, implementation errors can easily propagate. To facilitate reproducibility, we provide reference implementation of the InfoRMIA signal computation in Python below. The official implementation is also available in the latest release of PRIVACY METER.
Listing 1: Reference implementation of the InfoRMIA test statistic computation.
def run_informia! target_mcdel_idee int, all all_signals: np. ndarray, population_signals nd.rrar,,y, all memberships: ndarray,, num ceeference_modelle int, offline_ar #loat, ~> nd.ndaryyy:
Compute InfoRMIA membership sccres.
Args:
target_model_idk: Index of the target model.
all_signals: Softmax scores of all samples.
population_sigge siggallsssggnls:: Softmax scores of population samples. all all_menberships: Membership matrix for all models. num num_reference senfereenee_mdelss: models: Number of reference models. offline_a: Offline correction in coefficient used to approximate (x) using PLout.
15

---

Published as a conference paper at ICLR 2026
Returns:
Membership inference scores for all samples. Larger values indicate higher menbership likelihood.
Target model signals target_signals siinsignls target_model_ion
out signals get rmi out siggaag
all all_signals,
all_memberships,
target_model_idk, num rum_reeferenc_modelss
mean_cut_x np. moan (out_signals, axis=1) PLout (x)
mean_x =
((1 + offline_a) 2) mean_out_x
((1 offline_a) 2)
) Offline estimation of (x) according to RMIA
log (p(x|theta) / (x))
log_ratio_x ip. (target.ssigaaassss..s.
mean_x)
population_membershii. np. np.zeros_like.
populatiio__signals,
dtype-bool,
pepulation samples OUT
z_signals
target_model_idx get_rmia__ot_ssinnle.
2_out_signall
populatiin_siinall
eepulattio__membrriin target_model_ num_reference_model
mean out z mp.mean (z out signals, axis=1)
mean z
((1 + offline a) / 2) mean_out. ((1 offline_a) / 2)
prob_ration = rave))) a
mean_2
test statistici
log_ratio_x
np. sun (mean_2 2 no. (prob_ratio__
/ mean sum
return test_statistic
When computing log-likelihood terms, we recommend using the smallest epsi lon value that avoids numerical instability. Extremely small softmax probabilities may otherwise cause the loga- rithm to be dominated by the clipping constant. For clarity, no epsilon applied the reference implementation shown above.
E.4
OTHER DETAILS
When computing the InfoRMIA test statistics, we use the second last line in Equation because easier and more similar the computation of RMIA's test statistics.
16

---

Published as a conference paper at ICLR 2026
F DATASETS
F.1 AG NEWS
AG News (Zhang al. 2015) is news dataset that contains four categories of news articles. Its training set size is 120,000. In our experiment, we ignore the labels column and train au- toregressive models on the text only, identical to Mattern et al. (2023) and Tao & Shokri (2025). In implementation, max sequence length is set to 512. The preprocessing and loading script is adapted from the Pri- vacy Meter http::///....ahcocooooooooaaaaaa
use the datasett pacakge from huggingface to load the dataset. The
F.2 A14PRIVACY
The ai4privacy dataset we used is the pii-masking-300 300k variant, that can be access at https: //huggingfaace co/ wtt.:////aaaaa... httpst/////ppiacc/ppi...p
300k It is divided into two parts: OpenPI1-220k and FinPII-8Ok. The FinPlI has additional classes that are specific to the Finance and Insurance domains. In this dataset, there is a "privacy mask" column that marks the beginning and end location for each piece of private information. Thus, we can use this information categorize each token as private non-private. also assigns type to private substrings, such as last names or email addresses, enabling us to do more interesting analysis.
F.3 THE MIMIR BENCIMARR
MIMIR a benchmark based on the Pile (Gao et al. 2020) dataset, where non-members highly overlapped with any member sequence removed from the evaluation. Since the members and non-members are randomly shuffled before being split, MIMIR avoids the error of having a large dis- tributional shift between the sets (Maini et al.) 2024 Das et al. (0025). It also one of the most active benchmarks with official implementations of recent methods such as the MinK++ (Zhang al. 2025), ReCaLL (Xie et al., 2024), and DC-PDD (Zhang al. 2024).
F.3.1
EXPLANATIONS OF ALL ATTACK METHODS
We will briefly explain the score formulation of each attacking method in the MIMIR benchmark. The full details of each attack can be found at the respective papers. Note that in this benchmark, the higher the score is, the less likely a member.
LOSS (Yeom al. 2018): the average loss values of the sequence
Zlib (Carlini et al.
2021): the calibrated loss values by the entropy, estimated by the length
after zlib compression
Min-K % (hh et al. 2024]: the average of the bottom-k%% of token probabilities, and taking the negative (to align the score's 's sign with MIMIR's standard)
Min-K%++ % Zhang et al. (025 the average of the bottom-k of token probabilities calibrated by the mean and variance of each token position's 's softmax output distributions, and taking the negative
DC-PDD (Zhang al. 2024): the average token probabilities calibrated by token frequen- cies calculated on reference dataset, and take the negative
Ref (Carlini et al., 202Th: the average token loss gap between a reference model and the target model
F.3.2 WHY RECALL WAS EXCLUDED IN OUR TABLE
The ReCaLL attack pushed to the MIMIR benchmark is simplified version (according to the au- thor) and very problematic. There is an implicit information leakage about the membership labels in crafting the attack signals, making the result unreliable and unfair. By right, the attacker should not be able to distinguish any non non-member from any member, which means the attacker has no in- formation that can be used to tell non-members apart from members before the attack. However, the
17

---

Published as a conference paper at ICLR 2026
ReCaLL attack in MIMIR explicitly uses non-members in the evaluation set its "non-member" prefix to be prepended to all sequences, making the attacker aware of the membership label of certain non-members. Although the label information is not explicitly used in the attack, it is implicit information leak that can be used to tell apart the two sets. Hence, the current implementation the ReCaLL attack in MIMIR is incorrect. We will include it once the official implementation is fixed.
G ADDITIONAL RESULTS
G.1 MIMIR RESULTS
We provide more results on the MIMIR benchmark here. In particular, we use the ngram<0, split. Table 3 and 4 show the results on TPR @ 1% FPR and TPR@0.1 T5ble the AUCs. The results show that our method has strongest inference power (highest TPR at small FPRs), while achieving very competitive results on AUCs. It is also stronger than the prior reference-bart based MIA, Ref (Carlini et al. (021).
respectively, while
Table 3: TPR @ 1% FPR on MIMIR with deduped Pythia models. on MIMIR benchmark with deduped Pythia models. The Neighbor method is not included due its computational complexity and relatively inferior performance reported in prior works. ReCaLL is not included for reasons in Appendix F33.2 Ref method evaluated using the checkpoint of Pythia- 1ythia-160m after the first step the reference model. Our method (InfoRMIA is the loken-based InfoRMIA that does require additional population data. InfoRMIAI uses averaging to aggregate, while InfoRMIA2 uses min- k%, using the same hyperparameter l Min-K% and Min-K% +%++. Bold numbers the best, and the underlined are the best reference-based.
Wikipedia
Github
Pile CC
PubMed Central
Method
160M
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
160M
2.8B
6.9B
Loss
0.9
0.6
0.6
0.6
13.1
13.3
21.9
13.2
0.4
0.7
0.8
0.9
0.7
0.4
0.6
0.4
1.3
0.6
16.9
24.0
0.7
0.9
0.3
0.5
0.4
Min-K%
1.4
0.9
0.6
0.5
12.0
13.1
21.8
13.0
0.5
0.6
0.7
1.0
0.6
0.2
0.6
0.4
Min-K%++
1.2
0.7
0.6
1.0
11.2
12.8
18.1
12.8
1.1
1.1
1.2
1.5
0.6
0.5
0.6
DC-PDD
0.9
0.4
1.2
1.4
10.8
11.3
9.8
10.7
0.4
1.1
0.6
1.1
1.5
0.8
1.3
1.3
Ref
0.9
0.8
0.7
0.6
13.4
13.9
20.3
14.7
0.6
0.7
0.8
1.0
0.8
0.6
0.5
0.4
InfoRMIAI
0.8
0.9
17.7
21.2
1.2
0.9
0.4
InfoRMIA2
1.1
1.2
0.9
0.7
13.0
14.1
18.3
14.5
0.4
0.5
0.7
0.5
1.2
1.4
1.3
1.6
ArXiv
DM Mathematics
HackerNews
Average
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
Loss
0.7
0.7
0.4
0.8
0.5
0.5
1.1
1.1
0.9
0.7
0.6
0.8
2.5
2.4
3.7
2.5
Zlib
0.5
0.7
0.9
1.0
1.0
2.7
4.1
2.9
Min-K%
0.3
0.3
0.4
0.7
0.8
0.6
0.2
0.4
0.7
0.9
0.7
1.1
2.3
2.4
3.6
2.4
Min-K%++
1.1
1.9
1.2
1.4
1.0
1.0
1.2
1.0
0.7
0.5
1.1
0.7
2.4
2.6
3.4
2.7
DC-PDD
0.5
1.0
0.9
0.5
0.5
0.4
0.2
0.1
1.3
1.0
0.5
1.2
2.3
2.3
2.1
2.3
Ref
0.8
0.5
0.5
0.6
1.2
1.0
1.2
0.7
1.4
0.7
0.7
0.7
2.7
2.6
3.5
2.7
InfoRMIA1
0.3
0.4
0.6
1.3
1.5
1.2
1.7
2.9
3.7
2.9
InfoRMIA2
0.1
0.3
0.3
0.4
1.2
0.8
1.1
0.7
1.7
1.2
1.0
1.8
2.7
2.8
3.4
2.9
G.2 MIMIR RESULTS WHEN USING THE FIRST STEP CHECKPOINT OF THE TARGET MODEL THE REFERENCE
Instead of using the first step checkpoint of Pythia-16 1m00, ing to the target model, e.g., we use the Pythia-1- 1e:aa11.b::: sb:ttepl as the reference when attacking Pythia-1. 4b-deduped. We found in Table 6 and 8 that the difference in utility is minimal. Therefore, it might be better to stick to snapshots smaller models as reference models for cost- sensitive auditors. Moreover, using the first step checkpoint of the small LLM as the reference has
the checkpoint correspond-
18

---

Published as a conference paper at ICLR 2026
Table 4: TPR @0.1% FPR on MIMIR with deduped Pythia models.
Wikipedia
Github
Pile CC
PubMed Central
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
Loss
0.0
0.2
0.0
0.0
7lib
0.1
0.1
0.1
0.1
8.4
5.7
8.6
6.9
0.0
0.2
0.2
0.3
0.0
0.0
0.0
0.0
Min-K%
0.0
0.1
0.2
0.1
5.7
4.8
8.0
4.9
0.0
0.2
0.1
0.2
0.0
0.0
0.0
0.0
Min-K%++
0.0
0.0
0.1
0.0
6.1
3.5
2.1
0.2
0.1
0.3
0.0
0.0
0.0
DC-PDD
0.0
0.0
0.0
0.0
3.7
0.3
0.3
1.1
0.0
0.0
0.3
0.2
0.0
0.0
0.1
0.1
Ref
0.0
0.0
4.3
2.2
0.2
0.3
0.0
InfoRMIA1
0.1
0.2
0.2
0.2
0.0
0.0
0.6
1.0
0.1
0.1
0.1
0.1
0.0
0.1
0.3
0.3
InfoRMIA2
0.0
0.0
0.1
0.1
4.8
0.0
0.9
0.2
0.1
0.1
0.1
0.2
0.0
0.0
0.0
0.0
ArXiv
Mathematics
HackerNews
Average
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M 1.4B
2.8B
6.9B
Loss
0.1
0.0
0.1
0.0
0.0
0.0
0.8
1.2
0.8
Zlib
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.1
0.2
0.2
0.2
1.2
0.9
1.3
1.1
Min-K%
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.2
0.1
0.1
0.1
0.8
0.7
1.2
0.8
Min-K
0.0
0.0
0.0
0.0
0.1
0.0
0.5
0.2
0.2
0.0
0.1
0.0
0.9
0.5
0.8
0.4
DC-PDD
0.0
0.0
0.2
0.0
0.0
0.0
0.0
0.0
0.0
0.2
0.1
0.0
0.5
0.1
0.1
0.2
Ref
0.2
0.0
0.0
0.0
0.0
0.0
0.7
0.4
0.6
InfoRMIA1
0.1
0.0
0.0
0.0
0.1
0.3
0.3
0.6
0.0
0.0
0.0
0.0
0.1
0.1
0.2
0.3
InfoRMIA2
0.0
0.0
0.0
0.0
0.3
0.3
0.7
0.1
0.2
0.1
Table 5: AUC results on MIMIR benchmark with deduped Pythia models.
Wikipedia
Github
Pile CC
PubMed Central
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B 6.9B
Loss
50.2
51.0
51.7
51.6
63.7
65.8
71.2
67.6
49.5
50.1
50.1
51.3
49.9
49.8
49.9
50.5
Zlib
51.0
51.8
52.4
52.3
67.2
68.8
49.6
50.2
50.3
51.2
50.0
50.0
50.0
50.6
Min-K%
48.6
50.6
51.6
51.4
63.6
65.9
71.4
68.0
50.0
51.0
50.5
51.9
50.4
50.2
50.4
51.0
Min-K%+++
47.7
53.7
52.4
61.4
65.7
51.1
49.9
50.9
52.3
DC-PDD
49.0
50.6
52.4
51.8
64.9
66.2
71.4
69.0
49.6
51.1
51.2
51.9
50.5
51.0
50.6
51.1
Ref
50.0
50.8
51.6
51.4
63.9
66.0
71.4
67.9
49.4
50.0
50.0
51.2
49.8
49.7
49.8
50.4
InfoRMIA1
50.9
50.8
51.0
51.2
65.0
66.1
70.8
67.0
49.4
49.6
49.8
50.5
50.2
49.7
49.5
49.8
InfoRMIA2
50.0
50.3
51.1
51.1
63.5
65.3
70.6
66.9
50.6
51.1
50.8
51.7
51.4
50.4
50.2
50.7
ArXiv
DM Mathematics
HackerNews
Average
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M 1.4B
2.8B
6.9B
160M 1.4B
2.8B
6.9B
Loss
50.7
51.4
51.9
52.5
49.0
48.6
48.3
48.4
49.2
50.4
51.2
51.7
51.8
52.4
53.5
53.4
Zlib
50.0
50.8
51.3
51.8
48.2
48.0
48.1
49.6
50.2
50.9
51.0
52.0
52.6
53.6
53.4
Min-K%
50.0
51.2
52.2
52.7
49.4
49.3
49.1
49.3
50.2
$1.3
52.4
53.0
51.7
52.8
53.9
53.9
Min-K%++
48.7
53.1
52.8
50.0
50.9
51.1
52.3
51.3
54.4
54.6
DC-PDD
50.4
52.0
52.9
52.9
49.0
49.3
49.8
49.7
50.7
51.8
53.0
53.9
52.0
53.1
54.5
54.3
Ref
50.3
51.0
51.5
52.1
48.8
48.5
48.3
48.3
49.1
50.4
51.2
51.7
51.6
52.3
53.4
53.3
InfoRMIA1
50.3
51.1
51.1
51.5
48.0
47.6
47.9
47.9
50.4
50.7
51.0
51.3
52.0
52.2
53.0
52.7
InfoRMIA2
50.8
51.2
51.6
52.3
49.0
49.1
49.0
48.9
50.4
51.9
52.4
53.1
52.2
52.7
53.7
53.5
another benefit: if the checkpoint is not publicly available, training the small LLM for step is computationally cheap. Normal users can obtain the checkpoint and run the attack with low end computes and short training period.
19

---

Published as a conference paper at ICLR 2026
Table 6: TPR @ % FPR on MIMIR benchmark with deduped Pythia models when using the first
step checkpoint of the target model as the reference.
Wikipedia
Github
Pile CC
PubMed Central
Method
160M
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
2.8B
6.9B
Loss
0.9
0.6
0.6
0.6
13.1
13.3
21.9
13.2
0.4
0.7
0.8
0.9
0.7
0.4
0.6
0.4
Zlib
1.3
0.7
0.8
0.6
14.3
16.9
24.0
15.5
0.7
0.7
0.9
1.5
0.3
0.4
0.5
0.4
Min-K%
1.4
0.9
0.6
0.5
12.0
13.1
21.8
13.0
0.5
0.6
0.7
1.0
0.6
0.2
0.6
0.4
Min-K%++
1.2
0.7
0.6
1.0
11.2
12.8
18.1
12.8
1.1
1.1
1.2
1.5
0.6
0.4
0.5
0.6
DC-PDD
0.9
1.4
11.3
1.1
0.6
1.5
1.3
Ref
0.9
0.8
0.7
0.9
13.4
10.9
17.9
4.7
0.6
0.5
0.7
1.2
0.8
0.9
0.2
0.5
InfoRMIA1
0.8
0.7
1.0
17.5
21.0
14.9
1.2
0.9
0.7
1.0
0.7
InfoRMIA2
1.1
1.3
0.9
1.0
13.0
13.8
18.7
14.5
0.4
0.5
0.5
0.6
1.2
1.2
1.3
0.8
ArXiv
DM Mathematics
HackerNews
Average
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
Loss
0.7
0.7
0.4
0.8
0.5
0.5
1.1
1.1
0.9
0.7
0.6
0.8
2.5
2.4
3.7
2.5
Zlib
0.5
0.2
0.4
0.7
1.1
0.9
0.9
0.6
0.6
1.0
1.0
1.0
2.7
3.0
4.1
2.9
Min-K%
0.3
0.3
0.4
0.7
0.8
0.6
0.2
0.4
0.7
0.9
0.7
2.3
2.4
3.6
2.4
Min-K%+++
1.1
1,9
1.2
1,4
1.0
1.0
1.2
1.0
0.7
0.5
1.1
0.7
2.4
2.6
3.4
2.7
DC-PDD
0.5
0.9
0.5
0.5
0.4
1.3
1.0
0.5
2.3
2.1
2.3
Ref
0.8
0.3
0.6
0.5
1.2
0.9
1.0
0.3
1,4
1.0
0.8
1.0
2.7
2.2
3.1
1.3
InfoRMIA1
0.3
0.3
0.4
1.4
1.2
2.9
2.9
InfoRMIA2
0.1
0.3
0.3
0.3
1.2
0.9
1.2
1.0
1.7
1.3
1.0
1.7
2.7
2.8
3.4
2.8
Table 7: TPR @0.1% FPR on MIMIR benchmark with deduped Pythia models when using the first step checkpoint of the target model as the reference.
Wikipedia
Github
Pile
PubMed Central
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
Loss
0.0
0.2
0.0
0.0
7lib
0.1
0.1
0.1
0.1
8.4
5.7
8.6
6.9
0.0
0,2
0.2
0.3
0.0
0.0
0.0
0.0
Min-K%
0.0
0.1
0.2
0.1
5.7
4.8
8.0
4.9
0.0
0.2
0.2
0.0
0.0
0.0
Min-K%+++
0.0
0.0
0.1
0.0
6.1
3.5
4.6
2.1
0.1
0.2
0.1
0.3
0,0
0.0
0.0
0.0
DC-PDD
0.0
0.0
0.0
0.0
3.7
0.3
0.3
1.1
0.0
0.0
0.3
0.2
0.0
0.0
0.1
0.1
Ref
0.0
0.1
0.0
0.1
4.3
0.0
0.6
0.1
0.1
0.1
0.1
0.2
0.0
0.0
0.0
InfoRMIA1
0.1
0.1
0.2
0.3
0.0
0.0
0.4
0.1
0.1
0.1
0.1
0.1
0.0
0.1
0.3
0.3
InfoRMIA2
0.0
0.1
0.0
0.1
0.0
0.0
ArXiv
Mathematics
HackerNews
Average
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M 1.4B
2.8B
6.9B
Loss
0.1
0.1
0.0
0.0
0.8
0.8
Zlib
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.1
0.2
0.2
0.2
1.2
0.9
1.3
1.1
Min-K%
0.0
0.0
0.0
0.0
0.2
0.1
0.1
0.8
0.7
1.2
0.8
Min-K%+++
0.0
0.0
0.0
0.0
0.1
0.0
0.5
0.2
0.2
0.0
0.1
0.0
0.9
0.5
0.8
0.4
DC-PDD
0.0
0.0
0.2
0.0
0.0
0.0
0.0
0.0
0.0
0.2
0.1
0.0
0.5
0.1
0.1
0.2
Ref
0.2
0.0
0.1
0.0
0.0
0.0
0.7
0.1
0.1
InfoRMIA1
0.1
0.0
0.0
0.0
0.1
0.1
0.3
0.5
0.0
0.0
0.0
0.0
0.1
0.1
0.2
0.2
InfoRMIA2
0.0
0.0
0.1
0.2
0.7
0.1
20

---

Published as a conference paper at ICLR 2026
Table AUC results on MIMIR benchmark with deduped Pythia models when using the first step checkpoint of the target model as the reference.
Wikipedia
Github
Pile CC
PubMed Central
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
Loss
50.2
51.0
51.7
51.6
63.7
65.8
71.2
67.6
49.5
50.1
50.1
51.3
49.9
49.8
49.9
50.5
Zlib
51.0
51.8
52.4
52.3
65.6
67.2
72.2
68.8
49.6
50.2
50.3
51.2
50.0
50.0
50.0
50.6
Min-K%
48.6
50.6
51.6
51.4
63.6
65.9
71.4
68.0
50.0
51.0
50.5
51.9
50.4
50.2
50.4
51.0
Min-K%++
47.7
52.3
53.7
52.4
61.4
65.7
70.7
69.1
49.8
51.1
49.9
51.7
50.9
50.6
51.2
52.3
DC-PDD
49.0
52.4
51.8
66.2
51.1
51.2
50.5
51.1
Ref
50.0
50.9
51.6
52.1
63.9
65.4
71.4
66.9
49.4
50.0
50.2
51.3
49.8
50.0
49.5
50.5
InfoRMIA1
50.9
51.1
51.5
66.0
66.9
49.5
49.8
50.2
49.4
49.8
InfoRMIA2
50.0
50.4
51.7
51.6
63.5
65.3
70.5
66.9
50.6
50.9
$1.0
51.4
51.4
50.3
50.1
50.2
ArXiv
DM Mathematics
HackerNews
Average
Method
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
160M
1.4B
2.8B
6.9B
Loss
50.7
51.4
51.9
52.5
49.0
48.6
48.3
48.4
49.2
50.4
51.2
51.7
51.8
52.4
53.5
53.4
Zlib
50.0
50.8
51.3
51.8
48.2
48.1
48.0
48.1
49.6
50.2
50.9
51.0
52.0
52.6
53.6
53.4
Min-K%
50.0
51.2
52.2
52.7
49.4
49.3
49.1
49.3
50.2
51.3
52.4
53.0
51.7
52.8
53.9
53.9
Min-K%+++
48.7
51.2
53.1
52.8
49.9
50.0
50.3
50.2
50.9
51.1
52.3
53.7
51.3
53.1
54.4
54.6
DC-PDD
50.4
52.0
52.9
52.9
49.7
51.8
53.0
52.0
53.1
54.5
54.3
Ref
50.3
51.3
51.8
52.3
48.8
49.0
48.7
48.2
49.1
50.5
51.4
51.5
51.6
52.5
53.5
53.3
InfoRMIA1
50.3
51.2
51.6
47.7
50.4
50.7
51.1
52.0
52.3
53.0
52.8
InfoRMIA2
50.8
51.0
51.6
52.2
49.0
49.1
49.0
48.8
50.4
52.0
52.3
52.3
52.2
52.7
53.7
53.3
G.3 USING LATER CHECKPOINTS
REFERENCE MODEL
In this section, we provide experiment results on MIMIR when we use later checkpoint of the Pythia-168 60m. Given that the pretraining takes 143000 steps, we experimented with steps 10000 and 100000. Tables 9 14 are the respective results. Compared to Tables to the numbers show that using later checkpoints yields weaker attack performance. This aligns with our argument that we need the reference model be as OUT possible. Therefore, the earlier the checkpoint, the fewer training sequences it has seen, the better the attack performance is.
G.4 MIMIR RESULTS WITH THE GPT-NEO FAMILY
The MIMIR benchmark also supports testing with models from the GPT-Neo family. However, since intermediate checkpoints of GPT-Neos are released, we instead use the smallest pretrained model in the family, which the GPT-Neo- 2.7b models. Note that we have mentioned in the previous section that using fully pretrained model the reference model is bad, as the reference model completely IN, instead of being OUT. Hence, the numbers in Tables 15 17 are just for curious readers.
the reference model, attack the 1.3b and
H TOKEN-BASED ANALYSIS
In this section, provide some analytical results the token-based interface on finetuned mod- els on AG News and ai4privacy, when conducting offline token-level InfoRMIA with reference models.
21

---

Published as a conference paper at ICLR 2026
Table 9: AUC results on MIMIR benchmark for deduped Pythia models using the step 10k check- point of the 160m model. Note that the performance of InfoRMIA deteriorates the reference model becomes less OUT when using later checkpoint.
Wikipedia
Github
Pile CC
PubMed Central
Method
160M
1.48
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.88
Loss
50.2
51.0
51.7
63.7
65.8
71.2
49.5
50.1
50.1
49.9
49.8
Ziib
51.0
51.8
52.4
65.6
67.2
72.2
50.2
50.3
50.0
50.0
Min K%
48.6
50.6
51.6
63.6
65.9
71.4
50.0
51.0
50.5
50.4
50.2
50.4
Min-K%he+
47.7
52.3
53.7
61.4
65.7
70.7
49.8
51.1
49.9
50.9
50.6
51,2
DC-PDD
49.0
50.6
52.4
64,9
66.2
71,4
49.6
51.1
51.2
50.5
51.0
50.6
Ref
49.9
51.4
53.0
36.6
40.7
47.5
50.5
52.1
52.0
50.3
49.8
50.1
Info-RMIAI
48.8
50.4
51.5
41,9
43.3
51.0
48.5
51.1
51.0
49.7
49.0
49.0
ArXiv
DM Mathematics
HackerNews
Average
Method
160M
1.4B
2.81
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
Loss
50.7
48.3
51.2
52.4
53.5
7lih
50.0
50.8
51.3
48.2
48.1
48.0
49.6
50.2
50.9
52.0
52.6
53.6
Min-K%
50.0
51.2
52.2
49.4
49.3
49.1
51.3
52.4
51.7
52.8
53.9
Min K
48.7
51.2
53.1
49.9
50.0
50.3
50.9
51.1
52.3
51.3
53.1
54.4
DC-PDD
50.4
52.0
52.9
49.0
49.3
49.8
50.7
51.8
53.0
52.0
53.1
54.5
Ref
50.3
52.0
53.1
49.3
48.6
48.5
50.4
52.9
54.8
48.2
49.6
51.3
Info-RMIA]
50.5
51.2
51.7
48.2
47.8
47.7
50.8
52.5
53.4
48.3
49.3
50.8
Table 10: TPR @0.1% FPR MIMIR benchmark for deduped Pythia models using the step 10k checkpoint the 160m model. Note that the performance of InfoRMII deteriorates as the reference model becomes less OUT when using later checkpoint.
Wikipedia
Github
PubMed
Method
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2,8B
Loss
0.0
0.2
5.7
4.8
7.9
0.2
0.0
0.0
Ziib
0.1
0.1
8.6
0.2
0.2
0.0
Min-K%%
0.0
Min-K%e++
0.0
0.1
4.6
0.2
0.0
DC-PDD
0.2
0.8
Info-RMIAI
0.5
0.0
DM Mathematies
HackerNews
Average
Method
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2,8B
Loss
0.1
0.0
0.0
0.0
0.2
0.0
0.0
0.8
0.7
0.0
0.0
0.9
Min-K%
0.0
0.0
0.0
0.1
0.7
Min
DC-PDD
0.0
Info RMIAI
0.0
Token Info RMIA
25000
099m1
Count
Figure 5: Distribution of token InfoRMIA scores on AG News dataset.
22

---

Published as a conference paper at ICLR 2026
Table 11: TPR @ 1% FPR MIMIR benchmark for deduped Pythia models using the step 10k checkpoint the 160m model. Note that the performance InfoRMII deteriorates as the reference model becomes less OUT when using later checkpoint.
Wikipedia
Github
Pile CC
PubMed Central
Method
160M
1.4B
2.88
160M
1.4B
2.88
160M
1.4B
2.8B
16UM
1.4B
2.8B
Loss
0.9
0.6
13.1
13.3
21.9
0.7
0.8
0.7
0.4
Zilb
1.3
0.8
14.3
24.0
0.7
0.4
Min-K%
1,4
0.6
12.0
13.1
21.8
0.6
0.6
0.2
Min-KSe++
1.2
11.2
18.1
DC-PDD
0.9
10.8
9.8
1.1
0.8
3.0
1.2
3.9
DM Mathematies
HackerNews
Average
Method
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
Loss
0.7
0.4
0.5
1.1
0.7
2.5
2.4
Ziib
0.5
0.4
1.1
0.9
0.9
1.0
1.0
2.7
3.0
Min-K%
0.3
0.4
0.2
0.9
2.4
Min-K%e+
1.1
1.2
1.0
1.2
0.5
2.6
DC-PDD
0.5
1.2
1.1
1.2
Info RMEAI
1.9
Table 12: AUC results MIMIR benchmark for deduped Pythia models using the step 100k check- point the 160m model. Note that the performance of InfoRMIA deteriorates as the reference model becomes less OUT when using later checkpoint.
Wikipedia
Github
PubMed Central
Method
160M
1.4B
2.8B
160M
1.4B
2.8B
16UM
1.4B
2.8B
16UM
1.4B
2.8B
Loss
50.2
51.0
51.7
63.7
71.2
49.5
50.1
5U.1
49.9
49.8
49.9
Zlib
51.0
51.8
52.4
65.6
67.2
72.2
49.6
50.2
50.3
50.0
50.0
Min-K%
48.6
51.6
63.6
71.4
51.0
50.5
50.4
50.2
Min. KS
47.7
DC-PDD
49.0
71.4
51.2
Info RMIAI
40.5
HackerNews
Average
Method
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2,8B
Loss
50.7
51.9
49.0
48.3
49.2
50.4
51.2
52.4
Zilb
50.0
50.8
51.3
48.2
48.1
48.0
49.6
50.2
50.9
$2.0
52.6
Min-K%%
Min-K%e++
48.7
49.9
50.3
51.1
53.1
DC-PDD
50.2
55.3
Inf-RMMA1
23

---

Published as a conference paper at ICLR 2026
Table 13: TPR @0.1% 1% FPR MIMIR benchmark for deduped Pythia models using the step 100k checkpoint the 160m model. Note that the performance InfoRMII deteriorates as the reference model becomes less OUT when using later checkpoint.
Wikipedia
Github
Pile CC
PubMed Central
Method
160M
1.4B
2.81
160M
1.4B
2.88
160M
1.4B
2.8B
16UM
1.4B
2.8B
Loss
0.0
0.2
5.7
4.8
7.9
0.2
0.0
0.0
Zilb
0.1
0.1
8.6
0.2
0.2
0.0
Min-K%
0.0
0.2
5.7
8.0
0.2
0.0
0.0
Min-K%e++
0.0
0.1
4.6
0.2
0.0
DC-PDD
0.0
0.0
3.7
0.3
0.0
0.0
DM Mathemattess
HackerNews
Average
Method
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
Loss
0.1
0.0
0.0
0.2
0.0
0.0
0.8
0.7
Ziib
0.0
0.0
0.0
0.0
0.2
0.2
1.2
0.9
Min-K%
0.0
0.0
0.0
0.0
0.1
0.7
Min-K%e+
0.0
0.0
0.5
0.0
0.5
DC-PDD
0.0
0.0
0.1
Info RMEAI
Table 14: TPR @ 1% FPR MIMIR benchmark for deduped Pythia models using the step 100k checkpoint the 160m model. Note that the performance InfoRMIA deteriorates as the reference model becomes less OUT when using later checkpoint.
Wikipedia
PubMed Central
Method
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
16UM
1.4B
2,8B
Loss
0.9
0.6
13.1
13.3
21.9
0.7
0.8
0.7
0.4
Zlib
1.3
0.8
14.3
24.0
0.7
0.9
0,3
0.4
Min-K%
1.4
12.0
21.8
0.6
0.2
Min. KSee-
1.2
DC-PDD
0.9
9.8
0.8
Info RMEAI
1.0
4.6
0.9
HackerNews
Average
Method
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2.8B
160M
1.4B
2,8B
Loss
0.7
0.4
0.5
1.1
0.7
2.4
Zilb
0.5
0.4
1.1
0.9
0.9
1.0
1.0
2.7
3,0
Min-K%%
0.3
0.2
Min-K%e++
1.1
1.0
1.2
0.5
2.6
DDD
0.6
1.1
info-MMAA
1.3
24

---

Published as a conference paper at ICLR 2026
Table 15: AUC results MIMIR benchmark for GPT-Neo models. Note that the results here only for curious readers, because the reference model is completely IN, breaking the offline attack assumption.
Wikipedia
Github
Pile
PubMed Central
Method
1.38
2.7B
1.38
2.78
1.31
2.7B
1.3B
2.78
Loss
31.0
51.3
68.1
69.9
50.0
50.4
49.6
49.8
Ziih
51.7
51.9
69.6
71.3
50.0
50.5
49.7
49.9
Min-K%
51.2
68.2
70.1
50.3
50.7
50.0
50.1
Min-K16++
51.5
53.4
68.2
70.2
49.7
50.4
51.1
51.4
DC-PDD
51.2
50.7
50.3
51.7
46.9
52.5
49.3
50.0
Info RMEAI
51.5
49.1
Info-RMIA2
53.2
52.4
DM Mathematies
HackerNews
Average
Method
1.3B
2.7B
1.38
2.7B
1.3B
2.7B
1.3B
2.7B
Loss
51.1
51.5
48.6
48.5
49.9
50.2
52.6
53.1
Zlib
50.6
51.0
48.1
48.1
50.1
50.2
52.8
53.3
Min-K%
51.7
49.2
49.2
51.4
53.0
53.5
Min-K9e++
49.6
50.8
53.3
54.1
DC-PDD
47.7
Info RMIAI
50.5
RMIA2
54.0
Table 16: TPR @0.1% FPR on MIMIR benchmark for GPT-Neo models. Note that the results here are only for curious readers, because the reference model is completely IN, breaking the offline attack assumption.
Wikipedia
PubMed Central
Method
1.3B
2.7B
1.38
2.7B
1.3B
2.7B
1.3B
2.7B
Loss
0.1
4.7
0.1
0.2
0.0
0.0
Ziib
0.1
13.7
0.2
0.2
0.0
0.0
Min K%
0.1
4.8
0.2
0.2
0.0
0.0
Min-K%6+
0.2
0.0
DC-PDD
Info
0.2
HackerNews
Average
Method
1.3B
2.7B
1.38
2.7B
1.3B
2.78
1.3B
2.78
Loss
0.0
0.9
Zilb
0.0
0.0
0.0
2.0
Min-K%%
0.0
Min-K%
0.2
DC-PDD
Info-MMAA
0.0
Info RMIA2
0.2
25

---

Published as a conference paper at ICLR 2026
Table 17: TPR 1% FPR on MIMIR benchmark for GPT-Neo models. Note that the results here are only for curious readers, because the reference model is completely IN, breaking the offline attack assumption.
Wikipedia
Github
Pile CC
PubMed Central
Method
1.3B
2.7B
Loss
0.6
0.6
18.7
22.0
0.7
0,7
0.2
0.3
Zlib
0.5
0.6
20.1
22.2
0.7
0.8
0.3
0.3
Min K
0.6
0.5
18.9
22.3
0.7
0.5
0.3
0.6
Min-K%e++
0.6
0.5
19.0
21.9
0.8
0,9
0.5
0.5
DC-PDD
0.5
0.7
16.1
20.0
0.8
0,9
1.1
0.9
Ref
0.9
11
2.0
2.6
1.0
1.0
1.3
Info-RMIAI
0.8
0.9
2.8
3.0
1.0
0,9
1.0
0.7
Info RMIA2
0.9
12.5
0.9
0.5
0.6
ArXiv
DM Mathematics
HackerNews
Average
Method
1.3B
2.7B
1.3B
2.7B
L.3B
2.7B
1.3B
2.7B
Loss
0.9
0.6
0.5
0.4
0.7
0,9
3.2
3.6
Zlib
0.4
0.5
0.5
0.5
0.5
0,5
3.3
3.6
Min-K%
0.7
0.5
0.2
0.2
0.7
0.8
3.2
3.6
Min-K%e+
1.2
144
1.1
1.1
0.8
0.8
3,4
3,9
DC-PDD
1.2
0.7
0.3
0,3
L.1
1.6
3.0
3.6
Ref
1.9
0.6
1.4
0.8
1.3
1.3
Info-RMIAL
1.5
0.8
1.0
1.8
1.3
1.3
Info MMM-2
1.3
1.2
1.0
2.6
2.3
Summary statistics of token membership scores grouped by entity type on AG News, sorted by mean scores. Top-1% scoring tokens are called "high" tokens. "n_high" the number of high tokens, and high_rate" is the proportion of the tokens in each entity being high scoring. "None" entities are not nouns, which are unsurprisingly the majority.
Table 18:
entity
count
mean score
median_scoee score
p95
n h_high
high_rate
PERSON
2225
0.156000
0.103894
0.847365
50
0.022472
WORK SORKKOFFAA ARA
107
0.135550
0.056464
0.695698
3
0.028037
PRODUCT
75
0.122673
0.068920
0.854701
0.000000 0
FAC
136
0.119761
0.067262
0.894820
0.007353
LOC
161
0.117694
0.078282
0.729102
0,006211 1
TIME
159
0.115637
0.080404
0.621918
0.006289 1
ORG
4624
0.113697
0.073791
0.729737
0.016003 74
GPE
1587
0.107486
0.064042
0.634189
20
0.012602
QUANTITY
79
0.107218
0.076787
0.625024
0.012658 5
MONEY
1139
0.103070
0.081028
0.464732
0.004390
None
36188
0.094550
0.056949
0.646802
327
0.009036 2
EVENT
188
0,094198
0.051614
0.684139
0.010638
NORP
391
0.086809
0.063955
0.619991
0.010230 4
ORDINAL
134
0,081780
0.039014
0,571259
0.000000 4
CARDINAL
720
0,072828
0.051281
0.558225
0.005556
PERCENT
123
0.052848
0.037016
0,407925
0.000000 0
DATE
1798
0,048768
0.031457
0,501247
6
0.003337
LAW
5
0.005115
-0.096963
0.653592
0
0.000000
LANGUAGE
3
-0.149360
-0.130233
0.017094
0
0.000000
26

---

Published as a conference paper at ICLR 2026
Table 19: Summary statisties of token membership scores grouped by their privatc/non-priivtte status the ai4privacy dataset.
token
count
mean
std
min
10%
50%
90 %
max
Non-private
147411.
0.090224
0.303371
-4.658639 -4.478451
-0.088854
0.056127
0.304669
8.894643
Private
36340.0
0.076426
0.320213
-0.164783
0.048231
0.340272
7.925481
104
102
NONE STREET
LASTNAME1
CITY
US EMAIL GIVENNAME
STATE
IDCARD
TITLE
BOD POSTCODE
PASS
LASTNAME2
IP
TEL PASSPORT
TIME GIVENNAME:
SEX COUNTRY
DATE BUILDING
LASTNAME3
GEOCODRSS
Figure 6: Distribution of high scoring tokens according to their types in ai4privacy dataset. The
y-axis is the number of tokens in log scale.
27

---

Published as a conference paper at ICLR 2026
Seq 345 In0ex=330n alg=0.451, avg_priv=0.2443
1989. With the garimanjaly DeghoiallyyDotthall... her communication channel. w her P2TWC0501915 like badge honor in this virtual 794-5305 always ready strategic discussions gamers. Armed the opZ37", she fearlessly navigates through quests and challanges, embodying strangth determination, Joining her an this gaming advantur
Seq @ampll_index=144222 avg=0.440, avg pg__priv001121
813", entrr__date". "2049-11-210000000000 "entry, time": AM", "location"; :8866 "behaviors": ["Practiced
tolerance
techniques" "Used interpersonal effeciiveness skills", "Reviewed diary cards"), "reactions': :: Fell empowered seedeeeedd by distress tolerance practice", SSccessfully applied interpersonal skills in difficult situation", "Identified patterns diary card review" "J) ("entry Id", 3, Tuser Id";
"flinnagujnluzlvxde" "passportii"": "9"
Seq 234 (sample Index=8324, avg=0.367, avg. prix=0228)
palasingam will present case study highlighting the successful implementation of sustainable water management practices in the region. During the seminar, will also de into the challenges faced resource authorities, as illustrated by rodi sprugesci's reseerch the impact water Scarcity
cammunities. grmnnddtjoBB will offer valuable insights the legal implications transbound
Seq 113 (sampla_jndexe17333 0v3=0.353, wgg_priv=0.180)
they also create support school uniforms playing studants from diverse socioecanomic backgrounds. helps pravent discrimination based clatning brands." Username: Poulailion Comment:
onat Comment:
"Although uniforms restrict
expression
aelonging and school pride that creating
posilively impact overall school atmosphere."
"From teacher's ' perspect
Seq 233 (sample_ (sample_inexx=622222 Jndex:15283, avg=0.334, avg_prive00 03)2)
knowledge sharing sessions led by "Boogjo"* enhance curriculum ceherence. "PPooessionaal Development Activities:** "*Pope** to lead workshop on integrating technology into curriculum design. 2. Organize conference on culturally responsive teaching strategies guided by "*I "Lobada**, Establish peer ceservation groups to promote best practices in **Dnniso"* and "CClarkton" schools. 4, implement feedback mechanism utilizing "iyxmpwabq
Seq 491 (sample, 8189 0v3=0.331, avg. prgveriv))
stratégle commerce éiectronique constante évolution, réuss dans marché dynamique, actions objectifs pour prospérer
Business Plan e-commerce "Introduction""
objectifs Notre plan notre secteur commerce ligno. "*Stratégias clés** "Sogmentation Marché**:
Nous utilisetons les informations de nos clients pour diviser le marché en segments spécif
Seq 434 (sample_inder jediijiddxxo00 20020, avg avg-0.3300 0.330, avg.. nng_privinan)
Team Collaboration Platforms for Enhanced Pediatric Care Dear Team, our continuous efforts improve pediatric care services, we are excited to introduce new team collaboration platform that will streamline our communication and anhance patiert care outcomes, This platform aims to leverage technology to ensure efficient coordination among healthcare professionals and enhance the overall quality of care provided to our young patients. Key Features
Seq 38
wgg-0.326, wwg_priv=0.129)
4. 23/03/1962 Esperto legale sulle questioni delle spazio 5. 02/02/1965 Espertà tecnica in comunleazion satellitari 6. 18" tebbraio 1972 Kaporesentante del settore degii irvestimenti spaziali agesta/02 Consulente di sicurezza spaziale RESPONSABILI parti concordano sulle seguenti clausole: part impegnano rispettare normative spazieli nazionali internazionali. satelitare sarå regola
spettro
Seq [annsll_nddex7756,, avg=0.326, wva_priv=0.191
Date": "21st November 2022", "Severance Pay": "S10,300", "Working Notice Period": "2 weeks", "Vacation Pay Owed": "$1,500", "Lump Sum Payment*: *85,000", RReturn of Company Property Deadline": 7 days", "Confidentiality Clause": "Employee shall not disclose any confidential information after termination.) }}}
Seq 307 (sample_inden jnd3x-11138, av3:0.314, p
2K "Coomunlcation with therapist:"* 06457998 - Female: (CLIENT FEEDBACK] 3. "Contort eval during the session:-1 06457998 Female: [CLIENT FEEDBACK] 4. *'mpact of the therapy on your well well-being::* Q8457998 Female: CLIENT FEEDBACK] 5. "Suggestions for Female: (CLIENT FEEDBACK] --- "PPlease repeat the above format for all remaining clients)*
improvement:" 08457998
Figure 7: Top- memorized sequences by finetuned GPT-2 model ai4privacy dataset, identified by InfoRMIA, ranked by sequence-based membership scores. Some of them do not even have any private tokens. Others have disproportionately small private token average scores.
28

---

Published as a conference paper at ICLR 2026
Top 10 sequences by average private-token score
Seq 76 (sample_indee in1e__111dd1110 2vg=0.207, cvg_priv=0.687)
is": "Autismo moderato", "Specific_ Behaviors": "Aggressione fisica, difficoltà nell' nel'eesrrssionee verbale" MXXARL5507225 49
"Skarl "Murteza", "Diagnesis". "Disturbi dello spettro autistico specificat", "Speeiiff Specific_Beeaviios"" "Rumore eccessivo, interazione sociale" }33),{
Seq 377 (nmplle_ddex:1178,
ava_privm0.553)
renessicfo>
rasidants risks alcahol ellbs SuppotSuuppa Servicas:/hb> programs strupaling with alcohol dependency-<hMi cilscb-Reuulationnoh Implement policies
access counseling support alcoho advertising and availability in community-shhe </ul> <h2>inddvidual Responsibillieeshh22 <p>Each member of the community, including [Lodile]] [[Hatice]] and [[Jamrat]]
Seq 216 (sample_innde 9nd8x avg=0.172, avg_.rrv=00.51 0r51=
Betreff: Anfrage zur Rückmeldung Aapeliaticnspraxis Sehr geehrte Camen und Herren, ich hoffe, diese Nachricht erreicht Sie im besten Wohlbefinden. Gerne würde ich Ihr wertvolles Feedbacck zu meiner jurislischen Angelegenhe erheiten. Meine Kontaktdeten und weitere Deettalls des Antragstellers: ens: 1. Antragsttlllar Mohammed berhan
Informationan finden Sie unten. Datum der Anfrage: Mai 11,, 2065
Geschlecht:
Seq 223 (sample_indes incplli___oooiieee 127813, avg-0.154, avg_priv:D.479)
contribuisce plesmare futuro della nostra comunità, Le chiedo cortesemente portare suo 465345506 come documento alle fondamentale d'identià valido partacipare alle domanda chaarment per suo coinvoigimento impegno civico. Cordialmente, tuo inclusiva. disposizione Nome] tuo Ruclo]
rappresentanza democratica
(sample_indessee index:1694444, avg=0.176, avg. pr__priv=0.43))
RUG Costa Berrituaa ) "EmplayenNamee "EmployeelD": '24887', "TrainingProrrem" *Strategic Planning in Healtheare", "CompletionDate" "2023-03-05", "CertificatelD": '418-Calle Valla. Valdernueda Ranedo de Valdetudjar" }])
Management', "ComplationDatee"
Seq 583 Isample index=96225,, avg=0.099, avg avg__rivest 4335=
genden finden Sie die Protokolle sawie die detaillierten Aufzeichnungen der heutigen Vorlesung: 1. Teilnehmer STUDENT_ Benutzername: 1991M32 - Perspeklive: Zweite Person Plural 2. Teinnhhmer STUDENT H: Benutzernanee ii aree Perspekkive: Formale dritte Person Plural S. Tellnehmer STUDENT_GC C:
Banutzernmm:: 94kerrin Perspektive: Formale zwe te Person Plural 4.
Seq 171 [sample_ index=144775, avg=0.167, avg avg__rrieee _2_prx=00411)
covestion_contentt content": "In che modo gesiisce diritti di licenza per contenuti digitali?* ), ("questioo numtier "question, "ovestion_ccnteettt ": "Quali sono tendenze emergenti che impattano gestlone dei diritti digital??" }]), "applicant "iitle" "infante", "designation" "Avvocato Benior "country": "Switzerland", "specialty": "Online Privacy Law". "questions":
"question_
Seq 447 (sample__im 78987 avg=0.152, wgg_priv=00407
Commentaires suivi projet Service d' d'oncologie pédiatrique, Hôpital de Genéve Chers collégues, Nous souhaitons partager contributians chaque
cl-dessous détaillé l'équipe Nom Kusa ch Adresse 230.223..1122 Commentaire
projet oncologle pédiatrique,
avons
dornéés pour éveluer progression tra
Seg (sample_index 5966,, 0113,, avg_ wwg_priv=0.402
"aaml health_prommeeeee wwwpprrmmttonccommm commattionccmmalinn::: campaignt: "Campaña de Pramoción la Salud para Todos' "Involucra comunided
hábitos saludables y prevención de enfermedades:" [ "odos los miembres de comunidad" id: "Famentar la adopción de un estilo vids saludable conciencia sobre salud preventive' id: Talleres educativos, sesiones informativas, actividades al aire libre" id: "15-07-2023* id:
"15 08 2023" id: "9:55" id: "España"
Seq 417 [semple_index=166600 165877 avg=0.201, avg. vu__rrv=0.400)
Buen dia, me gustaria particigar el estudio de investigación que han mencionado clase. Quedo atento cualquier requerimiento adicional. *Corree Asunto: Horario de Cleses Remitente: Masculino N" Documento: V27969571 Mensaje: Hola, desearia conocer si pasible medificar horaria de pronta respuesta.
presantado confiictos
actividad extracumicuiar. Quedo espera
Figure 8: Top-10 10 sequences that have the highest average scores of private tokens by finetuned GPT-2 model on ai4privacy dataset, identified by InfoRMIA. Note that their sequence averages are much smaller compared to those in Figure and the average private token scores. This is aligned with our intuition that signals from private tokens can get diluted in long texts.
29


Reviews:
[{"review_id": "XRn3o6RVOq", "reviewer": "Reviewer_Vt48", "summary": "This paper presents InfoRMIA, a membership inference attack (MIA) designed specifically for large language models (LLMs), specifically based on the existing RMIA framework. Then the authors argue that sequence-level MIAs are fundamentally flawed for LLMs, as the privacy risk is localized to individual tokens rather than entire sequences. To address this, they propose a token-level privacy assessment framework that can more accurately identify privacy leakage at the token level.", "questions": "1. The organization of the paper could be improved. Specifically, Section 3.1 briefly introduces RMIA, without much detail, and even the problem statement and threat model are not clearly defined. The authors directly jump into the technical details of InfoRMIA without providing sufficient context. RMIA is a relatively new method, and readers may not be familiar with it, even those who are well-versed in membership inference attacks. \n\n2. Another issue linked to the unclear threat model is that it is not clear what the attacker's capabilities are. For example, I checked Appendix A, and saw RMIA requires a dataset that is drawn from the same distribution. Does InfoRMIA have the same requirement? And although this assumption is common in traditional MIAs, it needs further discussion in the context of LLMs. Specifically, for traditional MIAs, the models are usually trained on relatively small and homogeneous datasets, like training images on CIFAR-10. So it's clear what \"the same distribution\" means. However, LLMs are typically trained on massive and diverse datasets, making it less clear what \"the same distribution\" means in this context. For example, if we want to infer whether a medical text is part of the training data, are we supposed to sample other medical texts from the same distribution as the target text? And many sensitive data are quite rare, making it difficult to find similar data points from the same distribution. Therefore, I would like to see a more detailed discussion on this assumption and its implications for the practicality of InfoRMIA.\n\n3. The idea of token-level membership inference needs further discussion. I agree that certain tokens in a sequence may pose greater privacy risks than others. However, the definition of privacy leakage for individual tokens requires further discussion. The privacy risk of a token are also dependent on its context within the sequence. For example, the name of the patient in a medical report may be sensitive, but if it is appears in an author list of a research paper, it may not be sensitive at all. Similarly, a password is highly sensitive if it's surrounded by the corresponding login information, but if it appears in a random sentence, it may not be sensitive. Therefore, claiming sentence-level MIAs are fundamentally flawed may be too strong without further discussion.\n\n4. Experimental results in Table 1 are hard to interpret. I would expect that sampling more data would lead to better attack performance, and it's true for RMIA. However, for InfoRMIA, the attack performance remains identical even to four decimal places. It would be helpful to provide some explanation for this phenomenon.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Membership inference attack on large language models is a critical area of research.\n2. The proposed InfoRMIA method is effective.", "weaknesses": "1. The organization of the paper could be improved, particularly in providing sufficient context for RMIA and clearly defining the threat model.\n2. The threat model needs further discussion, especially regarding the assumption of having access to a dataset drawn from the same distribution.\n3. The idea of token-level membership inference needs further discussion, particularly regarding the definition of privacy leakage.\n4. Experimental results in Table 1 are hard to interpret.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HadQuG2mqE", "reviewer": "Reviewer_Kseq", "summary": "This work presents InfoRMIA an information theory centric method for membership inference. There are two key contributions: a continuous test static which is superior to a discrete statistic that RMIA uses. This improves AUC and TPR, at low FPR. The second contribution is a token level member inference framework. This is distinct from currently used sequence level methods. The results with a token level framework show that sequence level scores are not accurate due to conflation of private and non-private tokens within a sequence.\n\nThe work shows that this method outperforms RMIA and is less sensitive to population size. In addition the token level membership inference results that are stronger than sequence level membership inference. They especially point to sequences which are memorized with no private tokens over estimating the privacy risk, while also pointing to the challenges of using AUC as a privacy metrics", "questions": "1. How is p(z/theta) instantiated over the LLM implementations. Is this treated as an empirical distribution or logits over some token set? Please provide details around normalization, smoothing and detail how is KL term impacted by these?\n\n2. What is the runtime overhead when comparing sequence level methods on longer inputs and large vocabularies?\n\n3. For MIMIR experiments how sensitive are results to an early checkpoint, domain mismatch between reference and target?\n\n4. Would it be possible to include other reference based MIAs to support SOTA claims?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Better test statistic: Current work reframes RMIA’s setup as a composite hypothesis test, propose a log-likelihood ratio that decomposes into per-sample info gain term and a KL divergence term. This decomposition into memorization and generalization term provides a good interpretability/diagnostic lens,  importantly eliminates gamma, and removes sensitivity of populations set Z. \nThe derivation is sound and statistically rigorous. Computationally efficient and avoids scaling with large population sets.\n\nThe equation turns is smoother, information theory centric relaxation of RMIA equation, there by turns discrete counting into a counting BayesFactor score. This makes sense conceptually and is mathematically sound. \n\n2. Empirically the work shows that this improves AUC and TPR, at low FPR on tabular, image and text datasets. The overall results show that this superior to RMIA.\n\n3. Token level MIA enables a more granular signal and thereby enables more precise decision making related to membership inference. The work also argues and shows sequence level MI conflates private and non private tokens. Thus token level MIA yields a stronger MI.", "weaknesses": "1. The KL term assumes normalized distribution over data. However details around estimator or smoothing do not seem fully specified.  This could impact the KL term i.e. how sensitive is KL term to these choices\n\n2. RMIA seems to be the baseline comparison. Are there other MIAs with which the model can be compared to make SOTA claims?\n\n3. Token to sequence aggregation: performance depends on type of aggregation like min-k, mean. While the paper acknowledges this it will be good to show additional aggregators like trimmed mean and also try tuning free calibrations to reduce dependence on min-k", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wQE1zt1bQU", "reviewer": "Reviewer_hMRp", "summary": "This paper proposes a fine-grained member inference attack, specifically at the token level, called InfoRMIA, which calculates the leakage of individual tokens and aggregates them to the sequence level to achieve stronger inference capabilities.This paper conducts an in-depth analysis of the existing best MIA (RMIA), and then proposes a more principled statistical test method (InfoRMIA), and analyzes its advantages over RMIA This paper proposes a token-level MIA framework to better quantify the memory and information leakage of LLMS with finer granularity and more meaningful analysis", "questions": "The article proposes a new attack method, InfoRMIA, and the author clearly explains the reasons why it is superior to RMIA, reducing the dependence on the size of the dataset Z and improving the accuracy by using Bayesian factors. The author pointed out the shortcomings of the traditional MIA, such as information loss caused by global averaging, and proposed token-level member signal analysis, which is of great practical significance and value. The author pointed out that under the traditional MIA test, tokens with high member scores are more likely to contain less sensitive information, thereby highlighting the limitations and flaws of the AUC metric, that is, AUC to a greater extent reflects the member signals of non-sensitive tokens. These issues are all very valuable and worthy of in-depth consideration. Moreover, the author expounds these issues with clear logic and evidence, which is highly persuasive \n\nHowever, I do have the following comments:\n \n- Lack of more effective aggregation methods\nOn page 2, the author claims that aggregating from token-level metrics to sequence-level metrics would make it less significant for privacy assessment. However, the method used when migrating from the token-level to the sequence-level in the author's proposed approach is also a common aggregation method. Is this somewhat contradictory\n- Lack of a mechanism to distinguish tokens from the opponent's perspective\nOn page 5, the author states that the work of this paper can precisely locate information leakage, and the scenario proposed by the author is that users observe the member scores of each token and check whether these tokens are sensitive. I don't think this is the most appropriate approach, because in member inference attacks, the adversary needs to filter and confirm which tokens are truly sensitive, and this process should be completed automatically, as the amount of data could be extremely large. This requires the formulation of some kind of recognition algorithm to complete the screening of truly sensitive tokens. For tokens that will not disclose privacy, even if their member scores are high, they should still be excluded\n- The truly sensitive information is diluted or concealed(The same as the first point)\nOn page 6, the author mentioned better aggregation methods, but glossed over them due to their limitations and used common aggregation methods such as averaging and min-k. However, before screening out the truly sensitive tokens, simply averaging the tokens in the sequence obviously leads to information loss. The author also mentioned the shortcomings of the averaging method, but did not use a better aggregation method\n•No method for distinguishing between sensitive and non-sensitive tokens was proposed(The same as the second point)\nIn many places in the text, the author points out that the traditional sequence-level MIA has shortcomings because it ignores fine-grained member signals, and the signals of non-sensitive tokens may dilute or mask those of sensitive tokens. On page 7, the author points out the deficiency of the AUC metric, that is, AUC may reflect the signal of non-sensitive tokens to a greater extent. However, throughout the entire article, the author did not provide any effective methods for distinguishing sensitive tokens from non-sensitive ones. They merely claimed that users could check by themselves whether a token was sensitive, but this was clearly not from the perspective of an adversary. Therefore, I think the author should provide an automated algorithm to distinguish between sensitive tokens and non-sensitive tokens, making the article more persuasive\n\nMinor issues：\nDespite pointing out the limitations and flaws of the AUC metric, the results presented on page 8 show that the attack in this paper achieved a higher AUC value, which sounds somewhat strange", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clear theoretical framework construction\n- Clear and concise formula and algorithm expressions", "weaknesses": "- Lack of more effective aggregation methods\n- No method for distinguishing between sensitive and non-sensitive tokens was proposed\n- The truly sensitive information is diluted or concealed", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KPdoBkzV7h", "reviewer": "Reviewer_zEAx", "summary": "This paper introduces InfoRMIA, a novel membership inference attack (MIA) method for large language models (LLMs) that significantly improves upon the original Range Membership Inference Attack (RMIA). The authors propose a theoretically grounded approach based on information theory principles, using a continuous test statistic that eliminates the need for the hyperparameter γ required in RMIA. The paper demonstrates that InfoRMIA provides higher precision in membership scores and reduces computational overhead. Additionally, the authors conduct token-level analysis, revealing that personally identifiable information (PII) such as names of people and artworks is disproportionately more memorized by LLMs compared to other token types. The paper shows that sequence-level privacy metrics like AUC may overestimate privacy risks, as they often reflect memorization of non-private content, making them poor indicators of true privacy risk.", "questions": "Q1: Could you provide more detailed computational complexity analysis comparing InfoRMIA and RMIA, including execution time and memory usage measurements across different dataset sizes? This would strengthen the claim about reduced computational overhead.\n\nQ2: Would it be possible to expand the token-level analysis to include more diverse datasets across different languages and domains to better generalize the findings about PII memorization patterns?\n\nQ3: Could you investigate the reason behind the observation that private tokens sometimes show lower membership scores than non-private tokens in ai4privacy? This could have important implications for privacy risk assessment.\n\nQ4: How might the insights from your token-level analysis be used to develop more effective privacy-preserving training methods or defenses against membership inference attacks?\n\nQ5: Could you compare InfoRMIA with other state-of-the-art membership inference attacks beyond RMIA to better position the contribution of your work in the broader field?\n\nQ6: The paper mentions \"token-level framework\" but doesn't clearly explain the methodology for token-level analysis. Could you provide more details on how the token-level scores are calculated and how they relate to the sequence-level scores?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1: The paper provides a strong theoretical foundation for InfoRMIA, deriving a continuous test statistic based on information theory principles that is more precise than the discrete statistic used in RMIA.\n\nS2: The token-level analysis is a significant contribution that reveals insights not captured by aggregate metrics like AUC, showing that PII (personally identifiable information) is disproportionately more memorized by LLMs.\n\nS3: The paper effectively demonstrates that sequence-based frameworks overestimate privacy risks by showing that the most memorized sequences often contain no private tokens (Figure 2).\n\nS4: The experimental evaluation is comprehensive, showing InfoRMIA's superiority over RMIA across multiple standard datasets (Purchase100, CIFAR-10, AG News) and models.\n\nS5: The paper clearly explains why AUC alone is a poor indicator of true privacy risk in LLMs, providing a valuable insight for the privacy research community.", "weaknesses": "W1: The paper lacks sufficient analysis of InfoRMIA's performance across different LLM architectures and sizes. The experiments are limited to a few standard datasets and models, which may not fully represent the diversity of real-world LLMs.\n\nW2: While the paper claims \"reduced computational complexity,\" it does not provide specific quantitative comparisons of computational costs between InfoRMIA and RMIA, making it difficult to assess the practical significance of this claim.\n\nW3: The token-level analysis is limited to only two datasets (AG News and ai4privacy), which may not be sufficient to generalize the findings about PII memorization patterns across different domains and languages.\n\nW4: The paper does not discuss potential defense mechanisms against InfoRMIA or how the insights from the token-level analysis could inform the development of more privacy-preserving training methods.\n\nW5: The counterintuitive finding that private tokens sometimes show lower membership scores than non-private tokens (in ai4privacy) is not sufficiently explored, leaving an important observation underdeveloped.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ipmci0Wka0", "reviewer": "Reviewer_pY12", "summary": "This paper studies data memorization issues of LLMs through membership inference attacks. The paper proposes InfoRMIA, a membership inference attack stronger than the Robust Membership Inference Attack (RMIA). InfoRMIA extends membership inference attacks from the sentence level to the token level, providing a more fine-grained analysis of privacy.  Experimental results suggest that their method achieves new state-of-the-art performance and offers a more detailed perspective on memory leakage in large language models.", "questions": "1. I may miss some paper details. Could the authors clarify the experimental setup in more detail, including dataset splits, model architectures, and training configurations? How are the auditor tools used?\n\n2. Could the authors include additional experiments to better demonstrate the robustness and consistency of InfoRMIA’s performance?\n\n3. Could the authors discuss cases where InfoRMIA underperforms or performs similarly to RMIA? Will LiRA be considered as one extra baseline?\n\n4. Could the authors provide more intuition behind the theoretical derivations, especially in Section 4.2?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1.\tThe paper proposes studying privacy from a token-level perspective, which offers finer granularity compared to the traditional sequence-level analysis.\n\n2. Experimental results show that  InfoRMIA achieves a new state-of-the-art performance, surpassing prior baseline RMIA.", "weaknesses": "1.\tThe theoretical derivations in the paper are difficult to follow and would benefit from clearer explanations and additional intuition. For example, in Section 4.2, it is hard to understand what the authors are trying to emphasize, as the section directly presents a formula without sufficient explanation or intuition.\n\n2.\tThe paper lacks a sufficient explanation of the experimental setup. While Section 5 introduces the analysis tools used, Section 6 directly presents the results and conclusions without clarifying the chosen baselines or model configurations.\n\n3.\tIt would be better to include more experimental results to substantiate the claimed SOTA performance. Although Tables 1 and 2 show that InfoRMIA generally outperforms RMIA, the advantages appear marginal in several cases (e.g., AUC 0.822 vs. 0.821 on the ai4privacy dataset). Moreover, some results in the appendix suggest that InfoRMIA’s performance is not entirely stable. Providing additional experiments and analyses would help strengthen the paper’s main claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)