Membership Inference Attacks Against Fine-tuned Diffusion Language Models
Authors: Yuetian Chen
Kaiyuan Zhang
Yuntao Du
Edoardo Stoppa
Charles Fleming
Ashish Kundu
Bruno Ribeiro
Ninghui Li
Venue: ICLR 2026 Poster (2026)
Keywords: Membership Inference Attack
LLM
AI Privacy
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: SAMA: First membership inference attack exposing privacy vulnerabilities in Diffusion Language Models. Achieves 8× better detection than existing methods through robust sign-based aggregation of multiple mask configurations.

Abstract: Diffusion Language Models (DLMs) represent a promising alternative to autoregressive language models, using bidirectional masked token prediction. Yet their susceptibility to privacy leakage via Membership Inference Attacks (MIA) remains critically underexplored. This paper presents the first systematic investigation of MIA vulnerabilities in DLMs. Unlike the autoregressive models' single fixed prediction pattern, DLMs' multiple maskable configurations exponentially increase attack opportunities. This ability to probe many independent masks dramatically improves detection chances. To exploit this, we introduce SAMA (Subset-Aggregated Membership Attack), which addresses the sparse signal challenge through robust aggregation. SAMA samples masked subsets across progressive densities and applies sign-based statistics that remain effective despite heavy-tailed noise. Through inverse-weighted aggregation prioritizing sparse masks' cleaner signals, SAMA transforms sparse memorization detection into a robust voting mechanism. Experiments on nine datasets show SAMA achieves 30\% relative AUC improvement over the best baseline, with up to 8$\times$ improvement at low false positive rates. These findings reveal significant, previously unknown vulnerabilities in DLMs, necessitating the development of tailored privacy defenses.

Published as a conference paper at ICLR 2026
MEMBERSHIP INFERENCE ATTACKS AGAINST FINE-TUNED TUNED DIFFUSION LANGUAGE MODELS
Yuetian Chen', Kaiyuan Zhang', Yuntao Du', Edoardo Stoppa',
Charles Fleming?, Ashish Kundu', Bruno Ribeiro', & Ninghui Li!
'Department of Computer Science, Purdue University
Cisco Research
{yuetian, zhan4057, ytdu, estoppa, ribeirob, jinghui]8purduee edu [chflemin, ashkunda]Jcisco.com
ABSTRACT
Diffusion Language Models (DLMs) represent promising alternative autore- gressive language models, using bidirectional masked token prediction. Yet their susceptibility to privacy leakage via Membership Inference Attacks (MIA) re- mains critically underexplored. This paper presents the first systematic investi- gation of MIA vulnerabilities in DLMs. Unlike the autoregressive models' sin- gle fixed prediction pattern, DLMSs' multiple maskable configurations exponen- tially increase attack opportunities. This abillity probe many independent masks dramatically improves detection chances. To exploit this, we introduce SAMA (Subset-Aggregatee Membership Attack), which addresses the sparse signal chal- lenge through robust aggregation. SAMA samples masked subsets across progres- sive densities and applies sign-base based statistics that remain effective despite heavy- tailed noise. Through rnverse-weighter aggregation prioritizing sparse masks' cleaner signals, SAMA transforms sparse memorization detection into a robust voting mechanism. Experiments on nine datasets show SAMA achieves 30% rela- tive AUC improvement over the best baseline, with up to 8x improvement at low false positive rates. These findings reveal significant, previously unknown vulner- abilities in DLMs, necessitating the development of tailored privacy defenses.
Introduction
Large Language Models (LLMs) demonstrate transformative capabilities across diverse applica- tions (Team, 2024; Qwen, 2025; DeepSeek-AL, 2025). While autoregressive models (ARMs) based on sequential next-token prediction remain dominant, Diffusion Language Models (DLMs) are compelling alternative (Gulrajani & Hashimoto, 2023; Gong al., 2024; Ye et al., 2025b). These models, leveraging bidirectional context to reverse data corruption by reconstructing masked tokens, are increasingly adopted for their strong scalability, performance competitive with ARM counterparts, and their ability to address ARM-specific limitations like the reversal curse (Berglund et al., 2024). Recent industrial developments, such Google's 's Gemini Diffusion (DeepMind, 2025), further underscore the growing relevance of this paradigm, offering performance competitive with significantly larger autoregressive models.
rapidly emerging
The privacy risks associated with LLMs, particularly their propensity to memorize training data, have been extensively studied for ARMs (Carlini al., 2021: Mireshghallah et al., 2024; Merity et al., 2018; Hartmann et al., 2023). Membership Inference Attacks (MIAs) (Shokri et al., 2017), which aim to determine a specific data sample was part of a model's training corpus, are the primary benchmark for quantifying these risks, with numerous techniques developed to probe ARM vulnerabilities (Mattern et al., 2023: Duan et al., 2024; Fu et al., 2024). However, for the burgeoning class of DLMs, which operate distinct principles of iterative, bidirectionally-cnneee conditioone mask prediction, the extent and nature of such privacy vulnerabilities remain critically uncharted. Their unique architectural properties deviate significantly from ARMs, rendering existing MIA strategies

---

Published as a conference paper at ICLR 2026
(a)
(b)
The doctor
prescribed
[MASK]
2x
Target Model
symptoms
f
A(S)
[MASK] doccor
[MASKQ]
the
medication
afteer
[MASK)]
Inference
[MASK]] symptoms
[MASK]
Coctor
[MASK] [MASK]
Prosicted Membersile
-0.3
eF
medicasion
[MASK]
restewing
[MASK]
[MASK]
[MASK]
[MASK]
$
Reference Modelé,
Figure I: An overview of the SAMA. (a) An input sequence The doctor prescribed the medication
undergoes progressive masking over steps, accumu- lating masked positions. (b) The target and reference DLMSs reconstruction errors for masked to- kens [MASK] are tracked position-wis at each step ({(*)s=)) 1) Sign-based test statistics detect when specific mask configurations activate memorization, computing the fraction of configurations where reference loss exceeds target loss. This sign-based comparison, combined with inverse weighting. forms the membership score to differentiate training members from -ommmeeers.
after reviewing the patient symptoms
potentially ill-suited. The privacy implications of these distinctions have been largely overlooked. To the best of our knowledge, this work presents the first systematic investigation into the privacy risks of emerging DLMs through the lens of MIAs.
The absence of privacy analysis focused on DLMs constitutes an important gap and raises a key question: Do the unique paradigms of DLMs introduce novel vulnerabilities exploitable by MIAs? We focus on fine-tuning, where such privacy risks are typically amplified (Mireshghallah et al., 2022a; Fu al., 2024; Huang al., 2025; Meng al., 2025). In this work, we address this question through three key contributions:
1. We analyze how DLMss multiple masking configurations, unlike the single fixed pattern in ARMs, change the membership inference problem. The ability to probe many independent masks exponentially increases the chances of detecting memorization patterns learned during fine-tuning.
2. We empirically validate this heightened susceptibility and effectively exploit these unique char- acteristics by designing SAMA (Subset-Aggregated Membership Attack), an MIA framework that combines progressive masking with robust sign-based statistics statistics-addresstte addressing the fundamen- tal challenge that only sparse mask configurations reveal membership while most yield noise.
Through comprehensive experiments on state-of-the-art DLMs (i.e., LLaDA-SB- 2025). and Dream-vO-Based main splits in MIMIR (Duan et al., 2024), WikiText-103 (Merity et al., 2016), AG News (Zhang et al., 2015), and XSum (Narayan al., 2018)), we demonstrate that SAMA significantly out- performs a wide range of existing MIA baselines, thereby uncovering and quantifying critical privacy vulnerabilities.
(Nie al., (Ye et al., 2025a)) across diverse datasets (i.e., six different do-
The high-level overview of how SAMA works is shown in Figure 1, by systematically sampling diverse mask configurations across progressively increasing densities, then applying sign tests bust to sparse signals, we transform the detection problem from finding large signals in noise to identifying consistent patterns across multiple independent probes.
2 Preliminaries
This section describes the threat model under which SAMA operates, and provides background on DLMs, focusing on the mechanisms relevant to work.
2

---

Published as a conference paper at ICLR 2026
2.1 Threat Model
This paper investigates MIAs (Shokri et al., 2017) targeting DLMs, focusing on vulnerabilities intro- duced during -eeeitnnng. Throughout this work, and autoregressive models, respectively. The adversary aims to determine whether a specific text se- quence {are) E V4, where V is the model's token vocabulary and I. is the sequence length, was part of the training dataset Durain of target model Mbr- This is formalized as inferring the indicates x Draain. The adversary constructs an attack function ((x,Mrr that outputs a membership prediction p(x).
subscripts DF and AR to denote diffusion
binary membership status p(x) E {0, 1}, where μ(x)
We assume a grey-box access model, where the adversary can query the target model M with arbitrary inputs, including custom partially masked sequences. In response, the adversary receives detailed outputs such as predictive probability distributions or logits for specified token positions. However, the adversary cannot access internal parameters, gradients, activations. Following reference-basee MIA approaches (Watson et al., 2021; Mireshghallah al., 2022a; et al., 2024; Huang et al., 2025; Meng al., 2025), the adversary also has grey-box access to a reference model MBF. The ideal reference is the pre-trained base model from which MDr was derived, as this best isolates fine- seee-ttningsseeeee used, with implications discussed in Section D.6. Practically, this access model aligns with current open-weight deployment practices where fine-tuned weights are publicly released alongside base models. Furthermore, emerging DLM serving interfaces, such as LLaDA's in-filling demos (Hug- ging Face), inherently support the custom masked inputs required for this threat model.
memorization. When unavailable, alternative reference models can be
2.2 Diffusion Language Models
While autoregressive models predicting tokens sequentially represent the dominant LLM paradigm (Radford et al., 2019; Brown et al., 2020), DL.Ss offer distinct approach, approch,, gaining promi- nence in the language domain (Austin et al., 2023; Lou & Ermon, 2023; Shi et al., 2024a; Ou et al., 2024). These models typically learn to reverse a data corruption process, commonly implemented via iterative masking and prediction. This process starts with original sequence of length L, where randomly selected tokens are replaced by a special [MASK] token. We denote the set of masked positions as S S [L], where [L] {1, 2, L}. The model is then trained to predict the original tokens at positions in given the partially masked sequence.
Training Procedure. A notable example of DLM is LLaDA (Large Language Diffusion with mAsking) (Nie et al., 2025);
its paradigm detail the training formulation. The cen- tral component is a mask predictor, usually a Transformer architecture, which learns to predict the original tokens I; at the masked positions (i.e., i E S), conditioned on the unmasked con- text x-S := {0; : i E [4] S}. The model is optimized by minimizing the cross-entropy loss computed on mini-batches from the training dataset D For each training sequence x E D and randomly sampled mask configuration S, the loss is:
É(x, S)
Σ (x, iES
(1)
[S]
where p(z./x-s) denotes the probability assigned by model M to token æ; at position conditioned on the unmasked context x- and S is sampled with varying cardinalities to ensure the model learns reconstruct tokens under different masking densities. This training objective relies random sampling across the entire configuration space, where the model sees diverse, unpredictable masking patterns during training. This objective serves variational bound on the negative log-likelihood, grounding it in established generative modeling principles.
3 SAMA: Exploiting Bidirectional Dependencies in DLMs
DLMs process text through bidirectional masking mechanisms rather than the unidirectional ap- proach of autoregressive models. This architectural distinction creates fundamentally different mem- orization patterns during fine-tuning, which we investigate for membership inference vulnerabilities through proposed SAMA.
3

---

Published as a conference paper at ICLR 2026
3.1 Membership Signals in Autoregressive vs. Diffusion Models
We formalize how the architectural differences between ARMs and DLMs affect membership sig- nals, focusing on how each model type constrains the attacker's ability to probe for memorization. For text sequence x, define the membership signals as the loss difference between refer- ence model M and a fine-tuned target model M This difference captures memorization patterns introduced during fine-tuning.
In ARMs, the membership signals are deterministic and fixed by the autoregressive factorization:
AAR(x)= = LAR (x; LAR Lx: MAR AR
I Σ (x) (x))
where l log p(z;((<<)) represents the loss at position given only the preceding context. Cru- cially, this provides exactly one context configuration per token position- -he attacker observes only left-to-right dependencies, making any backward bidirectional relationships invisible to member- ship inference.
In DLMs, the membership signals depend on the chosen mask configuration S [[]:
(xF(x; LDF (x; 8, MM (x
(x (x,
(3)
where S) = - logp(x; represents the loss for predicting token given all unmasked
tokens in sequence
This DLM mask-configuration dependency creates both challenges and opportunities for member- ship inference. The challenge arises because an arbitrary mask configuration S chosen during infer- ence may not align with the masking patterns the model saw during training, resulting in weak or noisy signals. During fine-tuning, the DLM memorizes specific token relationships under particu- lar masking patterns; when a chosen configuration aligns even partially with these training patterns (e.g., sharing key masked positions preserving critical context tokens), the memorization acti- vates and the signals (x; S) becomes large. Conversely, configurations rarely or never seen during training yield weak signals dominated by noise.
However, this same property creates two distinct advantages for membership inference in DLMs. First, we gain multiple independent probing opportunities. Rather than extracting single fixed sig- nal A (x), we can collect a set of configuration-dependeet signals (x: S1). ADF S2),..., and if we sample enough diverse masks, we may uncover stronger membership signals than avail- able in ARMs. Second, the bidirectional context enables probing token relationships impossible in ARMs. Consider tokens x; and xr with a strong semantic relationship memorized during fine- tuming: in ARM, if j i, this relationship can never influence the prediction of 2; since I; is always masked, but in a DLM, we can probe this relationship through multiple configurations by masking only {ara} see how xy influences its prediction, masking only {xy } for the reverse, or masking both {2,, } to test their joint reconstruction. Each configuration potentially reveals different memorization patterns inaccessible unidirectional models.
To validate the hypothesis that membership signals in DLMs are related to the mask configuration, conducted controlled experiment on the ArXiv dataset. For balanced set of member and non- member samples, we evaluated 100 distinct random mask configurations per sample and analyzed the resulting distribution of signal strengths ADF (z; S). Figure 2 presents these findings. Panel (a) illustrates the aggregate signal densities, showing that while mon-member signals form symmetric distribution centered near zero, member signals exhibit distinct positive shift with heavier right tail. Panel (b) reveals the high intra-sample variance through violin plots of individual samples. The variance in signal strength caused simply by changing the mask configuration (a Rs 0.10) exceeds the average margin between members and non-members (6 s 0.06). This demonstrates that the membership signal is not a static property of the input text but fluctuates significantly depending on whether the mask hides specific "memorable" tokens. Consequently, single random mask is statistically likely to miss the sparse membership signal, necessitating a multi- coltiiionfiguuration probing strategy robustly distinguish members from non-members.
4

---

Published as a conference paper at ICLR 2026
(a)
(b)
0.75
Density
Aor(x:
0.50
0.25
1
0.00
-0.25
0.0
0.2
0.4
0.6
2
3
AoF (x; S)
Sample index
Members
Non-members
Figure 2: Empirical analysis of signal sparsity and configuration dependency. (a) The aggregate den- sity of signal strengths AnF(2; S) shows that member signals (red) are shifted positively compared to the more zero-centered non-nonneeen member noise (blue), yet possess significant overlap. (b) Violin plots for individual samples reveal that the intra-sample variance, fluctuations in signal strength caused solely by changing the mask configuration, is substantial. This high variance confirms that mem- bership signals sparse and confguration--ependenn motivating the need for robust aggregation over multiple masks rather than single-shot estimation.
3.2 Exploiting Sparse Membership Signals in DLMs
Traditional LLM MIAs (Fu et al., 2024; Xie et al., 2024; Wang et al., 2025; Duan et al., 2023) show limited effectiveness when applied DLMs, demonstrated empirically in Section 4.2. These methods naturally follow the training objective in Equation (1) by approximating the expected loss difference through Monte Carlo estimation:
DF A (x) (s) (x: S)),
where P(S) is the distribution over mask configurations used during training, and Apr S) is the loss difference for specific mask configuration as defined in Equation (3). In practice, this expec- tation is estimated by randomly sampling multiple mask configurations and averaging the resulting loss differences.
However, this averaging- approach is fundamentally limited in identifying membership sig- nals for two key reasons. First, within a single configuration, averaging over all masked tokens includes noisy predictions that obscure genuine memorization-not all masked tokens contribute meaningful signal, as the strongest loss reductions often stem from domain adaptation effects on domain-specifee tokens rather than instance-leve memorization. Second, across configurations, ran- dom sampling with simple averaging treats masking densities equally, ignoring that different densities operate at distinct signal-to-noise regimes: sparse masks provide stronger per-token sig- nals with fewer aggregation points, while dense masks offer weaker individual signals but more aggregation opportunities.
This motivates us to address both issues through a different aggregation strategy. Within each config- uration, instead of averaging over all masked tokens, we aggregate the sign of loss differences from multiple token subsets, as this binary decision is robust to outlier tokens that contribute primarily noise. Across configurations, extracting meaningful membership information requires careful consideration of how these signals vary with masking density. Different masking densities provide different signal-to-noise characteristies, which is analogous to how different noise levels in image diffusion models capture different aspects of the data (Ho et al., 2020; Song et al., 2020). We exploit this by progressively increasing the masking density, gathering evidence at multiple scales rather than attempting to explore all possible configurations.
Our method comprises three key components: (1) Robust subset aggregation: At each density level, we sample multiple token subsets and employ sign-based statistics to aggregate their signals, ensuring robustness against noise; (2) Progressive masking: We systematically increase masking density to capture signals at multiple scales, from sparse masks (strong per-token signals but fewer aggregation points) to dense masks (weaker individual signals but more aggregation opportunities); and (3) Adaptive weighting: We weight contributions from different density levels, prioritizing the cleaner signals from sparser masks while incorporating cumulative evidence from denser configura- tions when beneficial.
5

---

Published as a conference paper at ICLR 2026
3.3 Robust Subset Aggregation
At each progressive masking step {1,..., T} with mask configuration St S [Z] containing k masked positions, we obtain losses for all masked tokens through a single forward pass. However, directly averaging all by token losses is vulnerable to domination by extreme values, single token with exceptionally high or low loss can skew the entire signal. As we empirically demon- strate in Section E, the distribution of loss differences is governed by heavy-tailed statistics rather than Gaussian noise. Fine-tuning introduces domain adaptation effects that manifest as this long- tailed noise with occasional extreme values orders of magnitude larger than typical signals. These outliers often correspond to domain domain-specific tokens rather than instance-specioe memorization. Consequently, simple average is often overwhelmed by domain noise, whereas the true membership signal is consistent but sparse. To address this dual challenge, we employ two-part robust aggregation strategy: local subset sampling combined with sign-based aggregation.
Local Subset Sampling. We sample N random subsets of positions from St, where each subset Ut C. St, for n = ,,, contains m positions (typically m = 10) with m < kr For each subset U", we compute a localized loss difference:
AP; S,)
Σ (x. (x),
(5)
where i indexes positions in the sequence, and (x), (x, Si) are the reference and target model losses at position when sequence x is masked according to S..
vEZr"
This gives us N different local measurements rather than single global aggregate, each less suscep- tible to individual outlier tokens. This subsampling approach aligns with robust statistical methods where using multiple small random subsets provides stability against outliers (Rousseeuw & Leroy, 2003; Maronna et al., 2019). The principle is also employed in ensemble methods and stochastic op- timization, where aggregating over random subsets yields more robust estimates than using all data points (Breiman, 1996; Bottou et al., 2018). Each subset excludes different tokens, so an extreme value that dominates one subset's signal appears in only fraction of our N measurements, while consistent membership signals present across most subsets accumulate through aggregation.
Sign-Based Aggregation. Rather than using the magnitude of each localized loss difference, we transform each ADP into a binary indicator: (x) = [[ (x; S.) > 0). This transformation discards magnitude information, recording only whether the reference model has higher loss than the target model for that particular subset 2 While this might seem like discarding information, it actually provides robustness against heavy-tailed noise distributions.
For non-members, the target and reference models behave similarly since neither was trained on the sample. The loss difference D from subset Ltth is purely someeiess positive, sometimes negative, but centered at zero. Therefore, 8² equals with probability exactly 0.5, regardless of the noise distribution's 's properties. This holds even if the noise has infinite variance or no defined mean, robustness property that magnitude-base approaches cannot achieve.
For members, when we hit a mask configuration that activates memorization, the target model's 's loss drops below the reference model's loss, making ADr positive. Not every configuration activates memorization; most don't, but those that do consistently push 8D toward 1, By aggregating these binary indicators across our N sampled subsets, we compute (x = N ΣN (x)).
This principle that sign-based statistics remain efficient under minimal distributional assumptions is formalized in the Lehmmnn theorem (Hodges Jr & Lehmann, 201 I) and has been exten- sively validated in robust statisties literature (Huber, 1981: Peizer & Pratt, 1968). By combining local sampling with sign-based aggregation, we transform a single high-variance measurement into multiple robust binary decisions. This ensures that our membership detection at each masking den- sity is stable and reliable, accumulating evidence from the consistency of the signal rather than its magnitude, which is precisely what we need for DLMS where sparse activation means most config- urations contribute noise, but the few that reveal membership do reliably in their direction rather than their size.
3.4 Progressive Masking with Adaptive Weighting
Given the step-wise scores (3) from local sampling and sign-based aggregation, we now address how to effectively combine signals across different masking densities. As demonstrated

---

Published as a conference paper at ICLR 2026
empirically in Figure 2, the membership signal ADF S) exhibits high variance across different configurations. Relying on a single masking density or a static configuration creates a high risk of sampling a "null" configuration where the true membership signal is drowned by noise. To miti- gate this volatility, rather than fixing a single masking level, we progressively increase the masking density across 7 steps. Specifically, each step & f1,...;77 T), we set the masking density
t-1
Omin + T
(Amax Ominn)
(6)
where amin and amax (typically 5% to 50%) define the range of masking densities. This yields k2 [[.ar azz masked positions at step t.
This multi-scale approach exploits a fundamental trade-off in masking density. With sparse masks (low az), each prediction benefits from rich surrounding context, making memorization pattems more apparent when they exist-but we have fewer masked positions to aggregate. With dense masks (high ot), we gain more aggregation points, but each individual prediction becomes noisier as less context is available and domain adaptation effects become more pronounced. Intuitively, this principle mirrors established strategies in statistical hypothesis testing. Similar how multi-scale lesting improves detection power by examining data at various resolutions to find sparse signals (Arias-Castro Castro et al., 2011), our progressive masking captures memorization patterns that manifest differently across masking densities. Some memorization artifacts may only be detectable with sparse masks where strong contextual clues remain, while other patterns require aggregating evi- dence from many masked positions to overcome noise.
Empirical observation shows that signal quality degrades with masking density-early steps with sparse masks provide cleaner signals than later steps with dense masks. This motivates our adaptive weighting scheme:
T
T
N
SAMA(X) = Σ (x.3((x)
Σ
Wc
[x; (x;
> 0].
(7)
12
where W. Σ. 1/i is the inverse-step weight inspired by the use of harmonic means in robust statistics (Hedges & Olkin, 2014), S, denotes the mask configuration at step t, and (x;(x)) is the localized loss difference from Equation (5). This weighting prioritizes early steps where memorization signals are less contaminated by noise, while still incorporating evidence from all
1/a
scales.
3.5 Algorithmic Specification
We now present the complete SAMA algorithm that implements three-component strategy: pro- gressive masking, robust subset aggregation, and adaptive weighting. The SAMA algorithm op- erates in two distinct phases to extract robust membership signals from DLMs. We theoretically and empirically demonstrate in Appendix D.7 that this robust aggregation logic incurs negligible computational overhead compared to standard query-based baselines.
Phase I: Progressive Evidence Collection (Algorithm 1). This phase implements our progres- sive masking strategy, systematically increasing the masking density across T' steps from amin to Omax (typically 5% to 50% of tokens). At each step i, we determine the target number of masked positions k; and collect independent measurements through subset sampling uniformly without replacement.
Phase II: Sign-based Aggregation and Weighting (Algorithm 2). This phase transforms the raw loss differences into robust membership score. First, each difference A is converted to binary indicator B 1 | > 0), implementing our sign-based detection that remains robust to heavy-tailed noise. For each step t, we compute the average binary indicator 3, across all N samples at that density level.
7

---

Published as a conference paper at ICLR 2026
Algorithm I: Phuse I: Progressive Evidence
Algorithm 2: Phase II: Binary Aggregation
Collection
Weighting
Input:
Target model. Man. Reference model
Input: Difference collection D = {(t, D:)
Mhee, sequence length
<-
// Stepwise sign stats
Params:
Masking range (Amin, Amax]. steps T. samples N, subset size m
foreach
(t,
8, +
D <-
foreach A E E de
1....; T
Bp.apedd( 0j)
// Compute tarqat masking density for step
+- Mean (B.)
k (amin (0 0min))
B.append((t..) 3.)
D. [-
17 Compute invarse-step weights
// Sample mask positions for this atep
D 1/i
(1/1)/ fo // Eq.
+- Su)ple([4).k]]
// Final weighted aggregation
Xa Mask(x,
EE - Return: memhership score o [0.1]
// Compute loases
E [- loggug (z,(x-s) s,es
10
e <- lolgg (m:(x-s
// sample local subsets from masked pasitions for = I....,N. N 4t + Sample(S,, // Compute subset difference
Algorithm 3: SAMA: Main Procedure
Input: Target model Mar. Reference model
11
.M Text sequence
12
Params:
Masking range [Amin, Amax). Steps T. Sumples N. Subset size m
A" Eeer (e e!)
14
Dr.apeend D (tadd((t, D.))
Algorithm MMMr..Ber Quinn Amax, T, N, m)
Algorithm 2(D)
16 Return: difference collection D
Return: memhership score
4
Experiments
This section validates our proposed SAMA attack against state-of-the-ar DLMs, demonstrating its effectiveness across diverse datasets and comparing existing MIA methods.
4.1 Experimental Setup
We evaluate SAMA state-of-the-arr diffusion language models: LLLDA--B-Base (Nie al., 2025) and Dream-v0-7B-Base (Ye et al., 2025a), with their pre-trained versions serving as refer- ence models MR Fine-tuning was performed member datasets from six diverse domains of the MIMIR benchmark (Duan et al., 2024) following Zhang et al. (2025b); Chang et al. (2024); Antebi al. (2025), and three standard NLP benchmarks (WikiText- 103 (Merity al., 2016), AG News (Zhang et al., 2015), XSum (Narayan et al., 2018)) following Fu et al. (2024). The fine-tuned models maintain strong utility (see Section D.2).
SAMA uses T 16 progressive masking steps from Omin 5% to Qmax 50%, sampling N 128 subsets of size m 10 tokens at each step. We compare against twelve baselines: (1) Au- toregressive MIA methods: Loss (Yeom et al., 2018), ZLIB (Carlini et al., 2021), Lowercase (Carlini et al., 2021), Neighbor (Mattern et al., 2023), Min-K % (Shi et al., 2024b), Min-K (Zhang al., 2025a), ReCall (Xie et al., 2024), CON-ReCall (Wang et al., 2025), BoWs (Das al., 2025), and Ratio (Watson et al., 2021): ;);;;))))))) (2) Diffusion MIA methods adapted from image domain: SecMI (Duan al., 2023) and PIA (Kong et al., 2023). Evaluation follows standard MIA metrics (Carlini al., 2022): AUC and TPR at 10%, 1%, and 0.1% FPR. All baselines also utilize an identical budget of = 16 model queries per sample. Full experimental details are provided in Section D.I.
4.2 Main Results
Table presents the MIA performance of SAMA compared to twelve baseline methods across six diverse datasets from MIMIR, evaluated using AUC and TPR at various FPR thresholds (additional results Dream-v0-7B and three NL.P benchmarks in Section D.3).
8

---

Published as a conference paper at ICLR 2026
MIA performance (AUC, TPR@ 10%FPR. TPR @ 1%FPR, TPR 2.1 &FPR) across datasets. Each cell shows the mean with std. dev. as a subscript. The best results are highlighted.
Table 1:
ArXiv
GitHuub
HackerNews
MIAS
T@10%
T81S
T00.1%
T010%
T61%
160.1%
AUC
10%
1011
T60011%
Loss
0.506+
0.119
0.010. 0.000 0.551
4.01
0.161
0.036=
0.007
0.495
0.118
0.010
0.000
ZLIB
2.4
0.012 m 0.000 o0 0.561 1.01
0.205
0.045=m0
0.
0.486
0.0834.01
4.009
0.001=10
Lowerease
0.515
0.103=
0.013 m 0.001..00 0.579
0.178:4.02
0.044=001
m 0.483
0.074+..01
0.007.+.00
0.001
Min-K%
0.1022
0.012g on g:0
g.530
0.171
0.039=mm
0.007= m 2.
0.108
0.013
0.001
Min-XS++
g.485
0.095gmm 0.006g m 0.000 i 0.496
0.117
0.016
m 0.003
0.486 m
0.100 cn
g.
0.002
BoWs
0.519
0. = 0.01
0.000
0.656..02
0.306.4.02
0. 0.154_
0.059
0.527
0.128
0.009
0.001
ReCall
0.50
0.1322=
0.007
0.000
0.562
0.187++00
0.037=001
0.007
m 0.494
0.090
0.010+.00 0.000
CON-Recall
0
0.011
0 0.000 o 0.549
0.168
0.027_ m1 0.005 m 0.501 in
0.098 1.
g. 0.000 00
Neighbor
0.506
0.098.mm
0.012
01 0.001 o 0.478
0.072
0.008
on 0.0000
0.520
m
0.123
01 4
0.001
Ratio
0.597 1:01 0.181
0.023
0.001 1 m 0.743
0.355
0.081
0.017 m 0.575
0.146..001
0
0.000
SecMI
0.096 07 0.006 m 0.001 1 m 0.604
0.190
0.044_mm
m 0.019
0.523
0.125
0.001
PIA
0.525
0.099
0.01 = 0.000 on 0.571 0.131
0.012=00 00 0.001 m 0.494
0.086
0.0144.000
0.000
(Ours)
0.850 0.586_m3 a3 0.178 0.014
0.876 0.647 2: 0.259 M5 0.075 M5
0.657 +.0
0.215 0.027 0.003 09
PubMed Central
Wikipedia (en)
Pile CC
MIAs
T 10%
T@12
T00.1%
T@10%
10.1%
160.1%
AUC
1%
1%.0.1%
Loss
0.114 0.016
0.001
0.495
0.087
0.010
0.003
0.502
0.105
0.012
0.002_m0
ZLIB
0.4
0
0.005
1:0
0.495
0.093
0.008=
0.000
m 0.491
0.095
0.007
0.001
Lowercase
0.502++01
0.096
0t 0.002:
0 0.000
00 0.535
0.107
0.013
0.001
mm
0.518
0.1022..0
0.009
0.001 00
Min-K%
0.500++00
0.119
0.008
m 0.002+.00
0.482
0.070++.01
0.008=000
0.000=
0.491
0,095-+.01
0.008
0.001
Min-K%-+
0.109g
0.011 o 0.000 m 0.4888
0.118
0.004_20
0.000
0.491 1:00
0.1131
0.007
0.001
BoWs
1.489
0.100
m 0.002 m 0.000 o 1:
0.0884..0
0.003_
0.001 m 0.480
m
0.092...0
0.
0.001 09
ReCail
0.495++20
0.088+.0l
- 0.007
0.000...0 .. 0.506++.0
0.103
0.010--m:
0.000--.o
0.500+.01
0.096+.01
0.009
0.000 00
CON-Recall
0.498++02
0.119+.m2
0.009: 00 0.0004..0 00 0.498++.02
0.092++.01
0.000== mm 0,498++..
0.106+.01
0.009+.co
0.000- 00
Neighbor
0.
0.091
0.011
0.0000
0.504
0.101
0.009_
0.001
0.505
0.096
0.010
0.001
Ratio
0.05
0.128
0.018
0.003
0.653
0.184_.03
0.011
0.000
0.604
0.156
0.
0.002
SecMI
0.510
0.109
0.009:
0.008
0.522
0.101
0.015
0z
0.009
0.515
0.108
0.010
0.001
PIA
0.09
0.
10102
0.005
0.000
0.522
0.120
0.011
0.001
0.509
0.103
0.009
0.000
SAMA (Ours) 0. 0.442 0.132. 0.011 0.790 0.433 0.136 0.008 0.778 0,408 0.1 0.009
SAMA demonstrates substantial and consistent improvements across all metrics and datasets. On average, SAMA achieves AUC of 0.81, compared to the best baseline (Ratio) at 0.62 30% relative improvement. The superiority is particularly pronounced at low FPR thresholds, critical for practical deployment. For instance, TPR @ 1% FPR, SAMA achieves 0.16 average while the best baseline reaches only 0.04, representing a 4x improvement. On the GitHub dataset, where memorization is most pronounced. SAMA attains an AUC of 0.88 with TPR @ 10%FPR of 0.65, while the best bascline (Ratio) achieves 0.74 and 0.36, respectively.
Notably, most traditional MIA methods designed for autoregressive models perform near randomly (AUC = 0.50) on DLMs, confirming that existing approaches fail to capture the unique memoriza- tion patterns in diffusion-bart based architectures. Even methods specifically designed for the diffusion paradigm (SecMI, PIA) show limited effectiveness, with AUCs barely exceeding 0.52. This vali- dates our hypothesis that DLMS require fundamentally different attack strategies that account for their sparse senfiiuurationndeeeeeee membership signals. We also evaluate SAMA'S 's effectiveness against privacy-preser fine-tuning methods (Differential Privacy Liu et al. (2024) and LoRA Hu et al. (2021)) in Section D.5.
4.3 Ablation Study
Figure 3 shows the contribution of each SAMA component. Starting from a baseline loss attack (AUC 0.5), we progressively add: (1) reference model calibration, yielding 0.09-0.19 AUC im- provement by isolating cee--tnnnn-specifii memorization; (2) progressive masking, contributing modest 2-3% gains by capturing multi-scale patters; (3) robust subset aggregation, delivering the
9

---

Published as a conference paper at ICLR 2026
ArXiv
GitHub
HackerNews
PubMed Central
Wikipedia (en)
0.60
TPR @ 10% FPR
0.75
AUC
0.40
0.60
Calibration
Progressive
Weighting
10%
Figure 3: (Ablation) Impact of SAMA'S Core Components Across MIMIR Datasets. The bar charts are illustrated in AUC. The overlaid line plots show the corresponding TPR @ 10%FPR.
largest improvement (20-30% AUC increase) by robustly handling heavy-tailed noise through sign- based voting rather than magnitude averaging; and (4) adaptive weighting, providing 3-5% final refinement by prioritizing cleaner signals from sparse masks. The consistent pattern across datasets confirms that sign-based aggregation is the critical component for handling DLMs' sparse, noisy signals, with calibration and progressive strategies providing essential but smaller contributions. We provide an extended sensitivity analysis of hyperparameters (steps T, subset size m, number of subsets N) Section D.4.
5 Related Work
Our work SAMA builds upon research in three primary areas: MIA ARMs, MIA for image diffusion models, and the evolution DLMs. For an expanded discussion, please refer to Section C.
MIAS ARM and Diffusion-based Generative Models. Membership Inference Attacks (MIAs) (Shokri et al., 2017) have demonstrated particular vulnerabilities in fine-tuned Autoregres- sive Models (ARMs) (Mireshghallah al., 2022b; Fu et al., 2024; Zeng al., 2024; Puerto al., 2025). Standard MIA techniques often rely on loss analysis or reference model calibration (Wat- son et al., 2021; Mireshghallah et al., 2022a; Fu al., 2024), primarily targeting signals from the unidirectional, final-layer outputs of ARMs. This fundamentally differs from SAMA's 's approach to bidirectional diffusion models. Prior MIA research for diffusion models has predominantly focused on image generation, identifying vulnerabilities related to loss, gradients, and intermediate denois- ing steps (Hu & Pang, 2023; Pang et al., 2024; Matsumoto et al., 2023; Kong al., 2023; Duan al., 2023; Fu et al., 2023; Zhai et al., 2024). However, methods designed for pre-trained models often fail on fine-tuned models as they primarily capture domain adaptation signals rather than instance- specific memorization-- challenge that SAMA addresses through robust aggregation designed to isolate true membership signals.
Diffusion Language Models. The adaptation of diffusion principles (Sohl-Dickstein et al., 2015; Ho et al., 2020) to discrete language has led to Masked Diffusion Models (MDMs) (Austin et al., 2023). These models, exemplified by recent large-scale works like LLaDA (Nie et al., 2025) and Dream (Ye et al., 2025a), iteratively mask and predict tokens, establishing a viable alternative to ARMs. SAMA is specifically designed to exploit this core iterative mechanism.
6 Conclusions
This work presents the first systematic investigation of membership inference vulnerabilities in Dif- fusion Language Models. DLMs' bidirectional masking mechanism creates exploitable memoriza- tion patterns absent in autoregressive models. Our SAMA leverages this through robust subset aggre- gation and progressive masking, achieving over 10x improvement compared existing baselines. These findings reveal key privacy risks in emerging DLMs. Limitations: SAMA targets mask- mees--tredict diffusion models; its effectiveness on other diffusion paradigms remains unexplored. The attack re- quires reference models with compatible tokenizers and masking schemes (see Section D.6). While SAMA'S 's progressive masking is specialized for the bidirectional nature DLMs, the robust subset aggregation component is generalizable. The insight of using sign-based voting to filter heavy- domain noise could potentially enhance MIAs against autoregressive or masked language models in future work.
10

---

Published as a conference paper at ICLR 2026
Ethics and Broader Impact Statements
Our work introduces SAMA, potent membership inference attack that exposes privacy vulnerabil- ities in diffusion-based L.MSs by identifying training data. While this research highlights critical risks, all experiments were conducted strictly on public datasets (e.g., MIMIR (Duan et al., 2024), WikiText- 103 (Merity et al., 2016), AG News (Zhang et al., 2015), and XSum (Narayan et al., 2018)) to prevent any new privacy violations. By demonstrating SAMA's effectiveness, this paper reveals previously overlooked vulnerabilities specific to diffusion-basee LLMs. This research directly mo- tivates and informs the development of tailored privacy-preservee defenses for this emerging LLM class, contributing to safer Al model deployment.
Reproducibility Statement
To ensure reproducibility of our results, we provide comprehensive implementation details through- out the paper and supplementary materials. The complete SAMA algorithm is formally specified in Section 3.5 with all hyperparameters explicitly stated in Section 4.1. Full experimental de- tails, including fine-tuning procedures, model configurations, and evaluation protocols, are provided and Dream-v0- (ate--00---BBee)) and
in Section D.I. We use publicly available models (LLaDA-8B-Basl standard benchmarks (MIMIR, WikiText- 103, AG News, XSum) with data processing steps detailed in Section D.1. The mathematical formulation and theoretical foundations are presented in Section 2 and Section 3.1. Implementation code for SAMA and all baseline methods, along with scripts for reproducing experiments, will be made available at https: / thub. com/ com//trryl Strr233333//////33 SAMA. Additional results validating our findings across different models and datasets are provided Sec- tion D.3, demonstrating the robustness of our method.
References
Reza Yazdani Aminabadi, Samyam Rajbhandari, Minjia Zhang, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Jeff Rasley, Shaden Smith, Olatunji Ruwase, and Yuxiong He. Deepspeed inference: Enabling efficient inference of transformer models at unprecedented scale, 2022. URL
https://arxiv.. wwt. 2ttp:///2220cc2 00032.
Guy Amit, Abigail Goldsteen, and Ariel Farkash. Sok: Reducing the vulnerability of fine-tuned language models to membership inference attacks, 2024. URL httpp://eexxxi..rrrr /
0.481.
Sagiv Antebi, Edan Habler, Asaf Shabtai, and Yuval Elovici. Tag&tab: Pretraining data detec- tion in large language models using keyword-based membership inference attack. arXiv preprint arXiv:2501.08454 2025.
Ery
Emmanuel J Candès, and Yaniv Plan. Global testing under sparse alternatives: Anova, multiple comparisons and the higher criticism. 2011.
Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg- Structured denoising diffusion models in discrete state-spaces. In M. Ranzato, A. Beygelz- imer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.). Advances in Neural In- formation Processing Systems, volume 34, pp. 17981-17993. Curran Associates, Inc., htt.:paerrr..lleeppe///aa/ lesspaper/2021/
2021. URL https: http:///ooeeeddeeeeee ngs neweiis htcpps..cc.eeeeeee cc. wctpccer_eeeeeeeeeeeeeee fi File/958c550 /a/.//988535555 (wt.:330055547cccc88e777 par. pad.
Jacob Austin, Daniel D. Johnson, Jonathan Ho. Daniel Tarlow, and Rianne van den Berg, Structured denoising diffusion models discrete state-spaces, 2023. URL https:://axxiii www.eiiiv.....a.o.e.. / 21 2107. 03006.
Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Kor- bak, and Owain Evans. The reversal curse: Lims trained on "a b" fail to learn "b a", 2024. URL https::///// /tax/vivvvv... orgg /2399 9 12288.

---

Published as a conference paper at ICLR 2026
Léon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review, 60(2):223 311, 2018.
Leo Breiman. Bagging predictors. Machine learning, 24(2):1231140, 1996.
Tom Brown, Benjamin Mann, Nick Ryder. Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:187771901, 2020.
Andrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. Advances in Neural 2022.
Information Processing Systems,
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pp- 1633-2650, 2021.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. Mem- bership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1897-1914, 2022. doi: 10. 10.1109/SP462 14.2022.98336444
Hongyan Chang, Ali Shahin Shamsabadi, Kleomenis Katevas, Hamed Haddadi, and Reza Shokri. Context-aware membership inference attacks against pre-trained large language models. arXiv preprint (1t.::2409.137445. 2024.
Ting Chen, Ruixiang Zhang, and Geoffrey Hinton. Analog bits: Generating discrete data using diffusion models with self-conditioning. arXiv preprint arXiv:2208 042002, 2022.
Zixiang Chen, Huizhuo Yuan, Yongqian Li, Yiwen Kou, Junkai Zhang, and Quanquan Gu. Fast sampling via de-randomization for discrete diffusion models. arXiv preprint arXiv:2312.09193, 2023.
Debeshee Das, Jie Zhang, and Florian Tramèr. Blind baselines beat membership inference attacks for foundation models, 2025. URL https: :ttp:p:://aaee //arxiva.ivv////a org/ abs /206 16201.
DeepMind. Gemini diffusion - our state-of-the-art, experimental text diffusion model. https: / //deepmind. googl 1e e/ htt/d//l///////eee///e/.eee/llee http://emmiiiiiiiiieeieieeiaieee 2025 Accessed: 2025-11-27.
DeepSeek-AL. Deepseek-v3 v3 technical report, 2025. URL 19437.
Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, et al. Continuous diffu- sion for categorical data. arXiv preprint arXiv:221 1.5509,,, 2022.
Jinhao Duan, Fei Kong, Shiqi Wang, Xiaoshuang Shi, and Kaidi Xu. Are diffusion models vul- nerable to membership inference attacks?, 2023, URL https: h /ttp///vv...ooo/////////// 01316.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? In Conference on Language Modeling (COLM), 2024.
Jan Dubińíski, Antoni Kowalczuk, Stanistaw Pawlak, Przemystaw Rokita, Tomasz Trzciński, and Pawel Morawiecki. Towards more realistic membership inference attacks on large diffusion mod- els, 2023. URL https: /ttw/viiivv/////////////////aava www.aiiwww..g.....e./oeeeeeeeeee aa///30s///066 12983.
Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, and Tao Jiang. A proba- bilistic fluctuation based membership inference attack for diffusion models. arXiv preprint arXiv:2308. /2/43, 2023.
Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, and Tao Jiang. Practical member- ship inference attacks against fine- tuned large language models via self prompt calibration, 2024. URL https: / /aaxivv org/ abs 231 06062.
12

---

Published as a conference paper at ICLR 2026
Xiaomeng Fu, Xi Wang, Qiao Li, Jin Liu, Jiao Dai, Jizhong Han, and Xingyu Gao. OMS: One more step noise searching to enhance membership inference attacks for diffusion models, 2025. URL htttps://ppen/eeeet cpwnrevieereeeeeeeeeeeeeeeeeee.e net Forumiid-IRCos
Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky TQ Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman. Discrete flow matching. arXiv preprint arXiv:2407.15595, 2024.
Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models. arXiv preprint arXiv:2210.08933, 2022.
Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, et al. Scaling diffusion language models via adaptation from autoregressive models. arXiv preprint arXiv:24 17891, 2024.
Alex Graves, Rupesh Kumar Srivastava, Timothy Atkinson, and Faustino Gomez. Bayesian flow networks. arXiv preprint arXi::2388.07033 2023.
Ishaan Gulrajani and Tatsunori B Hashimoto. Likelihood- based diffusion language models. Ad- vances in Neural Information Processing Systems, 36:1669331117777 2023.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks, 2017. URL https: //apxiv./////i...////eve wwtp:/orggg////b. wwt.//0066...0 04599.
Xiaochuang Han, Sachin Kumar, and Yulia Tsvetkov. Ssd-Im: Seni-auttregressive simplex- based diffusion language model for text generation and modular control. arXiv preprint arXiv:2210. 2022.
Valentin Hartmann, Anshuman Suri, Vincent Bindschaedler, David Evans, Shruti Tople, and Robert West. Sok: Memorization in general-purpose purpose large language models, 2023. URL https:// arxiv.org abs 2310.
Yu He, Boheng Li, Liu Liu, Zhongjie Ba, Wei Dong. Yiming Li, Zhan Qin, Kui Ren, and Chun Chen. Towards label-only membership inference attack against pre-trained large language models, 2025. URL https: /aaxivv...iivva//aa org/ abs/2502. 18943.
Zhengfu Hc, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, and Xipeng Qiu. Diffusion-
bert: Improving generative masked language models with diffusion models. arXiv preprint arXiv:2211.15022 2022.
Larry Hedges and Ingram Olkin. Statistical methods for anta-snalysis. Academic press, 2014.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6855 2020.
Joseph L Hodges Jr and Erich L Lehmann. Estimates of location based rank tests. In Selected works of EL Lehmann, pp. 287-300. Springer, 2011.
Emiel Hoogeboom, Alexey Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans. Autoregressive diffusion models. arXiv preprint arXiv:2110.02037, 10.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling, Argmax flows
and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:12454-12444 2021b.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen Zhu, Yuanzhi Li. Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https: //arxivv.vee abs 2tt.//22160..666
Hailong Hu and Jun Pang. Membership inference of diffusion models, 2023. URL https:// arxiv. orgg abs /2/2//30 2301. 09956.
Zhiheng Huang, Yannan Liu, Daojing He, and Yu Li. Df-mia: A distribution-fie membership inference attack on fine-tuned large language models. Proceedings of the AAAI Conference Artificial Intelligence, 39(1)3433-311, Apr. 2025. doi: 10.1609/aaai.v399 32022. URL https: / /ojss. a wttpr/iinxx..pp/Aaa... http:///wwaaag..///aaaaaaaaaat/r
13

---

Published as a conference paper at ICLR 2026
Peter Huber. Robust statisties. Wiley Series in Probability and Mathematical Statistics, 1981.
Hugging Face. Llada - multimodalart space on hugging face. URL htth:: /
Nikhil Kandpal, Krishma Pillutla, Alina Oprea, Peter Kairouz, Christopher A. Choquette-Choo, and Zheng Xu. User inference attacks on large language models, 2024. URL htttp:://axii.
brg/abs/23100 09266.
Ouail Kitouni, Niklas Nolte, James Hensman, and Bhaskar Mitra. Disk: A diffusion model for structured knowledge. arXiv preprint arXiv:2312.0522 2023.
Fei Kong. Jinhao Duan, RuiPeng Ma, Hengtao Shen, Xiaofeng Zhu, Xiaoshuang Shi, and Kaidi Xu. An efficient membership inference attack for the diffusion model by proximal initialization, 2023.
URL attps::/aaaai /wwpivxxiivvv.///////eee/eaaaava org/ abs / 2005 18355.
Siqi Kou, Lanxiang Hu, Zhezhi He, Zhijie Deng, and Hao Zhang. Clims: Consistency large language models. arXiv preprint arXiv:2403. 00835, 2024.
Hao Li, Zheng Li, Siyuan Wu, Chengrui Hu, Yutong Ye, Min Zhang. Dengguo Feng, and Yang Zhang. Seqmia: Sequential-metric based membership inference attack, 2024. URL https: //ar //arxiv.co org/ abs 2207 15098.
Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion- Im improves controllable text generation. Advances in neural information processing systems, 35: 4328-4343, 2022.
Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Nan Duan, and Weizhu Chen. Text generation with diffusion language models: A pre-training approach with continuous paragraph denoise. In International Conference Machine Learning, pp. 21051-21064, PMLR, 2023.
Xiao-Yau Yang Liu, Rongyi Zhu, Daochen Zha, Jiechao Gao, Shan Zhong, Matt White, and Meikang Qiu, Differentially private low-rank adaptation of large language model using federated learning, 2024. URL https: / atw/vriiiivv... org/ abs /2t...12222133
Yiyong Liu, Zhengyu Zhao, Michael Backes, and Yang Zhang. Membership inference attacks by exploiting loss trajectory, 2022. URL https: //ts:v//aavvvv. org/ 22..
Aaron Lou and Stefano Ermon. Reflected diffusion models. In International Conference on Machine Learning, pp. 22675-22701. PMLR, 2023.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion language modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310.1683. 2023.
Zihao Luo, Xilie Xu, Feng Liu, Yun Sing Koh, Di Wang, and Jingfeng Zhang. Privacy-preservinn low-rank adaptation against membership inference attacks for latent diffusion models, 2024. URL https: / //w.xva.avvv//a////a//eaa/aaaav. org/ abs/ 2402 11989.
Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae, James Henderson, Iz Beltagy, Matthew Peters, and Arman Cohan. Tess: Text-to-text self-conditioned simplex diffusion. arXiv preprint AxXiv:3305.0837 2023.
Ricardo A Maronna, R Douglas Martin, Victor J Yohai, and Matías Salibián-Barmnn Robust statis- tics: theory and methods (with R). John Wiley Sons, 2019.
Tomoya Matsumoto, Takayuki Miura, and Naoto Yanai. Membership inference attacks against diffusion models. 2023 IEEE Security and Privacy Workshops (SPW), pp. 77-83. IEEE, 2023.
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Sachan, and Taylor Berg-Kirkpatrick. Membership inference attacks against language models via neigh- bourhood comparison, 2023. URL. https: //trxvvvvv///////////////////aal org abs/ 2230 684
14

---

Published as a conference paper at ICLR 2026
Matthieu Meeus, Igor Shilov, Shubham Jain, Manuel Faysse, Marek Rei, and Eves-Alexandre de Montjoye. Sok: Membership inference attacks on Ilms are rushing nowhere (and how to fix it). arXiv preprint arXiv:2406.1799 2024.
Chenlin Meng, Kristy Choi, Jiaming Song, and Stefano Ermon. Concrete score matching: General- ized score matching for discrete data. Advances in Neural Information Processing Systems, 35: 34532-34545, 2022.
Wenlong Meng, Zhenyuan Guo, Lenan Wu, Chen Gong, Wenyan Liu, Weixian Li, Chengkun Wei, and Wenzhi Chen. Rr: Unveiling IIm training privacy through recollection and ranking. arXiv preprint arXiv:2502.126555 2025.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07880 2016.
Stephen Merity, Nitish Shirish Keskar, and Richard Socher. An analysis of neural language modeling at multiple scales, 2018. URL https: //axivv. org/ /abs//bb/////aa 1803. 08240.
Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrine and Reza Shokri. Quantifying privacy risks of masked language models using membership inference at- tacks. arXiv preprint arXiv:2203.0392. 2022a.
Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, and Taylor Berg- Kirkpatrick. An empirical analysis of memorization fine-tuned autoregressive language models. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1516-1826, Abu Dhabi, United Arab Emirates, December 2022b. Association for Computational Linguisties. doi: 1..1863/v1/2022. emnlp-main. 1 19. URL https: : / aclanthooogoo org/ 2022 commp-pmainn.. 119/.
Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, and Yejin Choi. Can ilms keep secret? testing privacy implications of language models via contextual integrity theory, 2024. URL htttps://arxiv. wtgggba///////r/oo/ee/// 1.100.17888 17884.
Hamid Mozaffari and Virendra Marathe, Semantic membership inference attack against large 10218.
language models, 2024. URL
Shashi Narayan, Shay B Cohen, and Mirella Lapata. Don't give me the details, just the sum- mary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint 1808008808. 2018.
Shen Nie, Fengqi Zhu, Chao Du Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. Scaling up masked diffusion models on text. arXiv preprint arXiv:2410.187 2024.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models, 2025. URL https:// wexiv.org// absg//bs//t 2502 09992.
OpenAI. Gpt-4o system card, 2024. URL mnttps://arxiien www.aiwww...ogaaaaaaooaooo abss/////////// 2210221112222 21276.
Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. arXiv preprint arXiv:2406.037:5 00032236660076 2024.
Yan Pang and Tianhao Wang. Black-box membership inference attacks against fine-tuned diffusion models. arXiv preprint arXiv:2312.08207,, 2023.
Yan Pang, Tianhao Wang, Xuhui Kang, Mengdi Huai, and Yang Zhang. White-box membership inference attacks against diffusion models, 2024. URL https://arxiv. http:///axxx..o///////////// wttp.////rr/ab//........ 06405.
David B Peizer and John W Pratt. A normal approximation for binomial, f, beta, and other common,
related tail probabilities, i. Journal of the American Statistical Association, 13(324):1416-144 1968.
15

---

Published as a conference paper at ICLR 2026
Haritz Puerto, Martin Gubri, Sangdoo Yun, and Seong Joon Oh. Scaling up membership inference: When and how attacks succeed on large language models, 2025. bttpsttp:///axiiit abs/2411 00154.
Qwen. Qwen2.5 technical report, 2025. URL www.h:ttp///ee htt.:ttss//aaxx..or//ab////////
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenA/ blog, 1(8):9, 2019.
Machel Reid, Vincent J. Hellendoomn, and Graham Neubig. Diffuser: Discrete diffusion via edit- based reconstruction, 2022.
Pierre H Richemond, Sander Dieleman, and Arnaud Doucet. Categorical sdes with simplex diffu- sion. arXiv preprint 12X1::2210014784, 2022.
Peter Rousseeuw and Annick M Leroy. Robust regression and outlier detection. John wiley & sons, 2003.
Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. arXiv preprini arXiv:2406.07522, 2024.
Ali Satvaty, Suzan Verberne, and Fatih Turkmen. Undesirable memorization large language models: A survey, 2025. URL https: : wttpava.ivvv.../.. abs 2410. 02
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and general- ized masked diffusion for discrete data. Advances in neural information processing systems, 37: 103131-103167 2024a.
Weijia Shi, Anirudh Ajith. Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models, 2024b. URL httph:ttp/////...t. http:///xxvvvvoo/o abs / 0...1..00 16789.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models, 2017.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan. and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing, pp. 2256-2265. pmlr, 2015.
Yang Song, Jascha Sohl-Dickster Dickkstin, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint 1.Xiv:2011.134560 2020.
Robin Strudel, Corentin Tallec, Florent Altché, Yilun Du, Yaroslav Ganin, Arthur Mensch, Will Grathwohl, Nikolay Savinov, Sander Dieleman, Laurent Sifre, et al. Self-conditioned embedding diffusion for text generation. arXiv preprint (tti::2//100423:: 2022.
Haoran Sun, Lijun Yu, Bo Dai, Dale Schuurmans, and Hanjun Dai. Score-based continuous-time discrete diffusion models. arXiv preprint arXiv:2211.1675: 2022.
Shuai Tang, Zhiwei Steven Wu, Sergul Aydore, Michael Kearns, and Aaron Roth. Membership inference attacks on diffusion models via quantile regression, 2023. URL //aosii///vvv.. org/abs 111//3121200111 05140.
Meta Llama Team. The Ilama 3 herd of models, 2024. URL http:::/aassiv.orrg//eeeee// 2407. 21783.
Cheng Wang, Yiwei Wang, Bryan Hooi, Yujun Cai, Nanyun Peng, and Kai-Wei Chang. Con-recall: Detecting pre-training data in llm via contrastive decoding, 2025. URL https://arxiv. oxg/abs/2409 03363.
Lauren Watson, Chuan Guo, Graham Cormode, and Alex Sablayrolles. On the importance of diffi- culty calibration in membership inference attacks, arXiv preprint 1..044400 2021.
16

---

Published as a conference paper at ICLR 2026
Rui Wen, Zheng Li, Michael Backes, and Yang Zhang. Membership inference attacks against in- context learning. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS 24, pp. 3481-3495, New York, NY. USA. 2024. Association for Computing Machinery. ISBN 9798400706363. doi: 10.1 141/36666644433633306 URL. https: //doi. wrg/ro...0. /0. 355555655644..0 3690306
Tong Wu, Zhihao Fan, Xiao Liu, Hai-Tao Zheng, Yeyun Gong, Jian Jiao, Juntao Li, Jian Guo, Nan Duan, Weizhu Chen, et al. Ar-diffusion: Auto-regressive diffusion model for text generation. Advances in Neural Information Processing Systems, 36:39957-39974, 2023.
Yixin Wu, Ning Yu, Zheng Li, Michael Backes, and Yang Zhang. Membership inference attacks abs/
against text-to-image generation models, 2022. URL 00968.
Roy Xie, Junlin Wang, Ruomin Huang, Minxing Zhang, Rong Ge, Jian Pei, Neil Zhenqiang Gong, and Bhuwan Dhingra. Recall: Membership inference via relative conditional oogliiklssoods. arXiv preprint arXhv:2406.15966 2024.
Chenkai Xu, Xu Wang, Zhenyi Liao, Yishun Li, Tianqi Hou, and Zhijie Deng. Show-o turbo: Towards accelerated unified multimodal understanding and generation. arXiv preprint arXiv:2500.00445.. 2025.
Kaiwen Xue, Yuhao Zhou, Shen Nie, Xu Min, Xiaolu Zhang, Jun Zhou, and Chongxuan Li. Unify- ing bayesian flow networks and diffusion models through stochastic differential equations. arXiv preprint orXiv:2404.15766, 2024.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b, 2025a. URL https::///// http///kklll..eeee github. ht/bllg///////oo 200lllgg/2//////// 2025/ dreem. http://...lll...ubbbbb.p
Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, and Quanquan Gu. Diffussonn language models can perform many tasks with scaling and isstuction-finetunine arXiv preprint arXiv:2308. 2023a.
Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, and Mingxuan Wang. Dinoiser: Diffused condi- tional sequence learning by manipulating noises. arXiv preprint arXiv:2302.10025, 2023b.
Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, and Quanquan Gu. Diffusion language mod- cls can perform many tasks with scaling and instruction-finetunn 2025b. URL https: 12219.
/
Samuel Ycom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learn- ing: Analyzing the connection to overfitting, 2018
Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, and Dawei Yin. Exploring memorization in fine-tuned language models, 2024. URL https: /ttp//aavvvv///////////////a.av. org/ /abs////////// 211220110227 06714.
Shengfang Zhai, Huanran Chen, Yinpeng Dong, Jiajun Li, Qingni Shen, Yansong Gao, Hang Su, and Yang Liu. Membership inference on text-to-image diffusion models via conditional likelihood discrepancy. Advances in Neural Information Processing Systems, 37:74122-74146, 2024.
Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Frank Yang, and Hai Li. Min-k%++: Improved baseline for detecting pre-training data from large /2...2404.00.
language models, 2025a. URL httpss///xxxvvvvo http:b/b22444.03366
Kaiyuan Zhang, Siyuan Cheng, Hanxi Guo, Yuetian Chen, Zian Su, Shengwei An, Yuntao Du, Charles Fleming, Ashish Kundu, Xiangyu Zhang, and Ninghui Li. Soft: Selective data ob- fuscation for protecting Ilm fine-tuning against membership inference attacks, 2025b. URL https: /ettps://atte //te:i////ivv///// orgg alos/ 2200. 0424.
Minxing Zhang, Ning Yu, Rui Wen, Michael Backes, and Yang Zhang. Generated distributions all you need for membership inference attacks against generative models. In Proceedings of the IEFF/CVF Winter Conference on Applications of Computer Vision, pp. 2024.
17

---

Published as a conference paper at ICLR 2026
Xiang Zhang. Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text clas- sification. Advances in neural information processing systems, 28, 2015.
Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming- Yu Liu, Jun Zhu, and Qinsheng Zhang. Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling, arXiv preprinl arXiv:2409.02908, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing. Hao Zhang. Joseph Gonzalez, and lon Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023a. httptt:::// /ttp:::/aaaxvv.o/o//e abs/2300 05685.
Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation. ArXiv, abs/2302.05737, 2023b.
18

---

Published as a conference paper at ICLR 2026
A
Notation Summary
This section provides a summary of core mathematical notation used throughout the paper to ensure clarity and consistency. 'able 2 lists these symbols and their meanings in the context of SAMA.
B LLM Usage Disclosure
We used Large Language Models OpenAI (2024) to aid in polishing the writing of this paper. Specif- ically, LLMs were used to refine the abstract for clarity and conciseness, and to generate summary statements. All technical content, experimental design, analysis, and core contributions are entirely the authors' original work. The L.M served only as writing assistant for improving readability and presentation of already-developed ideas.
C Additional Related Works
MIA on ARMs. MIAS against ARMs initially showed limited success on pre-training data un- der rigorous IID evaluation (Duan et al., 2024; Meeus et al., 2024; Das et al., 2025), attributed to large datasets, few training epochs, and fuzzy text boundaries (Duan et al., 2024; Carlini et al., 2021; Satvaty et al., 2025). Early LLM MIAs adapted loss/perplexity thresholding (Yeom et al., 2018: Shokri et al., 2017) or reference model calibration (Watson et al., 2021; Mireshghallah et al., 2022a), with LiRA (Carlini et al., 2022) emphasizing low-FPR evaluation. Reference- free methods like Min-K% % (Shi et al., 2024b) and the theoretically grounded Min-K%++ (Zhang et al., 2025a) analyzed token probabilities. Other approaches compare outputs perturbed neighbors (Mattern et al., 2023). leverage semantic understanding (SMIA) (Mozaffari & Marathe, 2024), analyze loss trajectories (Liu et al., 2022: Li al., 2024), or target memorization via probabilistic variation (SPV- MIA) (Fu et al., 2024). Fine-tuned LLMS proved more vulnerable (Mireshghallah et al., 2022b; Zeng et al., 2024), leading attacks like SPV-MIA with self-prompt calibration (Fu et al., 2024) and User Inference targeting user participation (Kandpal et al., 2024). Extraction methods (Car- lini et al., 2021) and label-only attacks (He al., 2025; Wen et al., 2024) were also developed. Recent findings suggest MIA effectiveness against pre-trained LLMs increases significantly when aggregating weak signals over longer sequences (Puerto et al., 2025). However, nearly all existing LLM MIAs fundamentally probe signals derived from the final layer outputs or aggregated scores pertinent to the autorcgressive generation process.
MIA for Siltusion-Baseee Generative Models. Investigating MIAs in diffusion models initially centered on image generation, posing distinct challenges compared to GANs or VAEs (Duan et al., 2023; Carlini et al., 2021). White-box analyses confirmed vulnerability, identifying loss and like- lihood as viable signals (Hu & Pang, 2023) and suggesting intermediate denoising timesteps carry heightened risk (Matsumoto et al., 2023), with gradients potentially offering stronger leakage chan- nels (Pang et al., 2024). Despite this, mounting effective black-box attacks, especially against large- scale models like Stable Diffusion, proved difficult (Dubińíki al., 2023). Significant grey-box attacks emerged, leveraging access beyond final outputs. Reconstruction based methods showed promise, particularly for fine-tuned models (Pang Wang. 2023). Trajectory analysis, probing the iterative denoising path, yielded query- query-efficient attacks like PIA (Kong et al., 2023). Other black- box innovations bypassed shadow models by analyzing generated distributions (Zhang et al., 2024) or employing quantile regression (Tang al., 2023). Advanced methods aim for deeper insights, de- tecting memorization via probabilistic fluctuations (PFAMI (Fu et al., 2023)) or optimizing attacks through noise analysis (OMS (Fu et al., 2025)). Text-to-image models introduced further com- plexities and attack surfaces (Wu et al., 2022), exploited via techniques like conditional overfitting analysis (CLiD (Zhai et al., 2024)). These studies highlight the importance of the iterative process in image diffusion privacy but are tailored to its continuous state space and noise-based corruption, distinct from the mechanisms in discrete language diffusion.
Diffusion- Based Language Models. Adapting diffusion principles (Sohl-Dickstein et al., 2015; Ho al., 2020; Song et al., 2020) to discrete language required overcoming unique hurdles. Ap-
19

---

Published as a conference paper at ICLR 2026
Table 2: Summary of Core Notation.
Symbol
Meaning
General Symbols
text sequence (data record).
V
Model's token vocabulary.
L
Length of the sequence X.
[Z]]
The set 21, ...: L.} of all token positions.
ra
The i-th token in sequence x.
tei
Tokens preceding position i: I;-1):
Duain
Training dataset (member set).
(xx)
True membership status of for member, 0 for non-member).
x(x)
Predicted membership status of x.
A(x, MFF)
Attack function outputting membership prediction.
Model and Loss Notation
Mb
Target diffusion language model (fine-tuned).
MBF
Reference diffusion language model (pre-trained base).
MA
Target autoregressive model.
M
Reference autoregressive model.
1
mask configuration (set of masked positions).
x-S
Unmasked context: {æ; i E [L] SS).
()
Training loss for DLMs (Equation (1)).
p(x{(xs))
Probability of token z; given unmasked context (DLMs).
p(
Probability of token x; given preceding context (ARMs).
4(x
Loss at position in DLMs:
(xs).
(x, o Sx
Per-token losses for reference and target models.
SAMA Algorithm Symbols
Number of progressive masking steps.
Step index (1 S t $ 7).
[Ominn Oemax
Range of masking densities (e.g., [0.05, 0.5]).
Og
Masking density at step t.
ka
Number of masked positions step [T. as..
N
Number of samples (random subsets) per step.
Subset size (typically 10 tokens).
S,
Mask configuration step
The n-th sampled subset from S,. Loss difference for n-th subset. Binary indicator: 1 1 > 0).
A" n
Ba
Average binary indicator step l.
W:
Weight for step t: (1/t)/H where H/ = Σ/ 1/i.
H
Harmonic sum for normalization: ΣT= 1/i.
SAMA(x)
Final membership score (Equation (7)).
Altemative notation for final membership score.
D
Collection of loss differences across all steps.
D.
Loss differences collected at step t.
20

---

Published as a conference paper at ICLR 2026
proaches include diffusing continuous text embeddings (Li et al., 2022: Gong et al., 2022: Han
et al., 2022; Strudel et al., 2022; Chen et al., 2022: Dieleman et al.. 2022: Richemond et al., 2022;
Wu et al.. 2023; Mahabadi et al., 2023; Ye et al., 2023b) modeling continuous proxies for dis- crete distributions (Lou & Ermon, 2023; Graves et al., 2023; Lin et al., 2023; Xue et al., 2024), though often encountering scalability challenges relative ARMS (Guirajani & Hashimoto, 2023)- Discrete diffusion frameworks, operating directly on tokens (Austin et al., 2021; Hoogeboom et al., 2021b;a; He et al., 2022: Campbell al., 2022; Meng et al., 2022: Reid et al., 2022: Sun et al., 2022: Kitouni et al., 2023; Zheng et al.. 2023b; Chen et al., 2023; Ye et al., 2023a; Gat et al., 2024: Zheng al., 2024), offered an alternative path. Among these, Masked Diffusion Models (MDMs) (Austin et al., 2021), utilizing iterative masking and Transformer-based prediction, showed early promise (Lou et al., 2023; Nie et al., 2024), supported by theoretical work (Ou al., 2024; Shi et al., 2024a). Research focused on refining MDMs (Sahoo et al., 2024), integrating them with ARM pre-training (Gong et al.. 2024), and acceleration (Kou et al., 2024; Xu et al., 2025). This progress culminated in models like LLaDA (Nie et al., 2025) and Dream (Ye et al., 2025a), demonstrating that large-scale discrete diffusion models can achieve emergent L.M capabilities comparable to strong ARMs, establishing them viable, distinct LLM paradigm.
Despite the rapid maturation of diffusion- based text generation, particularly with capable models like LLaDA and Dream, their vulnerability to MIAs remains largely uncharted. Existing MIA tech- niques are ill-suited: those for ARMs target likelihoods from sequential generation, while those for image diffusion address continuous denoising steps. The unique discrete "mask-and-predite iterative mechanism of diffusion- stepwise analysis concepts proven effective in the (continuous) image domain (Duan et al., 2023; Kong et al., 2023), our work tackles this challenge directly.
LLMs necessitates novel MIA formulations. Inspired by
D Supplementary Experimental Results
This section provides supplementary materials complement the main paper. It begins with detailed breakdown of the experimental settings, including dataset specifics, model configurations, fine-tuning procedures, and comprehensive descriptions of baseline methods and evaluation metrics. Subsequently, we present extended results for our main comparisons and ablation studies across wider range of conditions. Further analyses delve into the impact of model utility, training duration, and reference model choice on attack efficacy. Finally, we explore potential defense mechanisms against SAMA and offer comparative perspective on the vulnerability of diffusion- based models versus their autoregressive counterparts.
D.1 Experiment Settings
This section provides comprehensive details regarding the experimental setup used to evaluate SAMA, supplementing Section 4.1 of the main paper. We describe our evaluation across nine di- verse datasets spanning scientific, code, and general text domains, with samples ranging from 1. 1,000 to 10,000 per dataset. Our experiments focus on two state-of-ther thrt----tttttttt-- diffusion language models fine- tuned using consistent hyperparameters. We compare SAMA against twelve baseline MIA methods, including ten adapted from autoregressive models and two designed for diffusion models, with all methods using 16 Monte Carlo samples for fair comparison. Performance is assessed through stan- dard MIA metries (AUC, TPR various FPR thresholds) and model utility measures (perplexity, LLM-as-a-Judge evaluation).
D.1.1 Datasets
Our experiments utilize datasets from two sources, summarized in Table 3. The primary evalu- ation uses six subsets from the MIMIR benchmark (Duan et al., 2024), following the 13_0.8 8 deduplication protocol from (Wang al., 2025). Additionally, evaluate three standard NLP tasks following Fu et al. (2024): WikiText- 103 (Merity et al., 2016), AG News (Zhang et al., 2015), XSum (Narayan al., 2018).
21

---

Published as a conference paper at ICLR 2026
Table
Dataset overview for MIA evaluation.
Dataset
Type
Train
MIMIR Benchmark
(13_0.6 8 split)
ArXiv
Scientific
1,000
1,000
GitHuh
Code
1,000
1 1,000
HackerNews
Forum
1,000
1,000
PubMed Central
Medical
1,000
1,000
Wikipedia (en)
Encyclopedia
1,000
1,000
Pile CC
Web Crawl
1,000
1 1,000
Standard NLP Tasks
WikiText-
Lang. Model
10,000
10,000
AG News
Classification
10,000
10,000
XSum
Summarization
10,000
10,000
D.1.2 Models and Fine-tuning
We investigate two state-of-the-aat ML:s: LLaDA-8B-Bast (Nie et al., 2025) and Dream-vO-Base 7B (Ye et al., 2025a). For reference models in calibrated attacks (including SAMA and Ratio), the pre-trained versions of these models. Fine-tuning employs Adam W optimizer with learning rate x 10-5, weight decay 0.1, batch size 48, and 4 epochs with early stopping (patience 3). Training uses 6 precision on 3 x NVIDIA A100 GPUs with DeepSpeed Stage optimization (Aminabadi al., 2022).
D.1.3 SAMA Configuration
SAMA employs progressive masking with T steps, increasing masking density from Ommnn 5% to ('max 50%. At each step, we sample N 128 random subsets of m 10 tokens, compute binary indicators based on loss differences, and aggregate using inverse-step weighting. All scores averaged over 4 Monte Carlo samples for stability.
D.1.4 MIA Baselines
We compare against twelve baseline MIA methods. For all baselines requiring model queries, scores are averaged over 16 Monte Carlo samples (matching SAMA'S progressive steps) to ensure stable estimates under the stochastic nature of diffusion models' masking process.
Autoregressive-based Baselines Adapted for DLMs.
Loss (Yeom al., 2018): The most fundamental baseline that directly uses a sample's reconstruc- tion loss as its membership score. For DLMs, we compute the average negative log legliikllhood of correctly reconstructing masked tokens across random mask configurations with 15% of tokens masked. Lower reconstruction loss indicates higher membership likelihood, as the model better predicts tokens has seen during training.
ZLIB (Carlini et al., 2021): This method compares the model's reconstruction difficulty against the text's inherent complexity, measured through compression. The membership score is the ratio the model's loss to the compressed length of the text using ZLIB compression (compression level 6). Lower ratios suggest membership, indicating the model finds the text casier to reconstruct relative to its complexity.
Lowercase (Carlini et al., 2021): This baseline exploits the observation that models memorize exact casing patterns during training. The score is computed as the difference between the reconstruction loss on lowercased text and the original text, both evaluated with 15% masking. Positive differences indicate potential membership, suggesting the model is sensitive the specific casing memorized.
22

---

Published as a conference paper at ICLR 2026
Min-KK % (Shi et al., 2024b): Originally designed for autoregressive models, this method focuses on the least confident predictions. For DLMs, we randomly mask 15% of tokens in each of 16 iterations, record the probability assigned to each true token masked positions, average these probabilities per token across iterations, and sum the smallest 20% of these averaged probabilities. Lower scores indicate membership, even the model's ' least confident predictions relatively accurate for memorized content.
Min-K %++++ ++ (Zhang et al., 2025a): An enhanced version that uses log-probabilities for improved numerical stability. Following the same procedure as Mi, we work with log-probabilities of true tokens. summing the 20% most negative values. A small constant added to probabilities before logarithm computation to prevent numerical errors.
BoWs (Das et al., 2025): A query-free method that analyzes textual patterns without accessing the model. Texts are converted to term frequency-inven document frequency (TF-IDF) representations with 5000 maximum features, ignoring terms appearing in fewer than 5% of documents. Random Forest classifier with 100 shallow decision trees (maximum depth of 2, minimum 5 samples per leaf) trained using 5-fold cross-validation. The membership score is the predicted probability of being training member.
ReCall (Xie et al., 2024): This method tests whether prepending non-member context affects recon- struction difficulty. We select 7 non-member text examples from external dataset, compute the reconstruction loss when these examples precede the target text, and compare to the baseline recon- struction loss without any prefix. The membership score is the ratio of baseline loss to prefixed loss. Higher ratios indicate membership, suggesting the model doesn't 't benefit from the irrelevant context when reconstructing memorized content.
(Wang et al., 2025): This approach contrasts the effects of member versus non- member prefixes. We compute reconstruction losses using 7 member prefixes (from presumed training data) and 7 non-member prefixes, then calculate the score as the difference between non- member-prefixed and cemmber-prefxeee losses, normalized by the baseline loss. Positive scores indi- cate membership, as the model benefits more from a relevant training context. Prefixes are matched by average length to minimize confounding factors.
CON-ReCall
Ratio (Watson et al., 2021): This calibration method isolates fine-tuning effects by comparing target and reference model losses. The score the ratio of reference model loss target model loss. Higher ratios indicate membership, as the fine-tuned model should show lower loss on its training data compared to the reference model.
Diflusion-speeific Baselines.
SecMI (Duan et al., 2023): Originally designed for image diffusion models, SecMI exploits the observation that member samples exhibit lower reconstruction errors across different corruption lev- els. For DLMs, adapt this by analyzing token prediction errors across multiple masking densities. The implementation proceeds as follows: We sample 5 masking ratios lincarly spaced between 10% and 80% of tokens. At each masking ratio, we apply deterministic masks (seeded by the sample content and step for reproducibility). then compute the cross-entropy loss for predicting the origi- nal tokens masked positions. The errors are aggregated using inverse step weighting, where early steps (lower masking ratios) receive higher weights (weight proportional to 1/(step+1)) 1)), as they typi- cally provide cleaner signals. The final score is the negative weighted average of these errors-lower reconstruction error indicates higher membership likelihood, as the model better reconstructs tokens it memorized during training.
PIA (Kong et al., 2023): Proximal Initialization Attack analyzes how masking affects the model's prediction consistency. For DLMss we implement this by comparing model outputs between un- masked and partially masked versions of the same text. The method first obtains the model's pre- dictions on the completely unmasked text (baseline), then creates a masked version with 30% of tokens replaced by mask tokens (using deterministic masking seeded by text hash for reproducibil- ity). We compute the difference between the two sets of predictions at the masked positions using cross-entropy loss difference. The intuition is that for member samples, the model maintains more consistent predictions even when context is partially obscured, resulting in smaller differences. The final score is the negative of this difference- smaller changes in predictions (lower error) indicate
23

---

Published as a conference paper at ICLR 2026
Table Model Utility Comparison LLaDA-8B- Pre-trained vs. Fine-tuned Stages. Lower Perplexity (PPL, 4) is better, higher LLM-as-ap a---------ddee (LLM Judge, 1) score is better.
Pre-trained Model
Fine-tuned Model
Dataset
Test PPL
Train PPL
LLM Judge Test PPL
Train PPL (4)
LLM Judge
(4)
(L)
(f)
(4)
(1)
ArXiv
13.327
13.217
0.475
12.043
9.648
0.742
Hacker News
27.207
20.908
0.490
18.527
14.786
0.685
Pile (CC)
17.705
17.369
0.525
17.489
14.012
0.748
PubMed Central
12.148
12.461
0.625
10.332
8.467
0.771
Wikipedia (en)
8.344
7.875
0.640
8.394
6.562
0.708
GitHuh
4.320
3.641
0.506
4.378
3.241
0.593
WikiText-103
8.878
8.783
0.400
7.195
6.684
0.644
AG News
28.419
28.984
0.415
21.036
16.329
0.663
XSum 12.393
12.543
0.590
12.214
11.658
0.705
Table 5: Model Utility Comparison on Dream-vO-Base-P7 Pre-trained vs. Fine-tuned Stages. Lower Perplexity (PPL., J) is better; higher L.LM-as-a-judge (LLM Judge, 1) score is better.
Pre-trained Model
Fine-tuned Model
Dataset
Test PPL (1)
Train PPL LLM Judge Test PPL
Train PPL
LLM Judge (1)
(L)
(t)
(1)
(1)
ArXiv
15.195
15.157
0.585
14.887
10.812
0.768
Hacker News
23.769
23.186
0.455
21.924
15.036
0.671
Pile (CC)
17.399
17.853
0.520
16.973
14.947
0.654
PubMed Central
11.087
11.368
0.650
11.416
8.702
0.792
Wikipedia (en)
10.478
10.293
0.680
10.125
8.493
0.763
GitHub
6.980
6.008
0.760
5.829
5.358
0.804
higher membership likelihood. Alternative error metrics (L1 or L2 norm between logits) can also be used, though cross-entropy typically provides the most stable signals.
All methods use consistent sequence truncation at 512 tokens and identical random seeds(42) for reproducibility.
D.1.5 Evaluation Metrics
MIA performance is evaluated using: (1) AUC: Area under ROC curve, where 1.0 indicates per- fect discrimination; (2) FPR@FPR: True positive rates at 10%, 1%, and 0.1 % false positive rates, assessing practical privacy risks. Model utility is verified through perplexity on held-out sets and LLM-as-a-Judge evaluation using structured prompts.
D.2 Performance of Target LLMs
To ensure that our fine-tuned diffusion- based LLMs maintain high utility and are not merely overfit- ting the training data, which condition that could artificially inflate MIA success (Yeom al., 2018), we evaluate their performance using two primary approaches: perplexity and an LLM-as-a- Judge framework (Zheng et al., 2023a), shown in Table 4 and 5.
Generally, both Test PPL and Train PPL values decreased after fine-tuning, indicating an enhanced understanding and generation capability of the models. Concurrently, the I.LM-as-ajjudge scores,
24

---

Published as a conference paper at ICLR 2026
Table 6: Additional MIA performance (AUC, @ 10%FPR, T FPPR TPR @ (FFRR
across WikiText- AG News, and XSum datasets. Each cell shows the mean with std. dev. as subscript. pest-perg results are highlighted.
AG News
XSum
MIAs
AUC
T@10%
T@1s
T60.1
AUC
1010
161%
T60.1%
AUC
Té 10%
1Ø1%
T@0.1%
Loss
0.518
0.122 or 0.014gmm
0 in 0.526..01 n1
0.135
0.015 os 0.002
0.532
0.138
0.002
ZLIB
0.525
0. 0.015 m 0.002 0.534
0.141
0.018 00 0.002 m 0.540
0.145
0.018
0.003_ 00
Lowercase
0.536::00
0.1362
0.017= 0.0022.m
0.542
0.148
0.019=m
00 0.003
0,548
0.152
0.020
0.003
Min-K%
0.522
0.125g
0.016: m 0.001
0.530
0.1382.01
0.017=m0
0.002
0.536
0.141
0.019
0.002
Min-K%+++
0.519
0.123=
0.014
0.001
0.527
0.136...01
0.016=
0.002=
0.5333
0.139
0.017
0.002
BoWs
0-501++01
0.105+
0,010-
0,000
0.508+..01
0.118++.01
0.012+
0.001+
0.515
0.1224..1
0.0134.60 0.001-
ReCail
0.124g
0.014 o 0.001 1 i 0.529
0.137
0.016
0.002
0.535
0.140
01 0.017
0.002
CON-Recall
0.517+t01
0.
0.013:
00 0.001
+.00 0.525 4.01
0.134-+.01
0.015=
0.002:
+.531+.00
0.137++.01
0.016+.00
0.002 n0
Noighbor
0.508.2.01
0.113:
0.011
00 0.001
0.00 0.516.g.01
0.126:2.01
0.014=200
0.001
m
0.522
0.129
0.015
0.001 00
Ratio
0.58
0.178
0.028 01 0.003 0 0.608
0.205
0.034 M1
0.004 m 0.616g..
0.212
0.036
0.005 00
SecM1
0.528
0.130
0.015 m 0.001 :: 0.536
0.143
2:01
0.017=200
0.002 m
0.542.
0.1477
0.018
0.002 00
0.524
0:127 0t 0.014 0 0.001 + oo 0.532
0.
0.040..00
0.016=
0.002: m
0.538
0.144
0.017
0.002 00
SAMA (Ours) a.7 0.415
0.108 nz 0.018 0.673 0.268
0.052_.. 0.007
0.682 0.277 0.055 0.008
which reflect the quality of the model's output as assessed by another I.M,, generally increased post-finettuning. This trend was observed across diverse datasets, including those from the MIMIR benchmark and standard NLP benchmarks such as ArXiv, DM Mathematics, Hacker News, Pile (CC), PubMed Central, Wikipedia (en), GitHub, WikiText-100 AG News, and XSum. These obser- vations collectively suggest that the fine-tuning stage effectively refined the models' abilities.
D.3 Extended Main Comparison Results
This section provides a more extensive evaluation of SAMA, detailing its performance with the LLaDA-8B-Bass model on additional benchmark datasets including WikiText- 103, AG News, and XSum. also presents comprehensive findings for the Dream-vO-Based model across six diverse datasets. These supplementary results offer broader perspective on SAMA'S 's robust performance and the privacy implications for diffusion- based L.MMs.
For the LLaDA-8B-Bail model, further analysis is presented in Table 6. On Wiki'Text 103, SAMA achieves an AUC of 0.782 with a TPR @1 &FPR of 0.108, substantially outperforming the second- best method (Ratio), which achieves 0.584 AUC. For AG News and XSum datasets, SAMA main- tains strong performance with AUCS of 0.673 and 0.682, respectively, though with more modest TPR @ %FPR values of 0.052 and 0.055. While the improvement margin is less dramatic than on other datasets, SAMA still consistently outperforms all basclines by approximately 10-20% in AUC, demonstrating its effectiveness across diverse text domains.
Turning to the Dream- v0- Base-BB model, Table 7 reveals strong MIA performance by SAMA de- spite the model's diffusion- based architecture. SAMA achieves exceptional performance on GitHub with an AUC of 0.806 and TPR @ &FPR of O. 168, representing a 45% improvement over the second- best method (ZLIB at 0.558 AUC). On Wikipedia (en). SAMA attains an AUC of 0.764 with TPR @ 1 FPFR of 0. 12, demonstrating robust attack efficacy. For ArXiv and Pile CC, SAMA achieves AUCs of 0.748 and 0.745, respectively, with corresponding TPR 1%FP values of 0.098 and 0.096. PubMed Central shows AUC of 0.732 with TPR @ I 1&PR of 0.081, while Hack- erNews exhibits an AUC of 0.615 with TPR@ I FPPR of 0.024.
D.4 Extended Ablation Studies
To assess the robustness of SAMA and the contribution its hyperparameters, we conduct a sensitiv- ity analysis the LLaDA-8B Ba-asse model using the ArXiv dataset. We vary three key parameters: the number of progressive masking steps (I), the subset size (m), and the number of sampled subsets (N). Figure 4 illustrates the impact of these variations AUC and TPR @ FRFPR
25

---

Published as a conference paper at ICLR 2026
Table 7: MIA performance on gream-v0-Base-7BB model across different datasets. Each cell shows the mean with std. dev. as subscript. Best-performing results are highlighted. Note the more irregular performance patterns compared to LLaDA models.
ArXiv
GitHuub
HackerNews
MIAs
AUC
T@10%
TeIs
T80.
AUC
1010%
T61%
T60.1%
AUC
10%
TØ1%
T@0.1%
Loss
g.498
0.103 in 0.009 mn 0.000 in 0.524
0.
0.018
0.002 0.491
io
0.000_
ZLIB
g.08
0.007 0.
0.000
0.558
0.168
0.028 m 0.014 m 0.507
0.1122.00
0.012
0.001=
Lowercase
0.5092:0:
0.097=
0.010=
0.001
0.498
0.104
0.014=m:
01 0.001
0.482 t.
0.086
0.007
0.001
Min-K%
0.53221.01
0.015
0.001
0.515
0.119
0.021 on 0.003
0.518g.m
0.121
0.016::::
0.002
Min: S++++
0.504
02 0.009
0.000
4.30
0.094_.00
0.01 =00 0.001 m 0.499
0.108-+.02
0.001
0.001
BoWs
0.519 00 0.011
0.000
0.656
0.306.+.02
0,154_..2
0.059
0.527
0.128-4.01
0.009+.00
0.001
ReCall
0.501 2:01 0.109 nz 0.009 m 0.0000 in 0.531 1.00
0.142
0.023 m 0.003
0.496..m
0.102 o1 1.
0.000
CON- CON-Recall
0.
0.098 0.0088 oz 0.000 i 0.523
0.
0.019_
on 0.002
0,488
mn
0.092 o1 0.008
0.000
Neighbor
0.506+.02
0.101 0.01 1 0.001 4.00 0.479
0.082++.0
0.007
01 0.000
0,546+-10
0.148-4.02
0,021+
0.001
Ratio
0.521+
0.
0.016
0.001
0.549
0. 0.162.+.03
0.029=
0.004=
0.535
0.141
0.018
0.001
SecMI
0
0.
10
0.010
0.000
0.537
0.025
0.004=
0.511
0.117
3
0.001=
PIA
0.
0.096 0.008
0.000
0.509
0.113
0.013
0.001
0.486
0.090
0.009
0.000
SAMA (Ours) 0.748:..1 0.412 m 0.098
0.011 2.01 0.806
0.498
0.168
0.042
0.615
0.186
0.024::.a
0.003
PubMed Central
Wikipedia (en)
Pile CC
MIAS
AUC
T@10%
Te1%
Te0.1%
AUC
T@10%
T81%
T@0.1%
AUC
Té 10%
T@1%
T@0.1%
0.489 4.01 0.096 a2 0.010= m 0.001 0.093
0.089
0.009
0.001
0.497
0.101 +.02 0.01
0.001 00
ZLIB
0.088gm m 0.006g m 0.000 m 0.519 o1 0.124
0.017
0.002 m 0.488 m
0.093 o1 0.008 0.000_
Lowercase
0.
0.093gm
0.00 0.000
0.509
0.099
0.010
0.001
0,514
0.103
0.009
0.001
Min-K%
0.496+.01
0. 10
0.009-
0.001
0.483+.01
0.077+.01
0.008-
0.000-
m 0,529
0.131 +.00 0.017+.01
0.002
Min-KS+++
0.491
0.103
0.008
0.000
0.501
+.01
0.107++00
0.007
-0 0.000--.
0.490
0.098++.0
0.007+.00 0.001 -00
BoWss
0.100 m 0.002 m 0.000 o 2.07
0.0844..m
0.003
m 0.001 m 0.480 i
0.0924.01
0.003
0.001 00
ReCail
0.4
0.090 ot 0.007 = oo 0.000 o 0.504
0.101
0.009=mm
m 0.000 m 0.498 in 0.095
0.008
0.000 00
CON-Recall
0.5184400
0.117
0.012:
0.001 0.493
0.09222.01
0.008
0.000 m 0.496
0.100
0.009
0.000 00
Neighbor
0.507 2:01
0.097
0.010=
0.
0.527
0.118
0.018
0.002
0.502
0.093
0.009
0.000
Ratio
0.114g
0.011
0.001 : m 0.523
0.121
0.014=m0
0.001
0.526 01
0.128
0.
0.001
SecM1
0,501 ot 0.102
0.009
0.000
0.514
0.109
0.011
m1
0.001
0,508
0.106 o.
0.010
0.001
PIA
0 0.092
0.006
0.000
0.507 1 0.
0.009
00 0.000
0,501
in 0.097
0.008 no 0.000
SAMA (Ours) 0.732 0.338__. 0.081
0.009 1.m 0.764
0.426...a
0.112_
0.016
0.745
0.382
0.096_..1 0.011
Impact of Progressive Steps (T), We observe strong positive correlation between the number of masking steps and attack performance. Increasing T from to 16 yields a significant improvement, raising the AUC from 0.741 to 0.805 and nearly doubling the TPR @1 F%FPR from 0.089 to 0. 174. While extending T' further to 48 achieves even higher performance (AUC 0,845), we select T = 16 as the default configuration balance attack efficacy with query complexity.
Impact of Subset Size (m). The size of the token subset plays a critical role in robust aggrega- tion. Extremely small subsets (e.g., m = 1) yield suboptimal performance (AUC 0.783), they lack sufficient context to distinguish membership signals from noise. Performance improves as m increases, reaching an AUC of 0.856 at m 32. This suggests that larger subsets can effectively capture coherent membership patterns in long-context domains like ArXiv. However, our default choice of m 10 remains a robust operating point that significantly outperforms the baseline.
Impact of Number of Subsets (N). SAMA demonstrates saturation effect with respect to the number of sampled subsets N. Performance improves rapidly as N increases from 1 (AUC 0.755) to 32 (AUC 0.803), demonstrating the necessity of sufficient sampling to estimate the robust mean. However, beyond N = 32, the performance plateaus, with N = 128 yielding a marginal gain to 0.805. This indicates that the sagn-based statistic converges quickly. Since these subsets are sampled offline with negligible computational cost, we employ N = 128 to ensure maximum robustness without incurring additional model queries.
26

---

Published as a conference paper at ICLR 2026
0.55
0.75
0.73
SAMA
0.70
0.70
0.65
0.65
0.50
0.69
10
30
16
52
64
0.30
SANA
0.30
0.20
0.20
0.20
0.15
0.30
9.05
0.00
10
30
0.00
1.6
32
0.00
02 64 120 256 Number of Subeents
Numbor of
Subeet Sin0
Figure 4: Sensitivity analysis of SAMA hyperparameters on LLaDA-8B-Baal (ArXiv). The top row reports AUC, and the bottom reports TPR @ I%FPR. We vary the number of progressive steps (left), subset size m (middle), and number of subsets N (right). The dashed red line indicates the performance of the best baseline (Ratio).
Table 8: Comparison of LoRA configurations and DP-LoRA privacy levels on ArXiv. Left: LoRA rank (r) impact. Right: DP-LoRA with varying privacy budget (e). Lower perplexity indicates better utility; higher AUC indicates stronger membership inference.
Trn. %
PPL
AUC
T@ 10%
PPL
AUC
T@10%
T@ 1%
4.02%
13.27
0.528
0.106
0.01
13.46
0.497
0.098
0.010
512
7.73%
12.95
0.571
0.142
13.18
0.502
0.109
0.012
1024
14.34%
12.61
0.638
0.218
10.0
12.74
0.548
0.128
0.015
2048
25.09%
12.38
0.726
0.321
20.0
12.35
0.607
0.176
0.024
Full
100.00%
12.04
0.850
0.586
N/A
12.04
0.850
0.586
0.178
D.5
Defending against SAMA
We evaluate parameter-efficien fine-tuning methods potential defenses against SAMA: stan- dard Low-Rank Adaptation (LoRA) and its differentially private variant (DP-LoRA). These methods naturally limit memorization by constraining the parameter space during fine-tuning, potentially re- ducing vulnerability to membership inference. We analyze the privacy-utility trade-off's of both approaches on the ArXiv dataset, demonstrating that while both defenses can significantly reduce SAMA'S 's effectiveness, they require careful calibration to maintain acceptable model utility.
D.5.1 Low-Rank Adaptation (LoRA)
To evaluate defense strategies against SAMA, we investigate Low-Rank Adaptation (LoRA) Hu al. (2021) fine-tuning the Ma-DA-B--Baa model using the ArXiv dataset. LoRA A reduces the num- ber of trainable parameters through low-rank decomposition (with rank and a 2r), potentially limiting memorization- key vulnerability exploited by membership inference attacks (Luo et al., 2024; Amit et al., 2024; Zhang et al., 2025b).
Table 8 (left) reveals clear privacy-utility trade-off. Lower ranks provide stronger privacy protec- tion: at r = 256 (4.02% trainable parameters), SAMA'S AUC drops substantially from 0.850 (full fine-tuning) 0.528, with TPR@ @ 10%FPR decreasing from 0.586 to 0.106. This privacy gain, how- ever, incurs a utility cost- pese-eexxxitty increases from 12.04 to 13.27. As the rank increases, model utility improves, but privacy protection weakens. At 2048 (25.09% trainable parameters), per-
27

---

Published as a conference paper at ICLR 2026
Table 9: Impact of temperature scaling on membership inference attacks. Higher temperatures increase output entropy, providing modest defense with no retraining required.
ArXiv
T
Method
AUC
TPR@ 10%
TPR@1%
1PR@0.1%
Ratio
0.597
0.181
0.023
0.001
SAMA
0.850
0.586_.m1
0.178
0.014
Ratio
0.588
0.172
0.021 2:01
0.001
1.1
SAMA
0.842..01
0.575
0.175
0.014
Ratio
0.575
0.163
0.019
0.001
1.2
SAMA
0.831
0.560
0.169 +.00
0.012
Ratio
0.534
0.135
0.014
0.000
1.5
SAMA
0.776
0.448 -01
0.108 -002
0.005
plexity improves to 12.38, yet SAMA's effectiveness rises significantly (AUC 0.726, TPR @ 10%FPR 0.321). The intermediate configuration (r = 1024) offers a balanced compromise with AUC 0.638 and perplexity 12.61, suggesting that practitioners must carefully calibrate LoRA rank based their specific privacy-utility requirements.
D.5.2 Differentially Private LoRA (DP-LoRA)
Differentially Private LoRA (DP-LoRA) Liu et al. (2024) provides formal privacy guarantees against MIAs like SAMA. Computing per-sample gradient norms for gradient ferential privacy-becomes computationally prohibitive for hillion-parameter models. DP-LoRA addresses this by restricting per-sample gradient operations, noise injection, and privacy account- ing the compact LoRA adapter parameters. We employ LoRA rank 1024 (with 2048, dropout 0.05), apply DP with target 8 = 1 10-5, gradient norm clipping at 1.0, and vary the privacy budget c.
for dif-
Table 8 (right) demonstrates DP-LoRA's effectiveness against SAMA on ArXiv. The baseline (N/A row) represents standard LoRA without differential privacy. Stringent privacy guarantees (E 0.01) virtually neutralize SAMA, reducing AUC from 0.850 to TPR @ 10%FPR plummeting from 0.586 to 0.098. This robust defense incurs utility penalty, creasing perplexity to 13.46. As relaxes, the privacy-utility trade-off becomes more favorable: at
random chance
1.0, strong privacy persists (AUC 0.512) with improved utility (PPL 13.18), while e = 10.0 achieves AUC 0.548 and PPL 12.74. Even with relaxed privacy (e 20.0), SAMA'S effectiveness (AUC 0.607, TPR @ 10%FPR 0.176) remains substantially below the unprotected baseline, while perplexity (12.35) approaches baseline performance.
These results confirm that DP-LoRA offers scalable, practical defense for diffusion-based LLMs, effectively mitigating membership inference risks across various privacy budgets. The persistent protective effect even at higher values demonstrates DP-LoRA's robustness, though optimal de- ployment requires careful calibration of the privacy budget balance protection against SAMA with acceptable model performance for the target application.
D.5.3 Temperature Scaling
Temperature scaling is often proposed as low-cost, inference-time defense that flattens the output distribution via = exp(z/T))/X) exp(2,/T) (Guo et al., 2017). However, our evaluation in Table 9 reveals that this method fails to offer a practical trade-off between security and utility.
At moderate levels (T [1.1, 1.2)). the defense offers false sense of security. The attack perfor- mance remains stubbornly high, with SAMA'S 's AUC dropping only marginally from 0.850 to 0.831 (a decrease of merely 2.2%). This resilience stems from the nature of window- haased aggrega-
28

---

Published as a conference paper at ICLR 2026
Table 10: Attack performance against SOFT defense on ArXiv dataset. SOFT configured with selection ratio = 0.3 and paraphrase ratio / = 0.5.
ArXiv
Defense
PPL
Method
AUC
TPR 10%
TPR@1
T.R@0.1%
Ratio
0.597
0.181
0.023
0.001+:00
Baseline
9.648
0.850
0,586 03
0.178
0.014
Ratio
0.497
00
0.011 =.001
0.001 1.00
SOFT
9.647
SAMA
0.499_.004
0.104
0.013_ 002
0.0022..00
tion: because the entropy increase is applied uniformly, the relative ordering of loss values within local windows persists, preserving differential signal between target and reference models.
When scaling is pushed aggressively to T 1.5, observe more tangible reduction in attack success. AUC falls by 8.7% to 0.776, and TPR @ 1&FPR decreases by approximately 39% (0.178 to 0.108). However, this regime is known to severely degrade generation quality, causing incoherence and repetition. Furthermore, SAMA proves far more robust to this perturbation than the global baseline. As T increases 1.5, the relative AUC advantage of our method over Ratio actually widens (from 1.42x to 1.45x), confirming that local aggregation is structurally better equipped to survive entropy-based obfuscation than global averaging.
D.5.4 Selective Data Obfuscation in LLM Fine-Tuning
We further evaluate the resilience of SAMA against the SOFT defense (Zhang et al., 2025b), which mitigates privacy risks by identifying and paraphrasing the most "memorable" training samples. Table 10 illustrates a complete collapse in attack efficacy on the ArXiv dataset. While SAMA ini- tially achieves dominant AUC of 0.850, the application SOFT suppresses this score to 0.499, effectively rendering the attack no better than random guess. This demonstrates that by altering the syntax influential records (approximately 15% of the data) without changing their seman- tics, SOFT successfully eradicates the specific token-level artifacts that SAMA exploits, all while maintaining the model's general utility.
D.6 Impact of Reference Model Misalignment
0.90
0.80
TPR @ 10% FPR
AUC
0.80
0.600
0.70
0.40
0.20
N/A
Severely Misaligned
Slightly Misaligned
Ideal Alignment
Accessibility of Reference Model
N/A
LLaDA Instruct
TPR
10% FPR
Dream Base
LLaDA Base
Figure 5: Impact of Reference Model Misalignment on SAMA Performance. Performance (AUC and TPR 10%FPR) shown for an ideal reference (LLaDA-8--Base) slightly misaligned ref- erence (LLaDA-8BB misalignment leads to reduction in attack performance, though SAMA still outperforms basclines with severe misalignment.
and severely misaligned reference (Dream-v0-77 (0-7B-l v0-7B-Base) Increasing
29

---

Published as a conference paper at ICLR 2026
This section investigates the robustness of SAMA when this ideal reference is unavailable, neces- sitating the use of alternative, potentially misaligned reference models. We maintain the LLaDA- 8B-Base model fine-tuned on the ArXiv dataset the target and evaluate SAMA'S performance (Ideal Alignment), the closely re-
using three distinet reference models: the ideal LLL.DA-8--BE (Slightly Misaligned), and the substantially different Pream-v0-7B-Bane (Severely Misaligned). LLaDA-8B-Instrul shares architecture and tokenizer with the target but dif- fers in instruction-tuning, affecting utility and prior knowledge. Dream-vO-7B-BB ----B-Base differs more significantly in architecture, pre- -re-training data, and tokenizer.
lated
The results, presented in Figure 5, reveal that SAMA'S 's performance degrades gracefully with increas- ing reference model misalignment. The slightly misaligned LLaDA-8B-Ing ---strutt reference causes a small drop in attack efficacy compared to the ideal LLaDA-8BB -8B-Base reference. However, the severely misaligned Dream-v -a-e---v----aaa--- reference leads to more critical reduction in performance, although the attack remains more effective than uncalibrated baselines. This performance drop can be attributed to two main factors. Firstly, greater divergence in the reference model's prior knowl- edge compared to the target model diminishes the accuracy of difficulty calibration. Secondly, dif- ferences in tokenization between reference and target models mean that masks applied during each step may cover different semantic units of the sentence, leading to reconstruction losses that reflect disparate aspects of the text and thus less effective calibration.
D.7 Computational Overhead
We provide a theoretical analysis of the computational cost of SAMA based on the algorithmic specification in Section 3.5, followed by empirical validation.
Theoretical Analysis. The computational complexity of SAMA is dominated by the model inference steps required to extract membership evidence. As detailed in Algorithm 1 (Phase I), the algorithm iterates through T progressive masking steps. Within this outer loop (lines 4-15), the target and reference models are queried exactly once per step (lines 9-10) compute the loss vectors ge and e for the masked sequence Xm- The subsequent robust subset aggregation involves an inner loop (lines 11-14) that samples N subsets and computes arithmetic differences. This inner loop operates entirely on the cached loss vectors derived from the single pair of forward passes. Consequently, the operations involving N, as well as the sign-based aggregation and adaptive weighting in Phase II (Algorithm 2), are purely CPU-bound arithmetic operations with negligible cost compared to the GPU-bound model inference. Therefore, the total query complexity of SAMA is O(T Cfwd), where Ciudt is the cost of a forward pass. This asymptotic complexity is identical to standard query- based baselines (such as Loss or Ratio), which also require aggregating results over T Monte Carlo samples to estimate the expected loss over the stochastic masking space of DLM.
Empirical Runtime Comparison. To validate this theoretical assessment, we measured the wall- clock time for performing attacks the LLDD--B-BBase model using the ArXiv dataset. To ensure a direct comparison, we fixed the query budget to T = 16 for both S SAMA and the query-based baselines (Loss and Ratio). As shown in Table 11, the additional CPU overhead from our subset aggregation logic is practically undetectable. The runtime of SAMA (Ih 32m 16s) is nearly identical to that of the Ratio baseline (Ih 32m 00s). In contrast, methods that require generating new text samples, such as Neighhorhood-basee attacks, incur significantly higher computational costs, taking over 8 hours to complete the same task. Furthermore, as shown in Section D.4, SAMA remains effective even at lower query budgets. With T = 4, SAMA achieves TPR @ F&FP of 0.1808 while reducing the runtime to approximately 23 minutes on the ArXiv dataset, effectively outperforming the full- hall-budget (T = 16) baselines while being 2 faster.
E
The Structure of Membership Signals: An Empirical Analy- sis
In this section, we provide a plausible explanation for the design choice of SAMA. We conducted an analysis of token-level loss differences. We utilized the LLaDA-8B model fine-tuned on the ArXiv
30

---

Published as a conference paper at ICLR 2026
Table 11: Runtime comparison of MIA methods on LLaDA-8B-Bar (fixed query budget T 16 for query- query-based methods). SAMA incurs negligible overhead compared to the Ratio baseline.
Dataset
Loss
Ratio
SAMA (Ours)
Neighborhood
Wikipedia (en)
39m 01s
32m 00s
32m 16s
8h 43m 27s
PubMed Central
55m 17s
37m 04s
36m
32m
HackerNews
54m 20s
1h 35m 08s
Ih 35m 36s
9h 13m 02s
64)
(b)
10a
10-1
10"
Density
P(N
10"
10
10-3
10
10-3
10-1
[Loss Difference]
Loss Difference
Members
Non-members
Gaussian
Figure 6: Empirical distribution of token-level loss differences. (a) Complementary CDF confirms the long-tailed behavior of the token loss. (b) Log-scale density plots reveal long-tailed distributions with subtle rightward shift for members. The difference is most pronounced in the consistent shift of the bulk, whereas the extreme right tail is dominated by shared domain adaptation effects.
subset of the MIMIR benchmark, evaluating both the fine-tuned target model MIP and the pre- trained reference model MBA on balanced sets of 1,000 member samples and 1,000 non-men non-member samples. For each masked sequence, we computed the token-level loss difference A t (". where positive values indicate that the fine-tuned model predicts the token with higher confidence than the reference model.
Statistical Characteristics. The analysis of over 260,000 token comparisons reveals that the loss difference distribution is governed by extreme events rather than Gaussian behavior. As shown in Figure 6(a), the data exhibits massive heavy tails. The excess kurtosis reaches 82.9 for members and 89.1 for non-members, far surpassing the zero value expected of a normal distribution. Furthermore, the distributions show significant positive skewness (7.5 for members and 7.6 for non-members). This statistical profile indicates that while the mean difference between members (0.032 and non-members (0.007 + 0.029) allows for some separability, the global average is highly susceptible to being skewed by rare, high-magnitude outliers.
Domain Membership Signals. Figure 6(b) illustrates that the right tail regions (high loss reduction) for both members and non-members are nearly overlapping. This observation suggests that tokens exhibiting the most dramatic loss reductions We hypothesize that these high- magnitude events correspond to domain domain-specific aokens-term. that appear frequently across the entire fine-tuning dataset (c.g.. formatting keywords or common domain terminology). The model adapts to these features globally, resulting in improved reconstruction for both member and non- member instances drawn from the same domain.
often poor indicators of membership.
To validate this hypothesis, we isolate tokens with the maximum mean loss differences and calculate their "Signal Strength." " defined as the ratio of the average member difference the average non- member difference. A ratio close to 1.0 implies the token provides no discriminative value. Table 12 presents several such tokens identified in the ArXiv dataset. Words like "Background," "Biogggay and "Joseph" exhibit large mean differences (> 0.39), yet their signal strength is remarkably close to 1.0 (e.g., 0,973 for "Background"), These tokens contribute disproportionately to the global loss magnitude, effectively masking the subtler, instance-specifie signals necessary for inference.
Implications for Attack Design. The strongest membership signals are not found in the greatest loss reduction tokens, but rather in the consistent, smaller shifts of instance-specify tokens. Standard attacks that rely on global averaging (like the Loss or Ratio baselines) are inherently noisy because they conflate these two signal types. By employing robust subset aggregation and sign-based voting.
31

---

Published as a conference paper at ICLR 2026
Table 12: Tokens with the highest loss differences often yield weak membership signals. The Signal Strength (== 1.0) indicates that the loss reduction is similar for both members and non-members, confirming these are domain adaptation effects rather than -peecic memorization.
Token
Mean Diff (A)
Signal Strength
Count
Biography
1.040
0.705
50
Background
0,671
0.973
25
Life
0,631
0.701
21
List
0.544
0.501
39
Description
0.399
0.526
Joseph
0.393
0.978
SAMA effectively filters out the maggitaae domain noise shown in Table 12 and amplifies the consistent, albeit weaker, signals of true membership.
32


Reviews:
[{"review_id": "dfK9vVL3zb", "reviewer": "Reviewer_aFkU", "summary": "This paper presents the first systematic investigation of Membership Inference Attack (MIA) vulnerabilities in fine-tuned Diffusion Language Models. It introduces SAMA, a novel attack framework that leverages progressive masking and robust sign-based aggregation to exploit the bidirectional dependencies unique to DLMs. The method demonstrates significant improvements over a wide range of baselines across multiple models and datasets.", "questions": "See the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed SAMA method is innovative since it is the first work to systematically study MIA risks for Diffusion Language Models.\n2. The experimental setup is comprehensive, evaluating SAMA on state-of-the-art DLMs across diverse datasets against multiple baselines. The results are compelling and the ablation studies effectively justify the design choices.\n3. The paper is written in a clear structure.", "weaknesses": "1. The SAMA framework involves multiple components (e.g., progressive masking, subset aggregation, adaptive weighting) and hyperparameters. While effective, the method is somewhat complex. A more intuitive explanation and stronger justification for the necessity of each component could enhance clarity.\n2. It would be good to propose or evaluate more defensive strategies to improve practical applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FMzeltOsy0", "reviewer": "Reviewer_VbcU", "summary": "This paper presents the first systematic investigation into the vulnerability of fine-tuned Diffusion Language Models (DLMs) to Membership Inference Attacks (MIAs). The authors identify that the bidirectional, multi-configuration masking mechanism of DLMs creates a fundamentally different and larger attack surface compared to autoregressive models (ARMs). To exploit this, they propose a novel attack framework named SAMA (Subset-Aggregated Membership Attack). SAMA leverages a progressive masking strategy to probe the model at multiple scales, combined with a robust subset aggregation method using sign-based statistics to handle sparse and noisy membership signals. Through extensive experiments on two state-of-the-art DLMs across nine datasets, the authors demonstrate that SAMA significantly outperforms a wide range of existing MIA baselines, revealing critical and previously overlooked privacy risks in this emerging class of models.", "questions": "1. The grey-box assumption in Section 2.1 is my primary concern. Could the authors clarify if the cited works for this threat model (Zhai et al., 2024; Duan et al., 2023; etc.) truly operate under this specific assumption for language models (i.e., querying with custom masks to get intermediate losses)? Or is this a stronger variant? How would the attack perform in a more realistic API-based scenario where only the final generated sequence and its token probabilities are available?\n2. Section 3.1 provides an intuitive analysis that multiple masking configurations in DLMs create a larger attack surface. While Section 3.2 points to the main results in Section 4.2 for empirical validation, is there any more direct experimental data that can specifically isolate and demonstrate how varying mask configurations ($S$) impact the membership signal strength ($\\Delta_{DF}(x;S)$)?\n3. The attack seems to rely on sampling a variety of mask configurations with the hope of \"hitting\" one that reveals a strong signal. Is there an analysis of how many samples are needed to achieve good performance? Specifically, how does the attack's performance scale with the number of progressive steps ($T$) and the number of sampled subsets ($N$)?\n4. The subset aggregation method relies on hyperparameters $N$ (number of subsets) and $m$ (subset size). How were the values ($N=128, m=10$) chosen? Could the authors provide a sensitivity analysis showing how these parameters affect the trade-off between attack accuracy and computational cost?\n5. The analysis in Section 3.1 is compelling, but it is presented without direct empirical support within the section itself. Could the authors include a small, illustrative experiment to support the claim that different mask configurations can reveal stronger or weaker signals for member vs. non-member samples?\n6. A minor point on presentation: the manuscript uses a large number of em-dashes for sentence construction, which can hinder readability. I would recommend the authors polish the text to improve flow and clarity, which would help lower the cognitive load for readers.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "-  The work is highly significant and original as it provides the first systematic study of Membership Inference Attacks against the emerging and important class of Diffusion Language Models. It tackles a critical and underexplored privacy problem.\n-  The quality of the work is high. The proposed SAMA attack is technically sound and specifically tailored to the unique properties of DLMs. The empirical evaluation is comprehensive, and the substantial performance gains over a wide range of baselines strongly support the paper's claims.\n-   The paper is written with clarity. The core concepts, especially the insightful comparison between the attack surfaces of ARMs and DLMs, are well-articulated, making the motivation for the proposed method easy to understand.", "weaknesses": "- The grey-box access model assumed in Section 2.1 appears to be quite strong. It posits that the adversary can query the target model with arbitrary, custom partially masked sequences and receive detailed outputs like logits for specific token positions. This may not be realistic in many practical scenarios. For instance, with closed-source models, an attacker might only be able to access the final output sequence and its token probabilities via an API. Conversely, with open-source models, an attacker would have full white-box access, including parameters and gradients, which is an even stronger (and different) threat model. The assumed threat model sits in a potentially unrealistic middle ground. The authors should provide a stronger justification for its practicality or discuss how the attack's performance might change under more constrained, API-based settings.\n-   While the components of SAMA are intuitive and empirically effective, their design is largely heuristic and lacks theoretical guarantees. For example, the analogy drawn between progressive masking and multi-scale analysis techniques like wavelet decomposition (line 291) is an interesting intuition but is not rigorously justified; the mathematical mechanisms and application domains are fundamentally different.\n-  A significant concern after reading the methodology is the computational overhead of SAMA. A standard loss-based attack on an ARM requires roughly one forward pass per sample (complexity $O(L)$). In contrast, the proposed SAMA algorithm requires multiple forward passe, e.g. one for each of the $T$ progressive masking steps. This suggests a complexity that is at least $T$ times higher. While the impressive performance gains might justify this extra cost, the authors should discuss this directly. A theoretical analysis and an empirical comparison of the runtime/computational cost of SAMA versus the baselines are essential for evaluating the practical viability of the attack.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WZE2MKP5bg", "reviewer": "Reviewer_7q5n", "summary": "This paper introduces SAMA (Subset-Aggregated Membership Attack), a novel membership inference attack tailored for diffusion-based language models (DLMs) in fine-tuning scenarios. Unlike traditional autoregressive models, DLMs generate text through iterative bidirectional masking, which exposes unique privacy vulnerabilities. SAMA exploits these characteristics by employing progressive masking and sign-based subset aggregation to amplify sparse membership signals that emerge under specific mask configurations. Through extensive experiments across multiple datasets and diffusion architectures, the authors show that SAMA achieves substantial improvements over established baselines (previously mainly designed for autoregressive models), such as loss-based and ZLIB attacks, particularly at low false-positive rates.", "questions": "1. Could SAMA be generalized beyond diffusion-based architectures, for example, to masked or autoregressive language models, by adapting the masking or sampling strategy?\n\n\n2. What is the computational overhead of SAMA in practice? Since it involves repeated masking and querying, quantifying its runtime or query cost relative to traditional MIAs would help assess its feasibility in realistic attack settings.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper focuses on a new problem by systematically investigating membership inference vulnerabilities in diffusion-based language models, a model family that differs fundamentally from autoregressive architectures.\n\n\n2. The proposed SAMA framework combines progressive masking and sign-based subset aggregation to effectively expose membership vulnerabilities.\n\n\n3. The experimental evaluation is extensive and well-controlled, spanning multiple datasets, and various MIA baselines.\n\n\n4. The ablation analysis is informative, isolating the contribution of each component in SAMA.\n\n5. The paper is well-written and easy to follow.", "weaknesses": "1. The proposed method is specialized for diffusion-based language models and depends on access to compatible reference models, limiting its generalizability to broader LLM architectures.\n\n\n2. The intuition behind progressive masking and subset aggregation remains somewhat abstract; a concrete example with simple proof or visualization would make the mechanism more interpretable.\n\n\n3. The method incurs high query complexity due to multiple mask configurations, raising scalability concerns for large models and long sequences (the $m$ can be large).\n\n\n4. The paper focuses primarily on empirical MIA results. Better to have more in-depth discussion on the concrete patterns or examples in the evaluation.\n\n\n5. It is better to discuss the potential defenses strategies for SAMA.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qiqlTNGNsM", "reviewer": "Reviewer_9XsJ", "summary": "The paper presents SAMA, a new Membership Inference Attack method tailored to Diffusion Language Models. The authors argue that the iterative masking mechanism in DLMs creates unique opportunities for membership probing and propose an attack that aggregates membership signals across progressively sampled masking configurations using robust sign-based statistics. Experiments on multiple datasets and DLM architectures show that SAMA outperforms prior MIA methods.", "questions": "- How realistic is it for an adversary to obtain access to both the fine-tuned DLM and its corresponding pre-trained reference model, particularly for proprietary or commercial systems?\n\n\n- What is the total number of queries required per target sample, and how does the attack’s performance scale with reduced query budgets?\n\n- Could common API defenses such as rate limiting, randomized outputs, or noise injection effectively thwart SAMA?\n\n- How much of SAMA’s reported advantage is attributable to increased query opportunities rather than algorithmic superiority?\n\n- Can the authors provide evidence or examples of Diffusion Language Models being used in real-world production or fine-tuning scenarios?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper identifies an emerging gap by studying MIAs on DLMs, which are indeed less explored compared to autoregressive models.\n\n- The formulation of the problem and discussion of bidirectional masking effects are insightful.\n\n- The use of sign-based aggregation to handle heavy-tailed loss noise is conceptually sound.", "weaknesses": "- The attack assumes the adversary has grey-box access to both the fine-tuned DLM and its reference pre-trained model. In practice, access to the exact pre-trained model is often unrealistic, especially for proprietary commercial models. In Appendix D.5, we also see that performance drops significantly when the reference diverges. This raises doubts about the practical feasibility and real-world impact of the attack.\n\n- Each attack step involves querying the target model with multiple masking configurations across many subset samples (T×N queries), which could easily require thousands of model queries per instance. This not only makes the attack computationally expensive but also highly detectable in monitored API environments. Standard defenses such as API rate limiting, randomization, or output obfuscation would likely render SAMA ineffective. \n\n- The comparison with baselines seems unfair. SAMA requires orders of magnitude more queries than existing MIA approaches, yet the paper reports results without normalizing for query cost. The reported AUC gains may primarily reflect this increased probing power rather than a superior inference mechanism. A fair evaluation should constrain all methods to the same query budget or analyze performance vs. the number of queries.\n\n- The work assumes Diffusion Language Models are widely deployed in practice. However, as of now, most production LLMs are still autoregressive (GPT, Claude, Gemini, etc.), while DLMs are largely at the research stage. Without stronger evidence that DLMs are being fine-tuned and deployed at scale, the urgency and real-world relevance of the attack remain limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)