Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution
Authors: Andreas Floros
Seyed-Mohsen Moosavi-Dezfooli
Pier Luigi Dragotti
Venue: NeurIPS 2025 poster ()
Keywords: Generative Models
Diffusion Models
Data Provenance
Primary area: deep_learning
TL;DR: We identify the origin of images—whether from external sources, a generative diffusion model or its training set—by analyzing patterns in their diffusion trajectories.

Abstract: Diffusion models have transformed image synthesis through iterative denoising, by defining trajectories from noise to coherent data. While their capabilities are widely celebrated, a critical challenge remains unaddressed: ensuring responsible use by verifying whether an image originates from a model's training set, its novel generations or external sources. We introduce a framework that analyzes diffusion trajectories for this purpose. Specifically, we demonstrate that temporal dynamics across the entire trajectory allow for more robust classification and challenge the widely-adopted "Goldilocks zone" conjecture, which posits that membership inference is effective only within narrow denoising stages. More fundamentally, we expose critical flaws in current membership inference practices by showing that representative methods fail under distribution shifts or when model-generated data is present. For model attribution, we demonstrate a first white-box approach directly applicable to diffusion. Ultimately, we propose the unification of data provenance into a single, cohesive framework tailored to modern generative systems.

Tracing the Roots: Leveraging Temporal Dynamies in Diffusion Trajectories for Origin Attribution
Andreas Floros Imperial College London andreas floros18@imperial...... uk
Seyed-Mohsen Moosavi-Dezfooli
Pier Luigi Dragotti
Apple
Imperial College London
smoosavi@@pple..com com
p aargotti@impeer all ac uk
Abstract
Diffusion models have transformed image synthesis through iterative denoising, by defining trajectories from noise coherent data. While their capabilities are widely celebrated, a critical challenge remains unaddressed: ensuring responsible use by verifying whether an image originates from model's training set, novel generations or external sources. We introduce framework that analyzes diffusion trajectories for this purpose. Specifically, we demonstrate that temporal dynamics across the entire trajectory allow for more robust classification and challenge the widely-adopted "Goldilocks zone" conjecture, which posits that membership inference is effective only within narrow denoising stages. More fundamentally, we expose critical flaws in current membership inference practices by showing that representative methods fail under distribution shifts when model-generated data is present. For model attribution, we demonstrate a first white-bon box approach directly applicable diffusion. Ultimately, propose the unification of data provenance into a single, cohesive framework tailored to modern generative systems.
Introduction
Generative modeling has seen major advancements with the rise diffusion models (Ho et al.) 2020). These models have become the standard for image synthesis, achieving state-of-the-arr performance via trajectories from noise to coherent data (Karras et al. 2024). However, their growing prevalence has raised significant concerns about privacy, security and accountability. For instance, sensitive or copyrighted data used during training can be memorized and inadvertently reproduced by the model
during inference (Carlini al., 2023] Somepalli et al. 2023 Gu et al. 2025). Furthermore, the
lack of mechanisms to attribute content to specific models has made it easier to distribute harmful or malicious outputs without repercussions (Wang et al.) 2023 Laszkiewicz et al., [0244] Liu et al.)
2025).
We are therefore broadly concerned with the following question regarding data provenance:
Given pretrained generative models and data points, what relationships, if any, exist between them?
Specifically, toward responsible generative modeling, our goal is determine whether an image is (i) part of model's training (member) set, (ii) novel sample from the model (belonging) or (iii) sourced externally (external). doing so, we propose to unify two traditionally separate tasks: Membership Inference Attacks (MIAs). which determine whether an image was part of model's training set (Shokri et al.) 2017 Yeom al. 2018), and Model Attribution (MA), which determines whether an image was generated by a specific model (Wang et al. 2023
Laszkiewicz et al. 2024).
Work done while Imperial.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

We begin our exploration of data forensics by first focusing on the MIA setting. The literature on this task for diffusion models has largely converged to likelihood thresholding approach, where the model's (negative) denoising loss is used as proxy (Matsumoto et al., 2023 Duan et al. 2023]. We identify two significant limitations with methods adopting this framework that, arguably, render them unworkable for practical membership inference. In particular, we show that such MIAs cannot distinguish between gooeel-generate and training data, limiting their applicability in auditing synthetic data (e.g.. for identifying memorization) data extraction attacks. More importantly, we show that such threshold-basee approaches may also underperform compared to naive baselines, that have no access to the underlying diffusion model and hence no real predictive power. We therefore argue that existing MIAs are, at best, strong non non-membership inference attacks (Carlini et al. (022).
The above-mentioned shortcomings of diffusion MIAs motivate us to depart from conventional approaches. Specifically, we challenge the widely-adopted and influential "Goldilocks zone" " conjec- ture of Carlini et al. (l.((220,, which posits that representations at the intermediate steps of denoising diffusion are most effective for membership inference. Intuitively, we expect that there a wealth of information that is encoded within diffusion trajectories, and we hypothesize the existence of hidden patterns in the temporal dynamics of diffusion that may be used for more robust classification.
Equipped with our proposed trajectory representations, we then revisit the problem of data provenance. With our method, we demonstrate an application data extraction, improved robustness to distribution shifts and first white- box model attribution method that is directly applicable to diffusion models.
2
Background
We give brief overview of diffuusion and the various data provenance tasks we concerned with. We refer the reader to Section 7 for a discussion on our work's 's position within the broader literature.
2.1 Diffusion-basn based generative models
Continuous formulation Diffusion defines a mapping between the data distribution, p. and a tractable distribution. Let xx()) : [0, T"] + 12" such that (0) ~ p and x(T") is normal. Specifically, for time f = 0 to & = T, we write the Itō Stochastic Differential Equation (SDE) (Song et al. (021
(1) where f(, : - g(): g.) : + R are appropriate drift, diffusion coefficients and w is the standard Wiener process. Given the above forward process, sampling performed from L T to t= 0, by modeling trajectories with reverse-time SDE (Anderson) 1982) as:
dæ = f(x, f)dt + g(f)dw,
dx = [f t) g(t)2V log (x)) + (t)ddw
(2)
where df < 0 and w is the reversed standard Wiener process. Crucially, under this interpretation, modeling the reverse diffusion trajectories requires knowledge of the score function Va log Pt (z).
Denoising diffusion probabilistic models In practice, one discretizes the above equations and considers families of SDEs with a tractable forward process. A common parameterization is given by DDPMs (Ho al. 2020),, which choose f(x, t) = (t) and g(t) = /3(t), where the function 3(1) is some noise schedule. By discretizing ( with DDPM, the forward probabilities become:
(xr(xx) = N(x;; v az))
where the subscripts denote discretization, = - 3r and ā. II 1- ag. As (x (x) is normal, the score may be approximated by application of Tweedie's formula (Efron] 2011):
Vz, P(.,) //0,B(a/aa.
(4)
where the expectation describes the minimum mean squared error Gaussian denoiser. Therefore, by (3) and 4 Denoising Score Matching (DSM) may be performed via noise-predicting neural network, co(-,t): R + R.,, that minimizes the quantity L, = ||co( ( Vāyz + /1 āzc, t) - cll
for all t. The complete DDPM optimization objective is then expressed as follows:
min E. (0.
(5)
For appropriate Xx, the above is equivalent to the Negative Evidence Lower Bound (NELBO) of the data (Ho et al. 2020). In this sense, DSM may also be reframed as likelihood maximization.
2

---

2.2 Data provenance
Membership inference The aim of this task is to determine whether a given data point was used in the training of machine learning model, i.e., whether it is a member. Traditionally, one assumes knowledge of the overall data distribution, p. but no knowledge of the specific training data. Moreover, other training details of the model are also assumed. Given this setup, /Shkki et al. (2017) proposed to develop MIAs via surrogates, whose aim is to closely reproduce the target model without knowledge of its member set. As these surrogate models are trained by us, the attacker, it is then possible to optimize the MIA supervised way, hoping that will also transferable to the target model.
Since then, follow-up work by Yeom et al. (0118) proposed MIAs via loss thresholding, where the intuition is that training data will naturally achieve smaller model loss compared to unseen data. In practice this approach simplifies the design space of the attacks, making them largely parameter-free.
Model attribution With the rise of increasingly capable generative models, the attribution of synthetic data has become problem of interest to the research community (Wang et al. 2202 Laszkiewicz al. 2024 Liu et al. 2025). Given such synthetic data, the aim here is identily the specific generative model that is responsible for producing it. Arguably, assuming the generators adequately capture the underlying data such that there are no noticeable distribution shifts, this is a task that may require white-box access to the systems for more reliable attribution. Representative works operating under this assumption have converged to reconstruction or inversion- the idea is to estimate model-specifer internal preimages of the samples that are more informative and distinguishable than the final outputs (Wang et al. 2023 Laszkiewicz et al. 2024).
methods, where
3 Threat model
We frame data forensics as game between model developers, whose interest is minimize liability, and an adversary wishing to uncover data and its origins. The adversary's goal is to extract meaningful features, f, that couple the diffusion model, cØ. and the data, in way that reveals their underlying relationship. To this end, we will consider a simple pipeline consisting of a feature extraction stage from an off- the-shelf diffusion model and a learning stage. For simple and standardized evaluation, we will adopt a linear probing model for classification to assess the quality of feature representations:
Ix:; <g) W f(:; Ea) +
Here, I represents logits for binary or ternary classification, depending the task consider, and W, b are parameters to be optimized based on the adversary's s capabilities. For clarity, we provide complete implementation details in Appendix A and state such capabilities explicitly below:
White-box access. Given well-trained generative systems, we argue that black-box methods will struggle distinguish the various classes of data under fair evaluation with no distribution shifts, necessitating the use of internal model signals for reliable classification.
Limited data access. Surrogate development, as proposed by Shokri et al. (017)), is often unrealistic and the alternative of method refinement at evaluation leads to inflated metrics. To properly develop forensics tools, we instead explicitly assume access to a small, potentially imbalanced or otherwise uncurated fraction of the member set (we will use 3.4%).
Note, the above-described threat model makes a non-standard assumption regarding data access and trades this for the surrogate framework of Shokri et al. (2017). Our position is that, while ours is strong assumption, is far milder than the alternative. We give two reasons to defend our choice.
On practical level, we argue that surrogate development is computationally prohibitive for modern generative systems, especially since competitive MIAs operating under this framework may train several such surrogates to approximate the target's behavior (Carlini et al. 2023 Pang et al. 2025).
More fundamentally, under the surrogate framework, it is difficult to justify or bound our assumptions. In particular, to reproduce modern generative systems, a detailed description of the training setup may required. However, important details such as pretraining, dataset composition, number of iterations and regularization are often omitted or otherwise difficult to infer even in open-source releases. We therefore argue that data provenance via surrogate development involves several hidden and intractable assumptions that make it harder to judge real data privacy and security risks.
3

---

Cillli-H ane
DOPK (Chiesst-H 256)
PTB0Q-200
log Te Ei ||co(arf.) f)
<
Codelau-H 206
-2
DDPM (ColebA 200)
FPFFQ-256
-1
(a) (L,) histograms
(h) Te. V histograms
-0-
-8
-10-
-12
200
Figure 1: Member, model-generated and external data loss (averages with solid lines) as function of t for CelebA-Hi -HQ DDPM. Figure 2: Visualization of CelebA- HQ DDPM features.
(c) (C.) I-SNE
(d) (1 V v... I-SNE
Table 1: TPRs / FPRs for MIA using model-generated data. The latter (TPR, FPR) are below the y = curve, i.e., worse than random.
Top are FPRs on external and bottom are FPRs on
Dataset
= 50
100
f 150
200
t= 250
300
46.4
44.7
41.7
42.8
42.4 67.3
45.6
CIFAR-10
56.3
60.1
61.1
62.2
62.2
59.6
60.7
64.3
66.6
66.6
65.7
48.3
41.5
34.2 78.9
31.0
85.1 / 29.2
29.2
CelebA-HQ 256
64.1
72.9 /
83.5 / 96.5
85.9
92.4
94.3
95.7
96.4
96.7
Do membership inference attacks work?
It is instructive to first review existing approaches to MIAs on diffusion models. Under the assump- tions established Section 3 these attacks are, arguably, useful only as filters for data extraction (Zhang et al. 2025). ). However, we will demonstrate that MIAs fail in the presence of model-generated data, limiting their potential in auditing model outputs or extracting training data. Even worse, will highlight further weakness and show how, under certain conditions, existing MIAs may underperform relative to naive baselines that provably have no real predictive power.
To this end, we consider the representative threshold-based approach of Yeom et al. (2018), which has inspired several recent works on diffusion MIAS (Carlimi et al. 2023 Duan et al.) 2023] Matsumoto et al. 2023) Kong et al. [224]] Tang et al. 2024 Zhai et al. (2024). For given sime-step. the idea idettify as members the samples that attain denoising loss, L,, below certain threshold ::
M(æ; = 1 1[c, < <], L, = llooVo,,a + /1 Vl-ā,,, () El/2,
6~ E~N10,
(7)
Typically, is chosen to demonstrate the True Positive Rate at a set False Positive Rate (TPR @ FPR) or adjusted for other metries, e.g., the Area Under the Curve (AUC) Attack Success Rate (ASR). t is reported to be a critical hyperparameter (Carlini et al., 2023) and commonly tuned via surrogates.
4.1 Representative attacks cannot audit synthetic data
For DDPM trained on CelebA-HO -HQ (Krras et al. 2018a) at 256 x 256 resolution, we plot L; in Figures [ 2a is clear that thresholding is not sufficient for reliable classification synthetic data consistently ly fools such detectors by attaining the lowest loss. We confirm our findings with further experiments Table T and also include a CIFAR-10 10 (Krizhevsky) 2009) DDPM in our analysis. Fundamentally, we attribute this phenomenon to the maximum likelihood interpretation of DSM, where the loss is equivalent to the NELBO of the DDPM (Ho et al. 2020). As hinted Figure this limitation motivates to explore more separable features, beyond thresholding, later in Section

---

Table 2: MIA (member-external) AUC, TPRs @ 1% FPR and ASRs. For the CIFAR-10 experiment, we use CIFAR-10. as the external set, where the naive baseline close to random guessing and there no distribution shifts. The CelebA-HQ experiment uses FFHQ as external data, where there clear distribution shifts, as reflected by the baseline's performance. In the latter case, the representative threshold-based approaches of Matsumoto et al. ) Kong al. (2024) underperform.
CIFAR-10
CelebA-HQ 256
Method
AUC
TPR 1% FPR
ASR
AUC TPR 1% FPR ASR
Naive (model-bling
52.2
0.0
52.0
94.4
60.1
86.6
Matsumoto et al. (2023
63.2
3.3
59.7
85.2
26.4
76.2
Kong et al. 2024
(PIA)
66.9
5.1
62.4
62.5
0.1
58.1
Pang al. 2025
(GSA2)
82.7
12.5
73.5
100.0
99.6
92.5
0.03
0.03
83
0.00
C
C.
Ous
-0.05
82
CSA
-0.03
300
500
1.0
1.5
2.0
% of member set
% of member set
(a) CIFAR-10
CelebA-HQ 256
(a) TPR @ 1% FPR
(b) AUC
Figure 3: Parameters from W in 6 correspond- ing different features against & for our MIAs.
Figure 4: MIA performance as a function of the data budget (% of member set) on CIFAR-10.
Table 3:
AUC for our MIAs (member-external) with {--) features CIFAR-10. We fix the number of queries to the diffusion model and consider different sampling strategies for 1. We observe significant performance gains when the time-steps cover the entirety of the diffusion process.
................................ 249
250,
499 t 500,
749
750,
999
0, 4, 996
68.5
68.1
54.7
50.7
71.2
4.2 Blind baselines may beat most membership inference attacks
We now focus on attacks under distribution shifts, where the member and external data may be distinguishable beyond the membership property For idealized external data, indistinguishable from the members barring membership. the literature often reports random baseline, i.e., 1% TPR @ 1% FPR and 50% AUC, ASR. Here, we propose potentially stronger, data-driven baseline that makes predictions without access to the diffusion model. In particular,
[
a standard ResNet18 (He) ctt al. (2166) classifier that is otherwise developed under identical conditions and with the same data budget as the other MIAs, i.e., 1000 members and 1000 external samples. Note, it is clear that such naive method cannot possibly generalize, since the membership property is necessarily tied to the model. Therefore, it is expected that tailored MIAs, with access the model, will outperform it.
To our surprise, however, the results of Table reveal that the representative threshold based attacks of Matsumoto et al. (2023); Kong et al. (2024) are brittle to distribution shifts. In particular, while we see reasonable performance on the CIFAR- 10 DDPM when the external data from CIFAR- 10.1 (Recht et al.] 2018 Torralba et al.) 2008). the experiments on the CelebA-HQ DDPM with FFHQ (Karras et al. 2018b) external data show that the naive baseline significantly outperforms them.
Note, the superior attack of Pang et al. (2025), namely Gradient Subsampling and Aggregation (GSA), based on loss thresholding and instead computes more involved feature vector as follows:
fx co) E, Vo, Cillē, concat(.... .erraat............. .,..... 0
(8)
where 0 is partitioned into subsets As GSA maintains competitive performance, our findings suggest failure specific to thresholding and further motivate us investigate more robust representations.
'These are interclass shifts. There is also the possibility of intraclass shifts, e.g., the development budget consists of "cat" members. but evaluation there exist "dog" members. See Table for some analysis this.
5

---

TPR @ 0.1% FPR
TPR 8 0.11% FPR
TPR @ @.0.1 1% FPR
Ours GSAg
99
98
1.6
0.6
1.2
1.2
% of member set
member
% of member
(a) CIFAR-IO
(b) CelebA-HQ 256
(c) CIFAR-100
Figure MIA TPRs 0.1% FPR as the data budget varies. See Appendix A for details.
Table 4: TPR @ 1% FPR for the MIA of Pang al. (2025)
(GSA2) and method (Lt, ViLt, VoCt.
t == 0, O..... 999). We report TPRs over ten random data splits that are not necessarily object-balanced. ie., there may be intraclass distribution shifts. The mean and standard deviation are the right.
Method
CIFAR-10
GSA2
9.7
9.6
11.7
8.0
9.8
8.2
7.1
8.8
11.1
6.6
9.1+1..64
Ours
18.3
10.9
11.3
14.5
9.4
11.9
11.3
13.0
12.4
7.4
12.0--2.93
CIFAR- 100
GSA Ours
3.7
2.8
1.5
1.9
2.7
1.8
4.4
4.0
2.2
1.2
2.6
9.4
7.4
9.1
7.7
14.0
6.9
10.2
12.7
10.6
6.4
9.4--2.50
Rethinking the Goldilocks zone conjecture
Conjecture 1. If is too large, and the noisy image is similar to Gaussian noise, then predicting the added noise easy regardless if the imput was in the training set; if t is too small, and so the noisy image is similar the original, then the task is too difficult. It hypothesized that there exists a "Goldilocks zone" for membership inference (Carlini et al. 2023).
Our experiments in Section fjsttify an exploration of richer features for more robust classification, beyond brittle thresholding. Inspired by the GSA method of Pang network signals via gradient information, and contrary to the influential "Goldilocks zone hypothesis Carlini et al. (2023), stated in Conjecture D propose extract internal signals relating to the underlying diffusion processes. In particular, we will now explore vulnerabilities arising due to global temporal dynamics that would otherwise be discarded previous approaches, hypothesizing that the paths traversed in high-dimensions ambient spaces are distinct for different classes of data.
(2025), which extracts internal
Carlini et al. (023) validated their conjecture CIFAR-10 MIAs, where they found that 300) optimal. To motivate our approach, we revisit this setting and demonstrate alternatives that make use of temporal context. Specifically, given fixed query budget to the diffusion model, we generalize the thresholding approach via concatenation of {=1) into feature vector meant to capture evolution The results using this proposed approach are in Table 3 with the setup of Section 4.2 Interestingly, while there exists a locally optimal region for t, in line with previous observations, we find that performance can be boosted significantly by uniformly distributing the time-steps over the entirety the diffusion process. We therefore propose the counterhypothesit that there exist global patterns encoded in diffusion trajectories, containing valuable information for data provenance.
In particular, having freed ourselves from loss thresholding, we augment our MIAs with gradient features, { C, 12. V [-1 with our complete feature extraction given in Algorithm| | and visualizations in Figures 2 3 Though we admittedly do not have theoretical basis for including the
gradients, we intuitively expect that they capture curvature information that useful for trajectory modeling, given that La measures the score matching error. Moreover, as explored in Appendix B we find they greatly improve our MIAs, making them competitive with GSA in Figures and Table 4
Note, our loss sequences are not native to DDPMs. The native forward process has independent Gaussian increments, with dependent overall noise different t. Instead, overall noise is completely independent of

---

LT-11
member
external
> softmax [W(+)
()+
VoColl VoCr-111
belonging
Figure 6: Our method. Given an image, we model its trajectory via features at different time-steps. We train classifiers on these representations to distinguish member, belonging and external data.
Table 5: Hyperparameters of our linear clas- sifier, defined in (o). We refer the reader to Appendix for further implementation details.
Algorithm 1: Trajectory feature extraction
Input: Data point
Parameters: Trained DDPM, co. with To
Output: Trajectory features f 0 for t E fo..... T 1 do
batch_size epochs
50
100
NO.II)
AdamW
learning_rate
le-3
L. - leal V V + 1,,,
weight_decay
10
f f U (,,, V VoC.
step_size
5 epochs
StepLR
return f
learning_rate_decaa
0.8
6 Toward origin attribution
Having investigated temporal dynamics in MIAs, we now generalize to Origin Attribution (OA), aiming to unify data provenance under cohesive framework. Arguably. in this context, there are two fundamental relationships between models and data, i.e., membership and belongingness. To this end, we elevate binary MIAs into ternary classification, with our overall implementation summarized in Figure 6 and Table 5 In what follows, we conduct experiments toward this more ambitious goal
6.1 Stronger membership inference attacks
Here, we show how our ternary OA may ad- dress the limitation discussed in 1.
including model-generated data during clas- sifier training. maintaining compliance with stated assumptions in Section 3 With this enhancement, we develop a method suited for OA and benchmark its performance in Figure S and Table 6 Importantly, we also demon- strate application to the more ambitious task of data extraction. Specifically. by in- specting system's misclassifications, collect subset of model generations that are classified members and investigate their similarity to the training data the model. We report our findings with this approach in Figure 7 where we filter 30k model-generaaed samples down 1.7k that resemble members, as quantified via the SSCD score (Pizzi et al.) 2022). With reference to Figure 7al clear that similarity is left-skewed for such misclas- sified data, validating our method.
generated Mischesiifod
0.2
0.4
SSCD Cosine Similarity
(a) Similarity histograms of generated images to members
A
CTTY
(b) Misclassified, generated
(c) Similar members
Figure 7: We investigate generated data that is mis- classified by our system and identify similar samples to members (all unseen by the classifier in training).
Following Wang et al, 2023), we refer to novel model-generated data as belonging, noting potential overlap of generated data and members when models memorize. Also note that have overloaded their OA term.
7

---

hare.
Thue hate.
Thue Poellive Tatt.
60
40-
modd--2nnd
Thue 10
40-
Trun
C.
Co O,C, 980
20-
30-
Co VC, 0- 909
10
100
10
100
100
110
100
Fabe: Posttor Rote
Fabso Positive Rate
Fabeo Positive Rate
Fabe: Posltive Rate
Member (CIFAR-10)
Belonging (CIFAR-10)
Member (CelebA-HQQ Belonging (CelebA-HQ)
Figure 8: OA ROCs, taking member or belonging classes as positives.
Table Average AUC, TPRs 1% FPR and ASRs for OA methods (memberr-helonging-eee
CIFAR-10
CelebA-HQ 256
Method
AUC TPR 1% FPR ASR
AUC TPR @ 1% FPR ASR
Naive (model-blind)
51.5
1.2
34.7
90.0
32.1
74.4
Our method time-steps
Lg VaCe VeL.
0, .1..... ................................ 999
71.6
6.9
54.9
95.2
52.7
84.4
0,3, 999
v V
V
85.0
14.3
69.4
98.8
79.5
92.7
0....... 999
v /
86.5
15.9
71.1
99.3
86.5
94.1
6.2
Model attribution diffusion models
Our analysis so far has centered around MIAs. We now specifically focus on attribution of synthetic data: Beyond membership, do trajectories also capture fingerprints useful for robust MA? To investigate, we train classifiers to distinguish DDPM, synthetic data and non- con--elonging, real data, with the same budget of 1000 external samples in Section 4.2) For assessing robustness, non-belonging data produced by other generative models, L.c., DDIM (Song et al., (021a), WaveDiff' (Phung al.; 2023) and DDGAN (Xiao al. 2022). In this sense, this is fine thee-graiedd task aiming to extract features that are capable of attribution to a specific model. Crucially, other generators are unseen during development, reflecting open- open-world setting (Laszkiewicz al. 2024).
evaluate
The results using the above-described setup are Table While our approach works overall, note is not without limitations. In particular, we observe failure cases on DDGAN and DDIM samples, where model-blind baseline may achieve higher conditional accuracy. However, referencing class- balanced accuracy in Table 7 the predictions of the model-blind classifiers are, overall, close to random guesses. We therefore caution that conditional accuracy is not sufficient to judge performance.
Actually, our analysis here leads us to question the viability of black- box methods for the task. Ar- guably, sufficiently capable generative models should produce samples perceptually indistinguishable from real data, motivating to explore model-internal signals. Despite this, we note that there are recent works claiming promising results even the black- box setting, by exploiting representations from foundational models (Liu et al. 2025). However, deem such approaches to be method- ologically impure and outside our scope as their performance may be attributed to the underlying pretraining. In particular, since foundation model development details are often opaque, it becomes difficult to assess effectiveness when a method's assumptions are unclear or otherwise unbounded.
It is also important to clarify that, while we are certainly not the first to explore the MA task in the white-box setting (Wang et al. [2023 Laszkiewicz al. 2024), our knowledge, we are the first to demonstrate white-box MA directly on diffusion. We stress that the above-mentionee works rely on reconstruction/ inversion techniques that have only been shown to work indirectly on latent or distilled models (Rombach et al.) 2022 [Song et al., (023). Specifically for the former, attribution is performed on autoencoders, saying nothing about the underlying diffusion model. Moreover, approaches based on reconstruction error may be fundamentally doomed in DDPMs, since perfect reconstruction is theoretically always possible, regardless of the sample, For example, see the inversion in [Song et al. (202 and also of affHHueemannSSpaa Spiccelglass al. 2024). which is applicable stochastic processes.

---

Table 7: Performance of DDPM MA. We show per-model and average (class- accuracy.
During development, classifiers only see DDPM- -ePM--eeeatted samples and non-belonging, real samples.
CIFAR-10
CelebA-HQ 256
Method
DDPM DDIM DDGAN WDif Avg DDPM DDIM DDGAN WDiff Avg
Naive (model-blind)
60.8
39.1
41.9
44.0 51.2
88.3
20.2
22.6
27.0 55.8
Our method
time-steps steps
C. VL.
0, 1, .....,,......................... 999 V
75.5
57.6
36.2
64.4 64.1
98.9
6.3
59.6
57.8
70.0
0,3, ................................ 999 V V
86.0
72.4
18.1
76.1
70.8
100.0
7.9
74.2
76.9 76.5
0, ........ 999 V
87.2
86.8
13.1
91.7 75.5
100.0
3.5
86.8
68.9 76.5
7
Discussion
We conclude with discussion of our study's limitations and with recommendations for future research based on our findings. We have taken first step in addressing general origin attribution. However, we believe that the task is far from solved, and we hope our work inspires the community make further advancements toward more practical and robust data forensics tools.
7.1 Limitations
Our attacks' times are comparable to diffusion inference, making our approach significantly slower compared to existing membership inference attacks. However, as we concerned with real-time application, this is not unworkable. More importantly, in our model attribution experiments of Section 6.2 observed failure
that require further investigation. In general, developed data provenance tools and evaluation should be made more robust, especially since our benchmarking was with controlled external sets. Other important aspects that warrant futher exploration include robustness to intraclass shifts, common and adversarial data corruptions as well as our approach's s generalization ability and extensions large, foundational generative systems.
7.2 Recommendations
Revisit the threat models As we have argued in Section 3 the traditional surrogate approach to membership inference (Shokri al. 2017) is viable for modern generative systems. Beyond the issues relating to computational complexity. the core concern is the implicit and opaque assumptions that this framework entails, which not usually met in practice. Granted, also makes strong assumptions regarding data availability. However, we argue that ours are more practical, better defined and quantified. In the same spirit, for the sake of methodological purity, also recommend against the use of foundational or otherwise pretrained models in the context of resource constrained and sensitive applications such model attribution.
proposed threat model
Embrace distribution shifts It is important to contextualize metrics with suitable baselines. As also shown in Section 4.2 distribution shifts may lift such baselines from random guessing to seemingly high-performing solutions (Das et al., 2025). While there is recent work that explores data sanitization mitigate these shifts (Dubiñski al. 2024), this approach risks discarding a still valuable, potentially informative part of the data and may end up crippling applicability. We therefore
argue for an alternative, simpler protocol, where distribution shifts are embraced and performance must be judged relative to appropriate, naive baselines that are developed under a fair setting.
Focus on data extraction Ultimately, as also discussed in Section 4 our position is that MIAs in the literature cannot prove membership, since they do not bound the FPRs. Specifically, under our stated assumptions in Section [hhng al. (2025) argue that membership inference is only useful as subcomponent of training data extraction attack (Carlini et al.) 2023). Indeed, while we have also performed an evaluation via standard AUC, TPR FPR and ASR metrics, we believe that it is our experiment in Figure that that is most insightful about real privacy and security risks. Going forward, we therefore advocate the membership inference community to forgo the conventional benchmarks in favor of more challenging and ambitious training data extraction task.

---

Acknowledgments and Disclosure of Funding
Andreas acknowledges support from NeurIPS Scholar Award. Seyed contributed to this work while at Imperial and remained involved after joining Apple. The authors declare no competing interests.
References
Anderson, B. D. (1982). Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-326.
Carlini, N., Chien, S., Nasr, M.. Song, S.. Terzis, A., and Tramèr, (2022). Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897-1914.
Carlini, N., Hayes, Nasr, M., Jagielski, M., Sehwag, V., Tramèr, F., Balle, B., Ippolito. D., and Wallace, E. (2023). Extracting training data from diffusion models. In Proceedings of the 32nd USENIX Conference on Security Symposium, SEC 233 USA. USENIX Association.
Das, D., Zhang. J., and Tramèr, F. (2025). Blind baselines beat membership inference attacks for foundation models. In ICLR 2025 Workshop on Navigating and Addressing Data Problems for Foundation Models.
Duan, J., Kong. F., Wang, S., Shi, X., and Xu, K. (2023). Are diffusion models vulnerable to membership inference attacks? In Proceedings of the 40th International Conference Machine Learning. ICML'23. JoL..
Dubiáski, J., Kowalezuk, A., Pawlak, S., Rokita, P. Trzciński, T., and Morawiccki, P. (2024). Towards more realistic membership inference attacks large diffusion models. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 4860-4869.
Efron, B. (2011). Tweedic's formula and selection bias. Journal of the American Statistical Association, 106(496):1602-1611 1602-1614.
Gu, X., Du, C., Pang, T., Ii, C.,, Lin, M., and Wang, Y. (2025). On memorization in diffusion models. Transactions Machine Learning Research.
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Ho, Jain, A., and Abbeel, (2020). Denoising diffusion probabilistic models. Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems, volume 33, pages 6840-6851. Curran Associates, Inc.
Huberman Huberman-Spiegelllla, I., Kulikov, V., and Michaeli, T. (2024). edit friendly DDPM noise space: Inversion and manipulations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12478-
Karras, T., Aila, T., Laine, S., and Lehtinen, (2018a). Progressive growing of GANs for improved quality, stability, and variation. International Conference on Learning Representations.
Karras, T., Aittala, M., Lehtinen, J., Hellsten, Aila, T., and Laine, S. (2024). Analyzing and improving the training dynamics diffusion models. In Proc. CVPR.
Karras, T., Laine, S., and Aila, T. (2018b). A style-based generator architecture for generative adversarial networks. 2019 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 4396- 4405.
Kong, F., Duan, J., Ma, R., Shen, H. T., Shi, X., Zhu, X., and Xu, K. (2024). An efficient membership inference attack for the diffusion model by proximal initialization. In The Twelfth International Conference on Learning Representations.
Krizhevsky, A. (2009). Learning multiple layers of features from tiny images.
Laszkiewicz. M., Ricker, Lederer. J., and Fischer, A. (2024), Single-model attribution generative models through linel-layer inversion. In Salakhutdinov, R., Kolter, Z., Heller, K.. Weller, A., Oliver, N., Scarlett, J., and Berkenkamp. F., editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 26007-26042. PMIR.
Liu, F. Luo, H., Li, Y., Torr, P., and Gu, J. (2025). Which model generated this image? a model-agnostic approach for origin attribution. In Leonardis. A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., and Varol, G., editors, Computer Vision ECCV 2024, pages 282-301. Cham. Springer Nature Switzerland.
10

---

Matsumoto, T., Miura, T., and Yanai, N. (2023). Membership inference attacks against diffusion models. In 2023 IEEE Security and Privacy Workshops (SPW), pages 77-83.
Pang. Y., Wang. T., Kang, X., Huai, M., and Zhang, Y. (2025). White-hox membership inference attacks against diffusion models. Proc. Priv. Enhancing Technol., 2025(2):398 415.
Phung, H., Dao, Q. and Tran, A. (2023). Wavelet diffusion models are fast and scalable image generators. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 10199 10208.
Pizzi, E.,, Roy, S. D.. Ravindra, S.N., Goyal, P., and Douze, M. (2022). A self-supervised descriptor for image copy detection. 2022 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 14512-14522.
Recht, B., Roelofs, R., Schmidt, L., Shankar, (2018). cifar- 10 classifiers generalize to cifar-10?
Rombach, R., Blattmann, A... Lorenz, D., Esser, P., and Ommer. B. (2022). High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684-10695.
Shokri, R., Stronati, M., Song, C., and Shmatikov, (2017). Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pages 3-18, Los Alamitos, CA, USA. IEEE Computer Society.
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., and Goldstein, T. (2023). Diffusion art or digital forgery? investigating data replication in diffusion models. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages
Song, J., Meng, C., and Ermon, S. (2021a). Denoising diffusion implicit models. In International Conference on Learning Representations.
Song. Y.. Dhariwal, P., Chen, M., and Sutskever, I. (2023), Consistency models. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org.
Song. Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. (2021b). Score-based generative modeling through stochastic differential equations. In International Conference Learning Representations.
Tang, S., Wu, S., Aydore. S., Kearns, M., and Roth, A. (2024). Membership inference attacks diffusion models via quantile regression. In Forty-first International Conference Machine Learning.
Torralba, A., Fergus, R., and Freeman, W. T. (2008). 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions Pattern Analysis and Machine Intelligence, 30(11)11958- 1970.
Wang. Z., Chen, C., Zeng, Y., Lyu, L.,, and Ma, S. (2023). Where did come from? origin attribution of Al-generated images. Seirty- Conference on Neural Information Processing Syxtems.
Xiao, Z.. Kreis, K.. and Vahdat, A. (2022). Tackling the generative learning trilemma with denoising diffusion gans. In International Conference on Learning Representations.
Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. (2018). Privacy risk in machine learning: Analyzing connection to overfitting. 2018 JEEE 31st Computer Security Foundations Symposium (CSF), pages 268 282, Los Alamitos, CA, USA. IEEE Computer Society.
Zhai, S., Chen, H. Dong, Y., Li, J., Shen, Q., Gao, Y.. Su, Н., and Liu, Y. (2024). Membership inference on text-to-image diffusion models via conditional likelihood discrepancy. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
Zhang. J., Das, D., Kamath, G., and Tramèr, F, (2025). Position: Membership inference attacks cannot prove that model was trained on your data, In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 333-345.
11

---

Table 8: Ablation study the choice of features for our MIAs. We t 0, 1, 999.
CIFAR-II 10
CelebA-HQ 256
AUC TPR @ 1% FPR ASR
AUC TPR @ 1% FPR ASR
PIA L. Vali VoL.
73.3
6.5
68.1
99.1
95.4
96.4
74.9
6.1
68.5
99.3
97.5
97.3
77.0
3.1
71.6
100.0
99.5
93.7
80.0
12.2
72.4
99.7
99.2 98.9
80.5
10.5
72.8
100.0
99.9 98.4
81.8
11.1
74.1
100.0
100.0 98.9
83.3
16.8
74.8
100.0
100.0 99.5
69.5
5.2
64.2
99.2
89.4 95.5
75.6
1.0
68.7
88.1
51.3 80.7
71.7
3.3 64.5
97.1
40.2 92.8
75.3
5.2 68.6
99.3
89.6 95.7
72.9
9.8 66.8
99.2
89.0 96.3
75.6
1.6 68.6
97.5
46.0
93.3
76.3
6.1
69.1
99.3
90.0
96.4
A Experimental setup
Sampling and feature extraction from the generative models was conducted on a Linux cluster with Quadro RTX 6000 GPUs. We then developed our framework on Windows laptop with GTX 1650 GPU. All demonstrations are on off-the-shelf DDPMs, according to Ho et al. 2020). By default, we budget 1000 seajectt-bllaneed samples per class for classifier development and evaluate on separate, similarly chosen datasets. CIFAR experiments use external sets (CIFAR-10.1/IO) val) with minimal interclass shifts, whereas CelebA-HQ experiments use FFHQ as the external set with notable shifts, After feature normalization to zero mean and unit variance based on training data statistics, we fix the hyperparameters in Table to optimize (6) via a cross-entropy loss. Hyperparameters are optimal (TPR @ 1% FPR on CIFAR-10) for Pang et al. (2025) and we did not perform search beyond that.
B The choice of features
We experiment with different features for modeling diffusion trajectories. From Ta- (8 feature combination yields the best results. When integrating PIA (Kong al., 2024), we replace E ~ No I) in L, with c0 0), making the features determinis- tic. However, we see that these features not performant counterparts. As a further exploration, we investigate whether trajectories reveal the object-class of samples CIFAR-10 in Table i.e., standard classification. Interestingly, this approach is much better than random guessing (~~ 10%).
Table 9: CIFAR- 10 classification with trajectory features.
time-steps
Lt
VeCi
VeLt
Accuracy
0.
999
26.5
999
31.4
their stochastic
0, 999
31.6
C Broader impact
The methods presented in this paper are developed ensure accountability and transparency in scenarios involving open-weight models with undisclosed training data. By enabling data owners determine whether their data was used in training model, these techniques support ethical development, protect intellectual property rights and enhance privacy by identifying memorization. Furthermore, they allow for the attribution of harmful or malicious content to specific models, contributing to greater safety and fostering trust in these technologies. The methods also present potential risks, such as enabling the deanonymization of data, exploiting privacy vulnerabilities or unfairly penalizing applications due to inaccurate attributions.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We study membership inference, model attribution tasks in diffusion models and propose to unify them for a more robust evaluation. We contribute a principled frame- work for data provenance, explore patterns in diffusion trajectories for this purpose and uncover flaws of existing approaches.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairess..
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
13

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
14

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
See supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https::/niis.cc)
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips. //wpps.ccppubii.. http:pptwpppdpe/////pppppp
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
15

---

7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: See Table 4
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative етгог rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
16

---

9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http://s///aaapp...cp/a//aahhaaa
Answer: [Yes]
Justification:
The research conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that there is societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
17

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All existing assets are properly cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
thcore
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
18

---

13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
See supplemental material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/cooe//model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
19

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy buctps://neure wwwparippps..coooeeeeeeeee for what should or should be described.
20


Reviews:
[{"review_id": "qSyeM556Gi", "reviewer": "Reviewer_HK2L", "summary": "The paper introduced a method for “Origin Attribution”, the goal of determining an image’s origin as one of (a) the training set, (b) generate by the model, (c) other/external. This goal is a generalization of two tasks: Membership Inference Attacks and Model Attribution. The introduced methods consists of exacting features from a given image that consist of a DDPM model gradients for that images over many time steps, and then training a linear probe on top of these features. The authors further study diffusion model dynamics, such as by exploring which time steps lead to features that are most valuable for MIAs.", "questions": "Can the authors address the issues mentioned in the \"Weaknesses\" section? I would be happy to raise the score if these weaknesses were sufficiently addressed.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n+ The general task of OA makes sense, though it would benefit from further motivation.\n+ The proposed method is easy to understand\n+ The results for MIA compare favorable to other works.\n+ I particularly enjoyed the motivation of the analysis for the “goldilocks zone” hypothesis and the insights drawn from it.\n\nWeaknesses\n- The paper has no related works section. While some related works are mentioned throughout (especially in intro and discussion), it would be highly valuable to have a standalone related works section that can provide further context. I’m sure the authors are concerned about page limit, so it should be ok (though probably not ideal) to put related works in the Appendix.\n- The authors mention: \"We address shortcomings of Wang et al. (2023); Laszkiewicz et al. (2024) which rely on model or layer inversion that are not easily adapted for diffusion. To our knowledge, we are first to demonstrate MA on DDPMs without assistance from pre-trained foundation models.” I have a few issues with this claim. First, both of these method are applying to diffusion models, so it seems their approach is adaptable to diffusion models (even if it not as natural) - for this the authors can compare to these works to show their approaches’ advantage. Second, based on my understanding, Laszkiewicz et al. (2024) does not use a pre-training foundation model as part of the underlying method (but feel free to correct me). Third, even if all other methods to use assistant from pre-training foundation models, it is not clear to me why this is an issue if these method work, since images foundation models will likely always be available.\n- The “robustness” evaluations of other MIAs seem somewhat unfair. The point the authors make is that these methods would categorize data generated by a model as part of their training datasets (as opposed to external). However, this is just using these methods for a purpose they are not meant to be used for. Also, in the case that the diffusion models “memorizes” a training datapoint and regurgitates it during inference, it feels closer to being a member than external.", "was_revised": true, "final_justification": "Thank you to the authors for their response. I raised the score.", "rebuttal": "Thank you for the review. We are pleased that you found our method easy to understand and that you enjoyed reading our motivation and analysis challenging the Goldilocks zone hypothesis. Please find answers to your questions below:\n\n1 Related work\n-\n\n**We have discussed related work in the area of data forensics in Section 2.2 (starting at line 87)**.\n\nFor Membership Inference Attacks (MIAs) the majority of the literature is on thresholding approaches and we focus on a subset of representative works (as we argue in the paper, thresholding is suboptimal for our purposes).\n\nFor Model Attribution (MA) we have also reviewed related work in the same section but, to our knowledge, no existing work has explored the task in diffusion models (please also see our point 2. below).\n\nOrigin attribution is a framework that we introduce so again, to our knowledge, there are no existing works.\n\n**We are open to an extended discussion on related works in the Appendix, as you suggest. If there are certain works that you feel we should discuss in detail, please do let us know and we will update the paper with them.**\n\n2 Existing methods and use of foundation models\n-\n\nWe are not claiming that [1], [2] rely on foundation models, these are two separate sentences. To avoid confusion, we will rephrase in the final version of the paper. We clarify below:\n\n**Applicability to diffusion** To our knowledge, there are no works in the literature that have demonstrated or explored white-box MA directly on diffusion models. As we have also stressed in the paper, **existing methods (such as the above-mentioned) only work for latent diffusion models where attribution is only performed on the VAEs and not on the underlying diffusion models**. For example, such approaches completely fail if two latent diffusion models share the VAE (as was done with early versions of Stable Diffusion). Importantly, both of **the above-mentioned works are based on the ideas of model/layer inversion. Such inversion/reconstruction-based methods are fundamentally doomed since zero reconstruction loss is always possible in diffusion [3] (see also interpretation of diffusion as a flow model).**\n\n**Regarding the use of foundation models**: **In a security and privacy-critical setting, such as that of data forensics, we care about making predictions based on a minimal set of assumptions**. Foundation models are trained on vast amounts of potentially undisclosed data and there is a risk of data leakage that leads to inflated metrics. We repeat the example we give in the paper regarding [4], which uses CLIP for black-box MA on Stable Diffusion (SD): SD was trained on a filtered version of LAION, where only images with a high CLIP score were included. This means that there are non-trivial implicit dependencies between SD and CLIP. Now, [4] uses CLIP to perform MA on SD, claiming that \"only a few images are available\", but CLIP's **foundational pre-training being aligned with the diffusion model is an implicit and significant assumption. In contrast, we do not rely on foundation models and make our assumptions explicit in the form of a limited data budget. It is in this sense that we claim that our appraoch is more reliable and our benchmarking more fair.**\n\n3 Robustness of MIAs\n-\n\nWe respectfully disagree with the comment that MIAs are not meant to distinguish between members and belonging samples. As motivated in the paper, the purpose of data forensics such as MIAs is to empower data owners to hold model developers accountable. For example, an artist wants to make sure that their work was not used to train the model. If an artist's work is used for this purpose, **memorized samples should not be confused with \"original\" creations by the model. Instead, memorization should be flagged as a data rights violation. Conversely, there is an element of originality in diffusion models and it is therefore not accurate to claim that all generated samples are copies of the training data. In this sense, our experiments on robustness to belonging data reveal a critical flaw of existing MIA frameworks that limits their applicability to the real world (e.g. searching for memorization or auditing belonging data).**\n\n---\n\nWe hope our responses have addressed your concerns and we remain open to further discussion if there are any further clarifications needed. Please consider raising your score if you find the above answers sufficient.\n\n\nReferences\n-\n\n[1] Wang Z., et al., 2023, Where Did I Come From? Origin Attribution of AI Generated Images\n\n[2] Laszkiewicz et al., 2024, Single-Model Attribution of Generative Models Through Final-Layer Inversion\n\n[3] Huberman-Spiegelglas et al., 2024, An Edit Friendly DDPM Noise Space: Inversion and Manipulations\n\n[4] Liu et al., 2025, Which Model Generated This Image? A Model-Agnostic Approach for Origin Attribution"}, {"review_id": "Tm4vGtxRFl", "reviewer": "Reviewer_13uc", "summary": "The authors propose a method to determine membership and/or attribution of images to diffusion models. The method extracts features from the entire diffusion trajectory, including loss values, input gradients, and parameter gradients across all time steps, then trains linear classifiers on these compressed representations. This “trajectory feature” departs from the popular “Goldilocks-zone” belief that only mid-range timesteps are useful.\n\nWith white-box access and <3.4 % of the training data, the method (i) outperforms prior loss-based and gradient-based MIAs on CIFAR-10 and CelebA-HQ, (ii) stays robust when external data are distribution-shifted, and (iii) can attribute samples from other generators.\n\nThe demonstration of the method is limited to small datasets like CIFAR and CelebA.", "questions": "Please look at the weaknesses section above.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1. First of all, this paper does a great job of showcasing that current threshold based MIAs have way too many false positives, and become almost meaningless under distribution shift.\n2. The method is able to solve two problems of membership inference and model attribution under the same hood.\n\n## Weaknesses\n1.  Experiments are limited to CIFAR-10 and CelebA-HQ-256; no ImageNet-level, text-conditioned, or latent-diffusion models are studied, so real-world generality is unclear.\n2. Per-image gradients at hundreds of timesteps are costly. What would the feasibility of this method look like?\n3. On CelebA-HQ, attribution accuracy against DDIM is only ~7 % when gradients are used, suggesting poor generalisation to some generators. More analysis of failure cases would be welcome.\n4. 3.4% of data is actually quite sizeable as we scale to datasets such as LAION. How would you consider scaling this to larger datasets, and different architectures?\n5. I do worry about the generalization of the method to OOD members, or when none of the pool of the çandidate member models is not the one that actually generated the data,. This is another class of false positives,", "was_revised": true, "final_justification": "I was concerned about the scalability of the method, among other concerns. This concern has been alleviated with some of the new experiments added by the authors.", "rebuttal": "Thank you for the review. We are pleased you appreciated our analysis on pitfalls of existing approaches and that you recognize the value of unifying the various tasks under a single framework. We address your concerns below.\n\n1 Generality of method and larger scale experiments\n-\n\n**To our knowledge, no prior work has rigorously evaluated membership inference at scales beyond what we have tested**: results are often reported on subsets of datasets. However, in our experiments the diffusion models are trained on the full datasets. For example, Table 3 in [1] (our stongest competitor) shows that their CIFAR-10 DDPM has 8k samples whereas we attack a DDPM trained on the full 50k sample CIFAR-10.\n\nWe are running more experiments (DiT, ImageNet). Due to resource and time constraints, it is unlikely that we will be able to complete these during rebuttal. However, we include benchmarking of our method and strongest competitor, GSA [1], on CIFAR-100. Specifically, we train the diffusion model on 50k samples of 100 different objects and set the data budget to 2% (1k). The external data is chosen from the 10k samples of the validation set of CIFAR-100, so we don't expect to see any distribution shifts. Again, **to our knowledge, our work demonstrates the largest scale MIAs, thoroughly evaluted under a controlled experimental setting with explicit and limited assumptions**:\n\n| CIFAR-100 MIA @ 2% budget | AUC        | TPR @ 1% FPR | ASR     |\n|---------------------------|------------|--------------|---------|\n| Matsumoto et al. [2]      | 58.9       | 1.4          | 56.7    |\n| Pang et al. (GSA) [1]      | 75.7       | 2.7          | 69.8    |\n| **Our method**            | **79.5**   | **10.8**     | **71.6**|\n\n\nWe have also benchmarked performance in the case of object imbalance, simulating a realistic scenario where one is dealt a member set for training the classifier that is not representative of the member distribution. **In all of these experiments, trajectory features outperform the strongest competitor by a large margin** (see also Table 10 and Appendix D for further details and motivation regarding this setting):\n\n| CIFAR-100, TPRs @ 1%FPR | Our method     | GSA        |\n|-------------------------|----------------|------------|\n| Split 1                 | **7.2**        | 3.4        |\n| Split 2                 | **6.1**        | 2.4        |\n| Split 3                 | **7.9**        | 2.5        |\n| Split 4                 | **6.2**        | 3.6        |\n| Split 5                 | **11.4**       | 6.2        |\n| Split 6                 | **5.5**        | 2.1        |\n| Split 7                 | **6.4**        | 3.2        |\n| Split 8                 | **7.1**        | 4.6        |\n| mean ± std              | **7.2 ± 1.85** | 3.5 ± 1.35 |\n\nNote, even in these experiments, which are relatively small scale compared to something like LAION, it is clear that the data forensics tasks are far from solved and still remain highly non-trivial. **Given that even at these scales robust data forensics remains a largely unsolved problem, our work demonstrates significant progress in the field that should not be discounted**.\n\n**Comments on diffusion variants** As mentioned in the paper, our focus on standard diffusion was intentional: it is the most widely studied and reproducible baseline. Extending to latent/conditional models introduces confounding variables (e.g., VAEs, text encoders), which should be carefully treated to ensure a clean ablation and proper benchmarking. As standard diffusion remains at the core of these more advanced constructions, we believe our analysis and insights remain highly relevant.\n\n2 Feasibility of method / feature extraction\n-\n\n**We provide discussion on complexity of our method in Appendix C of the paper**. We elaborate in more detail below:\n\nAs we acknowledge in our limitations, our trajectory features are more expensive than simple thresholding. Despite this, features can be extracted in a reasonable amount of time, even with a naive sequential implementation that iterates over each sample and time-step one at a time (e.g. ~29s to sequentially extract all features for a single image on an L40S GPU). **To accelerate inference, it is also possible to parallelize feature extraction, as we hint in Algorithm 1 of the paper**. Specifically, **PyTorch supports vectorizing per-sample gradients (e.g. with vmap) and one can parallelize over both time-steps and samples**. This is not something that we have explored in detail because inference time is not a priority. Ultimately, our position is that **real-time application is not critical in the tasks we consider and robustness is a far more important metric: one would rather correctly infer membership in 30s than make a false accusation instantly**. As we show in the paper, existing methods are brittle in practical scenarios and the robustness gains of our method more than make up for the overhead.\n\nAs a side note, one can subsample the trajectories (shown in Tables 5,6,7 and Figure 7), to explore a performance-time trade-off.\n\n3 Model Attribution (MA) generalization\n-\n\n**Conditional accuracy does not reflect a classifier's performance**. It is only meaningful to judge performance over all classes and data. For example, an ImageNet classifier that outputs a constant prediction might score perfectly when conditioned on a specific class but will score very poorly overall. Despite fluctuations in conditional accuracy, observe that **our MA with gradients consistently achieves the best overall accuracy. One can not draw conclusions about generalization from conditional accuracy**.\n\n4 Assumptions and scaling to larger datasets\n-\n\n**Our methodology is more scalable and practical than existing works (e.g. [1] [2], [3]) since we make far milder assumptions in our setup**. For example, as we mention in the paper, **in MIAs it is common to train shadow models to mimic the target DDPM, i.e., assume knowledge of training details of the diffusion models and knowledge of the member distribution to replicate the training.** In contrast, **we avoid this** and make our assumptions explicit in the form of a limited data budget, which is much more realistic and plausible. Of course, one would rather perform robust MIAs without assumptions but, to our knowledge, no such method is known (and perhaps this might be theoretically impossible).\n\n**Scaling to LAION?** As explained in point 1. of our answers, it is our position that robust data forensics, even in relatively small scale settings, is a largely unsolved problem. Again, **to our knowledge, there is no existing method that has been demonstrated to reliably operate at scales beyond what we have tested. We repeat that a rigorous data forensics analysis is lacking in the literature even in relative small scale settings such as the full CIFAR-10 dataset** (again, it is standard in the literature to consider diffusion models trained on a small subset of the data, see Table 4 of [1]).\n\n**As highlighted by recent works, [3], [4], the community is still far from reliably tackling data forensics at the scale of foundation models.**\n\n**That being said, when dealing massive datasets, a reasonable assumption is that we only care about a small subset at a time**. For example, an artist wishes to perform data forensics on a model trained on vast amounts of data but it is only a small percentage of that data that is of interest to them (e.g. art in their style or related data points). In such cases, one may condition on this subset to perform data forensics.\n\n5 Some more comments on generalization\n-\n\nWe are not sure what you mean by your last question 5 listed under weaknesses. It seems you are asking two separate things.\n\n**OOD members in MIAs** Consider a scenario where the diffusion model is trained on a diverse set of images (e.g. 10 or 100 object classes like CIFAR-10/100) and where the available members for developing our MIAs are limited and may contain class imbalances. For example, focusing on CIFAR, maybe we are dealt a batch of data that is mostly dogs and cats but there are no trucks or airplanes. If we understood your question correctly, you are concerned about such settings where the MIA makes prediction about potentially unseen or under-represented objects. **We have considered such a setting in our CIFAR-10 experiments, shown in Table 10 of Appendix D. We have also included results on CIFAR-100 in our rebuttal point 1. In all these experiments, we outperform the strongest competitor by a large margin.**\n\n**What happens when none of the models is responsible for the data?** We assume you're asking about MA of unseen non-belonging data. Note, in Table 6 our methods have only been trained on a budget of belonging samples (generated by DDPM, always available because white-box access is assumed). Samples from the other models are unseen during training of the classifiers. **Table 6 shows that, on average, our MA classifies generated data from other models (all unseen during training) correctly as non-belonging.**\n\n---\n\nWe hope our responses have addressed your concerns and that our additional experiments are convicing of the superiority and robustness of our approach compared to existing methods. Let us known if further clarifications are needed. Please consider raising your score if you find the above answers sufficient.\n\nReferences\n-\n\n[1] Pang Y., et al., 2024, White-box Membership Inference Attacks against Diffusion Models\n\n[2] Matsumoto et al., 2023, Membership Inference Attacks against Diffusion Models\n\n[3] Kong F., et al., 2024, An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization\n\n[4] Zhang et al., 2025 Position: Membership Inference Attacks Cannot Prove That a Model was Trained on Your Data\n\n[5] Das et al., 2025 Blind Baselines Beat Membership Inference Attacks for Foundation Models"}, {"review_id": "4x7CEuLNGz", "reviewer": "Reviewer_n36r", "summary": "This paper addresses the task for probing data provenance using the whole diffusion trajectories. Specifically, it presents one framework that performs membership inference and model attribution.\nImportantly, the paper finds that the “Godlikocks zone” assumption in previous works was not correct. Furthermore, the works explores the effect of distribution shifts on the model performance. \nThe proposed approach clearly outperforms previous works on various metrics for the considered tasks.", "questions": "- Why are foundational features only a problem? It is biased but it might also enable a better performance with a smaller data budget.\n- Why is data with high CLIP score a problem?\n- Why was the Goldilocks zone adopted? It would be helpful to rephrase the learnings and frame why this might be different to the proposed setting, in what regard the previous learnings were wrong.\n- How would the model perform with more / less data budget? Why 3.4%? A sensitivity analysis of this for more experiments would be helpful to better evaluate this.\n- What exactly is the model-blind classifier? Please refer or provide more details about it.\n- Why only a linear classifier? How would a non-linear model perform?\n- Why only such simple projection? Explain why this is reasonable and why more complex projection are not considered!\n- What are similarly chosen datasets (l.178)?", "limitations": "Yes, the limitations are discussed in the main paper.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths \n\nThe paper’s motivation is very clear and tackling a very significant topic considering the current developments of diffusion models.\nIn general, the writing is very clear with the paper having a well-organized structure, where hypotheses are followed by a through experimental analysis. The learnings are subsequently taking into account resulting in clear improvements compared to previous works. \nThe work does not stick to previous common strategies (Godlikocks zone, no hyperparameter optimization on test set) but instead questions and corrects those.\nThe work motivates very clean experimental setup removing confounder factors, which increases the confidence of the paper’s learnings.\nThe model performs well on the considered tasks.\n\nWeaknesses\n\nThe access to a few training samples is a strong assumption. The work would benefit from a better sensitivity analysis with respect to the data budget for all performed experiments.\nA lower data budget (than 3.4%) would reduce the overall accuracy values, potentially showing clearer gaps for some metrics (e.g. in Tab 5 and 6).\nWhile the work also mentions this as a weakness, it would be more impactful and helpful if the experiments were performed for larger models that are more commonly used.", "was_revised": true, "final_justification": "I support this work for acceptance since it addresses an important problem in the domain of generative models, it points out incomplete assumptions in previous works and demonstrates its effectiveness in solid experiments.", "rebuttal": "Thank you for the review. We are pleased that you found our setup and presentation clear. We address your questions below:\n\n1, 2 Why not features from foundation models / CLIP?\n-\n\n(We repeat part of our answer to HK2L, who asked a similar question) **In a security and privacy-critical setting, such as that of data forensics, we care about making predictions based on a minimal set of assumptions**. Foundation models are trained on vast amounts of potentially undisclosed data and there is a risk of data leakage that leads to inflated metrics. In the paper, we give the following example regarding [1], which uses CLIP for black-box Model Attribution (MA) on Stable Diffusion (SD): SD was trained on a filtered version of LAION, where only images with a high CLIP score were included. This means that there are non-trivial implicit dependencies between SD and CLIP. Now, [1] uses CLIP to perform MA on SD, claiming that \"only a few images are available\", but **CLIP's foundational pre-training being aligned with the diffusion model is an implicit and significant assumption. In contrast, we do not rely on foundation models and make our assumptions explicit in the form of a limited data budget. It is in this sense that we claim that our appraoch is more reliable and our benchmarking more fair.**\n\n3 Comments on Goldilocks zone hypothesis\n-\n\n**We are happy to further elaborate on this in the final version of the paper.** We include some comments here: Traditionally, in membership inference of discriminative models, a common approach is to infer membership based on thresholding the model's training loss. The Goldilocks zone hypothesis is stated in [2], where the thresholding idea is extended to diffusion models. However, since the true diffusion loss is averaged over all time-steps, for the sake of efficiency, [2] focuses on a single representative time-step. Since then, this approach has, to a large extent, become standard in diffusion model forensics. In contrast, **we show that there is rich information that is encoded in the trajectories that is discarded when performing simple thresholding. Our core argument, which we support with experiments, is that simple thresholding, while fast, is brittle and cannot be trusted in the security and privacy-critical scenarios that we care about in data forensics.**\n\n4 Effect of the data budget\n-\n\nPlease note that **we do have experiments on the effect of the data budget in Figure 5**: we plot performance as a function of the data budget percentage. Note, we consistently outperform the competitors even as the data availability varies. In our experiments, we limit the budget to at most 1000 member samples since we are interested in data forensics under minimal assumptions. For CelebA-HQ this amounts to 3.4%, hence that specific percentage.\n\n5 Model-blind classifier\n-\n\nAs mentioned in the paper, **the model-blind classifier is a neural network that operates directly on raw images, without access to the diffusion models and their features**. In our implementation, **we adopt a standard ResNet18 architecture for this**. The purpose of the model-blind baseline is to quantify the effect of distribution shifts, i.e., the \"hardness\" of the classification task, and to establish a lower-bound on the performance we would expect from a more sophisticated classifier that also takes into account the model. As you can see from the results **in Table 5, existing MIAs break down in the presence of distribution shifts to the point that this naive baseline outperforms them: this is one of our main arguments against simple thresholding**.\n\n6, 7 Why a linear model? Why not a more complex network?\n-\n\nWe again stress that the aim of data forensics is to make predictions based on a minimal set of assumptions. **In this context, where we assume a limited number of data points for training classifiers, we use a linear model to avoid overfitting**. A linear projection is a simple and objective way to assess the quality of extracted features (see, for example, the vast literature on unsupervised learning). More complex models might require extensive hyperparameter tuning and engineering, which is undesirable. Besides, we note that the simplicity of the linear model allows us to gain some insight on how features are weighted (see Figure 4) thus making the overall pipeline more interpretable.\n\n8 Similarly chosen datasets\n-\n\nBy this we mean that the evaluation set consists of data that has the same distribution and number of samples as the classifier's training set. We have included a detailed explanation of our experimental setup, including the choice of datasets in Appendix A.\n\nA comment on our assumptions and experiments at scale\n-\n\nHere we address comments made in the weakness section of your review. (We repeat part of our answer to 13uc, who had a similar question)\n\n**Assumptions** We claim that **our methodology is more scalable and practical than existing works (e.g. [3] [4]) since we make far milder assumptions in our setup**. For example, as we also mentioned in the paper, **in MIAs it is common to train shadow models to mimic the target DDPM, i.e., assume knowledge of training details of the diffusion models and knowledge of the member distribution to replicate the training**. In contrast, **we avoid this entirely** and make our assumptions explicit in the form of a limited data budget, which is much more realistic and plausible. Of course, if possible, one would rather perform robust MIAs without any assumptions but, to our knowledge, no such method is known (and perhaps this might be theoretically impossible).\n\n**Experiments at larger scale**  To our knowledge, no prior work has rigorously evaluated membership inference at scales beyond what we have tested: results are often reported on subsets of larger datasets. However, in our experiments the diffusion models are trained on the full datasets instead, reflecting a more realistic scenario. For example, Table 3 in [4] shows that their CIFAR-10 DDPM has 8k samples whereas we attack a DDPM trained on the full 50k sample CIFAR-10 dataset.\n\n**As highlighted by recent works, [6], [7], the community is still far from reliably tackling data forensics at the scale of foundation models. Given that even at relatively small scales robust data forensics remains a largely unsolved problem, our work demonstrates significant progress in the field that should not be discounted**.\n\nStill, to further strengthen the paper, we are currently running more experiments. Due to resource and time constraints, it is unlikely that we will be able to complete them during rebuttal. However, we include benchmarking of our method and our strongest competitor, GSA [4], on the full CIFAR-100 dataset below. Specifically, we train the diffusion model on 50k samples of 100 different objects and set the data budget to 2% (1k samples). In this case the external data is chosen from the 10k samples of the validation set of CIFAR-100, so we don't expect to see any distribution shifts. Again, **to our knowledge, our work demonstrates the largest scale MIAs, thoroughly evaluted under a controlled experimental setting with explicit and limited assumptions**:\n\n| CIFAR-100 MIA @ 2% budget | AUC        | TPR @ 1% FPR | ASR     |\n|---------------------------|------------|--------------|---------|\n| Matsumoto et al. [5]      | 58.9       | 1.4          | 56.7    |\n| Pang et al. (GSA) [4]      | 75.7       | 2.7          | 69.8    |\n| **Our method**            | **79.5**   | **10.8**     | **71.6**|\n\n\nWe have also benchmarked the performance in the case of object imbalance, simulating a realistic scenario where one may be dealt a member set for training the classifier that is not representative of the member distribution. **In all of these experiments, trajectory features outperform the strongest competitor by a large margin** (please refer to Table 10 and Appendix D for further details and motivation regarding this setting):\n\n| CIFAR-100, TPRs @ 1%FPR | Our method     | GSA        |\n|-------------------------|----------------|------------|\n| Split 1                 | **7.2**        | 3.4        |\n| Split 2                 | **6.1**        | 2.4        |\n| Split 3                 | **7.9**        | 2.5        |\n| Split 4                 | **6.2**        | 3.6        |\n| Split 5                 | **11.4**       | 6.2        |\n| Split 6                 | **5.5**        | 2.1        |\n| Split 7                 | **6.4**        | 3.2        |\n| Split 8                 | **7.1**        | 4.6        |\n| mean ± std              | **7.2 ± 1.85** | 3.5 ± 1.35 |\n\n\n---\n\nWe hope that you found our answers satisfying and we believe that our additional experiments are convicing of the superiority and robustness of our approach compared to existing methods. We remain open to discussion if there are any further clarifications needed.\n\nReferences\n-\n\n[1] Liu et al., 2025, Which Model Generated This Image? A Model-Agnostic Approach for Origin Attribution\n\n[2] Carlini N., et al., 2023, Extracting Training Data from Diffusion Models\n\n[3] Kong F., et al., 2024, An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization\n\n[4] Pang Y., et al., 2024, White-box Membership Inference Attacks against Diffusion Models\n\n[5] Matsumoto et al., 2023, Membership Inference Attacks against Diffusion Models\n\n[6] Zhang et al., 2025 Position: Membership Inference Attacks Cannot Prove That a Model was Trained on Your Data\n\n[7] Das et al., 2025 Blind Baselines Beat Membership Inference Attacks for Foundation Models"}, {"review_id": "otoXRwbviD", "reviewer": "Reviewer_c42K", "summary": "This paper endeavours to ensure responsible use of diffusion models and propose to unify membership inference and model attribution into a single framework - origin attribution (OA). It does this by identifying the limitations in the existing membership inference methods that existing methods fail under distribution shifts or when model-generated data is present, then propose to use characteristics in diffusion trajectory in DDPM to classify data origins. Experiments are conducted on DDPMs.", "questions": "1. Can you provide additional clarification regarding the practical significance of the assumptions this paper make (i.e., the use cases) as mentioned in the weaknesses section?\n2. Would the proposed method works in more practical and widely used diffusion frameworks?\n3. Can you provide more insights and analyse of how characteristics of diffusion trajectories are leveraged?\n4. Can you show more experimental results to compare with current state-of-the-arts across all three tasks (MIA, MA, memorization) that this paper claims to tackle?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n1. This paper tackles important issues that contribute to responsible AI. Specifically, it spans over the field of membership inference, model attribution, and memorization in diffusion models.\n2. In addition, this paper works towards unifying the aforementioned field into a single, cohesive framework.\n3. It is motivated by the gap in MIA in the literature that existing methods often fail for model-generated synthetic data, and the gap in MA that there is a lack of a white-box MA approach.\n\nWeaknesses:\n\n1. It would be better if more explanations regarding how often the assumed application scenario for the proposed general problem of origin attribution (OA) can be observed. As the authors acknowledged, the assumption is that the proposed methods only work when (1) model owners have not disclosed the training setup and data details, and (2) a small fraction of the training samples is available. I am concerned about the practical significance of the latter assumption, as it is mostly the case that users do not have a clue about the training samples used, let alone having a small fraction of them. \n2. This paper delves into the privacy issue in the framework of DDPM. However, the most widely used diffusion models are often latent diffusion models, DiT models, flow-based models, etc. It would be interesting to see how the method performs in these, as their generations are more widespread, thus privacy protection is more practically significant.\n3. Despite the motivation of unifying MIA and MA being well-supported, I find the method section less clearly explained, especially the section regarding the core insight of how characteristics of diffusion trajectories are leveraged. More explanations and analyses need to be conducted to make it more convincing.\n4. For the MIA results, the paper only compares its performance with GSA to demonstrate effectiveness. For the other areas that this work also claims to tackle (MA and memorization), it is also suggested to provide more comprehensive comparisons to existing works. The experiments are quite insufficient in their current state.", "was_revised": true, "final_justification": "The authors have provided a detailed rebuttal containing additional clarifications and results. These have well-addressed my concerns, so I have raised my rating accordingly.", "rebuttal": "Thank you for your review. We are pleased that you appreciated our arguments for the unification of data forensics into a single framework. We address your concerns below:\n\n1 Assumptions\n-\n\n(We repeat part of our answer to 13uc, who asked a similar question) **Our methodology is more scalable and practical than existing works (e.g. [1] [2], [3]) since we make far milder assumptions in our setup**. For example, as we mention in the paper, **in MIAs it is common to train shadow models to mimic the target DDPM, i.e., assume knowledge of training details of the diffusion models and knowledge of the member distribution to replicate the training.** In contrast, **we avoid this** and make our assumptions explicit in the form of a limited data budget, which is much more realistic and plausible. Of course, one would rather perform robust MIAs without any assumptions but, to our knowledge, no such method is known (and perhaps this might be theoretically impossible).\n\n**When do the assumptions hold & use cases?** As explained above, knowledge of part of the member set is a weaker and more plausible assumption than being able to replicate diffusion model training. To give an concrete example, **our assumptions hold in the case of the Stable Diffusion where it is known that LAION was used as the member set. In contrast, shadow model training is out of the question at such a scale.**\n\nNote, **even if the training set is (partly) known, simply searching through it becomes very challenging at large scales so our framework could be useful even in this context.**\n\nFurthermore, while we have mostly motivated data forensics from an outsider's perspective, **model developers may also want to develop such forensic tools to ensure fair use of their systems and to comply with regulations** (assumptions trivially hold in this case).\n\n2 Demonstration in more diverse / larger settings\n-\n\nWe are currently running more experiments (DiT, ImageNet). Due to resource and time constraints, it is unlikely that we will be able to complete these during rebuttal. However, we include benchmarking of our method and our strongest competitor, GSA [1], on CIFAR-100. Specifically, we train the diffusion model (with the flow-matching framework) on 50k samples of 100 different objects and set the data budget to 2% (1k). The external data is chosen from the 10k samples of the validation set of CIFAR-100, so we don't expect to see any distribution shifts. Again, **to our knowledge, our work demonstrates the largest scale MIAs, thoroughly evaluted under a controlled experimental setting with explicit and limited assumptions**:\n\n| CIFAR-100 MIA @ 2% budget | AUC        | TPR @ 1% FPR | ASR     |\n|---------------------------|------------|--------------|---------|\n| Matsumoto et al. [2]      | 58.9       | 1.4          | 56.7    |\n| Pang et al. (GSA) [1]      | 75.7       | 2.7          | 69.8    |\n| **Our method**            | **79.5**   | **10.8**     | **71.6**|\n\n\nWe have also benchmarked performance in the case of object imbalance, simulating a realistic scenario where one is dealt a member set for training the classifier that is not representative of the member distribution. **In all of these experiments, trajectory features outperform the strongest competitor by a large margin** (see also Table 10 and Appendix D for further details and motivation regarding this setting):\n\n| CIFAR-100, TPRs @ 1%FPR | Our method     | GSA        |\n|-------------------------|----------------|------------|\n| Split 1                 | **7.2**        | 3.4        |\n| Split 2                 | **6.1**        | 2.4        |\n| Split 3                 | **7.9**        | 2.5        |\n| Split 4                 | **6.2**        | 3.6        |\n| Split 5                 | **11.4**       | 6.2        |\n| Split 6                 | **5.5**        | 2.1        |\n| Split 7                 | **6.4**        | 3.2        |\n| Split 8                 | **7.1**        | 4.6        |\n| mean ± std              | **7.2 ± 1.85** | 3.5 ± 1.35 |\n\nWe stress that, even in these experiments, which are relatively small scale compared to something like LAION, it is clear that the data forensics tasks are far from solved and still remain highly non-trivial. **Specifically, as highlighted by recent works, [5], [6], the community is still far from reliably tackling data forensics in large scale models. Given that even at these relatively small scales robust data forensics remains a largely unsolved problem, our work demonstrates significant progress in the field that should not be discounted**.\n\n**Comments on standard diffusion vs more advanced constructions** As we also mention in the paper, **our focus on standard diffusion was intentional: it is the most widely studied and reproducible baseline**. Extending to latent/conditional models introduces confounding variables (e.g., VAEs, text encoders), which should be carefully treated to ensure a clean ablation and proper benchmarking (see [5], [6] and our discussion on robustness with belonging data and with distribution shifts, prior benchmarking is brittle as it fails to bound false positives). Besides, as standard diffusion remains at the core of these more advanced constructions, we believe our analysis and insights remain highly relevant.\n\n3 How are characteristics of diffusion trajectories leveraged?\n-\n\nWe will revise the method section and elaborate further. We include some comments here whcih we hope answer your question.\n\nOur main argument is that **there is rich information encoded in the diffusion trajectories that gets discarded by simple thresholding**. Prior work [4] focused on selecting a single time-step and they intuitively found that there is a Goldilocks zone for it where the best performance is achieved. Since then, the Goldilocks zone hypothesis and simple thresholding have become standard in diffusion data forensics. Our work disproves the hypothesis directly, as shown in Table 4 of the paper. Ultimately, the intuition is that **we exploit privacy and security vulnerabilities in the temporal dynamics of diffusion, not just the information at a single step. To get a sense of how the signals at different time-steps are leveraged by our classifiers, please see Figure 4 of the paper**. Notably, the weights of our classifiers are not localized in t, and we observe significant values spanning the majority of the trajectories.\n\nIf you have any specific suggestions as to how we can make the intuition even more clear with additional experiments and analysis please do let us know and we will update the paper. \n\n\n4 Comparison with SOTA\n-\n\n**We have included more experiments and comparisons in point 2 of our rebuttal above.**\n\n**Comparison with SOTA MIAs** Please note that **we have compared our MIAs with several existing works [1], [2], [3] as well as a black-box baseline (to quantify distribution shifts) in Table 5. Most methods adopt a threshold-based approach that we have repeatedly shown is not robust**, e.g. [2], [3], so we expect any other works that use the same ideas to perform similarly. **Notably, we consistenly outperform the top competitor [1] (in the paper and in additional experiments given above)**. If there is a specific work that you would like us to discuss we are happy to do so.\n\n**Comparison with MA methods: As you also acknowledge in the strengths section of your review, \"there is a lack of a white-box MA approaches\" for diffusion in the literature.** We have discussed this point in our rebuttal to HK2L's review and in the background section of our paper (starting at line 107, also see discussion starting at line 263). **To our knowledge, there is no existing white-box MA for diffusion.  As for black-box methods, you can see in Table 6 that we outperform them under a fair setting (top row).** Of course, if you are aware of existing white-box MA methods that work directly on diffusion models, please let us known and we would be happy to comment on them.\n\n**Comments on memorization search** We are not sure what you mean by comparing to SOTA in this category. To clarify, identifying memorized samples by filtering generated data is, to our knowledge, not a standard benchmark in membership inference literature. Actually, **as we experimentally prove in our paper (e.g. Figures 2, 3, Table 3), existing MIAs cannot be used for this purpose as they cannot distinguish between belonging and member data. The fact that our work enables this kind of application while existing MIAs completely fail precisely demonstrates the utility and practical use of our unified approach to data forensics**.\n\n---\n\nWe hope our responses have addressed your concerns and we believe that our additional experiments are convicing of the superiority and robustness of our approach compared to existing methods. We remain open to further discussion if there are any further clarifications needed. Please consider raising your score if you find the above answers sufficient.\n\nReferences\n-\n\n[1] Pang Y., et al., 2024, White-box Membership Inference Attacks against Diffusion Models\n\n[2] Matsumoto et al., 2023, Membership Inference Attacks against Diffusion Models\n\n[3] Kong F., et al., 2024, An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization\n\n[4] Carlini N., et al., 2023, Extracting Training Data from Diffusion Models\n\n[5] Zhang et al., 2025 Position: Membership Inference Attacks Cannot Prove That a Model was Trained on Your Data\n\n[6] Das et al., 2025 Blind Baselines Beat Membership Inference Attacks for Foundation Models"}]

Author rebuttal:

Meta-review: This paper showed that one can use features extracted from the sampling trajectories of a DDPM model to address the task of "origin attribution": i.e. distinguishing whether a given image came from (a) the training set, (b) generate by the model, (c) other/external. The reviewers all felt that the paper was worth of acceptance because (1) the solution is simple & effective, (2) it challenges the prevailing belief that only part of the trajectory is useful for origin attribution (the "goldilocks zone") and (3) the problem is important. While I agreed with the reviewers on these points, I did feel like the paper could have put some more detail into why the features they've selected are the best features for the task. The loss features are motivated as a natural generalization of the prior thresholding methods (which seems reasonable), but the gradient features were less obvious (they work and the curvature story is okay - but we could surely think of other features of the trajectories, and it's not at all obvious that gradients are the right ones to choose). That said, I agree with the reviewers that the paper is interesting and worth appearing at the conference.
Decision: Accept (poster)