Title: Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution
Venue: NeurIPS 2025
Accepted: yes
Keywords: Generative Models

Diffusion Models

Data Provenance
Abstract: Diffusion models have transformed image synthesis through iterative denoising, by defining trajectories from noise to coherent data. While their capabilities are widely celebrated, a critical challenge remains unaddressed: ensuring responsible use by verifying whether an image originates from a model's training set, its novel generations or external sources. We introduce a framework that analyzes diffusion trajectories for this purpose. Specifically, we demonstrate that temporal dynamics across the entire trajectory allow for more robust classification and challenge the widely-adopted "Goldilocks zone" conjecture, which posits that membership inference is effective only within narrow denoising stages. More fundamentally, we expose critical flaws in current membership inference practices by showing that representative methods fail under distribution shifts or when model-generated data is present. For model attribution, we demonstrate a first white-box approach directly applicable to diffusion. Ultimately, we propose the unification of data provenance into a single, cohesive framework tailored to modern generative systems.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper endeavours to ensure responsible use of diffusion models and propose to unify membership inference and model attribution into a single framework - origin attribution (OA). It does this by identifying the limitations in the existing membership inference methods that existing methods fail under distribution shifts or when model-generated data is present, then propose to use characteristics in diffusion trajectory in DDPM to classify data origins. Experiments are conducted on DDPMs.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1. This paper tackles important issues that contribute to responsible AI. Specifically, it spans over the field of membership inference, model attribution, and memorization in diffusion models.\n2. In addition, this paper works towards unifying the aforementioned field into a single, cohesive framework.\n3. It is motivated by the gap in MIA in the literature that existing methods often fail for model-generated synthetic data, and the gap in MA that there is a lack of a white-box MA approach.\n\nWeaknesses:\n\n1. It would be better if more explanations regarding how often the assumed application scenario for the proposed general problem of origin attribution (OA) can be observed. As the authors acknowledged, the assumption is that the proposed methods only work when (1) model owners have not disclosed the training setup and data details, and (2) a small fraction of the training samples is available. I am concerned about the practical significance of the latter assumption, as it is mostly the case that users do not have a clue about the training samples used, let alone having a small fraction of them. \n2. This paper delves into the privacy issue in the framework of DDPM. However, the most widely used diffusion models are often latent diffusion models, DiT models, flow-based models, etc. It would be interesting to see how the method performs in these, as their generations are more widespread, thus privacy protection is more practically significant.\n3. Despite the motivation of unifying MIA and MA being well-supported, I find the method section less clearly explained, especially the section regarding the core insight of how characteristics of diffusion trajectories are leveraged. More explanations and analyses need to be conducted to make it more convincing.\n4. For the MIA results, the paper only compares its performance with GSA to demonstrate effectiveness. For the other areas that this work also claims to tackle (MA and memorization), it is also suggested to provide more comprehensive comparisons to existing works. The experiments are quite insufficient in their current state.\n\nQUESTIONS:\n1. Can you provide additional clarification regarding the practical significance of the assumptions this paper make (i.e., the use cases) as mentioned in the weaknesses section?\n2. Would the proposed method works in more practical and widely used diffusion frameworks?\n3. Can you provide more insights and analyse of how characteristics of diffusion trajectories are leveraged?\n4. Can you show more experimental results to compare with current state-of-the-arts across all three tasks (MIA, MA, memorization) that this paper claims to tackle?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the task for probing data provenance using the whole diffusion trajectories. Specifically, it presents one framework that performs membership inference and model attribution.\nImportantly, the paper finds that the “Godlikocks zone” assumption in previous works was not correct. Furthermore, the works explores the effect of distribution shifts on the model performance. \nThe proposed approach clearly outperforms previous works on various metrics for the considered tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths \n\nThe paper’s motivation is very clear and tackling a very significant topic considering the current developments of diffusion models.\nIn general, the writing is very clear with the paper having a well-organized structure, where hypotheses are followed by a through experimental analysis. The learnings are subsequently taking into account resulting in clear improvements compared to previous works. \nThe work does not stick to previous common strategies (Godlikocks zone, no hyperparameter optimization on test set) but instead questions and corrects those.\nThe work motivates very clean experimental setup removing confounder factors, which increases the confidence of the paper’s learnings.\nThe model performs well on the considered tasks.\n\nWeaknesses\n\nThe access to a few training samples is a strong assumption. The work would benefit from a better sensitivity analysis with respect to the data budget for all performed experiments.\nA lower data budget (than 3.4%) would reduce the overall accuracy values, potentially showing clearer gaps for some metrics (e.g. in Tab 5 and 6).\nWhile the work also mentions this as a weakness, it would be more impactful and helpful if the experiments were performed for larger models that are more commonly used.\n\nQUESTIONS:\n- Why are foundational features only a problem? It is biased but it might also enable a better performance with a smaller data budget.\n- Why is data with high CLIP score a problem?\n- Why was the Goldilocks zone adopted? It would be helpful to rephrase the learnings and frame why this might be different to the proposed setting, in what regard the previous learnings were wrong.\n- How would the model perform with more / less data budget? Why 3.4%? A sensitivity analysis of this for more experiments would be helpful to better evaluate this.\n- What exactly is the model-blind classifier? Please refer or provide more details about it.\n- Why only a linear classifier? How would a non-linear model perform?\n- Why only such simple projection? Explain why this is reasonable and why more complex projection are not considered!\n- What are similarly chosen datasets (l.178)?\n\nLIMITATIONS:\nYes, the limitations are discussed in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a method to determine membership and/or attribution of images to diffusion models. The method extracts features from the entire diffusion trajectory, including loss values, input gradients, and parameter gradients across all time steps, then trains linear classifiers on these compressed representations. This “trajectory feature” departs from the popular “Goldilocks-zone” belief that only mid-range timesteps are useful.\n\nWith white-box access and <3.4 % of the training data, the method (i) outperforms prior loss-based and gradient-based MIAs on CIFAR-10 and CelebA-HQ, (ii) stays robust when external data are distribution-shifted, and (iii) can attribute samples from other generators.\n\nThe demonstration of the method is limited to small datasets like CIFAR and CelebA.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. First of all, this paper does a great job of showcasing that current threshold based MIAs have way too many false positives, and become almost meaningless under distribution shift.\n2. The method is able to solve two problems of membership inference and model attribution under the same hood.\n\n## Weaknesses\n1.  Experiments are limited to CIFAR-10 and CelebA-HQ-256; no ImageNet-level, text-conditioned, or latent-diffusion models are studied, so real-world generality is unclear.\n2. Per-image gradients at hundreds of timesteps are costly. What would the feasibility of this method look like?\n3. On CelebA-HQ, attribution accuracy against DDIM is only ~7 % when gradients are used, suggesting poor generalisation to some generators. More analysis of failure cases would be welcome.\n4. 3.4% of data is actually quite sizeable as we scale to datasets such as LAION. How would you consider scaling this to larger datasets, and different architectures?\n5. I do worry about the generalization of the method to OOD members, or when none of the pool of the çandidate member models is not the one that actually generated the data,. This is another class of false positives,\n\nQUESTIONS:\nPlease look at the weaknesses section above.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduced a method for “Origin Attribution”, the goal of determining an image’s origin as one of (a) the training set, (b) generate by the model, (c) other/external. This goal is a generalization of two tasks: Membership Inference Attacks and Model Attribution. The introduced methods consists of exacting features from a given image that consist of a DDPM model gradients for that images over many time steps, and then training a linear probe on top of these features. The authors further study diffusion model dynamics, such as by exploring which time steps lead to features that are most valuable for MIAs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n+ The general task of OA makes sense, though it would benefit from further motivation.\n+ The proposed method is easy to understand\n+ The results for MIA compare favorable to other works.\n+ I particularly enjoyed the motivation of the analysis for the “goldilocks zone” hypothesis and the insights drawn from it.\n\nWeaknesses\n- The paper has no related works section. While some related works are mentioned throughout (especially in intro and discussion), it would be highly valuable to have a standalone related works section that can provide further context. I’m sure the authors are concerned about page limit, so it should be ok (though probably not ideal) to put related works in the Appendix.\n- The authors mention: \"We address shortcomings of Wang et al. (2023); Laszkiewicz et al. (2024) which rely on model or layer inversion that are not easily adapted for diffusion. To our knowledge, we are first to demonstrate MA on DDPMs without assistance from pre-trained foundation models.” I have a few issues with this claim. First, both of these method are applying to diffusion models, so it seems their approach is adaptable to diffusion models (even if it not as natural) - for this the authors can compare to these works to show their approaches’ advantage. Second, based on my understanding, Laszkiewicz et al. (2024) does not use a pre-training foundation model as part of the underlying method (but feel free to correct me). Third, even if all other methods to use assistant from pre-training foundation models, it is not clear to me why this is an issue if these method work, since images foundation models will likely always be available.\n- The “robustness” evaluations of other MIAs seem somewhat unfair. The point the authors make is that these methods would categorize data generated by a model as part of their training datasets (as opposed to external). However, this is just using these methods for a purpose they are not meant to be used for. Also, in the case that the diffusion models “memorizes” a training datapoint and regurgitates it during inference, it feels closer to being a member than external.\n\nQUESTIONS:\nCan the authors address the issues mentioned in the \"Weaknesses\" section? I would be happy to raise the score if these weaknesses were sufficiently addressed.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-15"}