Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation
Authors: Tong Xie
Haoyu Li
Andrew Bai
Cho-Jui Hsieh
Venue: Accepted by TMLR ()

Abstract: Data attribution methods trace model behavior back to its training dataset, offering an effective approach to better understand ``black-box'' neural networks. While prior research established quantifiable links between model output and training data in diverse settings, interpreting diffusion model outputs in relation to training samples remains underexplored. In particular, diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships in previous contexts, posing a significant challenge to extend existing frameworks to diffusion models directly. Notably, we present Diffusion-TracIn that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep. This trend leads to a prominent bias in influence estimation, and is particularly severe for samples trained on large-norm-inducing timesteps, causing them to be generally influential. To mitigate this bias, we introduce Diffusion-ReTrac as a re-normalized adaptation that retrieves training samples targeted to the test sample of interest, enabling a localized measurement of influence and considerably more intuitive visualization. We demonstrate the efficacy of our approach through various evaluation metrics and auxiliary tasks, outperforming in terms of specificity of attribution by over $60\%$.

Published in Transactions on Machine Learning Research (06/2024)
Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation
Tong Xie
tongaie Sucla. edu mooyuli02@uchh edu andrewbai@e. ucla edu chohsich @ Bes, vcla. edu
Haoyu Li
Andrew Bai?
Cho-Jui Hsieh?
DDpartment of Mathematics, 2 *Department of Computer Science
University of California, Los Angeles
Reviewed on OpenReview: https: openreview. net/ per/forum? id- P3Lyun
Abstract
Data altribution methods trace model behavior back to its training dataset, offering an effective approach to better understand "black-box" neural networks. While prior research established quantifiable links between model output and training data in diverse settings, interpreting diffusion model outputs in relation to training samples remains underexplored. In particular, diffuusio models operate over sequence of timesteps instead of instantaneous input-output relationships in previous contexts, posing a significant challenge to extend ex- isting frameworks to diffusion models directly. Notably, we present Diffusion-Tracln that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep. This trend leads to prominent bias in influence estimation, and is particularly severe for samples trained on large-norm-inducing timesteps, causing them to be generally influential. To mitigate this bias, we introduce Diflusion-Re as re- normaaalleed adaptation that retrieves training samples targeted to the test sample of interest, enabling localized mcasurement of influence and considcrably more intuitive visu- alization. We demonstrate the efficacy of our approach through various evaluation metrics and auxiliary tasks, outperforming in terms specificity of attribution by over 60%.
1 Introduction
Deep neural networks have emerged to be powerful tools for the modeling of complex data distributions and intricate representation lcarning. However, their astounding performance often comes at the cost of interpretability, leading to an increasing research interest
better explain these "black-box" methods. mastadce-based interpretation is one approach to explain why a given machine learning model makes certain predictions by tracing the output back to training samples. While these methods have been widely studied in supervised tasks and demonstrated good performance (Koh & Liang 2017 Yeh et al. 2018 Pruthi et all 2020), there is limited exploration of their application in unsupervised settings, especially for generative
models [Kngmaa & Welling 2013 Goodfellow ct al. 2020 Ho et al. 2020 In particular, diffusion models
represent state-of-the-arl vancement. in generative models and demonstrate remarkable performance in a variety applications such as image generation, audio synthesis, and video generation (Kong et al.) 2020
Dhariwal & Nichol, 2021 Ho Salimans 2022: Saharia al. 2022a Hertz ct al.) 2022 Li et al. 2022
Ho al. 2022). The prevailing generative agents in creative arts also call for fair attribution methods to
acknowledge the training data contributors (Rombach et al. 2022 Ramesh et al. 2022 Saharia al. 2022b
Zhang et al.) 2023 Brooks et al. 2023). Nonetheless, the interpretability and attribution of diffusion models
remain an under-explored area
Georgiev et al. 2023
Dai & Gifford 2023).
"Equal contribution

---

Published in Transactions on Machine Learning Research (06/2024)
Compared to traditional supervised settings, the direct extension of instance- interpretation to diffusion models is challenging due to the following factors. First, the diffusion objective involves an expectation over the injected noise E ~ (0, I), hence a precise computation is impractical. Second, diffusion models operate over a sequence of timesteps instead of instantancous input-output relationships. Although each timestop is weighted equally during the training process, we observe that certain timesteps can exhibit. high gradient norm. This means the gradient of the diffusion loss function with respect to model parameters is dominantly large relative all other timesteps (Figure D). As most austancebbased explanation models utilize this first-order gradient information, such biased gradient norms can propagate its domination into the influence estimation for diffusion models. In practice, timesteps are often randomly sampled during training. This introduces arbitrariness in influence calculation, since training sample that happens to be trained on certain timesteps may exhibit higher-Chan-usual gradient norms, and thus be characterized as "generally influential" to various completely different test samples.
We present Diffusion-Trachi and Diffusion- ReTTrac to demonstrate and address the existing difficulties. Diffusion-Tracln is designed extension of TracIn (Pruthi et. al.] 2020) to diffusion models that incorpo- rates the denoising timestep trajectory. This approach showcases instances where influence estimation is biased. Subsequently, we introduce Iiffusion-ReTTere as a re-normalization of Diffusion-Traan to alleviate the dominating norm effect.
Our contributions are summarized as follows:
Propose Diffusion- as designed extension to diffusion models that incorporates and effectively approximates the timestep dynamics.
2. Identify and investigate the timestep-inee inddued gradient norm bias in diffusion models, providing preliminary insights into its impact on influence estimation.
3. Introduce CatiasionnReTrras to mitigate the timestep-induced bias, offering targeted data attribution.
Mustratet and compare the effectiveness of the proposed approach address the gradient norm bias on auxiliary tasks.
2 Related Work
Data attribution methods trace model interpretability back the training dataset, aiming to answer the following fundamental question: which training samples are most responsible for shaping model behavior?
This proves valuable across wide range of domains, such outlier detection, data cleaning, data curating, and memorization analysis (Khanna et al. 2019 Liu et al. 2021 Kong et al. 2021 Lin et al. 2022 Feldman
2020 van den Burg & Williams 2021 The adoption of diffusion models in artistic pursuits, such as Stable Diffusion and its variants, has also gained substantial influence (Rombach et al.) 2022 Zhang et al. 2028
This then calls for fair attribution methods to acknowledge and credit artists whose works have shaped these models' training. Such methods are also vital for addressing associated legal and privacy concerns (Carlini
et al.
2023
Somepalli et al. 2023
2.1 Influence Estimations
Influence functions quantify the importance of training sample by estimating the effect induced when the sample of interest is removed from training (Koh & Liang. 2017). This method involves inverting the Hessian of loss, which is computationally intensive and can be fragile in highly non-convex deep neural networks (Basu et al. 2020). Representer Point is another technique that computes influence using the representer theorem, yet relies on the assumption that allribution can be approximated by the final layer of neural networks, which may not hold in practice (Yeh et al.) 2018). For diffusion models, the application of influence functions is significantly hindered by its computational expense while extending the representer point method is ambiguous due the lack of natural "final layer." " Pruthi et al. (2020) introduced TracIn to measure influence based first-order gradient approximation that does not rely on optimality conditions.
2

---

Published in Transactions on Machine Learning Research (06/2024)
Recently, TRAK is introduced as attribution method for large-scale models, which requires a designed ensemble of models and hence is less suitable for naturally trained models (Park et al. 2023).
In this paper, we extend the Tracin framework to propose an instance-based interpretation method spe- cific to the diffusion model architecture. To address the challenges arose, we present Diffusion-ReTras that re-normalizes the gradient information and effectively accommodates the timesteps dynamics. Previous works have utilized similar re-normalization techniques to enhance influence estimator performance in su- pervised settings. RelatIF reweights infiuence function estimations using optimization objectives that place constraints global influence, enabling the retrieval of explanatory examples more localized to model pre- dictions (Barshan ct al. 2020). Gradient aggregated similarity (GAS) leverages re-normalization to better identify adversarial instances (Hammoudeh & Lowd, 2022). These works align well with our studies in understanding the localized impact of training instances on model behavior.
2.2 Influence in Unsupervised Settings
The aforementioned methods address the fundamental question in supervised settings, where model behavior may be characterized in terms of model prediction and accuracy. However, extending this framework to unsupervised settings is non-trivial due to the lack of labels or ground truth. It is challenging to quantify the influence of specific training samples on model behavior, as there lacks a straightforwait metric to gauge the change in accuracy or relevance of the output.
Prior works explore this topic and approach to compute influence for Generative Adversarial Networks (GAN) (erashita et al. 2021) and Variational Autoencoders (VAF) (Kong & Chaudhuri, 2021). Previous work in diffusion models quantifies influence through the use of ensembles, which requires training multiple models with subsets of the training dataset, making it unsuitable for naturally trained models (Dai & Gifford 2023). Journey-TRAK applies TRAK (Park et al. 2023) to diffusion models and attributes each denoising timestep individually (Georgiev et al. 2023). . As the diffusion trajectory spans numerous timesteps, this approach results in a multitude of altribution scores for a single image, posing challenges to settings where a holistic single-shot attribution is more interpretable and desired. Additionally, D-TRAK presents counter-intuitive findings suggesting that theorctically unjustified design choices in TRAK result in improved performance, emphasizing the need for further exploration in data attribution for diffusion models (Zheng et al. 2023). These works are complementary to our studies and contribute to more comprehensive understanding of instance-based interpretations in unsupervised settings.
3 Preliminaries
3.1
Diffusion Models
Denoising Diffusion Probabilistic Models (DDPMs) Ho al 2020 are a special type of generative models that parameterized the data as Po(xo) = po (Noer) drir,, where æ1...., 2T are latent variables of the same dimension as the input. The inner term po (our)) is the reverse process starting at the standard normal p(ær)= = (xrr; 0, I), which is defined by a Markov chain:
T po( p(xT) II po(
where p(#+-1/2;) -; t), ()), The reverse process learnable and fixed a Markov Chain based variance scheduler $1...., Br. Notably, efficient training is achieved by stochastically selecting timesteps for each sample. This means samples are trained on different timesteps in epoch, allowing for coverage of the trajectory over time without the need to process each timestep sequentially. DDPM further simplifies the loss by re-weighting each timestep, leading to the training objective used in practice,
Lsimpte (0) Emg [d(e, ea solaaro + /11 (1-)ee),, ov.ee,
(2)
where N( T) and d is the loss function such I distance.
3

---

Published in Transactions on Machine Learning Research (06/2024)
3.2 Tracin
TracIn is proposed as efficient first-order approximation of training sample's influence. It defines the idealized version of influence of a training sample to a test sample z' as the total reduction of loss on z' when the model is trained on 2. For tractable computation, this change in loss on the test sample is approximated by the following Taylor expansion centered at the model parameters wy
{(wN+1, x') - ((w,: = Ve(w),=') 2') (Wx+1 - wx) + O[I wxll
(3)
If stochastic gradient descent (SGD) is utilized in training, then the parameter update is measured by Wk+1 - wi = -1); V Therefore, the first-order approximation of Ideal-Influence is derived to be
TracIn(z,2") = Σ The Ve(We,,,)) 2') Ve(WE, 2),
(4)
summing iterations à where the particular training sample 2 is utilized. To reduce computational costs, TracIn approximates the influence with saved checkpoints from training to replay the entire training process. Typically, 3 to 20 checkpoints with a steady decrease in loss are utilized, reducing the excessive storage of checkpoints while effectively capturing training. It. is also expected that the practical form of TracIn remains the same across variations in training, such as optimizers, learning rate schedules, and handling of minibatches (Pruthi al. 2020
4 Extending Tracin to Diffusion Models & Challenges
4.1
Iiffusion-Trachn
In this section, present Diffusion-Trran TracIn to provide efficient extension of TracIn designed for diffusion models. Specifically, two adjustments keen to diffusion models critical to enable this extension. First, the diffusion objective is an expectation of denoising losses over different timesteps 1 t T. Second, the objective involves an expectation over the added noise e ~ NO0 7). To address these challenges, we first apply Tracin conditioned on each timestep t, then we compute a Monte Carlo average over m randomly sampled noises c.
From equation 2 it possible to treat. the diffusion model learning objective as combination of T loss functions. If we denote L,(0,, r, = (0, e, == d(e, co + (11 a,)e, to be distinct loss function each timestep & and noise, Subsequently, we compute TracIn influence score over each of the timestep t
treat Lsimple an expectation over all the Le & E [0,
TracIn(z,2',0) E. Σ mh (O L.)
k:zz =2
(5)
Σ K:2g =2
ne (0x, L,
m
where L Velu (Ok, Curain 2), train the training timesteps, and ésrain is the noise utilized when training the sample This enables true replay of the training dynamics. Then we define Iiffusion-Trachhe to be the expectation over 7 timesteps to provide an one-shot attribution score covering the full diffusion process,
Diffusion- Difcusion-Traclnn,,, z²)= E, (TracInn (1,,,
TracIn(z, (2, &',))
(6)
Σ
Th (o)
Σ
mhe
(0,,(02, L.
12 mf
4

---

Published in Transactions on Machine Learning Research (06/2024)
The practical form of Sulfusion-Tracllee also employs training checkpoints, suggested by Pruthi et al. (0200) to enhance computational efficiency,
T
Diffusion-Traclnn.... Diffusion- TracIn(2,2))
me mf
(0L,(Ok.
(7)
t=1
where is the number of checkpoints.
4.2 Timestep-inducer Norm Bias
To capture the influence training samples on the model, an attribution method A R x R R assigns attribution score O to each training sample 2; E D. Therefore, an idcal and fair attribution method should derive its results based on properties intrinsic to training samples. This ensures the method's outcomes accurately capture the inherent characteristics and significance of each training sample within the dataset.
In most machine learning models, a dominating gradient norm can be largely attributed to the training sample itself. For example, outliers and samples near the decision boundary may exhibit. higher gradient norms than usual. However, while sample-induces gradient norms are informative for influence estimation, we observe that the variance in gradient norms for diffusion models can also be an artifact of the diffusson training dynamics. We identify the dominating norm effect, which refers phenomenon that the variance in samples' loss gradient norms being largely affected by diffusion timesteps. The timestep artifacts sub- sequently propagate into influence estimation, introducing 'noise' that overwhelms the genuine signal from samples, dominating and rendering the attribution results somewhat arbitrary.
Using a diffusion model trained on Artbench-2 with 10,000 samples as illustration, we demonstrate the presence of such sinestep-induced norm bias by showing:
1. Statistically significant correlation exists between training samples' gradient norm and their timestep.
For each individual sample, its gradient norm is highly dependent on the timestep.
3. Manipulating the timestep in influence estimation significantly alters attribution results.
Definition 1. We define the highest norm inducing timestep for sample z to be
(m)x(2) arg max LL, Ctrain 2)|1-
Norm vs. Timestep. We examine the distribution of training samples' loss gradient norm and the training timestep (Figure D) This is conducted for multiple check- points, each yiclding distribution displaying norms at its specific stage of the training process. The distributions demonstrate notable upward trend that peaks at, in this case, the later timestep region (i.e. closer to noise).
1800
Nerm Ba-aasa
6000
This suggests that samples whose training timestep falls within the later range tend exhibit higher norms.
2000
We further examine the impact and quantify the relation- ship between loss gradient norms and training timesteps. Using randomly selected samples { measure the correlation between the proximity of sample z''s train- ing timestep to m=;)(z;) and ranking of z's norm among the entire training dataset. The detailed procedure is in- cluded in Algorithm[] [ This is conducted for every check- point used to compute influence. As an illustration, Fig- ure shows checkpoint with correlation of 0.7 and
Figure 1: Correlation of Timestep and Norm. We plot the norm ranking and distance between training timestep to tmax for 50 randomly selected samples. The resulting correlation 0.7 and the lin- ear regressor (red) has a slope of 6.038.
5

---

Published in Transactions on Machine Learning Research (06/2024)
3.5
3.0
2.5
2.5
Gradlent Norms
Gradient Norms
2.0
2.0
1.5
1.5
1.0
1.0
0.5
0.5
0.0
200
800
Training Timesteps
Training Timesteos
(a) 2 at checkpoint 100
(b) Artbench-2 at checkpoint 200
Figure 2: Samples' Norm Training Timestep. We plot the and timestep of 2,000 randomly selected training samples. We observe that loss gradient norms tend to increase when the training timestep falls the later range (towards noise). This upward trend is consistent at other checkpoints tested. The sample with the largest norm (red) and smallest norm (green) are shown; no exceptional visual patterns are noticed. Additional distributions for others well open-sourced models included in Appendix A.1 and A.2
p-value 1.38 10-7 A linear regressor also fitted the 50 data points, giving a slope of 6.038 and p-value 2.55 10-8 This suggests a statistically significant positive correlation between the norms and training timesteps. Consequently, it indicates notable training timestep-induced norm bias that could well dominate over sample-induced norms, which will then propagate into influence estimation.
Varying Timestep for a Single Sample. We further analyze the norm distribution for an individual training sample. At a given model chockpoint, we compute the loss gradient norm for a fixed sample a at every timestep. We plot the norm distribution for randomly selected samples and observe similar trends. Figure
3 shows the distribution for an example training data three different. model stages. The highest norm-inducing region at these three checkpoints all falls within the later timestep range, regardless of where the training timestep is sampled at. The further implication is that for each sample, its loss gradient norm is highly dependent on the chosen timestep. Tt is also observed that within the same epoch, various samples share similar trends in norm distribution. This suggests a systematic pattern (e.g. artifact of diffusion learning dynamics) beyond individual instances, supporting the intuition that over-reliance on gradient norms may not be ideal.
Timestep Manipulation. We further illustrate the timestep-induced bias by exploring the susceptibility of inffuence estimation the manipulation of timesteps. We conduct the experiment 500 training samples that are characterized by Diffusion- TracIn "uninfluential" to random test sample (i.e. influence score is close to 0, neither proponent nor opponent). For each uninfluential sample 24, we compute its influence using (z,) instead of the original training timestep. The result shows that after deliberately modifying timestop, the ranking of the magnitude of influence for these samples increases by 4,287 positions on average. Given that there are only 10,000 images the dataset, this notable 42.8% fluctuation indicates that the linestep-induced bias is significant enough flip a training sample from uninfluential opponents. It further indicates that iiffusion-Traclnt attribution results could well arise from timestep- induced norms in general. Such findings highlight the polential vulnerability within Diflusion-Tracln a data attribution method, emphasizing the need for more robust influence estimation techniques.
proponents
The preceding sections empirically show that a training sample's loss gradient norm is highly dependent on timesteps, which produces consistent trend in norms based on learning dynamics. As the timestop utilized in influence estimation is stochastically sampled in training, instances receive varying degrees of such timestep-induced norm. This bias is particularly evident in samples whose training timestep falls close

---

Published in Transactions on Machine Learning Research (06/2024)
12-
0.2
2.4
223
433
3.00 900
1330
200
400
900 059
233
430
170
(8) Checkpoint 100
(b) Checkpoint 200
(c) Checkpoint 300
Figure 3: One Sample's Norms Varying Timestep. Example of norm distributions for randomly selected sample is shown. On each checkpoint, the norm distribution is skewed later timesteps for each individual sample. This trend exists regardless of where the actual training timestep (red) is sampled at.
to fmax. leading them to be "generally influential" (Figure 6). These findings suggest samples' norms are a suboptimal source of information, calling for special attention to the handling of gradient norms.
5 Method
5.1
Timesteps Sparse Sampling
The extension TracIn to diffusion models, as formulated in Equation ? involves an expectation over T timesteps to replay the training process. However, since the number of timesteps can be large in practice (e.g. 1,000), optimization toward computational efficiency becomes imperative.
To address this, employ reduced subset of timesteps selected to encapsulate essential information of the sequence, while retaining the diffusion learning dynamics. We leverage n, evenly-spaced timesteps S = fti,te t2,...,tn. } ranging from 1 to T for the test sample z' and define
m
Diffusion- Traioonn-rraannnaaa2 z')
TH
VoLr(Oe, L,
(8)
EES
where the timesteps ip e approximates the full diffusion trajectory.
5.2 Diffusion- Mantiion-ReTraa
As shown in Section 4.2 the direct application of TracIn to diffusion models leads to a strong dependence on timesteps and the corresponding norm induced for the training sample. The prevalence this dominating norm effect calls for modification of the approach when applying to diffusion models. We thus propose Diffusion-ReTTes as ro-normalized adaptation mitigate this effect.
Intuition. By Cauchy-Schwarz inequality, it can be noticed from Equation that
TracIn(z, 2/) Σ nhl VellV,
Hence, training samples with disproportionately large gradient norms tend to have significantly higher in- fluence score [TracIn(z,22) z')]. This suggests that these samples are more likely to be characterized as cither a strong proponent or opponent to the given lest sample z', depending on the direction alignment of Vε(w,, z') and Vl(w, 2). As identified in Section 4.2 the loss function component L, from certain timesteps is more likely to have larger gradient norm. Hence varying degrees of "timostop-induccd" norm biases, stemming from systematic learning dynamics, emerge and affect the influence estimations.

---

Published in Transactions on Machine Learning Research (06/2024)
Approach. Since the gradient norm is not exclusively a property of the sample, ideal Instance-bsse interpretation should not overestimate the influence of samples with large norms or penalize those with small norms. In fact, this dominating norm effect can be introduced by the timesteps for both test sample x' and each training sample 2, whose loss gradient norms are computed using timestep t and train respectively.
For test sample z', the gradient information ELes Ex= VoI4(Ok, z') derives from an expectation over the ne sub-sampled timesteps S = (t,,... J. Therefore, influence estimation inherently upweights timesteps with larger norms and downweights those with smaller norms. For each training sample 2, the timestep train stochastically sampled during the training process, hence incorporating varying degrees of timestep- induced norm bias. To this end, we propose Diffusion-Re which introduccs normalization that reweights the training samples to address the dominating norm effect. We normalize these two terms and define
f. = [لl
L=(0,2.c) Then we define L and (0, 2, [
Diffusion MeresioonReeTieee Re'Tacc(ze0) (2)
Th
(eLi(0n,', L.
(9)
k= T21
LE
The bias introduced to influence estimation due timestep-induced norms is thus mitigated. In this way, minimize the vulnerability that the calculated influences are dominated by training samples with a disproportionately large gradient norm arising from stochastic training.
6 Experiments
To interpret model behaviors through training samples, the proposed data attribution method A encom- passes two natural perspectives: test-influence A(2, 2') and self-influence A(2, z). Specifically, test-influence measures the influence of each training sample z on a fixed test sample z', entailing a targeted analysis on a specific test sample. On the other hand, self-influence estimates the influence of a training sample on itself, providing rich knowledge about the sample itself and gauges its relative impact on the model's learning.
We evaluate Diffusion-Trran TracIn and ReTTrac under these two objectives. The experiments provide evidence of the dominating norm bias in Diffusion- TracIIn attributions, presenting instances where this effect may be unnoticed in one objective yet gives rise to substantial failure in another. We address the following questions:
1. Image Tracing: training data through lesl-influence?
How effective is each method at attributing the learning source of an image to the
2. Targeted Attribution: In general datasets, how does Diflusion-ReTrac outperform Diffusion- Tracin by addressing the loss gradient norm bias?
3. Outlier Detection: Why might the timestep-induced bias be unnoticed in detecting outliers or atypical samples by calculating self-inntuuence?
6.1 Image Tracing
One fundamental role of data attribution methods is to trace the model's outputs back to their origins in the training samples. This idea is also utilized for analyzing memorization (Feldman 2020), a behavior where the generated sample attributed to a few nearly identical training samples. In essence, Image source tracing helps pinpoint specific training samples that are responsible for a generation. Thus we evaluate our methods on the question: Given test sample, which instances in the training dataset is the model's knowledge of the test sample derived from?
Sctup. We begin by attributing a diffusion model trained on combination of the entire 5,000 samples from CIFAR-10 airplane subclass and 200 samples from MNIST zero subclass (Krizhevsky et al.) 2009 LeCun & Cortes 2010). Given a test sample of MNIST zero, it expected that the 200 MNIST samples in the training datasct serve as ground truth for the image source. Similarly, a test sample of CIFAR-planc should be attributed the 5,000 CIFAR training samples. We thus obtain an accuracy score by measuring the correctly attributed proportion among the top-k influential sample.
8

---

Published in Transactions on Machine Learning Research (06/2024)
1.000
Billoinnlleeeer DiflciosTaann
0.975
Random
0.900
Pecciin
Pestsio
0.925
6.900
0.875
6.850
Top
(a) Precision MNIST zero test samples
(b) Precision CIFAR-plane test samples
Figure 4: Image Tracing Accuracy. We measure precision by evaluating the ratio of correctly attributed samples among top & proponents. While (a) shows both methods successfully traced MNIST test samples, (b) shows that Diffusion- Tracin fails to attribute CIFAR-plane test samples since the outlier MNIST samples with large norms are characterized as strong proponents, giving low precision for small k. Note that random attribution is strong baseline in (b) since the 5,000 CIFAR-planes are more likely to be proponents compared to the 200 MNIST samples.
Results. The precision for Diffusion- Tracon and ReTrac are reported in Figure While both methods successfully altribule the MNIST test samples to the 200 MNIST training samples, Diffusion-Trach is sig- nificantly worse than Re Rear and the random baseline in attributing CLFAR-plane test samples. It frequently characterizos MNIST samples as top proponents for CIFAR planes. This aligns with the expectation that Diffusion-Tracln tends to assign high influence to training samples with large norms, in this case, the outlier MNIST zeros. Further analysis of these influential zeros indicates that their training timestep falls exactly at or close to tmaz. This further amplifies these outliers' norms, exacerbating the bias introduced. In contrast, ReTrac successfully attributes both MNIST-zero and CIFAR-plane test samples. Comparison the additional baseline of influence functions (Koh & Liang] 2017) is also included in appendix C
Visualization. The attribution results are shown in Figure Upon closer examining Diffusion-Ti Traccn attribution results, the set of MNIST zero samples exerting influence on plane is relatively consistent across various CIFAR plane test samples. For instance, the MNIST sample with highest influence on the two generated planes is identical. For the CIFAR-plame samples that Diffusion- TracIn successfully attributes (without influential MNIST zeros), there still appear to be generally influential planes. This phenomenon is alleviated in Re Trara with sets of influential samples being more distinct and visually intuitive. Additionally, the CIFAR-plane instances with high influence (e.g. among the top 200) to MNIST test samples tend to be planes with black backgrounds, which to extent also resemble the MNIST zero. Visualization for these proponent planes is included in Appendix
D.3 It is also worth highlighting that Diffusion-ReTTen identifies potentially memorized samples for the generated image, such as the last row in Figure
6.2 Targeted Attribution
We then provide a comprehensive analysis of the influential samples retrieved by CeffusionTTrclli and Re Trac on various datasets. Compared the previous settings, this experiment minimizes the effects of unusually large "sample-induced" gradient norms due to the deliberately introduced outliers. This experiment further compares the capability of Dillusion--Trcln and Re in tasks with different emphases or objectives.
Setup. We compule Lest-influence on diffusion models trained on 1). Tiny ImageNet containing 100,000 images with 64 x 64 resolution (Le & Yang 2015). 2). CelebFaces Attributes (CelebA) subset containing 100,000 images with 128 128 resolution (Liu et al. 2015), 3). Artbench-2 consisting of "Post-impressionns' and "ukiyo-e" subclasses (Liao et al. 2022; Zheng al. (023), each with 5,000 images and resolution 64 64, and 4). CIFAR-10 (Krizhevsky et al. 2009) consisting of 50,000 images with resolution 32 x 32.

---

Published in Transactions on Machine Learning Research (06/2024)
Diffusion-Re Trac
Diffusion-Train TracIn
0.00514
0.00405
0.00958
0.0034
0.00351
0.00324
0,00270
0.00272
0,00264
0.0025
0.00251
0,00245
0.00399
0,0027
0.00269
0.00268
0,0025
0.00252
0.00316
0.00305
0.00293
0.00277
0.00275
0.00263
0.00254
0,00252
0.0025
0,00248
0.00248
0.00241
Figure 5: Image Tracing on Outlier Model. We show the attribution results on 4 test samples, using both training (leftmost top 2) and generated samples (leftmost bottom 2); influence scores stated below each image. While both methods successfully attribute MNIST zero lest samples, Diffusion- incorrectly characterizes MNIST training samples as influential to a CIFAR-plane test sample. This notably mitigated with Diflusion-ReTraee
Results. To quantify the targetedness of an attribution method, assess the prevalence of generally infiuential samples. For a list of n random test samples, we analyze the top-k influential samples identified by the two methods, each giving us total of n x k samples. Within the set, we report the proportion of lower score implics more overlapping (i.c. generally influential samples). Wo
distinct instances in Table note that Cetuusion-Taacle yields extremely homogeneous results, where the top influential training samples for diverse set of test images are exactly the same. This is particularly evident in CelebA which is comprised entirely of human faces and ArtBench-2 containing just two subelasses, where the image categories less diverse and thus more susceptible to the bias induced. In these cases, Diffusion- TracIn results in only about 30% of unique samples while Re retrieves up to 95.6%, significantly outperforming in terms of targetedness of attribution.
much
Top 10
Top 50
Top 100
Top 10
Top
Top 100
D-TracIn
0.586
0.474
0.439
0.293
0.261
0.248
Tiny ImageNet
ArtBench-2
D-ReTrac
0.900
0.841
0.823
0.812
0.646
0.605
D-TracIn 0.313
0.273
0.269
0.725
0.663
0.636
CelebA
CIFAR-10
D-ReTrac
0.956
0.903
0.876
0.856
0.800
0.768
Table 1: Targeted Attribution. This table shows the average proportion of unique samples retrieved over 16 test samples. Caitusion-Trachn extracts fewer unique samples compared to Snffusion-Reeraas
Visualization. From Figure it is visually evident that Diffusion-Tracl Traccnn retrieves numerous generally in- fluential training samples. The same set of samples exhibits large influences test samples that are com- pletely different (e.g. in terms of subclass or visual similarities such as color and structure). In Tiny Im- ageNet, for instance, the training sample of nail with white background is characterized as influential to both the red building (row 1) and the bottle (row 3), which may visually counterintuitive.
1.0
05
Figure 7: Timestep and Generally Influential. A generally influential sample whose training timestep falls exactly within (x) region.
Further analysis of these generally influential samples reveals that their associated timestep tends to be close
10

---

Published in Transactions on Machine Learning Research (06/2024)
Diffusion-B ReTrac
Diffusion- Traclnn
Figure Targeted Attribution. We show attribution test samples each from Tiny ImageNet (top) and Artbench-2 (bottom). The top proponents shown. The generally influential images indicated in red box. It is visually evident that Diffusion- Re'Trac provides more distinct attribution results. More visualizations of CIFAR-10 and CelebA can be found in appendix D.1
to fmax- As an illustration, Figure 7 shows the distribution norm vs. timestep for an example generally influential instance from Artbench-2. This specific sample emerges as influential at checkpoint 80, which coincides with its training timestep falling close to Tmax- It then becomes generally influential as shown in Figure 6 (first proponent Iiffusion-ReTrrcc.. The revised approach retrieves proponents that bear greater visual resemblance to the test samples, highlighting Re ReTrac's targeted attribute and reinforcing that dissimilar test samples are more likely to be influenced by a distinct set of training samples.
the last row). This phenomenon is notably mitigated after normalization in
6.3 Outlier Detection
Influence cstimation is often used to identify outliers that doviate notably from the rest of the training data. Intuitively, outliers independently support the model's learning at those sparser regions of the input space which they belong, whereas learning of the typical samples is supported by a wide range of data. Hence in ideal influence method, outliers tend to exhibit high self-influence, indicating that they exert high contribution in reducing their own loss. Because of such outlier-induced norm, observe that biased est imattiono may easily go unnoticed in this common metric of outlier detection.
Setup. We extend the experiment conducted on the CIFAR-MNIST dataset mentioned in section 6.1 We compute the self-influence of cach of the training instances, and sort them by descending order. Since the 200 MNIST samples are outliers that independently support region, we evaluate whether our methods assign high self-influence to the 200 samples of MNIST zero.
11

---

Published in Transactions on Machine Learning Research (06/2024)
Result. The results show that even after normalization, ReTr retains the ability to identify outliers. This further supports that certain variations in loss scale between timesteps are arbitrary and do not convey relevant signals (Table 2). Nonetheless, the bias introduced by diffusion timesteps is unnoticed in this experiment. Since outliers naturally exhibit larger norms compared to the typical inlicrs, the timestep- induced norm becomes more obscure confounding factor and hence less subtle in the computation of self-influence.
Top 100
Top 200
Top 300
Diffusion-Trrcln 0.880
0.880
1.000
Diffusion- Meniiionn-eeeee Trac
0.860
0.845
1.000
Table 2: Outlier Detection. This table measures the proportion of MNIST samples among top-k identified samples with the highest self-inftuence. The performance of these two methods comparable. Both methods assign high self-infuence to the 200 MNIST outliers out of the 5,000 CIFAR. planes.
Visualization. Examining high-ranking samples shows that the airplane samples with high self-influence (among the top 200) have large contrast and atypical backgrounds compared to airplane samples with low self-influence. Overall, instances with high self-influence tend exhibit high visual contrast or are difficult to recognize. This observation consistent with patterns revealed in previous work on influence estimation for VAE [Kng & Chaudhuri 2021. Visualization for plane samples with high self-influence is included in Appendix D.2
7 Conclusion
In this work, we extend the data attribution framework to diffusson models and identify a prominent bias in influence cstimation originating from loss gradient norms. Our detailed analysis clucidates how this bias propagates into the attribution process, revealing that gradient information harbors undesired bias caused by diffusion model dynamics. Subsequent experiments validate Drfa as an effective attempt to mitigate this effect, offering fairer and targeted attribution results.
Limitations and future work. Since our proposed methods estimate influence by replaying the training process, adherence to the theoretical framework necessitates the usage of timesteps utilized during training, potentially limiting the scope of attribution. In addition, a theoretical explanation of the nrrge-norm-indueeee timesteps would offer greater insights into the underlying causes and facilitate tailored solutions. While renor- malization effectively addresses the dominating norm effect and "gencrally influential" samples, exploring more refined normalizing strategies and further examining the gradient alignments may also be beneficial. Appendix B provides an overview of the efficacy and trade-offs of various normalization methods.
Furthermore, analysis of other potential confounding factors would provide more comprehensive under- standing of what constitutes fair attribution method. Exploring attribution methods in latent diffusion models (Rombach et al. 2022) and scenarios involving corrupted data such as Ambient Diffusion (Daras et al. 2024) presents an intriguing avenue for future research, as it paves the way for understanding in more complex settings and the development of more interpretable methods for a wide range of applications.
8 Acknowledgements
This work is partially supported by NSF 2048280, 2331966, 2325121, 2244760 and ONR -00014-23-1-2300.
12

---

Published in Transactions on Machine Learning Research (06/2024)
References
Naman Agarwal, Brian Bullins, and Elad Hazan. Second-order stochastic optimization for machine learning in linear time. Journal of Machine Learning Reseorch, 18(116):1 40, 2017.
Elnaz Barshan, Marc-Etienne Brunet, and Gintare Karolina Dziugaite. Relauif: Identifying explanatory
training samples via relative influence. In International Conference on Artificial Intelligence and Statistics, pp. 1899-1909. PMLR, 2020.
Samyadoep Basu, Philip Pope, and Soheil Feizi. Influence functions in deep learning are fragile. arXiv preprinl ar Xiv:2006. 14651, 2020.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18392 18402, 2023.
Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 28), pp. 5253-5270, 2023.
Zheng Dai and David K Gifford. Training data attribution for diffusion models. ar preprint arXiv:2306.0P 02174, 2023.
Giannis Daras, Kulin Shah, Yuval Dagan, Aravind Gollakota, Alex Dimakis, and Adam Klivans. Ambient diffusion: Learning clean distributions from corrupted data. Advances in Neural Information Processing Systems, 36, 2024.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-879M 2021.
Gongfan Fang, Xinyin Ma, and Xinchao Wang. Structural pruning for diffusion models. Advances in neural information processing systems, 36, 2024.
Vitaly Feldman. Does learning require memorization? short tale about long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pp. 2020.
Kristian Georgiev, Joshua Vendrow, Hadi Salman, Sung Min Park, and Aleksander Madry. The journey, not the dostination: How data guides diffusson models. 2023.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11): 139 144, 2020.
Zayd Hammoudeh and Daniel Lowd. Identifying training-set attack's target using renormalized influence estimation. In Proceedings of the 2022 ACM SIGSAC Conference Computer and Communications Security, pp. 1367-1381, 2022.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yacl Pritch, and Daniel Cohen-Or. Prompt-to- prompt image editing with cross attention control. arXiv preprint arXiv:2208.010 2022.
Jonathan Ho and Tim Salimans. Classifier-free diflusion guidance. arXiv preprinl ar 125998 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840 13:6840-6851, 2020.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, al. Imagen video: High definition video generation with diffusion models. Xiv preprint or arXim:2210.0020 20.0220,,, 2022.
13

---

Published in Transactions on Machine Learning Research (06/2024)
Rajiv Khanna, Been Kim, Joydeep Ghosh, and Sanmi Koyejo. Interpreting black box predictions using fisher kernels. In The 22nd International Conference on Arlificial Intelligence and Statistics, pp. 3382- 3390. PMLR, 2019.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint crXiv:1312.6114, 2013.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885 1894. PMLR, 2017.
Shuming Kong, Yanyan Shen, and Linpeng Huang. Resolving training biases via influence-based data rela- beling. In International Conference on Learning Represenlations, 2021.
Zhileng Kong and Kamalika Chaudhuri. Understanding instance-based interpretability of variational auto- encoders. Advances in Neurol Information Processing Systems, 34:2400- 2412, 2021.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for andio synthesis. arXiv preprint orXiu:2009.0970) 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 777)3, 2015.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. URL httt::/yann. com/exdb/mniste
Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori Hashimoto. Diffusion-Im improves controllable text generation. Advances Neural Information Processing Systems, 35:4328 4343, 2022.
Peiyuan Liao, Xiuyu Li, Xihui Liu, and Kurt Keutzer. The artbench dataset: Benchmarking generative models with artworks. arXic preprint arXin:2206. 1404, 2022.
Jinkun Lin, Anqi Zhang, Mathias Lécuyer, Jinyang Ti, Aurojit Panda, and Siddhartha Sen. Measuring the effect of training data deep learning predictions via randomized experiments. In International Conference Machine Learning, pp. 13468 13504. PMLR, 2022.
Zhuoming Liu, Hao Ding, Huaping Zhong, Weijia Li, Jifeng Dai, and Conghui He. Influence selection for active learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
9274-9283, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. Tn Proceedings of the IEEE international conference on computer vision, pp. 3730-33 0730,,378,-70 2015.
Sung Min Park, Kristian Georgiev, Andrew llyas, Guillaume Leclere, and Aleksander Madry. Trak: At- tributing model behavior at scale. Xiv preprint ar Xiv:2300.14486, 2023.
Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advonces in Neural Information Processing Systems, 33:19920-19930, 2020.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint orXiv:2204.000 .000440..11.... 1(2):3 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lec, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pp. 1-10, 2022a.
14

---

Published in Transactions on Machine Learning Research (06/2024)
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text- to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479-364944 2022b.
Gowthami Somepalli, Vasu Singla, Micah Goldbhum, Jonas Gciping, and Tom Goldstein. Diffusion art digital lorgery? investigaling data replication in diffusion models. In Proceedings of the IBEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 6048-648800 6058, 2023.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. ar preprinl arXiv:2010.0805 2020.
Naoyuki Terashita, Hiroki Ohashi, Yuichi Nonaka, and Takashi Kanemaru. Influence estimation for genera- tive adversarial networks. arXie preprint arXiv:2101.08367 2021.
Gerrit van den Burg and Chris Williams. On memorization in probabilistic deep generative models. Advances in Neural Information Processing Systems, 34:27916 27928, 2021.
Chih-Kuan Ych, Joon Kim, Ian En-Hsu Yen, and Pradcep K Ravikumar. Representer point selection for explaining deep neural networks. Advances in newral information processing systems, 31, 2018.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3836 3847, 2023.
Xiaosen Zheng, Tianyu Pang, Chao Du, Jing Jiang, and Min Lin. Intriguing properties of data attribution on diffusion models. Xivie preprint Xiv:223311 200000000000 2023.
15

---

Published in Transactions on Machine Learning Research (06/2024)
Appendix
A Tinestep-Induced Bias
A.1 Norm vs. Timestep
To demonstrate that diffusion timesteps have significant impact on loss gradient norms, we plot the distribution of 2,000 randomly selected training samples' norms and their training timesteps. Visualization for the distribution is shown in Figure 8 There is notable upward trend that peaks the later range of the timesteps (i.c. timesteps closer to noise), suggesting that samples trained during these later timesteps tend to exhibit larger norms. Additionally, it is also observed that the trend in norm distribution gradually diminishes at the model convergence. This further supports that such variance due to timestep is an artifact of the training dynamii, rather than a property the training sample. However, Diffusion-Trralle utilizes gradient information throughout the entire learning process instead of focusing solely on those near convergence. This approach is due the tendency of the latter to contain minimal information, resulting in inevitable trend in norms affecting influence estimation. This also motivates the renormalization technique in Diffusion- Rrar.
24
(a) Checkpoint 100
(b) Chockpoint 200
(c) Checkpoint
Figure 8: Norm Distribution. We plot the loss gradient norm and training timestep of 2,000 samples. The distributions checkpoints 100, 200, and 300 all demonstrate an upward trend. This suggests that samples whose training timesteps fall within the later timestep region tend to have a larger norm.
staing Iiherie 131
bod Le Dutee
04
10
(a) Checkpoint 100
(b) Checkpoint 200
(c) Checkpoint 300
Figure Varying Timestep for Single Sample. The norms sample #0 are computed at different timesteps. The distribution obtained at checkpoints 100, 200, and 300 all demonstrates similar trend that peaks at the later timestep region.
16

---

Published in Transactions on Machine Learning Research (06/2024)
A.2 Norm Distributions for Various Models
To show that this observation is consistent in various diffusion models, Figure 10 plots the norm distribution on the following diffusion models: 1). trained on Tiny-ImageNel, 2). trained on CelebA, 3). open-source
Google's Pretrained DDPM on CTFAR-10.
Tmy-inagelie 500,000 Stepps
Tey-knagahee 8,0030000 Stapes
CelebA 800,000 Steps
Pretrained gusgknldgmmafatt
(a) IingeNe
(b) Tiny-InageNet.
(c) ColebA
(d) Pretrained CIFAR-10
Figure 10: Norm Distribution on Additional Models. We plot the loss gradient norm and limestep for three additional models. There is notable region of timesteps with significantly larger norms, although the region may differ depending on model learning dynamics.
Why is there correlation between norms and timesteps, where is it coming from?
Our hypothesis regarding this trend is primarily rooted in the learning dynamics inherent to diffusion models. Specifically, previous works have suggested that later timesteps in diffusion models tend to learn structure, while carlier timesteps focus on capturing finer details Fang et al. (2024). We speculate that this observed correlation between gradient norms and training timesteps may stem from similar dynamics, as evidenced by the variability in regions of high-norm induction across different stages of learning. Our work identifies and presents an initial investigation into this issue, further theoretical exploration would be beneficial.
A.3 Varying Timestep for Single Sample
We further show that for a fixed training sample, the gradient norm with respect to its loss computed at different timesteps varies significantly. This reinforees the effect of training timestep in the estimation of influence, indicating that each sample receives a varying degree of bias since the training timestep is stochastically sampled. An example norm distribution for a fixed sample at different checkpoints is shown in Figure
A.4 Correlation
We provided quantitative analysis addressing the question: If the training timestep of sample falls closer to (x), does also have relatively larger norm compared to the rest of the training dataset? To analyze the relationship between the stochastically chosen training timestep and the sample's overall norm ranking among the rest, we obtain a correlation score by i). compute the distance between a sample's training timesteps berai and the timestep that yields the maximum norm (x),, ii). the ranking of this sample's s gradient norm among all the training samples, and īī)). calculate a Spearman Rank correlation score between distance and ranking (Algorithm Figurel in the main text shows a visualization of the measured correlation.
Algorithm 1 Training Timesteps and Norm Ranking
V1 + 0
+ 0
for +- 0 to N do:
x +- k-th training sample
V/ 91 (x) (train()))
1/2 5/2 U Ra)k(r)
end for
return spearman-rank(yy.n
17

---

Published in Transactions on Machine Learning Research (06/2024)
B
Intensity of Normalization
Diffusion-Retrar
(Guided Normalization)
Diffusion- Tracin
O C
Figure 11: Comparison of Normalization Intensity. This shows the attribution results and associated trade-offs from three distinct approaches (non-differentiating, guided, and no normalization) for 16 test samples on the leftmost.
What are the consequences of suppressing all norms through re-normalization techniques?
We note that the normalization tochnique in Diffusion-Retrac may involve an inherent tradc-off, it dimin- ishes the significance of certain critical timesteps by treating all norms equally. In an attempt to address this issue, we here explore the possibility of utilizing "Guided Normalization..
Guided Normalization. As our experiments suggest, the norm of a training sample is comprised of two major components: (1). natural norm that is fully dependent on the training sample's properties, and (2). dynamic norm that is highly dependent. on timestep; and norm what considered to be biasing the attribution. Guided Normalization an attempt to only remove such timestep-induced dynamic norms, while maintaining the natural norm informative for attribution.
We fix a timestep & (e.g- the highest norm-inducing timestep at that checkpoint), and compute the norm for every training sample at t. Since the effect of timestep is now controlled for all samples, this resulting variation in norms is sample
Hence we utilize this variance between samples to guide us in answering the question how much to normalize for each sample? One potential approach is to set a lower threshold X. And the training sample with the highest norm (norm,....) remains unchanged, while subsoquent samples are gradually normalized in proportion their respective norms. This normalization process continues until the sample with the lowest norm is adjusted to (A X norm Guided Normalization is transition in between Tear and TracIn, aiming at alleviating the dominant influence in Diffusion- -iffusion-TraaIn, while retaining a precise portion of the loss gradient norm according to each sample.
Improvements and Trade-off. The comparison attribution results shown in Figure I| demonstrates this perspective of transition, where the proponents become incrcasingly repetitive methods from left right (e.g. generally influential proponent in rows 2-4 for Diffusion- . And the outliers MNIST- zero samples with larger norms characterized influential to CIFAR-planes. Similarly, the trade-off
18

---

Published in Transactions on Machine Learning Research (06/2024)
associated with non-differentiating normalization in ReTrac is also evident in the attribution of MNIST- zeros, where visually similar CTFAR-planes are assigned as top proponents. It is also worth noting that in Diffusion-Tracln, if a training sample 2 has a timestep that falls in a large-norm inducing region (such as timesteps close to 1000 in 2igure then this sample often emerges as one the top proponents for most test samples the calculation. We observe that this could be the case even when z only has a large-norm timestep in one checkpoint, which means that this effect is strong enough to dominate the attribution results from previous checkpoints.
Overall, since did not observe immensely convincing improvement with guided normalization, ultimately opted for the ReTrac normalization approach as a simple mitigation for the overly dominant influence that the vanilla Diffusion-Tracli exhibits. However, further refinement of Guided Normalization and other strategies for optimal normalization may be an interesting direction to explore.
C Comparison with Influence Functions
We evaluate the performance of Juffusion-RReTrasr and Diffusion- Ierfusion-TracIn against the Influence Functions (Koh & Liang 2017), one of the best-known influence estimators. It considers how a model changes if training instance z; weight infinitesimally perturbed by e;.
I(2, Bwest) V Vo (2rest: 2t) H)"
(10)
More intuitively, this approach estimates the leave-one-out influence of 2, on the test sample. However, directly applying influence functions is computationally intensive as it requires inverting the Hessian, and is particularly impractical due to the sheer number of paramcters in diffusion models. To overcome this limitation, adopted the approximation method Linear time stochastic second-order algorithm (LiSSA) proposed in the work AAgawwa et al. (2017), which offers a more feasible approach for computing sample influences in large-scale diffusion models. Figure 13 shows that influence functions result in much lower precision for Image Tracing proposed in Section 6.1
1.0
1.000
0.925
0.900
0.4
0.850
Drfiainn-ee Per Tac DVflsien-rrainn
0.825
fonction Bandom
25
30
2000
3200
4000
5000
(a) Precision on MNIST zero samples
(b) Precision on CIFAR-plane test samples
Influence Functions
Influence Functions
0
6
Figure 13: Influence Functions Attribution. This shows the top 6 proponents retrieved for the test samples. There notable cases where CIFAR-planes attributed to MNIST-zeero and vice
19

---

Published in Transactions on Machine Learning Research (06/2024)
D Supplemental Visualizations
D.1 Artbench-2 and CIFAR-10 Datasets
We provide further visualizations comparing the altribution of Diflusion-ReTrac and Culfusion-Traclnn Fig- ure 14 shows results for 1). CelebFaces Attributes (CelebA) subset containing 100,000 images with 128 x
128 rosolution (Liu et al. 2015), and 2). CIFAR-10 Krizhevsky al. 2009) consisting of 50,000 training
samples with resolution 32 x 32.
Consistent with observations in the main text and data reported in Table ! Diffusion- TracIn retrieves a set of highly homogenous training samples that are generally influential, while ReTrac significantly addresses this issue dve-nnomn by certain samples. This enhancement is particularly notable in datasets like Artbench-2 and CelebA, which have less diverse training sample categories (i.e., two subclasses in Arthench-2 and exclusively human faces in CelebA). To be specific, among the top 10 influential samples for CelebA test samples, Diffusion-Tracke only results in 31.3% of unique images while ReTrac retrieves 95.6% of unique images, significantly outperforming in terms of largetedness of attribution.
Diffusion- Re Trac
Diffusion-Tracln
(a) CelebA. When the dataset is less diverse, Diffusion- TracIn retrieves many generally influential samples.
Diflusion-Tracln
Diffusion-Re ReTrae
Diffusion- Trachn
(b) CIFAR-10. Top test samples of the subclass "automobile, "deer" and bottom 2 of "frog".
Figure 14: Targeted Attribution. We attribute the 8 test samples (leftmost) using both Diffusion- Tracnnn and Diffusion-ReThas The top proponents are shown. The generally influential images that appear multiple times for different test samples are indicated in red. It is visually evident that Diffusion ReTrac provides more distinct attribution results.
20

---

Published in Transactions on Machine Learning Research (06/2024)
D.2 CIFAR- with High Self-influence
Self-influence used to identify outliers in the training dataset. While Diffusion-Tracln TracIn and ReTrac assign high self-influence most of the 200 MNIST samples, certain CIFAR-plane samples also received high self- influence scores and are ranked among the top 200. These plane samples tend to have dark backgrounds high contrast, which are also visually distinct from typical samples in the CIFAR-plane subclass (Figure 15
Figure 15: CIFAR-planes with High Self-Influence. These four samples are assigned high self-influence scores by both Diffusion-Tracln TracIn and ReTrac. They are visually distinct from the typical plane samples in the training dataset.
D.3 CIFAR-Planes Influential to MNIST Samples
The auxiliary task of Image Source Tracing pinpoints specific training samples that are responsible for the generalion of a test sample. For an MNIST zero, while most of the retrieved proponents are MNIST zeros, some planes are also assigned high influences (Figure 16). We noticed that these planes are visually distinct from the other, and visually resemble the MNIST samples. They tend to exhibit a black background and the planes are centered the middle, which highly resembles the layout of MNIST zeros. This further proves the effectiveness of Diffusion- Reeoone identifying highly influential samples.
63
89
100
111
156
157
173
174
190
195
199
200
204
206
208
209
Figure 16: CIFAR-planes Influential to MNIST Samples. These CIFAR-Planes assigned high influence scores to an MNIST zero test sample. The attribution results are descending order and the corresponding ranking for each sample is labeled.
21

---

Published in Transactions on Machine Learning Research (06/2024)
E Timestep Selection
E.1 Effectiveness of Timestep Sparse Sampling
To approxinate the expectation over timesteps in equation efficiently, linearly spaced timesteps over the denoising trajectory are used. Empirically, we demonstrate the effectiveness of this sampling approach by comparing the Diffusion-Tracln and ReTrac results obtained using our sparse sampling method against those obtained through the full expectation of all T timesteps.
Specifically, we evaluate the Spearman's Rank Correlation between the attribution results derived from the two approaches. To ensure the reliability of our findings, we report the average correlation over 16 test samples. It is observed that Spearman's rank correlation consistently exceeds 0.99 across various sparsity levels (i.e., using 50/100/200 evenly spaced timesteps), for both TracIn and ReTrac. This suggests that our approach, leveraging sparse sampling, yields attribution results that closely align with those derived from full expectation. It is also worth noticing that fixing one timesteps per checkpoint is unfaithful to the true altribution result, where the correlation drops to 0.73 for Diffusion-Re Trac and 0.72 for Cnfuusion-Trcllle
The correlation scores are organized in the table 3
Fixed one
50
100
200
Jiffusion-Trachn 0.720
0.995
0.998
0.999
CiffusiionReTTral 0.730
0.995
0.997
0.999
Table 3: Consistency to Full Expectation. Spearman's rank correlation between attribution results derived using sparse timesteps subsampling and full expectation over the entire diffusion trajectory.
Fixed one
50
100
200
Full
Diffusion-Tracln 0.807
0.838
0.842
0.843
0.842
Diffusion-ReTrass 0.940
0.949
0.950
0.949
0.950
Table 4: Performance on Outlier Model. Accuracy of Diffusion- Tracon and Re'Trac on 16 CIFAR-plane test samples, varying the number of expectation timesteps sampled.
We then evaluate the accuracy drop on the CIFAR-MNIST outlier model as in section 6.1 by accessing the number of CTFAR planes ranked among the top 200 influential when attributing a plane test sample. We observe that compared to full expectation, the performance drop is negligible when averaging more than 50 timesteps. Similarly, notable decline in accuracy is observed when employing only one timestep (either fixed at 500 or selected randomly) per checkpoint. The result is shown in table 4
E.2 Efficiency of Timestep Sparse Sampling
We then evaluate the efficiency improvement the timesteps subsampling approach compared to the full expectation. Utilizing subsample of 50 evenly spaced timesteps reduces the computational cost by approx- imately 20-fold, compared the full expectation operation. Overall, the findings show that our proposed sparse sampling approach significantly reduces computational time while maintaining high fidelity attribution results, offering a compelling alternative to full expectation methods.
Time (In seconds)
50
100
200
Full
Diflusion-Trraln
27.55
57.67
100.01
509.41
Riflusion-ReTrae
28.87
53.39
105.09
508.97
Table Computational Time. Measurement of computational efficiency for Diffusion- TracIn and Diffusion- ReTrac utilizing evenly spaced timesteps and full (1000) timesteps.
22

---

Published in Transactions on Machine Learning Research (06/2024)
F Implementation Details
F.1 Model Details
The diffusion models utilized the experiments are trained with Diffusion Denoising Implicit Model (DDIM) (Song et al. 2020) architecture, using 1,000 denoising timesteps and 50 inference steps. Stochastic gradient descent (SGD) and Adam optimizers used and tested for the attribution results. Additionally, training and experinentations are conducted on Nvidia A10G and RTX A6000 GPUs.
We note that our approach should remain consistent across variations of diffusion models, since the methods are designed based on the training process which is largely unaffected by differences in inference procodures. Furthermore, our methods may also be modified and derived to accommodate the variations in training, such as the optimizer used. Similar the original work presented in TracIn, the practical form of the attribution method expected remain the same across these variations (Pruthi al. 2020).
F.2 Checkpoint Selection
Influence estimation susceptible to the choice model checkpoints, particularly because model behavior can vary significantly over the training period. It is optimal to select checkpoints with consistent learning and steady decline in loss, which indicate reliable learning stages. Checkpoints that are too carly in the modcl's learning stage can exhibit fluctuating gradient. information due to initial instability, while checkpoints near model convergence offer limited insights into the attribution. Influence estimation at these early or late epochs of the learning process can introduce noisc and compromise the accuracy of attribution results.
Attribution methods that rely on loss gradient norm information are also particularly sensitive to checkpoint selection. We observe that certain samples may exhibit an unusually large norm at specific checkpoints. When these checkpoints are used in Diffusion-Trachn such samples emerge as generally influential with notably high influence on various test samples, overshadowing attribution results from previous checkpoints. This effect is mitigated in Diffusion-ReTrail due to re-normalization, reducing the method's susceptibility to dominating loss gradient norms.
23


Reviews:
[{"review_id": "Oo0fXVd3Fq", "reviewer": "Reviewer_WxYC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Well extension of TracIn for the diffusion model considering the training temporal dynamic of diffusion.\n2. This work provides comprehensive experiments to show the strong correlation between the timesteps and the magnitude of the gradient norm, indicating the necessity of an influence estimation method resilient to timestep-induced gradient norm bias for the diffusion model.\n3. Well written and interesting.\n\nWeaknesses:\n1. The proposed Diffusion-ReTrac implicitly assumes that influence estimation from each timestep contributes equally as the normalized gradients share the same gradient norm. However, for each training sample, its influence on the test sample may vary through each timestep. The gradient norm difference might be from 2 things: (1)  gradient norm bias mentioned in the paper and (2) samples contribute differently during training at different timesteps. Could you provide more evidence to support your choice?\n2. The outlier detection result that Diffusion-ReTrac performs worse than Diffusion-TracIn comes from the same reason mentioned in weakness 1.  The role of critical timestep with abnormal self influence value is weakened. However, if considering the more precise norm of each timestep (instead of the same norm),  this drawback above might be alleviated.\n3.  Lack of comparison between Diffusion-ReTrac and related methods, e.g. similar influence re-normalization/reweight manner.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes a new data attribution method Diffusion-ReTrac for the diffusion model, using a similar manner of TracIn but considering the influence estimation bias from different timesteps. The key observation is that the magnitude of the loss gradient norm is highly correlated with the timestep of the sample. While naive TracIn extension Diffusion-TracIn only includes the expectation over timesteps and noise $\\epsilon$ with the existence of diffusion temporal bias,  Diffusion-ReTrac addresses the dominating gradient norm effect by additional introducing normalization for gradient terms. The paper provides experimental results demonstrating the effectiveness of Diffusion-ReTrac on the diffusion model compared with TracIn naive extension.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. More experiments for answering questions mentioned in weakness part will be great, e.g. show whether the influence ranking of a sample is consistent through timesteps (Diffusion-ReTrac only includes 1 timestep) with the normalization manner of Diffusion-ReTrac.\n2. Additional experiments for more potential baselines.", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4tEWxcJniq", "reviewer": "Reviewer_Hnoh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* The attribution problem in the context of generative models is very timely and I believe it will be in the interest of the TMLR audience.\n* The experimental finding of the correlation between the norm of the loss gradient and the timestep at which a point appeared during training is interesting.\n* The authors do a great job in validating this experimental finding through a series of ablations.\n* The proposed objective is reasonable and seems to perform very well in the settings in which it was evaluated.\n\nWeaknesses:\n\n* The experimental evaluation is kind of limited. The authors only study toy datasets. This type of work would have a lot more impact if more realistic datasets were considered, such as LAION. \n* The method assumes not only access to the training dataset but also access to the training timesteps that were used for each example. This seems kind of limited. \n* The authors only compare to one baseline. Further, this baseline was proposed by the authors themselves. I am not very familiar with the literature, but aren't there more works to compare to? Or other natural baselines to consider?\n* I did not find any theoretical justification for the experimental finding regarding the correlation between the loss gradient norms and the training timesteps. Is this experimental finding universal? Where is it coming from?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors study the problem of data attribution in diffusion models. The main contribution of this paper is the experimental finding that the time-dependent training of diffusion models leads to increased loss gradient norms for training examples that were seen under high noise. This norm bias throttles the performance of the standard extension of the TracIn framework to the diffusion setting. To mitigate this, the authors propose a simple modification of the TracIn extension to the diffusion setting that re-normalizes the norms taking into account the training timesteps at which the examples appeared. This seems to lead to increased experimental performance.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I think it would strengthen significantly the submission if the authors:\n\n* Apply their method to more meaningful datasets, such as LAION.\n* Show data attribution for open-source, pre-trained models such as Stable Diffusion (XL).\n* Include more baselines.\n\nI would be further curious to see:\n\n* How this method performs for diffusion models that are trained with corrupted data, as in Ambient Diffusion. Training under corruption has been proposed as a way to mitigate memorization of training examples. It would be interesting to see whether the proposed attribution method works in this case and whether the phenomenon of increased gradient norms carries to this setting.\n* if the method works for latent diffusion models.", "broader_impact_concerns": "I do not have any broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nm1GfHoUbq", "reviewer": "Reviewer_D4pE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. The idea of generalizing data attribution methods to diffusion models for image source tracing is interesting.\n2. The observation of the time-induced gradient norm bias is insightful for other research, e.g, improving training stability.\n\nQuestions\n1. In equation (4), the definition of the summation range $k:z_k=z$ is not clear to me. Is it a condition to select checkpoints $w_k$? I wonder why not every checkpoint could be used in computing the latter gradient term in the summation? Is that some checkpoint has never been updated by the gradient computed with the training sample $z$?\n2. Since the timestep sparse sampling is an efficient approximation technique, it might lead to reduced or less consistent performance. I'm curious about how much quicker it is relative to Diffusion-TracIn. Additionally, I'd like to know the performance impact of omitting it from Diffusion-ReTrac.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents two attribution scores to estimate the influence of training samples on the loss of testing samples in diffusion models. The paper proposed to use expectation (Diffusion-TracIn) and gradient norm normalization (Diffusion-ReTrac) to address the timestep-induced norm bias observed when applying conventional data attribution score (TracIn) to the diffusion loss.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Since the timestep sparse sampling is an efficient approximation method, it might lead to reduced or less stable performance. I'm curious about the performance of omitting it from Diffusion-ReTrac. Additionally, I'm curious about how much quicker it is relative to Diffusion-TracIn.\n2. The TracIn with different fixed timestep would be a valid baseline in experiments.\n3. In section 4.2, before the itemized list, in the sentence “the presence of such timestep-induced norm by showing” should be “timestep-induced norm bias”\n4. In equation (4), the definition of the summation range $k:z_k=z$ should be clarified.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper tries to develop data attribution methods to interpret diffusion models. Different from traditional models, diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships as in previous contexts. This paper proposes Diffusion-TracIn to incorporate this temporal dynamics, and further proposes Diffusion-ReTra to overcome limitations of Diffusion-TracIn.

The reviewers mostly agree on the value of the proposed problem and solutions. They raised some questions mainly on experiments, limitations and some technical details. The rebuttal address all the problems, and the reviewers all feel positive of the paper after additional experiments and explanations in the rebuttal. I think the paper is a first exploration of data attribution in diffusion models, and is worth of publishing.

I request the authors to fully address the reviewer comments into the final revision. I see the authors have added multiple changes and extra experiments in the rebuttal, I would suggest the authors at least to mention them in the main paper and put details in the appendix.
Decision: Accept (poster)