Title: Distributional Training Data Attribution: What do Influence Functions Sample?
Venue: NeurIPS 2025
Accepted: yes
Keywords: influence functions

data attribution

training data attribution

unrolled differentiation

deep learning theory
Abstract: Randomness is an unavoidable part of training deep learning models, yet something that traditional training data attribution algorithms fail to rigorously account for. They ignore the fact that, due to stochasticity in the initialisation and batching, training on the same dataset can yield different models. In this paper, we address this shortcoming through introducing _distributional_ training data attribution (d-TDA), the goal of which is to predict how the distribution of model outputs (over training runs) depends upon the dataset. Intriguingly, we find that _influence functions_ (IFs), a popular data attribution tool, are 'secretly distributional': they emerge from our framework as the limit to unrolled differentiation, without requiring restrictive convexity assumptions. This provides a new perspective on the effectiveness of IFs in deep learning. We demonstrate the practical utility of d-TDA in experiments, including improving data pruning for vision transformers and identifying influential examples with diffusion models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new framework called distributional training data attribution for evaluating the influence of training data on machine learning models, particularly in the context of stochastic training methods.\nThe authors argue that traditional training data attribution methods are insufficient because they do not account for the randomness inherent in deep learning model training, which arises from factors like random weight initialization and stochastic batching. \nThis randomness means training on the same dataset can produce different models.\nThe core idea of d-TDA is to shift the focus from predicting a single, deterministic change in model output to predicting how the entire distribution of model outputs changes when a training example is removed or altered. \nThis approach allows for a more comprehensive understanding of influence, capturing effects on not just the mean outcome but also its variance or other distributional properties.\nKey contributions:\n1. The paper formally defines the problem of data attribution in stochastic settings as predicting changes in the distribution of model weights or outputs. \nThis new concept quantifies a training point's importance based on how its removal affects the output distribution, using metrics like the Wasserstein distance. \nThe paper demonstrates that this can identify influential examples missed by traditional mean-based methods.\n2. The paper shows how unrolled differentiation can be used to estimate changes in the output distribution without the need for expensive retraining.\n3. The paper rigorously shows that classical influence functions, a popular but poorly understood tool in deep learning, naturally emerge from the d-TDA framework under certain assumptions. \nIt proves that IFs can be seen as a long-training-time limit of unrolled differentiation and as an optimal way to map between Boltzmann distributions representing the model before and after data removal. \nThis provides a new theoretical grounding for their effectiveness in non-convex settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The primary strength of this paper is its introduction of d-TDA. This is a fundamentally new and important way to think about data attribution that directly confronts the stochasticity inherent in modern machine learning. \nInstead of treating randomness as a nuisance to be averaged away, the authors reframe it as a central object of study. \nThis conceptual shift from predicting a deterministic change to predicting a change in the distribution of model outputs is a significant and timely contribution to the field.\n\n2. The d-TDA framework allows for a much richer understanding of influence. \nA key finding is that some data points have a minimal effect on the mean of a model's output but can drastically alter its variance or other distributional properties. \nThis is demonstrated effectively in a 1D regression task where the example most influential by Wasserstein shift was different from the one most influential by mean shift, and its removal sharply increased prediction uncertainty. \nThis ability to identify variance-inducing examples is a powerful new tool for debugging models, improving their stability, and understanding generalization. \n\n3. A major contribution is providing a new, rigorous mathematical motivation for the surprising effectiveness of influence functions in deep learning. \nThe authors show from two complementary perspectives that IFs emerge naturally from the distributional framework. \nThese derivations are significant because they do not require the restrictive convexity assumptions of traditional IF theory, making them far more applicable to non-convex deep learning models. \n\nWeaknesses:\n1. The paper claims that its framework, particularly the use of IFs, unlocks d-TDA for large-scale models like diffusion and language models. However, the experiments presented are on smaller-scale problems (UCI datasets, MNIST with an MLP). \nWhile the efficient forward-mode computation of the unrolled response is a notable contribution to efficiency, the paper does not empirically demonstrate that the full d-TDA approach is computationally feasible or provides meaningful insights on the massive models where attribution methods are arguably most needed. \n\n2. The theoretical results, particularly Theorem 2 regarding the convergence to IFs, rely on a set of assumptions (A1-A6). \nWhile the authors argue these are less restrictive than prior work, their applicability to all practical deep learning scenarios could be questioned. \nFor example, A6 assumes that gradient flow trajectories converge \"sufficiently fast\" to local minima. \nThe extent to which the complex, high-dimensional, and often chaotic dynamics of large neural network training satisfy these conditions remains an open question. \nThe paper acknowledges and discusses these assumptions, which is good practice, but they remain a limitation on the proven generality of the claims. \n\n3. The paper presents two distinct mathematical motivations for IFs, one based on the limit of SGD dynamics and the other on transport maps between Boltzmann distributions. \nWhile both are elegant and converge on the same result, the paper does not explicitly discuss the relationship between them. \nAre they simply parallel justifications, or is there a deeper connection between the long-term behavior of SGD and the properties of the Boltzmann distribution that this work implicitly relies on? \nClarifying this would strengthen the theoretical narrative.\n\nMissing citations:\n1. The journey, not the destination: How data guides diffusion models. Arxiv 2024\n2. Intriguing Properties of Data Attribution on Diffusion Models. ICLR 2024\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the idea of distributional data influence (d-TDA), which characterizes the potential distribution of the result model when removing some training samples from a stochastic training algorithm. The authors present an efficient d-TDA method based on unrolled differentiation and evaluate its ability in estimating the ground truth distribution change. Finally, the paper presents a lot of empirical analysis of the estimation results and provide a rigorous analysis on how the well-known influence function naturally arose in this setting.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. The paper generalizes training-data attribution from single-model effects to shifts in the entire distribution of models induced by training randomness, which is a natural and interesting setting.\n2. The exposition is clear and technically rigorous, with some insightful theoretical analysis.\n\n**Weakness:**\n1. Lack of numerical results: Most experiments are conducted only on toy regression tasks, and the biggest dataset/model tested is on the MNIST dataset. This raises question on how practical is the current methodology to the current large machine learning models.\n2. The paper currently does not make any experimental comparisons of the distributional method to standard (non-distributional) attribution (perhaps some downstream tasks such as dataset pruning or noisy-label detection?), so the claimed advantage remains speculative. If the new distributional setting offers little improvement while significantly increasing computational cost, its practical value remains doubtful despite its theoretical appeal.\n\nQUESTIONS:\nSee weakness and some more questions:\n\n1. In line 150, typically how many trained models are needed to accurate estimate the overall distribution? I would like to clearly understand the estimator's cost.\n2. The current unrolled differentiation framework assumes SGD training, will the faithfulness of the result be severely damaged if trained with other optimizers such as AdamW or Muon?\n\nThis is a very interesting work, but my main concern still revolves around the practical usability of this framework comparing to the traditional non-distributional ones. If the author could add some experiments or comparisons to better illustrate this point I am happy to raise my score.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the problem of training data attribution (TDA), often also called \"influence estimation.\" The goal of TDA is to estimate the impact of removing a training datapoint on the model parameters or some function of the model parameters. The paper notes that previous methods have ignored or glossed over the randomness present in training modern machine learning models via the random initialization and batch ordering. In particular, existing TDA methods focus on providing a point estimate that estimates the influence for a datapoint given a particular random seed. The paper proposes to instead compute *distributional* TDA (d-TDA or distributional influence), which computes the influence under all possible random seeds. In synthetic experiments, the paper shows that distributional influence can provide different answers than does regular influence. The paper then proves that influence functions (the typical method for estimating regular influence) are in fact a limit of distributional influence in that they approximate the d-TDA distribution.\n\nSTRENGTHS AND WEAKNESSES:\nI found this to be a technically sound paper that computes an interesting quantity: the distributional influence of a datapoint. As far as I'm aware, this is a novel contribution that hasn't been considered before. And the paper successfully demonstrates an example where the use of distributional influence results in different estimates than the use of regular influence, which is an important step for this type of algorithm being practically useful.\n\n**Overall motivation**\n\nOne of the biggest issues I see with the paper is that it's unclear why one would want to compute distributional influence. As far as I'm aware, one is generally interested in influence of a datapoint over the finished end model so that, e.g., one can wonder about whether a datapoint is negatively contributing to its performance (for the purposes of re-running the training) or if a datapoint was valuable to its performance (for the purposes of compensating data providers). I don't see a clear argument for anyone being interested in knowing how valuable a datapoint was for a hypothetical model with different number of layers, order of layers, activation functions, number of training epochs, or other hyperparameters. And I see random seed as just another hyperparameter that defines the particular model one gets after training. So why is it of interest to get results under some distribution of this hyperparameter? I don't think the paper currently makes this case, but instead points out that distributional influence is different, and then concludes that this means that it is meaningful. For example:\n\n- Figure 3 shows that regular influence functions capture different information than d-TDA. The paper concludes that \"This showcases that d-TDA captures meaningful information about the training data missed by previous notions of influence.\" I agree Fig 3 shows it captures different information, but I don't see why Fig 3 shows this information is *meaningful* for any tasks.\n- \"The ability to predict how different training examples impact the distribution over training runs may be practically useful. For example, one could identify examples to remove to reduce variance, improving model repeatability over training runs.\" Why is this practically meaningful? At first thought, variance over training runs sounds like a good thing because it allows us to find more local optima, some of which may be better than our initial training run. So why should a practitioner care if they can consistently get to the same local optimum using different random seeds? They already know how to get to that local optimum.\n\nI also have two other points about the motivation of the paper:\n\n- I'm not sure exactly what the takeaways from Theorem 3 are. I would say that Thm 3 shows that, for Boltzmann distributions, IFs are optimal to use. But why do I care about Boltzmann distributions? How does this relate to a user thinking of using IFs for some model in practice?\n- I can't tell what the paper's position on IFs is. Earlier in the paper, it seems to be that they're not good because they're not measuring distirbutional influence. But then, in the final paragraph before the conclusion, the paper seems to turn around and say that the whole paper motivates the use of IFs. So is the paper providing anything besides suggesting the use of IFs? Maybe its algorithmic contribution is to use unrolled differentiation, but I believe previous work has covered this (see Luo et al. (2023)).\n\n**Related work**\n\nThe work of Luo et al. (2023) is highly related to this paper and its discussion of unrolled differentiation. Luo et al. (2023) study the approximation of LOO in models fit with SGD, and essentially use similar unrolled differentiation techniques as proposed in Eq (6). So I think the quote from line 164 of the current paper, \"To the best of our knowledge, this is the first application of such techniques\" should be updated.\n\nThe paper notes that previous works like [20] argued that IFs are not good for predicting leave-one-out retraining for deep learning models. The paper claims that this is incorrect, and LOO is in fact just noisy, rather than IFs being bad. Is the claim here that d-TDA applied to the situations in [20] would work well? I think this is a major empirical claim that would be pretty impactful if true. But big claims need big evidence, and there's no discussion or empirical evidence backing this up; it feels like investigating this claim could be an entire paper on its own. I think this paragraph either needs to be removed or needs significant expansion.\n\n\n**Unclear things**\n\nI was unclear about what some of the experiments were trying to show, as well as found a few other places where things were unclear; I've detailed the issues below:\n\n- The algorithm proposed to compute d-TDA is on lines 150-155. I'm not clear on what the algorithm is. The key Step 3 says \"using these two sets of (correlated) samples...\", but what are the two sets? It seems like only step 2 describes a set of samples.\n- Figure 2 is supposed to be an initial empirical explanation of d-TDA, but I wasn't clear what Figure 2 is displaying. First, what is \"query sample\" in the legend? Are the models one-dimensional here? Because it seems like the leftmost figure is supposed to be plotting models. Next, the caption states that the middle figure shows that the exact retraining samples are closer to predicted samples than samples from the original model. But the center figure seems to only be showing two datapoints per color here. To show thing $a$ is closer to thing $b$ than is thing $c$, I would think we need three points on the plot ($a,b,c$) for each color. Finally, on the right, I'm not sure what each of the terms in the legend means.\n- Figure 4 also uses the term \"query point,\" and I'm not sure what it means here either. I tried searching the text for the word \"query\", but it's only used once: in the caption of Fig 2.\n- Figure 5 (top) claims to show measurements predicted by unrolled differentiation versus changes predicted by IFs. But I only see one set of lines here. Are the dashed lines supposed to be one unrolled differentiation or IFs? But then why do only some of the lines have corresponding dashed lines? Also why is the top line so thick?  Finally, I see only three lines, but there are five lines listed in the legend.\n- Figure 5 bottom looks like it has a bunch of things overlapping. Are these supposed to correspond to the different learning rates?\n- Sometimes $k$ seems to index datapoints and sometimes $i$ seems to index datapoints.\n\n\n**Theoretical development**\n\nI thought there were a few small issues with the theoretical development:\n\n- Assumption A1 says that the loss function gradients $\\nabla \\ell_k$ are \"Lipschitz continuous and bounded\". Bounded for each $k$? Or uniformly? Also, is it true for neural networks that their Hessian is bounded and Lipschitz?\n- \"If the [model] weights converge to a saddle point ... the final parameters will become sensitive to any infinitesimal perturbation of the loss function.\" I'm pretty sure this isn't true. Consider the loss $L(\\theta) = \\sum_{n=1}^N (x_n - \\theta)^3$. If $x_1 = \\dots = x_N = 0$, then there is a saddle point at $\\theta = 0$, and this is still a saddle point even after you remove any given $x_n$.\n- I didn't see any motivation for assumptions A4 and A5. Should we expect these to hold up in cases where we're typically using SGD? I see the discussion in 291-298 describes why they are useful for the proof, but why are they reasonable?\n- Duplicate assumption numbers are used -- there are two A1, A2, A3, A4, and A5's in the paper.\n- The second A2 states \"the nonzero eigenvalues of the Hessian are uniformly bounded away from zero.\" How could this be violated? The Hessian is of finite dimension, so the minimum non-zero eigenvalue is bigger than zero.\n- Theorem 3 uses the term \"minimum manifold,\" but I don't think this was defined.\n\n\n**References**\n\nYuetian Luo, Zhimei Ren, and Rina Barber. Iterative Approximate Cross-Validation. ICML. 2023\n\nQUESTIONS:\nMy main questions are:\n1. When is it practically meaningful to compute distributional influence?\n2. What is the paper's position on the use of IFs?\n3. How is the paper proposing to compute IFs? If the answer is the algorithm on lines 150-155, what exactly is going on in this algorithm?\n\nThank you!\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Distributional Training Data Attribution (d-TDA), a new framework for attributing model behavior to training examples under stochastic training dynamics. Unlike classical methods like influence functions that assume deterministic training and focus on mean effects, d-TDA models the full distribution over model outputs induced by random initialization and batch sampling. The authors define distributional influence as the change in output distribution (e.g., mean, variance, Wasserstein distance) when a training point is removed. They propose an efficient approximation using unrolled differentiation, avoiding retraining by computing a first-order response that estimates distributional shifts. The paper also provides new mathematical justifications for influence functions, showing they arise as limiting cases of d-TDA under certain conditions. Empirical results demonstrate that d-TDA captures richer signals—such as uncertainty changes—that traditional methods miss. This work redefines data attribution by treating randomness as a feature, not a nuisance, and opens up scalable and principled analyses for deep learning models.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths and Weakness\nStrength:\n+ highly theoretically interesting\n+ theoretically sound\n+ new insights into many aspects of influence function\n\n\nWeakness\n- the IF calculation to become even more computationally expensive\n- it is not clear how could we sample seeds for *actual* distribution, which leads to concern whether the ground-truth distribution is achievable\n- little evidence to show the distribution is really necessary for data attribution in practice\n\nQUESTIONS:\nQ1: Do you think the actual distribution of influence function is achievable in practice? \n\nQ2: Whether you think the estimation of $\\theta^*(D\\z_k)$ is always necessary, since we sometimes just need to estimate the training samples contribution to a specific prediction on a testing sample. For example, I am aware that paper which bypass such estimation for empirical convenience, which can locate the real problems in public data sets.\n\n> Debugging and Explaining Metric Learning Approaches: An Influence Function Based Perspective (NeurIPS 2022)\n\nIn their approach, they just find a $\\theta$ where a training sample and a testing sample are mostly likely to be correlated. \n\nQ3: I like your interpretation on the leave-one-out training, but what is the useful measurement for real-world experiment of data attribution?\n\nQ4: In practice, how your d-TDA can be significantly different from classical TDA to identify problematic training data samples?\n\nLIMITATIONS:\nIt is overall a wonderful paper with solid theoretical contribution and insights. However, it seems to be limited in answering how this work can empirically support real-world application.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}