Capturing the Temporal Dependence of Training Data Influence
Authors: Jiachen T. Wang
Dawn Song
James Zou
Prateek Mittal
Ruoxi Jia
Venue: ICLR 2025 Oral (2025)
Keywords: data attribution
Primary area: interpretability and explainable AI
TL;DR: We introduce data value embedding, a novel framework for real-time data attribution technique that approximates trajectory-specific leave-one-out (LOO) error.

Abstract: Traditional data influence estimation methods, like influence function, assume that learning algorithms are permutation-invariant with respect to training data. However, modern training paradigms—especially for foundation models using stochastic algorithms and non-convergent, multi-stage curricula—are sensitive to data ordering, thus violating this assumption. This mismatch renders influence functions inadequate for answering some critical questions in current machine learning: How can we differentiate the influence of the same data contributing at different stages of training? More generally, how can we capture the dependence of data influence on the optimization trajectory during training? To address this gap, we formalize the concept of \emph{trajectory-specific leave-one-out (LOO) influence}, which quantifies the impact of removing a data point from a specific iteration during training, accounting for the exact sequence of data encountered and the model's optimization trajectory. However, exactly evaluating the trajectory-specific LOO presents a significant computational challenge. To address this, we propose \emph{data value embedding}, a novel technique enabling efficient approximation of trajectory-specific LOO. Specifically, we compute a training data embedding that encapsulates the cumulative interactions between data and the evolving model parameters. The LOO can then be efficiently approximated through a simple dot-product between the data value embedding and the gradient of the given test data. As data value embedding captures training data ordering, it offers valuable insights into model training dynamics. In particular, we uncover distinct phases of data influence, revealing that data points in the early and late stages of training exert a greater impact on the final model. These insights translate into actionable strategies for managing the computational overhead of data selection by strategically timing the selection process, potentially opening new avenues in data curation research.

Published as a conference paper at ICLR 2025
CAPTURING THE TEMPORAL DEPENDENCE OF TRAIN- ING DATA INFLUENCE
Jiachen T. Wang" Princeton University
Dawn Song UC Berkeley
James Zou Stanford University
Prateek Mittal Princeton University
Ruoxi Jia Virginia Tech
ABSTRACT
Traditional data influence estimation methods, like influence function, assume that learning algorithms are permutation- permutation-invariant with respect training data. However, modern training paradigms, especially for foundation models using stochastic algorithms and multi-stage curricula, are sensitive to data ordering, thus violating this assumption. This mismatch renders influence functions inadequate for answering a critical question in machine learning: How can we capture the dependence of data influence on the optimization trajectory during training? To address this gap. formalize the concept of trajectory-speciie leave-one-out (LOO) influence, which quantifies the impact of removing data point from a specific iteration during training, accounting for the exact sequence of data encountered and the model's optimization trajectory. However, exactly evaluating the trajectory-specifie LOO presents a significant computational challenge. To address this, propose data value embedding, novel technique enabling efficient approximation of trajectory speccffc LOO. Specifically, we compute a training data embedding that encapsulates the cumulative interactions between data and the evolving model parameters. The LOO can then be efficiently approximated through simple dot-product between the data value embedding and the gradient of the given test data. As data value embedding captures training data ordering, it offers valuable insights into model training dynamies. In particular, phases of data influence, revealing that data points in the early and late stages of training exert greater impact on the final model. These insights translate into actionable strategies for managing the computational overhead of data selection by strategically timing the selection process, potentially opening new avenues data curation rescarch[
uncover distinct
INTRODUCTION
Data influence estimation aims provide insights into the impact of specific data points on the model's 's predictive behaviors. Such understanding is crucial not only for model transparency and accountability (Koh & Liang] 2017) but also plays a significant role in addressing Al copyright debates (Deng & Ma] [023] Wang et al., 2024) and facilitating fair compensation in data marketplaces (Tian al.] 2022). The majority of data influence estimation techniques focus on measuring the counterfactual impact of training data point: how would the model's behavior change if we removed a specific training data point?
LOO Influence. This counterfactual impact is often characterized by the Leave-One-Out (LOO) influence, which has a long history and is frequently utilized in various fields such as robust statistics (Cook & Weisberg 1980), generalization analysis (Bousquet & Elisseeff 2002), and differential privacy (Dwork et al. (0066. Inheriting from this rich classical literature across various domains, the LOO influence in data influence studies is typically defined as LOO(2"; 2 ((el)) ;= ((A(D), (val))
"Correspondence to Jiachen T. Wang and Ruoxi Jia (cianheovang@@rriccttooo ruoxi j aa edu).

---

Published as a conference paper at ICLR 2025
2000)), ((A(D) \. (2)))),(0a))) ie., the model's loss change on a validation data (1aa)) when the training data point 2" is removed from the training set D. Here, A is the learning algorithm. For ease of analysis, traditional literature usually assumes that the learning algorithm A is permutation-invante with respect to the training set DD meaning that the order of data points does not affect the learning outcome (Bousquet & Ellissefff (002). This assumption holds for models with strongly convex loss functions trained to converge. Within this framework, researchers have developed efficient methods to approximate LOO. Influence function (Koh & Liang 2017), which uses first-order Taylor expansion to estimate the LOO, emerging as the most prominent approach. Numerous follow-up works have further improved its scalability for large models and datasets (Guo al. 2021 Schioppa et al.) 2022]
Grosse et al. 2023
Choe et al. 2024).
However, modern training algorithms, particularly those used for foundation models, increasingly deviate from the permutation-invarian assumption. This deviation arises from both the non-convex nature of neural networks and the multi-stage training curricula that do not run to convergence. In particular, due to the immense size of datasets, large language models (LLMs) often undergo just one training epoch, meaning each data point is encountered only once during training. Consequently, training data order significantly shapes the influence of data points on the final model (Epifano) eet al. 2023 Nguyen et al. 2024). Due to their underlying assumption of permutation-invant the order-dependence of data influence in modern training paradigms is not accurately reflected by influence functions. For example, they assign identical influence scores to duplicate training points, regardless of their position in the training sequence.
Therefore, in this work, we argue that designing a data influence estimation technique relevant to the modern ML context requires rethinking how the counterfactual impact should be defined. Towards that end, we formalize the concept of trajectory-specift LOO, which characterizes the loss change resulting from removing a data point from the specific iteration it used during training. In contrast to the traditional LOO, trajectory -ppeccffc LOO explicitly accounts for the exact sequence of data encountered, considering the timing of target training point being trained on. An accurate evaluation of
LOO would enable us to answer many important questions that are impossible to address with influence functions. For instance, how does a data point's impact vary depending on its entry timing in the training process? How do later points affect the influence of carlier points?
However, exactly evaluating the trajectory-specific LOO presents a significant computational chal- lenge. To address this, introduce data value embedding. a novel data influence estimation framework designed for approximating properties at the same time: (1) accounting for training dynamics and reflecting how the data order impacts model training: (2) scale efficiently to the setting of foundation models, and is faster than the current most efficient implementation of influence function; (3) enable real-time attribution for any query without necessitating model retraining or prior access to validation data.
sejjectory-speciffi LOO. Our approach achieves several nice
Technical novelty. Our proposed data value embedding framework computes compact representa- tion for each data point that encapsulates the cumulative effect of subsequent training. The influence scores for any test instance can be approximated with a simple dot product operation between the test gradient and the data value embedding, enabling real-time computation of data influence scores. To improve the scalability of computing data influence embedding, we develop a suite of techniques for efficient computation and storage of data value embeddings. In particular, we introduce the inftuence checkpointing technique, which enables the parallel computation of data value embeddings at multiple checkpoints. This not only enhances computational efficiency but also allows tracking of how a fixed data point's value changes during the training process.
Empirical insights. Through data value embedding, we obtain several novel empirical insights into the training dynamics of foundation models. We identified three distinct regimes of data influence (Figure I (a)): very brief high-influence region at the start, much longer low-influence basin, and a region in the later training stage with gradually increasing influence, resuming to a high level. We show that performing online data selection solely in the early and late high-influence regions (less than half of the training duration) can achieve performance improvements on par with selecting data throughout the entire process (Figure (b)). Moreover, performing data selection (Fan et al., 2024) only in the first very brief high-influence region, lasting less than 4% of the training duration, can achieve 50% of the performance gain enabled by continuous selection. Since online data selection usually incurs significant computational costs, our findings suggest viable way of managing this overhead by strategically timing the selection process. By focusing data
2

---

Published as a conference paper at ICLR 2025
selection efforts on these critical phases, substantially improve training efficiency without compromising model performance. These temporal insights can potentially embark on new avenues of research on budget-limited data curation.
Data Influence Score of Data Points from Different Traning Iterations
25
Perplexity Reduction Compared Ne Selection
Pyntis-410M GPT2 Small
Low-Impact
Gradual Ascending
Data Influence Score
0.125
Basin
2.0
Parplexity Reductio
172g
1.74
0.100
1.5
1.55
1.62
1.21
0.075
1.06
1.0
0.96
0.050
0.025
High-Impact
0.000
Wormup Phose
0.000 0D
10000
20000
30000
40000
50000
No Selection Threughout
Earty Phone
Lete Phose
Early
Training Batch Idx
Teaiinn Stages that Perform Online Data Selechor
Figure 1: (a) Average data influence scores computed from data value embedding per training batch, measured against the final model's loss on Pile's validation set. Setting: Pythia-410M trained on 1 1% of Pile. (b) Comparison of online data selection strategies for training Pythia-410M, on Pile. All strategies use gradient cosine similarity to Pile's 's validation set to select high-quality training batches (Fan et al. 2024), and only differ in the training stages during which advanced batch selection is applied (random selection otherwise).
TRAJECTORY-SPECITE LEAVE-ONE-OUT INFLUENCE
In this section, we formalize the definition of trajectory-specity LOO which was originally introduced in Hara al. (019) 'SGID-inftuence" Consider data point =" that is included in the training process during the 1, iteration. Let B3 denote the training batch at iteration f. In standard SGD, the model parameters are updated as 0c+1 = 0g - ni Σ.EB, Vl(O,, 2) for t 0, ....., . 1, where ne is the learning rate at iteration I. We are interested in the change in the validation loss {(Or, z =(val)) when the data point 2 E B,, removed from iteration t.. In this counterfactual scenario. the parameter updates proceed as 0% = Gr, Th, ExeB, Mx*) VE(O,,, 2) and 0/ = 0g EEB V((0), for l. + 1, ,,... 1.
Definition 1 (Trajectory-Specift LOO (Hara et al. 2019)). The trajectory-specins leave-one-out for data point z" at iteration la with respect to validation point z(val) is defined as
TSLOO(t) ((1;a))))) E(0r,
2(aa))
Discussion. TSLOO quantifies the change in validation loss resulting from removing =" during the specific training run determined by the sequence of mini-batches and random initialization. TSLOO explicitly depends on the timing of when the data used and models the interaction effects between data points. For instance, it can show how the introduction of a certain type of example (e.g.. a challenging edge case) might amplify or diminish the influence of previously seen, related examples. Moreover, identical data points contributing at different stages of training can receive different value scores. A data point introduced early in training might have a significantly different impact compared to the same point introduced later, as the model state evolves. However, traditional methods like influence functions do not capture these temporal dynamics. The influence function is defined as VoE(f, 2²) where Hg the Hessian with respect to the full training loss. Because IF depends solely on the final state of the model, it invariably assigns the same influence value to identical z"s, regardless of their position in the training sequence.
[F (ma)) ;= (e, 2 2vaa))
Related works (extended version in Appendix A). Data attribution methods primarily fall into two categories: LOO-based methods and Shapley value-based methods. While Shapley value-based methods (Ghorbani & Zou, 2019) offer elegant theoretical interpretation, they typically require expensive model retraining, which limits their practical applicability. As a result, LOO-based methods such influence functions (Koh & Liang, 2017) have gained more attention due their computational efficiency. However, many studies have demonstrated that influence functions can be
highly unreliable when applied to deep learning models (Basu et al. 2020 Bae al. 2022 Epifano
3

---

Published as a conference paper at ICLR 2025
et al. 2023). In this work, argue that TSLOO provides more appropriate attribution framework
for deep learning, particularly in the context of foundation models. Various research communities
have independently explored Taylor expansion boadse technique (Section 3.1) for approximating
TSLOO for different purposes (Hara et al., 2019 Zou et al. 2021: Evron et al. [022]] Wu et al.) 2022
Luo al., 2023 Wu et al. 2024/Ding
2024). However, practical adoption has been hindered
by computational demands. This work proposes new method that overcomes the computational bottlenecks in approximating TSLOO for large-scale models.
DATA VALUE EMBEDDING
While trajectory-specifei LOO offers clear benefits for understanding data influence in modern ML, its computation presents significant challenges. Exact computation is not feasible, as it would require removing data point from specific training iteration and re-initiating the entire training process. To address this challenge, we introduce the concept of data value embedding.
3.1 PRELIMINARY: UNROLLING THE EFFECT OF A TRAINING DATA POINT IN SGD
Recall that we denote the final model as OT and the counterfactual model as 8y. which is ob- tained by removing 2" from t--th training iteration. We introduce interpolation between Or and T by defining +1 (E) = Or, Th, ELEB,, \(=*) VE(O,,, =) - T/, (1 e)Ve =*) and (c41(e) = 0s(e) 77 EzeB, V((O=(E), 2) for subsequent iterations. Note that 0r(0) Or and (1) 8,. Analogous to influence function-based approaches, approximate the change in validation loss using a first-order Taylor expansion around E 0: 2(va)) E(Or, (ea)) Ff(00,z(vatl)) (0)
Interestingly, the derivative (0) satisfies recursive relation
de
de s=0
detailed in Appendix C.I and we can obtain a well-established approximation from the literature:
T-1 II Jer,+1
E(67 (val)) - e(Or, 2 (va))) = ne. Ve(Or, 2((l)) T
(I - me H&)
Ve(Oe,
(1)
where H3 = ELER, (2e(On =) is the Hessian and is the identity matrix. In data attribution literature, this approximation in first appears in Hara et al. (2019) and has also been utilized in Chen et al. (2021) and Bae et al. 2024). It estimates the influence of removing 2" from the t- iteration on the validation loss (Ør, 2 (val)) at the final iteration. The product term IIk= +1 (I encapsulates the cumulative effect of the original data point's removal as it propagates through the entire training process. Notably, similar product terms appear frequently in related domains, including continual learning and deep learning theory (Zou et al. 2021 Evron al. 2022 Wu et al. 2022
H₂))
2024 Ding et al., 2024).
3.2 DATA VALUE EMBEDDING
Building on ( we extract the test-data-independent components and define "data value embedding" for training point 2" Br, as
T-1 II (I- (1-mHH) Ve(0t,,")) Lkt,++
(VEmb(ta) = 1T.
This embedding encapsulates the cumulative effect of a training point across the entire learning trajectory. By precomputing and storing these data value embeddings during after the training phase, we enable highly efficient computation of data influence scores. Specifically, for any given test point ((va)), the influence a training point 2 can be quickly determined by simply computing the dot product V 2 (eal)) (t) (x*). Vector dot products are among the most computationally efficient operations, especially when executed modern GPU hardware, which is optimized for such parallelized vector operations. Precomputing the data value embeddings eliminates the need for costly retraining or the availability of test data in advance, making the computation of data influence nearly instantancous. This is particularly advantageous in real-world scenarios such as data marketplaces, where rapid, on-demand data attribution is critical.
4

---

Published as a conference paper at ICLR 2025
nescale subsequent troining ot
(
Inf. batch
of from batch
1=0
t=1
Figure 2: An illustrative example of data value embedding for a 2-step training. The influence of a training point z" on a test point ((va)) can be obtained by projecting its data value embedding on (maa))ss gradient vector at the final checkpoint.
Approximation Error Bound. In Appendix C.2 we derive new theoretical analysis the approxi- mation eтror associated with the unrolled differentiation estimator for non-convex loss functions. We demonstrate that when the learning rate schedule satisfies 7) O(1/vt) with the maximum learning rate scaling as ((1/VT)- common choice in the literature (Vaswani 2017 the approximation remains uniformly bounded and is independent of the total number of training steps T. While the proof relies on certain assumptions to abstract the complexities of real-world implementation, the theoretical result still implies the method's applicability in practical model training.
EFFICIENT COMPUTATION AND STORAGE OF DATA VALUE EMBEDDING
While the data value embedding approach offers a promising solution for real-time data attribution that incorporates training-specific factors, its practical implementation faces significant computational and storage challenges. The computation of DVEmb non-trivial, requiring per-sample gradient calculations and per-step Hessian computations. Moreover, each DVEmb,(2* has the same dimen- sionality as the model parameters, making infeasible to store individual embeddings for each training data point on the disk. To address these challenges, we develop series of techniques that significantly enhance both the computational and storage efficiency of data value embedding.
4.1 RECURSIVE APPROXIMATION OF DATA VALUE EMBEDDING VIA GENERALIZED GAUSS-NEWTON MATRIX
We show that data value embedding can be computed recursively, beginning from the final training iteration and working backward, when using the Generalized Gauss-Newton (GGN) approximation for the Hessian matrix. This naturally gives rise a backward computation algorithm for (VEmb
A widely-adopted approximation for the Hessian matrix Hę is the Generalized approximation H, ΣxEB, Ve(O,, 2) VE(O,, 2) particularly in the context of cross-entropy loss (Martens 2020). The GGN approximation is extensively used in various machine learning algorithms because captures the essential curvature information of the loss landscape while remaining com- putationally feasible. For further details, see Appendix C.4 Under this approximation to H4, the following shows that we can compute DVEmb (z)) for any zt E B., if the data value embeddings of data points from later training iterations (i.c., DVEmb (2)) for t 2 + 1) is available.
(GGN)
Theorem 2. Given generalized Gauss-Newton approximation H, R Σ=es, Ve(O,, VE =) we have
(t) T/t, (((0,,,²*) nt.
Σ
Σ LEEB
Ve(O,, 2*)) DVEmb(t)
The proof is deferred Appendix C.3
5

---

Published as a conference paper at ICLR 2025
Interpretation. Theorem provides crucial insights into the interactions between training data points throughout the model training process. When two points x and 2 are similar, their gradient similarity term Vl(0,, 2) =E((0t,=*') increases, indicating stronger interaction between these points. To illustrate this phenomenon, consider training a language model where an early data point =" contains content about "quantum computing". The influence of =" the final model varies depending on the subsequent training data: if multiple similar "quantum computing" data points appear in later iterations, z influence on the final model diminishes, as these later examples could teach similar concepts to the model. Conversely, if 2 remains one of the few "quantum computing" examples throughout training, it maintains a stronger influence on the final model.
Overview of the remaining sections. Theorem 2 suggests the possibility of a backpropagation algorithm for computing data value embeddings, contingent on the availability of per-sample gradient vectors for all training data. To make this approach practical for large-scale applications, we address two key challenges the following sections: (1) Efficient computation and storage of per-sample
gradient vectors for all training data (Section 4.2) (2) Efficient computation (Sections 4.3) and
parallelization (Section 4.4) of data value embeddings using Theorem 2 Additionally, we discuss practical extensions and considerations for real-world scenarios (Appendix C.10p.
4.2 STEP 1: STORE PER-SAMPLE TRAINING GRADIENT INFORMATION AT EACH ITERATION
During model training, we additionally store the per-sample gradient for each data point in the training batch. However, this approach presents significant computational and storage challenges: (1) Storage: Let denote the number of model parameters. Each gradient vector has dimension p. requiring Bp) disk space, where B [B,] is the batch size. This effectively corresponds to storing millions of model-size vectors. (2) Efficiency: Computing per-sample gradients necessitates
separate backpropagation for each = E B., increasing computational cost by a factor of B.
Avoiding per-sample gradient computation & full gradient storage (detailed in Appendix C.S
To mitigate both issues, we leverage a gradient decomposition and take advantage of the computations
already performed during backpropagation (Wang al., 2025 Choe et al.) 2024). By expressing gradients as the outer product of activations and output derivatives, only a single backpropagation on the aggregated loss is required to compute per-sample gradients, preserving the usual training speed, Additionally, instead of storing the full gradient vectors, we store the decomposed components, potentially reducing the storage requirement OTTB for non-sequential data.
Random projections for large models. For large-scale foundation models with billions of parameters, apply random projections to further compress the stored gradient information. Using projection matrices, we project the activations and output derivatives to a lower-dimensional space. This approach significantly reduces storage needs (T where p is the projected dimension, while still capturing essential gradient geometric information.
We acknowledge that deriving theoretical multiplicative guarantee here is challenging, given that the data value embedding itself is linear combination that could be zero. However, our ablation study in Appendix E.5 demonstates that our approach relatively more robust compared to influence functions across different projection dimensions. These results provide strong evidence of the robustness of our method in practice, and we leave the theoretical guarantee as future work.
4.3 STEP 2: BACKPROPAGATINGG DATA VALUE EMBEDDING
Having established the method for storing projected gradient vectors, we now proceed to describe the backward computation algorithm for data value embeddings. For case of presentation, we continue to use full gradient vector notation. However, in practical implementations, we use the projected gradient vectors for efficient storage. That is, Vol E RD in the subsequent contents.
According to Theorem 2 an equivalent expression for DVEmb (t-) (z*) is given by
(t-)
DVEmb = T/. Ve(O,,, 2²) ne, Ve(0,,,2")MM)
where M (c) = Et-1+1 (EEER, (DVEmb (2) (2)))()..)
At a high level, our algorithm computes (t) for each l, from T down to 0, while maintaining a running matrix ((.) T throughout the backpropagation process for algorithm efficiency.

---

Published as a conference paper at ICLR 2025
Backward algorithm from the final iteration. We initialize M(7-1) 0 as the data value embedding coincides with the training gradient for the last iteration. For to = T 1..... O. we recursively compute: (1) The data value embedding for each 2' E B,, (VE (t.)
=
"l, Ve(0,,,,') - Te, ( VE(,,,, z*), and (2) Update the weighting matrix after computing all embeddings for the current iteration: ( 1) = M(+)) + E.res, (2" detailed algorithm pseudocode can be found in Algorithm
A
Computing data value embedding on a per-layer basis. Moreover, by adopting an assumption similar to that in EK-FAC regarding the independence of gradients across different layers, we can compute data value embeddings on a per-layer basis. This approach significantly reduces the compu- tational and memory costs. The assumption of layer-wise independence is common in the literature on influence functions (Grosse et al, 2023), as it enables tractable analysis and efficient algorithms for deep neural networks. While this approximation neglects cross-layer gradient correlations, it is often justified because intra-layer interactions tend to dominate in practice, Treating layers independently thus strikes favorable balance between computational feasibility and approximation accuracy.
The primary computational cost of our algorithm stems from matrix multiplications and additions in updating data value embeddings and the weighting matrix, resulting in (BTT²) floating-pont point operations (flops). However, if we compute the data value embedding per layer, flops improve to O(BTp" p" (L) where L is the number of layers. The update of the running matrix (( requires (B /L2) memory. In comparison, regular model training requires O (BTp) flops and O(p) memory, where is the number of model parameters. Consequently, Algorithm incurs significantly lower costs compared to regular training. We further note that the influence function method requires computing the per-sample gradient for each training data point on the final model, which is effectively equivalent to epoch of training. As result, both the memory requirements and flops for the influence function method are at least equivalent to those of model training, which are much larger than our algorithm's 's requirements. (2) Storage: Each DVEmb (2))) (z*) has dimension O resulting in total storage requirement of O(TT)) for data value embeddings across all training points. While this can be substantial, disk storage is relatively inexpensive in modern computing environments. Moreover, the reduced dimensionality achieved through projection significantly mitigates the storage burden compared to storing full-dimensional embeddings. summary of the complexity comparison with the most efficient implementation of the influence function (Choe et al. 2024) is provided in Table 2 in Appendix C.9
Complexity analysis. (1) Computational & Memory:
4.4 PARALLELIZED EXTENSION FOR INFLUENCE EMBEDDING COMPUTATION (OVERVIEW)
The backpropagation algorithm introduced in Section 4.3 operates with a runtime complexity of
O(T), as it sequentially computes (VEmb for T 1,..., O. While being significantly more efficient than the influence function, which requires re-computing all training gradients on the final model (see Section and Table it can still be costly for long training periods. Here, we introduce influence checkpointing, a parallelized extension for Algorithm [
Influence Checkpointing. We reduce computational costs by allowing concurrent computation of data value embeddings at multiple checkpoints during training. By selecting K evenly spaced training steps, we can efficiently compute data value embeddings for each intermediate checkpoint in parallel. By carefully computing and storing necessary results, we can efficiently reconstruct the data value embedding for the final model. This reduces the overall computational cost by K times. The detailed algorithm description, pseudocode, and complexity analysis are deferred to Appendix C.7
Data Value Dynamies During Training. In addition its computational benefits, the influence checkpointing algorithm enables powerful capability: tracking the evolution of data influences throughout the entire model training process. If the intermediate checkpoints o,,...., 0tx- was saved-a common practice in foundation model pretraining can analyze how the influence of a fixed data point changes on different intermediate checkpoints. As a result, we gain a more imee-griind and dynamic view of how the influence of fixed data point propagates to the subsequent training steps, providing deeper insights into the model's learning behavior over time. This capability opens up avenues for understanding and optimizing machine learning model training.
7

---

Published as a conference paper at ICLR 2025
Groeddinad Ielanno 7aaetee poodies (.04400))
Greeedimah Dasa Catedding s. 22
Urocedteeh Infereie Papcien
30rarmee 2.767 prtste 2.6536-31
pivdac 011
Erabottig
6,0010
0.030
9,000
Iufluncc Uitection
0,002
0,0002
0.00)
6.0003
0.100)
-0.0005
Dal.
0.003
(a)
-0,0000
00.00148.50188 LAKEI D9206L0010 D012
Groandmmns LOO Seom
(b)
-0.00]
Guouallo
0,500
0.061
(c)
-0.022
Crumadieea 100 Storn
0.221 BL1422
(d)
-0,00 C.000 0,005 5,10 Ocousitath EGO Same
Figure 3: The correlation between ground-truth LOO when the MLP is trained for 3 epochs and the estimation obtained by (a) the data value embedding method and (b) the influence function for single epoch removal. (c) and (d) present the corresponding correlations for all-epoch removal. Additional
results for models being trained for a longer time can be found in Appendix F.2
EXPERIMENTS
In this section, we evaluate the effectiveness of our proposed data value embedding method. First, assess its fidelity in accurately reflecting data importance using small-scale experimental setups (Section 5.I), as well as its computational efficiency (Section (.2). We then apply data value embedding to analyze the training dynamies during foundation model pretraining (Section 55. and Appendix The implementation details and additional results are deferred to Appendix
5.1 FIDELITY EVALUATION
To validate the effectiveness of our proposed data value embedding algorithm, assess its accuracy in approximating TSLOO scores. Additionally, in Appendix E..2I we compare to a variety of data attribution baselines on the standard benchmarks of mislabel data detection and data selection.
Computing ground-truth LOO requires retraining the model multiple times, each time excluding a single data point while keeping all other training specifics, such as batch order, unchanged. Given the computational intensity, we conduct our experiments on the MNIST (LeCun et al.) 1989) using a small MIP trained with standard SGD. We consider two settings: (1) Single epoch removal, where data point is excluded from training during single epoch but still in other training epochs. Here, we remove the data point from the last epoch. (2) All-epoch removal, where a data point is excluded in all epochs. In this case, the approximation provided by data value embedding is obtained by summing the data value embeddings of the data point from all epochs, as discussed in Appendix C.10
Figure 3 shows that data value embedding has a high Spearman correlation with the ground-trut LOO. This superior performance consistent across both settings. We note that the influence function scores remain constant for both settings, as influence functions do not account for specific training runs and cannot differentiate between single- and multi-epoch removals. Moreover, influence function exhibits a very weak correlation with LOO, phenomenon that has been reported in many literature (Segaard et al., 2021 Basu et al. 2020 Bae et al. 2022; Epifano et al. (023).
5.2 COMPUTATIONAL EFFICIENCY
In this section, we compare the storage, memory, and computational efficiency of data value embed- ding with LoGRA (Choe et al. 2024), the most efficient implementation of the influence function so far. LoGRA first computes per-sample training gradients on the final model for all training data points D, where D represents the dataset. Like our algorithm, LoGRA also uses random projection and stores the projected Hessian-adjuste gradient "1,'V((0T, =²) to the disk, and the influence function can be computed via dot-product with test data gradient.
Table 1 shows the result of computing data influence for Pythia-410M trained on 1 1% of the Pile dataset. Both algorithms first compute and store dlessian-adjusted gradients/data value embedding, and then compute the data influence with respect to any given test point. As we can see, LoGRA and data value embedding have similar disk storage requirements, as both approaches save vectors of dimension for each data point. For peak GPU memory in the storage step, LoGRA requires recomputing gradients for all training data on the final model 0r, which is effectively equivalent to one epoch of model training. In contrast, the data value embedding computation algorithm operates
8

---

Published as a conference paper at ICLR 2025
only on projected vectors, which takes much less GPU memory (0.84 vs 63.6GB). Consequently, the computational efficiency for computing data value embeddings is also much higher (over 15x faster). When computing data influence, since both approaches simply take the dot product between test data's (projected) gradient and H Vi(0r, =*) (VEmb(()2 or data value embedding, the GPU memory usage and efficiency are the same.
Storing y Vi((0, > / data value embedding
Compute Influence (dot-product)
Storage 170GB
Peak GPU Mem.
Throughput
Peak GPU Mem.
Throughput
LoGRA
63.6GB
41.6
16.3IGB
640
Data Value Embedding
171GB
64.6GB / 0.84GB
667.52
16.31GB
640
Table 1: Memory and compute efficiency analysis for LoGRA (Choe et al., 2024) and data value embedding. Throughput is measured as the number of data points per second for storing and influence computation. The experiment is conducted on one A100 GPU with 80GB VRAM. The projection dimension is set to 1024. *Since data value embedding technique contains two different steps in storing relevant information for data attribution (storing gradient during training & compute and store data value embedding after training), we include the peak GPU memory usage for both steps.
5.3 ANALYZING TRAINING DYNAMICS OF FOUNDATION MODELS
In this section, we showease data value embedding as a powerful tool for analyzing the training dynamics of foundation model pretraining with Pythia-410M trained on 1% of Pile dataset as an example. Results for additional datasets/models and the analysis for fine-tuning are in Appendix E.3
Value of training data from different stages in LLM pretraining. We first visualize the distribution of data influence scores on the final model across different training batches. For a fair comparison, we normalize the influence scores for each batch by their learning rate. Figure 1 (a) illustrates the results for training Pythia-410n on the Pile dataset. As we can see, the data influence on the final model can be categorized into three distinct regimes: (1) High-impact Warmup Phase: This phase occurs during the very early training stage and is characterized by exceptionally high data influence scores. It corresponds
brief window at the onset of training where the loss reduces rapidly. (2) Low-impact Basin: This regime spans the carly-to-middle training stage, where data influence scores are significantly lower. This period coincides with slowdown in the rate of loss decrease, transitioning into a phase of relative stability. (3) Gradual Ascent: In this phase, we observe that the later a data point participates in the training. the higher its influence score becomes.
Explanation: (1) Parameter initialization and warmup training are important for final model performance. During the very early stages of training, the gradient norms are large, which leads to significant parameter updates. Furthermore, the subsequent gradients' magnitude de- crease rapidly, causing data points from the High- tigh-impact Warmup Phase to maintain substantial influence throughout the training process, even as their immediate impact diminishes over time.
Figure 4 visualizes this phenomenon. The purple curve shows that training data points from the High- impact Warmup Phase, while experiencing large drops in influence as training progresses, still main- tain higher influence than later data points. This observation aligns with the well-known effect that model initialization and/or warm-up training plays crucial role in training performance (He et al. 2015; Hanin & Rolnick 2018), effectively initial- izing model parameters and gradually preparing the model for more complex learning tasks. (2) In- fluence saturation from future data. As training progresses into smoother loss regime, the gradient norms become relatively stable and decrease slowly. This makes the influence decay from subsequent training much more significant for these data points compared to those from the High-Impace Warmup Phase. Since earlier data points experience more future training iterations, their influence decreases more over time. The red curve in Figure 4 demon-
Data Influence on Intermediate Checkpoints
Train Batch Idx
2x
1000-2000
5000-6000
20000-23000
Influerce Score
10000-11000
25000-26000
10
High Empect
Wormpp
10
Gredusl Accanding
Lne-Zepect
1010
5000
10000
15000 20000 25000 30000
Training Steps
Figure 4: Evolution of influence scores across training checkpoints. The x-axis shows train- ing iterations, and the y-axis shows the average influence of training examples on each check- point. Examples are grouped according to the iterations they are being trained on.

---

Published as a conference paper at ICLR 2025
strates this trend, showing influence scores for these points gradually decreasing during training and eventually falling below those of later training data points. One might initially think this phenomenon connected to catastrophic forgetting, where the model appears to "forget" the influence of data from earlier training phases as it progresses. However, we note that a data point's influence score decreases the most when future data points are similar to it, which is different from catastrophic forgetting. Intuitively, if future points are identical, the presence of the earlier data point in training becomes less relevant the model's behavior. more detailed explanation is deferred Appendix
E.3
Implications for data selection strategies. These observations suggest that for pretraining, data selection is most critical during the very early and later stages of training. To validate this insight, we train Pythia-410M on Pile with different online data selection strategies, as shown in Figure (b). Specifically, we use an online data selection strategy (adapted from |aan et al. (2024) that forms each training batch by selecting data points whose gradients align well with those from a validation batch sampled from Pile (see Appendix F.3.2 for details). This selection process requires computing gradient similarities, introducing significant overhead at each iteration where it is applied. Therefore, identifying the most critical training phases for applying this selection process becomes crucial for computational efficiency. Remarkably, Figure (b) demonstrates that the performance of strategy where we only perform data selection in the first 2000 iterations and after 20000 iterations closely matches the performance when data selection is performed in all iterations. Moreover, it reduces computational costs by more than 5 times. This corroborates our practical insights for designing efficient data selection strategies LLM pretraining: by focusing data selection efforts on the critical early and late stages of training, we can potentially achieve optimal model performance while significantly reducing computational overhead.
CONCLUSION AND LIMITATIONS
In this paper, we introduced Data Value Embedding, a novel approach to data attribution tailored for foundation models. Our method addresses critical limitations of existing techniques by capturing the temporal dynamics of training and enabling real time attribution without the need for model retraining. The experiments demonstrate the efficacy of data value embedding in providing accurate and efficient data influence scores and unveiling unique insights into the training dynamies of foundation models.
SGD as a proxy for Adam. The data value embedding in ( is specifically tailored for SGD. It is not directly extendable to other popular optimizers like Adam due to their normalization terms. Nonetheless, using SGD proxy for Adam allows for efficient data influence estimation, which is the approach that is usually adopted in practice and has proved to be effective in our experiment, providing practical and effective solution for the current scope of our work. While using proxy for Adam has proved to be effective in our experiment, extending data value embedding to Adam and other optimizers remains exciting direction for future research.
Training curriculum design. Our findings on the varying influence of data points across training stages suggest the potential for designing optimal training curricula. Future work could explore leveraging data value embedding to design curricula that maximize learning efficiency. This could involve dynamically adjusting the presentation order and frequency of data points based on their predicted influence at different training stages.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This work is supported in part by the National Science Foundation under grants IIS-2312794, IIS- 2313130, OAC-2239622, CNS-2131938, CNS-2424127, Amazon- Virginia Tech Initiative Efficient and Robust Machine Learning, the Commonwealth Cyber Initiative, Cisco, OpenAl and Google.
We thank Tong Wu, Meng Ding, Haizhou Shi, and Weida Li for their helpful feedback the preliminary version of this work.
REFERENCES
Juhan Bae, Nathan Ng, Alston Lo, Marzyeh Ghassemi, and Roger B Grosse. If influence functions are the answer, then what is the question? Advances in Neural Information Processing Systems, 35:17953-17966 2022.
Juhan Bae, Wu Lin, Jonathan Lorraine, and Roger B Grosse. Training data attributiin via approximate unrolling. Advances in Neural Information Processing Systems, 37:66647-66686, 2024.
Elnaz Barshan, Mare-EEienne Brunet, and Gintare Karolina Dziugaite. Relatif:
Identifying explanatory training samples via relative influence, In International Conference on Artificial Intelligence and Statistics, pp. 1899-1909. PMLR, 2020.
MS Bartlett. Approximate confidence intervals. Biometrika, 40(1/2): 1953.
Samyadeep Basu, Philip Pope, and Soheil Feizi. Influence functions deep learning are fragile. arXiv preprint arXiv:2006. 14651, 2020.
Olivier Bousquet and André Elisseeff. Stability and generalization. The Journal of Machine Learning Research, 2:499-5266 0266,,522220002 2002.
Hongge Chen, Si Si, Yang Li, Ciprian Chelba, Sanjiv Kumar, Duane Boning, and Cho-Jui Hsich. Multi-stage influence function. Advances Neural Information Processing Systems, 33:12732- 12742, 2020.
Yuanyuan Chen, Boyang Li, Han Yu, Pengeheng Wu, and Chunyan Miao. Hydra: Hypergradient data relevance analysis for interpreting deep neural networks. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 7081-7089, 2021.
Sang Keun Choe, Hwijeen Ahn, Juhan Bae, Kewen Zhao, Minsoo Kang, Youngseog Chung, Adithya Pratapa, Willie Neiswanger, Emma Strubell, Teruko Mitamura, et al. What is your data worth to gpt? Ilm-scale data valuation with influence functions. arXiv preprint arXiv:240S. 2024.
R Dennis Cook and Sanford Weisberg. Characterizations of an empirical influence function for detecting inffuential cases in regression. Technometrics, 22(4):495-508, 1980.
Junwei Deng and Jiaqi Ma. Computational copyright: Towards a royalty model for ai music generation platforms. arXiv preprint arXiv:
2023.
Junwei Deng, Ting-Wes Li, Shichang Zhang, and Jiaqi Ma. Efficient ensembles improve training data attribution. arXiv preprint arXiv:2405. 17293, 2024.
Meng Ding, Kaiyi Ji, Di Wang, and Jinhui Xu. Understanding forgetting in continual learning with linear regression. In Forty-first International Conference on Machine Learning. 2024.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in
private data analysis. In Theory of cryptography conference, pp. 265-2655282 2844. Springer, 2006.
Jacob Epifano, Ravi P Ramachandran, Aaron Masino, and Ghulam Rasool. Revisiting the fragility of influence functions. Newral Networks, 022:5115889 2023.
Itay Evron, Edward Moroshko, Rachel Ward, Nathan Srebro, and Daniel Soudry. How catastrophic can catastrophic forgetting be in linear regression? In Conference on Learning Theory, pp. 4028 4079. PMLR, 2022.
11

---

Published as a conference paper at ICLR 2025
Simin Fan, Matteo Pagliardini, and Martin Jaggi. Doge: Domain reweighting with generalization estimation. In Forty-first International Conference on Machine Learning, 2024.
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 2020.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2 (tt::2/01.000 00002,,000 2020.
Amirata Ghorbani and James Zou, Data shapley: Equitable valuation of data for machine learning. In International Conference on Machine Learning, pp. 2242-2251. PMLR, 2019.
Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, et al. Studying large language model generalization with influence functions. arXiv preprint arXiv:2308 2iti::2308.03996, 2023.
Han Guo, Nazneen Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong. Fastif: Scalable influence functions for efficient model interpretation and debugging. In Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing, pp. 10333-10350, 2021.
Boris Hanin and David Rolnick. How to start training: The effect of initialization and architecture. Advances in neural information processing systems, 31, 2018.
Satoshi Hara, Atsushi Nitanda, and Takanori Machara. Data cleansing for models trained with sgd. Advances in Neural Information Processing Systems, 32, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034- 2015.
Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclere, and Aleksander Madry. Data- models: Predicting predictions from training data. arXiv preprint arXiv:2202.00N: 0020000.0022..... 2022.
Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nezihe Merve Gurel, Bo Li, Ce Zhang, Costas J Spanos, and Dawn Song. Efficient task-specific data valuation for nearest neighbor algorithms. Proceedings of the VLDB Endowment, 2019.
Pang Wei Koh and Percy Liang. Understanding black- predictions via influence functions. In International Conference Machine Learning, pp. 1885-1894, PMLR, 2017.
Frederik Kunstner, Philipp Hennig, and Lukas Balles. Limitations of the empirical fisher approx- imation for natural gradient descent. Advances in neural information processing systems, 32, 2019.
Yongchan Kwon, Eric Wu, Kevin Wu, and James Zou. Datainf: Efficiently estimating data influence in lora-tuned Ilms and diffusion models. In The Twelfth International Conference on Learning Representations, 2023.
Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard, and Lawrence Jackel. Handwritten digit recognition with a prok-prronggttion network. Advances in neural information processing systems, 2, 1989.
Yuetian Luo, Zhimei Ren, and Rina Barber. Iterative approximate cross-validation. In International Conference on Machine Learning, pp. 23083-23102. PMIR, 2023.
James Martens. New insights and perspectives on the natural gradient method. Journal of Machine Learning Research, 21 21(146): 2020.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint 1609::6609077800 2016.
Elisa Nguyen, Minjoon Sco, and Scong Joon Oh. A bayesian approach to analysing training data attribution in deep learning. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Published as a conference paper at ICLR 2025
Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry. Trak: attributing model behavior at scale. In Proceedings of the 40th International Conference on Machine Learning, pp. 27074-27113, 2023.
Daryl Pregibon. Logistic regression diagnostics. The annals of statistics, 94):7055-724, 1981.
Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33: 19920-19930) 2020.
Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pp. 8179-8188 2022.
Nicol N Schraudolph. Fast curvature matrix-vector products for second-order gradient descent. Neural computation, 14(7):123-1738,, 2002.
Anders Søgaard et al. Revisiting methods for finding influential examples. arXiv preprint arXiv:2111.04683 2021.
Zhihua Tian, Jian Liu, Jingyu Li, Xinle Cao, Ruoxi Jia, and Kui Ren. Private data valuation and fair payment in data marketplaces. arXiv preprint arXiv:2210.08723, 2022.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Jiachen T Wang and Ruoxi Jia. Data banzhaf: robust data valuation framework for machine learning. In International Conference on Artificial Intelligence and Statistics, pp. 3388-64211 PMLR, 2023a.
Jiachen T Wang and Ruoxi Jia. note on" efficient lask-specific data valuation for nearest neighbor algorithms". arXiv preprint 0.X4422304.04258, 2023b.
Jiachen T Wang. Zhun Deng, Hiroaki Chiba-Okabe, Boaz Barak, and Weijie Su. An economic solution to copyright challenges of generative ai. Technical report, 2024.
Jiachen T Wang, Prateek Mittal, Dawn Song, and Ruoxi Jia. Data shapley in one training run. In The Thirteenth International Conference on Learning Representations, 2025.
Jingfeng Wu, Difan Zou, Vladimir Braverman, Quanquan Gu, and Sham Kakade. The power and limitation of pretraining-finetuning for linear regression under covariate shift. Advances in Neural Information Processing Systems, 35:33041-33055 2022.
Jingfeng Wu, Difan Zou, Zixiang Chen, Vladimir Braverman, Quanquan Gu, and Peter Bartlett. How many pretraining tasks needed for in-context learning of linear regression? In The Twelfth International Conference on Learning Representations, 2024.
Yizi Zhang, Jingyan Shen, Xiaoxue Xiong, and Yongchan Kwon. Timeinf: Time series data contribution via influence functions. arXiv preprint arXiv:2407. 15247, 2024.
Eric R Ziegel. The elements of statistical learning, 2003.
Difan Zou, Jingfeng Wu, Vladimir Braverman, Quanquan Gu, and Sham Kakade. Benign overfitting of constant-stepsize sgd for linear regression. In Conference Learning Theory, pp. 4633 4635. PMLR, 2021.
13

---

Published as a conference paper at ICLR 2025
A EXTENDED RELATED WORKS
Here, we provide a general overview of the literature that is relevant to the trajectory-specifie LOO. We refer the readers Appendix of Wang al. (2025) for the overview of the related works for Data Shapley and other data attribution techniques.
A.1 LOO INFLUENCE VS LOOCV
is important to distinguish our LOO influence measure from traditional Leave-One-Out Cross- Validation (LOOCV) (Ziegel 2003). While both involve removing individual data points, they serve different purposes and yield different interpretations. LOOCV is a model evaluation technique that estimates generalization performance by averaging prediction errors held-out examples, where smaller errors indicate better model performance. In contrast, LOO influence measures how removing specific training point affects the model's behavior validation data, quantifying each training example's importance to the learning process. While LOOCV requires training N separate models to evaluate generalization (where is the dataset size), LO0 influence focuses on understanding the counterfactual impact of individual training points on model behavior. This distinction is crucial as we aim to understand data importance rather than model performance.
A.2 INFLUENCE FUNCTION AND FRIENDS
Influence function Koh & Liang 2017) has emerged important tool for interpreting and analyzing machine learning models. As the influence function requires computing the Hessian inverse, many subsequent works are focusing improving the scalability of the influence function for large-scale models (Guo et al., 2021| Schioopa et al. 2022] Grosse et al., (023). More recently,
Kwon et al. (2023) developed an efficient influence function approximation algorithm that is suitable for LoRA fine-tuning, and Zhang et al. (2024) extends the influence function to time-series datasets. In similar spirit to us. Chen et al. (2020) multi-stage extension influence function trace fine-tuned model's 's behavior back to the pretraining data. However, they changed the original loss function and added a regularization term to account for intermediate checkpoints. The most closely related to our work is Choe et al. (2024). Similar to us, they also make use of the low-rank gradient decomposition and random projection to enable efficient computation and storage per-sample gradient. However, their approach still requires computing per-sample gradient vectors for all training data on the final model checkpoint, which is effectively equivalent to one model retraining and takes significantly longer time than data value embedding.
Influence function and Newton step. The influence function formula bears a striking resemblance to the Newton step in optimization, both involve the product of an inverse Hessian and gradient. As result, much data attribution literature simply describes the influence function as "one Newton step". There is subtle difference in their expressions and motivations. For large dataset, Newton step approximates the influence function under certain regularity conditions. A single Newton step from the full model parameters toward the leave-one-out solution was proposed measure by Pregibon (9981) and has since been leveraged for data attribution (Park et al. 2023). The high-level argument is that while obtaining the exact leave-one-out model requires fully minimizing the cave-one-out loss, taking one Newton step from the optimal model 6 trained on the full dataset often provides sufficient information distinguish the relative influence of different data points.
influence
14

---

Published as a conference paper at ICLR 2025
B LIMITATIONS OF THE EXISTING DATA ATTRIBUTION TECHNIQUES FOR FOUNDATION MODELS
B.1 INFLUENCE FUNCTION
Influence functions (Cook& & Weisberg, 1980 Koh & Liang 2017) are classical technique from robust statistics, adapted for machine learning to measure how the removal of a single data point affects the performance of trained model. Influence functions quantify the sensitivity of model's predictions to specific data points, offering insights into the importance of individual training samples. In the machine learning framework, they are particularly useful for diagnosing model behavior, understanding dataset quality, and identifying mislabeled or harmful data points. The core idea of influence functions is to approximate the effect of removing data point from the training set without needing to retrain the model. Instead of actually excluding a point and retraining, influence functions leverage the model's final parameters and compute the impact of point's removal based the gradient and Hessian inverse of the loss function at the final model state. Formally, the influence of a training data point 2; on the loss E(O, (()a)) at a validation point (val) defined as:
IF (1)) Vol(0, (vat) T "H''V Vol 2s)
where 0 is the final model parameter after training, H = + Σ> Vie(O, zi) is the Hessian of the total training loss 8, Vol(o, (vvl)) and Več(O, 2.) are the gradients of the loss the validation point and the training point, respectively.
Limitation:
Neglecting Training Phases and Unrealiable Approximation to LOO. A key limita- tion of influence function techniques is their exclusive focus on the final model parameters, thereby ignoring the intermediate dynamics of the training process. By assessing data contributions solely based on the final trained model, influence functions fail to capture how each data point influenced the model's updates throughout training. This narrow focus introduces inaccuracies, as it overlooks the cumulative effects of model fluctuations during the training iterations. Consequently, influence functions can be less accurate in evaluating data contributions, particularly in large-scale models where the training process plays significant role. For instance, in modern training paradigms for large language models (LLMss,, models are typically pretrained on a broad corpus and subsequently fine-tuned on specialized domains. Influence functions, however, cannot differentiate between the impacts of data points during pretraining and fine-tuning phases. Relying solely on the final model parameters after fine-tuning, they miss how pretraining data contributed learning general language structures or how fine-tuning data adapted the model to specific domains. This inability to account for different training stages results in incomplete and often noisy estimates of data contributions, thereby reducing the precision of attribution in multi-stage training processes.
Moreover, our analysis in Section D demonstrates that the influence function approximates the expected data influence across different training trajectories only under overly simplistic conditions, which are often violated in practice. These conditions, such as assuming identical intermediate model checkpoints and Hessian matrices, almost never hold in real-world training scenarios where model evolve significantly. This highlights the inadequacy of influence functions accurately capturing data contributions, underscoring the necessity for more comprehensive data attribution methods that consider the entire training trajectory.
Neglecting Training Phases Necessitates Unreasonable Assumptions and Often Require Model Retraining, Additionally, the focus on the final model necessitates assumptions of convergence and strong convexity ensure reliable results. In many real-world settings, where models are non-convex and may not fully converge, these assumptions are often violated, leading to further inaccuracies in the data contribution estimates. As the influence function score is often found to be highly noisy in practice (Basu el al.) 2020) Søgaard et al., [022T] Bae et al.) 2022 Epifano et al. 2023). it typically necessitates multiple model retraining to produce reasonable results (Deng et al.
2024 which can
undermine their original computational efficiency advantage.
B.2 IN-RUN DATA SHAPLEY
In-Run Data Shapley Wang et al, 2025) is a data attribution technique designed to evaluate the contribution of individual data points during single training of machine learning models. It
15

---

Published as a conference paper at ICLR 2025
builds the traditional Data Shapley framework, which stems from cooperative game theory. The Shapley value, originally proposed by Lloyd Shapley in 1953, distributes total utility fairly among all contributing players based on their marginal contributions. Applying this concept to machine learning, Data Shapley attributes the contribution of each data point in a training dataset by assessing its influence model performance. However, standard Data Shapley methods face limitations in scalability because they require numerous retraining iterations on different data subsets. These computational demands make them impractical for large-scale models such foundation models. To address these challenges, In-Run Data Shapley was introduced as a scalable alternative that avoids the need for repeated retraining. Instead, it leverages the iterative nature of model training, specifically neural networks, where parameters are updated in small increments. By tracking gradient updates at each training step, In-Run Data Shapley calculates the contribution of individual data points toward the final model without retraining. It approximates the Shapley value using local utility functions tied to specific gradient updates and extends these to the full training process, capturing cumulative contributions. This method reduces the computational overhead to a level comparable with standard training runs while maintaining the theoretical fairness and interpretability of Shapley values.
Limitation: Requirement of Validation Data in Advance. One of the key limitations of In-Run Data Shapley is its reliance on the availability of validation data prior to the start of training. The technique calculates data contribution by examining the impact of training points measured against the validation set. Thus, access to this validation data throughout the training process is necessary to compute meaningful Shapley values at each iteration. This restriction can limit the applicability of In-Run Data Shapley in scenarios where validation data is not immediately available, such as in certain real-time learning environments or when the validation is defined only after training. Potential workarounds, such as saving intermediate model checkpoints to calculate contributions post-tres post-training, add complexity to the process and might be unreliable.
model performance as
16

---

Published as a conference paper at ICLR 2025
C ALGORITHM DETAILS
C.I DERIVATION DETAILS FOR SECTION 3.1
Suppose 2* is data point that participates in the training during the first iteration. Denote B; the training batch in the f-th itcration. For standard Stochastic Gradient Descent (SGD), we have:
0
02
7K Σ VE(Ok,2)
(3)
ZEB
for A = 0, ...... T 1, where 7 is the learning rate at iteration
For validation data 2(vat), we aim estimate the change E(Or, (val)) by removing =" from the first iteration. Specifically, we want to estimate E(0TT 2(val)) E(Or, 2 (wol)) where:
8 = 00 110
Σ
Ve(On, 2)
(4)
=EBoM==1
and
0 1
VE(0%;=)
(5)
ZEB
for k = T...
To approach this problem, we define an interpolation between 0T and 8y:
0, (e) == 0n 7/0
FE(0o,=) 7o - 5) E) =*)
(6)
=EBo\(=")
where (r()) is defined accordingly. Note that (r) = 0T and (1) Br.
By taking the first-order Taylor expansion at = = 0, we have: E(0 (ool)) E(Or, (00r)((vl)) (vva)) ((Or (1), ((val)) (0), 2(val))
2
e(OT (2), (((T())2((v)) 2(val)
πe
==0
(((al))((aal) 00,(5)
Oe
1==
Now, we derive (e)
by observing the following recursive relation for A: 2 1:
ds
00 r0)+
00k(e)
(0x
V'E((A(EE,2)
(8)
O=
O=
2EB
(0x(E) (I - mh H+))))
(9)
Oe
where = V=P(Ox(E), 2) is the Hessian and is the identity matrix. Additionally, for the first iteration where 2" participates, have
(8)
No Ve(60, 2*)
(10)
de
Expanding the recursion and substituting it back into original expression, get: ((0r(1), (()) 20 (val))
Oe ((Or(c), 2(aa)) l==
T The H
= 7o Vo 2 (naf)
T
Ve(00,z))
cumalative effect
This final expression gives an estimate of the influence of removing 2" from the first iteration on the
loss on (val) at the final iteration. The term I (I n)s H₂) represents the cumulative effect of all training iterations the initial influence. This product captures how the impact of the initial change propagates through the entire training process, accounting for the learning rate and the training data at each subsequent step.
17

---

Published as a conference paper at ICLR 2025
C.2 ERROR GUARANTEE FOR SHROLLING-BASER APPROACH
In this section, we derive the approximation error guarantee of the unrolling differentiation estimator
(0r(s) E = 0 1/ls
T-1
AO
II k=
(I - Th nHH =²(0t,;="),
de
for non-convex loss functions. A very loose bound for ||0 07- -0- has been derived in Hara et al. (201)). Here, we improve the error bound by additionally considering the decay of the learning rate and the spectral norm of Hessian matrices training progresses. Notably, establish uniform bound on the gap.
Assume that E(2; 0) is twice differentiable with respect to the parameter 0, and we train the model for iterations. We make the following assumptions:
1. Learning Rate Schedule: The learning rate 7) at iteration follows the schedule 7/2 Timas
where Tmmax = C- for some constant C. Justification: The decaying learning rate schedule VT
Umax a common choice in neural network training in famous literature (Vaswani 2017). Th vt This schedule allows for larger step sizes during the initial phases of training, facthitating rapid convergence, while gradually reducing the step sizes to fine-tune the model parameters
and ensure stability as training progresses. The max learning rate Vluax =
ensures
that the cumulative step sizes remain bounded over T iterations, which is crucial for deriving meaningful error bounds. This approach balances the trade-off between exploration and convergence, making it well-suited for training deep neural networks where maintaining stability is essential.
2. Hessian Spectral Norm Decay: There exists a constant > such that the Hessian matrices satisfy H. < A I for all 1. Justification: The assumption that the spectral norm of
Vi
the Hessian matrices decays as H, 3 A I is grounded in the observation that, as training progresses, the optimization landscape often becomes flatter around minima. This reduction in curvature implies that the Hessian's 's eigenvalues decrease, leading to smaller spectral norms. Such behavior is typical in many deep learning scenarios where initial training steps navigate regions of high curvature, followed by stabilization in flatter regions as the model converges. Additionally, this assumption aligns with empirical findings deep learning literature O. where Hessian's 's spectral norm has been observed to decrease over time, thereby facilitating more stable and efficient convergence. By incorporating this decay, we account for the diminishing influence of curvature parameter updates, which is critical for tightening the error bounds in our analysis.
V
Under these assumptions, we proceed to derive a uniform bound on the approximation error ||0r- 8 A0_ |]. This bound provides theoretical guarantees for the effectiveness of the unrolling- based approach in estimating the influence of removing a training data point on the final model parameters. The derivation leverages the decaying learning rate and the diminishing spectral norm of the Hessian matrices to tighten the error bounds compared to previous work (Hara et al. 2019).
Theorem 3. Assume that E(z; 0) is twice differentiable, that the Hessian Vi((2; 0) is L-Lipschitz continuous with respect O, and that the gradient norm is bounded, i.e., o[ 0)1 G f all Mmnee
and f. Furthermore, assume that the learning rate nh at iteration follows the schedule n Vt.
where 1/max F for some constant > O. Then, for the unrolling differentiation estimator AO A0_.. the approximation error satisfies
32 (1 - 07) A0_
(11)
Proof. By Cauchy's Mean Value Theorem, for each iteration (to...., 1), there exists r E [0, 1] such that for 0 = r0', + (1 r)0,. we have
(ve((z;0') vol(z;0.)) H -0,),
BEB
18

---

Published as a conference paper at ICLR 2025
where H. = ===EEB,, ())(200)) Define Z, (I 17. H, and Z = (I (I- -.) Then, we have
8xt1 - 0.1 = (o 02) + 77 (H, - H - 0.) =Z,(6' -6a) + Der
where D3 := (H - -H: - 02). Recursively applying these equalities over & E (ta...., T-1H, we obtain
T-1 T-1
0 - 0T = 20_
Σ
II Z.D.
s=t, k=st1
Hence, the approximation error is given by
T-1 T-1
11(0r 0p) 20 -0-
Σ
Z.D.
kmst1
To bound this, we proceed as follows. Given the learning rate schedule T): A
Dmus Vt
and the
VTt
assumption that H, 3 S I, we have
A
CA
| 1| H& S $1
+
T
+
Vi
KVT
For large and & norms as
1 2 the term
is small. Thus, we can bound the product of the
KVT
T-1
CA KVT k=
CA T-1 VT k=s+1
II
7 |l S exp
Σ
Sexp
h
k=s+1
Using the harmonic series approximation,
Σ1
Σ
< ln
T
<
In(T).
kms+1
Thus,
T-1
CAInT
II k=s+1
/ S exp
SeOs
VT
Therefore, we have
T-1 |
Σ II s=ty Ams+1
ZeD,
S on
Next, we bound ||D,]]:
|D (H, H;) 0.)1 175 H. H;
- 110; 0z11.
Since () is L- L-Lipschitz continuous with respect 0, we have
H. 1 H < LI 0-ll-
Additionally, we have
4GC V8 VT
1101, - 0.1l < 2
GG = 2G Σ
TTI
1=1
where we used the bound Σi=1 1 < 2
Thus,
(AGC V3 2
= r V
|D.|| < n.b.
VT
71.
where = C
19

---

Published as a conference paper at ICLR 2025
Substituting this bound into the sum, obtain
T-1 T-1
Σ II
DD
cecA
CA
T-1
v
T
k=s+1
We now evaluate the summation:
T-1 Σ
V8 5
1
2
T..
2 3
71.3
T1.5
71.5
3
sm1
where we used the bound Therefore,
Σ V S 8
T..
I Da k=s+1
s CAT
2 32
PC
3 33
C.3 COMPUTING DATA VALUE EMBEDDING RECURSIVELY
Theorem 4 (Restate for Theorem 2
Given generalized Gauss-Newton approximation H, 2
ELEB, VE(O,, VE((,,) T. we have
T-1 Σ
DVEmb() = ne Ve((0,,))) 2*) TJ
Σ A
(Ve(Ok, VE((1,,")) (V(()..))VV((((=2)) DVEmb(() (2)
k=++1
Proof.
DVEmb (2")
T-1 II - Hs) Lk=++1
17
vl(0,,2")
T-1
II (I - m H&)) A=++2
(I- 11 1 1
1/t
II (I (1-7HH) Hz) Lkt++2
I- Th41 Σ 2((0+1)) velli,z))
11
zEB
II (I - mo-mHH))) ve(Omn12)) ve(0++1,=) (V((0.)
II (I H&)
vl(0+,z)) 1/t Σ
11
M|++1
Lk=+2
sEB
k==+2
1h
II (I H)H
ve((0,2") 1T
Σ (V((0++122) Ve(O,,,")) TV((,, (V2)b((+))
Lk=++2
ZEB
T-1
Ve(O,,
11
Σ k=++1
(ve(Ok, 2) Ve(O,, DVEmb (x) AE
Σ
The transition from the penultimate the final line involves generalizing the summation over B to include all batches from t + to T - 1, effectively unrolling the recursive computation. In other words, the "data value embedding" for data points /th iteration can be approximated by its gradient subtracted by a linear combination of the data value embedding in the later iterations, where the weight of each embedding is determined by the gradient similarity 2()0).2) Ve(0,, 2*).
D
20

---

Published as a conference paper at ICLR 2025
C.4 GENERALIZED
APPROXIMATION TO HESSIAN
In this section, we justify the use of Generalized Gauss-Newton (GGN) as the approximation to the Hessian matrix. Similar derivation can be found in many literature and textbooks, such as Bartleu 1953); [chraadolphi (2002). This approach has also been used in other data attribution and
optimization techniques for approximating Hessian matrices (Martens
2020 Kwon et al. 2023
Grosse et al. 2023).
The cross-entropy loss function for classification with one-hot encoded labels is defined as:
I(y - Σ ys log(Si)
where = [y1 .c......c] is the one-hot encoded true label vector and f f2........ fc] the vector of predicted probabilities from the model. In this paper, we restrict our focus to the cross-entropy loss, as it is the most commonly used loss function, and many LLMs are pre-trained with the cross-entropy loss function.
By chain rule, the derivative of T. with respect to fa is 0L 2t. Since is a one-hot vector, only the correct class & has y/ while all other y; for + k. This simplifies the gradient to:
āf.
1 0
VoL
fu 00
Thus, the gradient depends only on the derivative of fk (the predicted probability for the correct class) with respect to 0. The Hessian H is the second derivative of the loss with respect to 0:
a H V = 00 (VoL)= = 30
1 854 fk
05k 00
05k 00
T
0 Ln
00
=
fx
002
Applying the product rule and assuming the second derivative when fü is approximately linear in near the current parameter values), the Hessian simplifies to:
0h is negligible (which is common 307
0fk
H ~
F2
00
00
Moreover, this approximation matches the outer product of the gradient of loss with respect to model parameter
1
0
Ohk a0
VoLVoL T
a0
Therefore, the gradient outer product exactly approximates the Hessian matrix under the assumption that 8² is negligible.
a62
Remark 1. We note that the specific definition of GGN depends how we split the loss function. As highlighted by Kunstner et al.) (2019). the GGN is ambiguous as it crucially depends on the "split" given by the inner and outer functions. For the cross-entropy loss, the canonical split treats the loss as a, (b) = loog p(yn ()) and b, (8) = (zn, 0). which allows the GGN to capture curvature information to some extent. However, the alternative split used in our paper may not retain useful second-order information. Nevertheless, most recent works in influence function approximation have used this approach for approximating the Hessian thus far (with the exception of Grosse al. (2023)). ). An important open question is understanding the gap between this approximation and Hessian regarding the task of data attribution.
C.5 GRADIENT DECOMPOSITION TECHNIQUE
To mitigate the computational cost from per-sample gradient computation, leverage gradient decomposition and take advantage of the computations already performed during backpropagation (Wang et al.) 2025 Choe al. 2024). We illustrate this technique with simple linear layer, where the output is aW, with W [idxxe xd2 being the weight matrix, a (a ..... (Bi)T as the
input, and (5(1) ,...,,(BB), (B)) representing the pre-activation tensor. For non-sequential data,
21

---

Published as a conference paper at ICLR 2025
E IR Rxd, S RRdd, Denote sample batch B = f2,,.... }. By chain rule, we can express
the gradient of individual loss g(+) E(w, 24) with respect to W as
ag(a)
aeli) is(1)
Og(9)
ag(+)
al
(12)
ow
OW
as(i)
Og(t)
where := E (0) is the aggregated loss, and the last step is because other data points' losses ag have no dependency on s; Note that the individual's output gradient Jy(r =
āe(ty is readily
de
available during the backpropagation pass in terms of Therefore, this method requires only single backpropagation on é, maintaining the training speed equivalent to standard training.
In terms of storage improvement, rather than storing the full gradient vectors SW E E4 xda for 860)
each data point 2;, we instead store the smaller pair (a(*) BF
Suete E Rdd+ This reduces memory
requirements from B) to B) for non-sequential data. For sequential data where a E BxSxi s E RBxSx, if S² > d1d2, it is more memory-efficiett to directly store the per-sample gradient vectors, so the storage requirement remains as B)
C.6 RANDOM PROJECTIONS FOR LARGE MODELS
For large-scale foundation models with billions of parameters, even the reduced storage of O( B) can be substantial. In such cases, we apply random projections to compress the stored information further. We use two projection matrices, Pa E Rx and Ps Rrxda, to project and ds to lower dimensional space I' respectively. The projected gradient can then be reconstructed directly from
the projected activations and output derivatives: (Pa X9 Ps)) (a &
0f
(P.a) & P 156
C.7 PARALLELIZED EXTENSION FOR INFLUENCE EMBEDDING COMPUTATION
The backpropagation algorithm introduced in Section for computing data value embeddings operates with runtime complexity of (T)), it sequentially computes DVEmb for t, T' 1-1,..., O. While being significantly more efficient than the influence function, which requires re- computing all training gradients on the final model (see Section 5.2 and Table 2 it can still be costly for long training periods. Here, we present inftuence checkpointing technique, parallelized extension for Algorithm I This extension reduces computational cost by enabling concurrent computation of embeddings at multiple checkpoints throughout the training process. Besides the computational efficiency benefits, it also enables the study of how the influence of individual data points evolves throughout model training, providing valuable insights into the learning process.
Influence Checkpointing. We pick K evenly spaced training steps 0 < t1 < < We then concurrently execute the backpropagation algorithm for value embedding, initiating from each of these intermediate steps. This process yields data value embeddings for each corresponding intermediate checkpoint 8,, ..,...., -ex. 00x- We extend notation for data value embedding and denote DVEmb (t. (t)))))) (2*) as the data value embedding of 2" E B,, for the intermediate checkpoint
O,, Note that DVEmb DVEmb (T) for the final model, and we must have I, < le, as later training batches cannot influence carlier checkpoints.
Consider initiating the backpropagation algorithm in Section 4.3 at step fr and stop at step fz-1, we will obtain data value embeddings DVEmb (VEmb(t,-+t) (t,++tz) for t2 = t 1.-1..... tt - 1. We additionally denote ((te-ta) = _ (I n, H.). From the definition on Equation (2) the final data value embedding (VEmb (T. (2*) can be computed from (f.
as follows:
DVEmb (t. (1) (2*) DVEmb (1. (1)=))(())))))))))))))))))<))( ((()7)
(13)
(=*), we additionally store the matrix K ((+rr++t)) between steps t--1 and te during the backpropagation for each tz. Consequently, for any t, such that te, S to < too+1, we have (tr-)) = I K(
Hence, to recovery of (t. (T. -T)
((+) allowing us to compute (V (t)-)) based
on [3). A detailed algorithm pseudocode is provided in Algorithm [ The complexity analysis of this algorithm is the same as the original data value embedding algorithm in Table but the actual runtime is being reduced by factor of K due parallelism.
22

---

Published as a conference paper at ICLR 2025
Data Value Dynamics During Training, In addition its computational benefits, the influence checkpointing algorithm enables a novel capability: tracking the evolution of data influences through- out the entire model training process. If the intermediate checkpoints ......., 0xx_1 was saved- common practice in foundation model pretraining-ww can analyze how the inftuence of indi- to < 1.
vidual data points changes as training progresses. Specifically, for any training step is where compute the data value embedding (fE)))((=+ for any checkpoint to + 1 as DVEmb (VEmm((.-7))( (z*) DVEmb(.-<te (II ((te1--tt)) This formulation allows us to estimate data influence scores not only for the final model checkpoint Or but for any intermediate checkpoints o..- As result, gain data influences evolve during training, providing deeper insights into the model's learning behavior over time. To our knowledge, this is the first data attribution method offer such a principled and practical framework for studying data influence dynamics throughout the training process. This capability opens up new avenues for understanding and optimizing machine learning model training.
fine-grained and dynamic view of how
C.8 PSEUDOCODE
Algorithm Backpropagation for computing data value embedding from the final checkpoint
Require: Training steps learning rates {n { training data gradients 2))
1: // Initialization
2: M(T-1) 0.
3:
4: // Recursion steps
5: for t = down to 0 do
6: =o B, do
7:
DVEmb!"( mp Ve((..)) V((O,(
8: if then ( + ( + EzeB DVEmb (E)
9:
10: return (DVEmb( (x))
23

---

Published as a conference paper at ICLR 2025
Algorithm 2 Parallel Influence Checkpointing for Data Value Embedding
Require: Training steps T, number of checkpoints K, learning rates {m T-11 loss gradients (V((O,,))) Hessians {H.
Ensure: Data value embeddings (DVEmb (+) (2)
Select K evenly spaced checkpoints 0 to < t1 < ... =tk=T
2: for & = 1 to K do
3:
Run BACERRPPGAATSSCCMNNTC....
4:
5: // Compute final embeddings
6:
for == to K do
7: for t, = + to 1 do
8: for 2 B,, do
(VEmb((A) (2) + DVEmb M K 1-
9:
10:
return (DVEmb (1)
11:
12: procedure BACKPROPAAATESSEEENNTT 4))
13:
Initialize and (1) as in the original algorithm k((s-ta)) +-
14:
for / la down la do for = E B, do
16:
DVEmb (1-+l) (2) 17 - 2)
18:
t > fu then ( ( + Σ EEB (L
19:
20:
K (t-+to) K! (1+)-16)) H,H,))
return (DVEmb
(z)) (1a-)t))
24

---

Published as a conference paper at ICLR 2025
C.9 COMPLEXITY SUMMARY
In this section, we compare the storage, memory, and computational efficiency of data value embed- ding with LoGRA (Choe et al., 2024), the most efficient implementation of the influence function to date. LoGRA is currently the only method that supports real-time, on-demand data influence computation when new test data is introduced. Similar to our algorithm. LoGRA initially computes per-sample training gradients on the final model for all training data points x E D, where D repre- sents the dataset. It then stores the projected Hessian-adjusted gradients VE 2*) for each =", and also assumes layer gradient independence.
While the random projection step in LoGRA is akin to our approach, LoGRA's requirement to recompute gradients for all training data on the final model Or is computationally intensive, effectively equivalent to one epoch of model training. In contrast, data value embedding captures the training data gradients during the original training process. As discussed in Appendix C.10 the training and disk storage can be handled asynchronously. This means that the gradient storage step in the data value embedding algorithm does not incur additional efficiency costs.
Storing H, Viier, > data value embedding
Compute Influence (dot-product)
Storage ((P)))
Memory
FLOPS DDp+ DI L TB/W/L/OTTE/
Memory
FLOPS Buaia
LoGRA
O(p)
O((Bas + B (p)
Data Value Embedding
(TT
O
O( Duin)))
BusBeainl
Table 2: Summary of the storage, memory, and FLOPS complexity for LoGRA (Choe et al. 2024 the most efficient implementation of the influence function to date. Here, p denotes the model dimension, á is the projected dimension, T represents the number of training iterations, and B is the batch size. (P) is the dataset size, with the relationship TB = [D] x #epochs. L is the number of layers. Breat and Birain refer to the test and training batch sizes during influence computation, respectively, which are independent of the batch size B used during model training. *Since the data value embedding technique involves two distinct steps for storing relevant information for data attribution (storing gradients during training & computing/storing data value embeddings after training), we include the complexity for both steps. For the gradient storage step, the complexity refers the additional cost beyond regular training.
C.10 PRACTICAL CONSIDERATIONS & POTENTIAL EXTENSIONS
In this section, we discuss some practical extensions and considerations for implementing data value embedding for real-world scenarios.
Optimizing 1/O operations for seamless training. During each training iteration, computed gradient representations need to be transferred from GPU to CPU and then written to disk. To prevent this process from blocking the main training loop, we implement several optimizations: (1) Asynchronous 1/O operations: To avoid the gradient storing process blocking the main training loop, make GPU operations and GPU-CPU transfers asynchronous by using CUDA streams. This allows GPU computations to continue while data is being transferred. We also offload the disk write process to a separate thread or process, allowing the main training loop to proceed without waiting for disk operations to complete.
(2) Gradient accumulation: Instead of writing gradients disk after every iteration, we can accumulate them over multiple iterations and then write them in bulk. This reduces the frequency of disk I/O operations, improving overall efficiency.
Approximating data value embeddings from checkpoints alone. In situations where only in- termediate model checkpoints are accessible and per-training-step (projected) gradient vectors are unavailable-sut such as when modifying the training loop's implementation is impossible or when disk storage is limited-we we can adapt our approach by assuming that there only one gradient update step between each checkpoint, similar to assumptions made in other data attribution literature (Pruthi et al., 2020). Under this assumption, we compute the gradient for each training point at the checkpoint immediately following its corresponding training iteration. These estimated gradients then used to execute the backpropagation algorithm, enabling the computation data value embeddings without requiring gradient saving during the original training run.
Dataset-level attribution through embedding aggregation. In practical applications, stakeholders often require valuation at the dataset level rather than for individual data points. To address this need,
25

---

Published as a conference paper at ICLR 2025
natural extension of approach is to compute the data value embedding for dataset by summing the data value embeddings of all constituent data points from the same source. This method offers significant advantage over the summation of expected LOO scores, as it inherently accounts for complex inter-data interactions throughout the training process. However, data value embeddings derived based on first-order Taylor approximations. While these approximations for estimating small perturbations to the model, making them suitable for predicting the effects of removing individual training points, their accuracy may diminish when aggregating over larger sets of data points. The potential discrepancy between individual- accuracy and dataset-level aggregation presents an interesting avenue for future research.
accurate
26

---

Published as a conference paper at ICLR 2025
D INFLUENCE FUNCTION AS AN INADEQUATE APPROXIMATION FOR THE EXPECTED LEAVE-ONE- OUT SCORE
Expected LOO. The expected LOO is an alternative traditional LOO that has been discussed in the past literature (Feldman & Zhang 2020). The expected LOO is the trajectory-specinn LOO averaged over all possible training runs characterized by different random initializations and mini- selections. Formally, it is defined as EL.OO(= ((va)) == IR,,, [((() ( ((ea)) ((Or(w), (
where (w) and (w) denote the final model parameters obtained with and without 2", respectively, under the randomness w which encodes the choices training batch order and parameter initialization. While the expected LOO offers a general assessment of data point's influence by averaging over multiple training runs, it may obscure the variability introduced by stochastic training dynamics. In contrast, by accounting for factors such random initialization and mini-batch selection in specific run, we argue that the trajectory-speeifi LOO provides a fine- fine-graing grained assessment of a data point's s impact for the trained model. This particularly important in practical scenarios, such deploying specific model for production, where stakeholders are interested in the valuation of data with respect that specific deployed model rather than the general learning algorithm.
While the influence function provides valuable insights, it overlooks the specific training trajectory. This raises the question: Can the influence function be interpreted as an estimate of the trajectory- specific leave-one-oul score?
We consider the following training batch sampling process. Let a == (Bo.... BB--1) represent a fixed sequence of training batches formed from the leave-one-out dataset D 2². The training point z" is uniformly likely to be added to any one of the training batches o. Additionally, denote (t-) (Bu. ... ... B., {2")...... Br-1) as the training batch sequence where x" is incorporated into the t. batch. Let o(o denote the model parameters at the k-th iteration when training with batch sequence ((,), and let H(ol!t)) denote the Hessian matrix at the k-th iteration when training on sequence of (t..)
When the specific training iteration t, where the training point of interest 2" is added is unknown, it is natural to estimate the expected influence score across all possible scenarios. The expected influence score for =" based on the unrolling differentiation approximation is given by:
T-1
( vecolotin)
E T-1] The Ve(ol (val))) T
II lkt,+1
(I ne nH
(14)
Theorem 5 (Influence Function Approximation). Under the following assumptions: (1) Model Approximation: 8 g(olto)) x Or for all t, and k = O....,T - (2) Hessian Approximation:
H (1))
~ H(+) = EEED V2l(Or, z) for all k; (3) Constant Learning Rate: tt = 1 for all O. the expected influence score in Equation standard influence function formulation for large T:
(4) simplifies and converges to the
14 = Vi(Or, 2((0))((vat))
Σ (2((0r,2) SED
ve(Or,2")
Implications. The derivation demonstrates that, under the stated approximations, influence function effectively approximates the expected influence score derived from the unrolling differentiation approach as T becomes large. This approximation indicates that the influence function may not fully represent the true leave-one-out score because relies on simplifying assumptions-such as approximating all model checkpoints and Hessian matrices to be identical often do not hold in practical training scenarios. In real-world settings, model parameters evolve significantly throughout training, learning rates typically scheduled to change over time, and the Hessian matrices can vary considerably between iterations. These factors undermine the validity of the assumptions, thereby limiting the effectiveness of the influence function an approximation for the leave-one-out score.
27

---

Published as a conference paper at ICLR 2025
Proof. Assume that all we have access to is the final model checkpoint 0y := ofo(tei) for a specific realization where f = 4 ~ [T 1]. Under this assumption, the best approximation make is:
(e olol x4 OT
for any and k o,..., T Additionally, approximate the Hessian matrices as:
(
(2((0T,2)
(15)
ZED
and assume a constant learning rate 7)- = for all 0, T - 1.
With these approximations, Equation simplifies to:
T-1
14
B1 (VE(Or, T
II Amttat1
(I - n Ve(OT;2")
(16)
(VE(OT,2(vat)) Beangr-11 ())mn())))--a- 1o Ve(0T,2")
(17)
T-1
T ve(er. s(al)) T
Σ
(I- nH (+) 1 )-) Ve(01,2")
(18)
ve(Or. 2(val))T T ()
ve(0r,2")
(19)
vi(0r, ( (
(=((0,22)
vi(0r,2*)
(20)
EED
The approximation in the fourth step arises from summing the geometric series of matrices. For sufficiently small and T large, we have ET-((I (I n = (n)
28

---

Published as a conference paper at ICLR 2025
E ADDITIONAL EXPERIMENTS
E. BASELINE & IMPLEMENTATION DETAILS
Fidality Evaluation (Section 5.1. Given the computational intensity, we conduct our experiments on subset (10%) of the MNIST using an MLP with two layers with 128 neurons in the hidden layer. We train the model with standard SGD with a learning rate 10 for 10 epochs. We randomly pick 100 data points and compute their ground-truth trajectory-specifi LOO score. For the single epoch removal, we remove the data point from the last epoch. For this experiment, we do not use random projection and use the full gradients. For the comparison with influence function, we use the state-of-the-ard art implementation from LoGRA (Choe et al. 2024) with the damping term set to be 10 3 following Bae et al. (2022).
Large-scale Experiments in Section 5.2
5.3 and E.4 Our experiments focus on two language
models: Pythia-410M and GPT2-Small. We train these models on two commonly used datasets in the literature for large-scale language model training: (1) A 1 1% subset of the Pile dataset (Gao et al. 2020), and (2) Wikitext- 103 (Merity et al. 2016). We note that our choice of model architecture size primarily constrained by the available GPUs in current setup. However, this limitation does not diminish the significance of our findings. With enough computational resources (e.g.. 8 H100 GPUs), our method is readily applicable to perform data attribution for billion-scale model training.
For both settings, the sequence length is set to 1024. The learning rate is set at maximum of 3 x 10-4, We use AdamW as the optimizer with a weight decay of 0.1, and beta values set to 0.9 and 0.95. Gradients are clipped at a maximum value of 1.0 to maintain stability during training. The batch size is set to 16, with a learning rate warmup of 2000 iterations followed by cosine decay.
For all experiments, for storage reasons, we compute and store projected gradients and data value embedding on linear layers of the model only, with the projection dimension set to 1024 per layer. However, we stress that this is not restricted by computation but by disk storage limit.
E.2 ADDITIONAL RESULTS FOR FIDELITY EVALUATION (SECTION 5.1
Evaluation on more epochs. Here, we show additional results for the fidelity experiment in Section 5.1 where the model is being trained for a longer time (10 epochs), in which case the model is closer to convergence. Figure 5 that even in this case, data value embedding still has a high Spearman correlation with the ground- LOO in both settings, and the influence function exhibits almost no correlation with the LOO score.
Greaadmntt vi Data Vakas EabbuARig
Croaadtrtth VS Inficence Finedas
Greendrrchh DatA Wate Ratediing
Crosedtrat Infarsoo Peretee pradder
Datn Valte Tabbothiie
0.004
Iullessee Purtiio
EL004
0.003
DAUA Vallee Sastesthiie
Infowmsee Pusriioo
0.004
6.000
0.002
0.002
0.00
C.002
0.000
0.003
-0.01
0.001
0.000
6,000
-0.02
-0.001
(a)
-0 -0.00-3.000 00 0.000 6,00 0.004
(b)
-0,000
0.030
0.502
0.004
(c)
-0.02 -0.01 0.00
901
(d)
-0.000 0.005 0.000 0.002 0.0L0
Figure 5: The correlation between ground-trute -tuuh LOO when the MLP is trained for 10 epochs and the estimation obtained by (a) the data value embedding method and (b) the influence function for single
epoch removal. (c) and (d) present the corresponding correlations for all-epoch removal.
Evaluation on different architectures. To demonstrate that our method's effectiveness extends beyond simple MLPs, we evaluate data value embedding on CNN architecture consisting of two convolutional layers (with 32 and 64 filters, respectively, each followed by 2x2 max pooling) and a final linear layer. We train the model on MNIST using SGD with learning rate 10-2 for 10 epochs. Following the same experimental setup as with MLP, we randomly select 100 data points and compute their ground-truth trajectory-specifie LOO scores for both single-epoch and all-epochs removal settings. Figure 6 shows that data value embedding maintains strong correlation with ground- truth LOO scores, achieving Spearman correlations of 0.818 for single-epoch removal (Figure 6 (a)) and 0.682 for all-epochs removal (Figure (b)). These results demonstrate that our method can effectively approximate data influence across different neural architectures.
29

---

Published as a conference paper at ICLR 2025
Grouddruth Data Value Eubeeddng (Speacueo Corveltion: 0.818
Grovadrruth Data Value Erahediizng
[Spearmen Cocteiatio: 0.052
20)
Data Valle Enbeddiing
0.0005
Data Value Emibedring
0.002
0.00d
0.000
-0.002
-0.006
-0.0015
-0.008
-0.001
0.000
0,001
-0.005
0.000
(2)
Groundtreth LOO Score
(b)
Groundtrath LOO Score
Figure 6: Scatter plot showing the correlation between ground-truth LOQ and data value embedding
method when training a small CNN on MNIST for 10 epochs, where (a) is for single epoch removal (b) for all-epoch removal setting.
E.2.1 EFFECTIVENESS ON MISLABELED DATA DETECTION AND DATA SELECTION
In addition to comparing our results with ground-truth training run-specific LOO, we further evaluate the performance of our data value embedding algorithm in the task of mislabeled data detection and data selection, two standard benchmarks in data attribution literature. We compare several data attribution baselines, including cettraininn-basele Data Shapley (Ghorbani & Zou 2019), KNN- Shapley (Jia et al. 2019), Influence Function (Koh & Liang] 2017). Trak (Park et a 2023). Empirical Influecce Functions (Feldman & Zhang] 2020), and Datamodels [yyas al. (022).
Experiment settings. We use Saaggeet-preetaiing ResNet18 as the architecture in the experiment. Given the computational intensity of retraining- methods, we conduct our experiments on subset of 1,000 samples from CIFAR-10 dataset. We use Adam with a learning rate 0.001, weight decay of Ie-4, and label smoothing of 0.1 over 50 epochs. The learning rate reduced by a factor of 0.1 every 10 epochs. The batch size is set to 64. For retraining- based techniques (Retraining based Data Shapley, Empirical Influence Functions, Datamodels), we estimate the corresponding attribution scores with 1000 model training runs. For Trak, we set the projection dimension to be 2048. For KNN-Shapley, we set K and use the features extracted from the last linear layer of ResNet18. Both experiments included 10% random label noise to reflect the challenges of real-world data.
The mislabeled data detection and data selection benchmark here mainly serves to evaluate the fidelity of our algorithm in settings where wround-truth LOO computation is infeasible. However, stress that data value embedding is not specifically designed for those tasks. Rather, it is developed as an interpretability tool and a mechanism for real- cal-time data valuation, with potential applications in data marketplaces and addressing Al copyright disputes (Wang et al. 2024).
Remark 2.
I. Mislabeled Data Detection.
Table showws that KNN- ENN-Shapley achieves the highest accuracy in detecting mislabeled data, likely due to its sensitivity to label inconsistencies. Retraining betrai baased methods (Retraining-base based Data Shapley, Empirical Influence Functions, Datamodels) exhibit the lowest performance, which can be attributed the inefficiency of Monte Carlo sampling and the inherent stochasticity during retraining, as discussed in Wang & Jia (2023a). Among techniques requiring only a single training run. Trak underperforms relative to other methods. This observation aligns with findings from original paper (Park et al.) 2023), which suggests that ensemble methods are often necessary for optimal performance. Notably, data value embedding and influence function achieve comparable performance, outperforming all other techniques except KNN-Shaple The strong performance of these methods likely stems from their deterministic nature, which provides more consistent and reliable results.
II. Data Selection.
Table 4 demonstrates that Data Value Embedding outperforms all existing data valuation methods in the task of data selection. Retraining- methods (Data Shapley, Empirical Influence Functions, Datamodels) show limited effectiveness due to the high variance introduced by Monte Carlo sampling and learning stochasticity. While the influence function and Trak do not require model retraining, their performance is constrained by assumptions that often do not hold in practice, such as model convergence and strong convexity. KNN-Shapley provides stable valuation results. However, it assigns similar scores to similar data points, potentially reducing dataset diversity among the selected data subset. In contrast, Data Value Embedding considers both data characteristies and temporal
30

---

Published as a conference paper at ICLR 2025
Method
Performance (Mean = Std)
Data Shapley (Ghorbani 7ou
2019
0.582 (0.029)
Empirical Influence Function (Peldman Zhang 2020
0.552 (0.017)
Datamodels (Ilyas el al. 2022
0.520 (0.008)
KNN-Shapley Jia ct al. 2019 Wang & 2023b
0.760 (0.018)
Trak (Park et al
2025
0.511 (0.012)
Influence Function (Koh & Liang 2017
0.654 (0.054)
Data Value Embedding (ours)
0.667 (0.031)
Table 3: AUROC scores of mislabeled data detection task with various data attribution techniques on CIFARI 10 dataset. The higher the AUROC score is, the better the method is. The results are across three different training runs (the randomness comes from construction of corrupted datasets), where we show the standard deviation in O)
ordering in training, allowing similar data points to receive different scores based on when they appear in the training sequence. This temporal awareness helps maintain dataset diversity while identifying valuable samples.
20
40%
60%
%
Random
0.350 (0.010)
0.461 (0.010)
0.525 (0.004)
0.559 (0.003)
Data Shapley (Ghorbani Zou 2019
0.317 (0.047)
0.468 (0.010)
0.527 (0.004)
0.570 (0.008)
Empirical Influence Function (Fcldman Zhang 2020
0.342 (0.004)
0.466 (0.01))
0.530 (0.009)
0.568 (0.010)
Datamodels (Tlyas el al. 2022
0.342 (0.004)
0.465 (0.004)
0.534 (0.010)
0.559 (0.005)
KNN-Shapley (Jia al. 2019
0.354 (0.017)
0.478 (0.007)
0.525 (0.015)
0.563 (0.005)
Trak (Park el al. 2023
0.329 (0.021)
0.443 (0.030)
0.517 (0.016)
0.572 (0.009)
Influence function (Koh Liang 2017
0.320 (0.033)
0.450 (0.028)
0.530 (0.015)
0.580 (0.004)
Value Embedding (ours)
0.391 (0.007)
0.518 (0.008)
0.566 (0.005)
0.604 (0.009)
Table 4: Test accuracies when training ResNet18 on high-value data points selected by various data attribution techniques. To be able to compare with techniques that require model retraining, for each training run we randomly sample size-1000 subset of CIFARIO 10 dataset (with 10% data points being mislabeled). The results are across three different training runs (the randomness comes from construction of corrupted datasets), where we show the standard deviation in O.
E.3 ADDITIONAL DISCUSSION AND RESULTS FOR SECTION 5.3
F. EXPLANATION OF INFLURNCE TREND
1. Parameter initialization and warmup training are important for the final model perfor- mance. The blue curve in Figure (b) illustrates the trend of average training data gradient norm throughout the training process. We observe that gradient norms are typically large and unstable during early training (t, S 2000). As training progresses, these norms decrease rapidly, leading to significant reduction in the eigenvalues of the Hessian matrix H, ~ ExEB, Ve(O,, VE(E,, =. Consequently, when ((( is significantly larger than later training gradients, the norm of data
value embedding | +1 (I - 1/: H.) V()O,, remains substantial. This results in early-stage data points maintaining significant influence until the end of training. Figure 7 (a) further illustrates this phenomenon. The purple curve shows that training data points from the High-impact Warmup Phase, while experiencing large drops in influence as training progresses, still maintain higher influence than later data points. This observation aligns with the well-established effect that model initialization and/or warm-up training plays crucial role in training performance, effectively initializing model parameters and gradually preparing the model for more complex learning tasks.
Influence saturation from future data. As the model enters a relatively smooth loss regime (t, > 2000 in Figure 7 (b)), the training data gradient norm decreases much more slowly. In this phase, the magnitude deflation effect from I (I 7/e H,) remains significant for relatively small t., while the training gradient norm | )I does not differ significantly between earlier and later
||VÉ(0..)] for t, t,. creating low-impact basin during the carly-to-midee training stage. In this basin, influence scores are lower than those of data points from both the very early and later training stages. The red curve in Figure 7
training points. This results in [ (I (HH.)(((8))
31

---

Published as a conference paper at ICLR 2025
(a) demonstrates this trend, showing influence scores for these points gradually decreasing during training and eventually falling below those of later training data points. This pattern aligns with the phenomenon of catastrophic forgetting, where the model appears to "forget" the influence of data from this middle phase as training progresses. One might initially think this phenomenon is connected catastrophic forgetting, where the model appears to "forget" the influence of data from earlier training phases as it progresses. However, we note that a data point's 's influence score decreases the most when future data points are similar to it, which is different from catastrophic forgetting. Intuitively. if future points are identical, the presence of the earlier data point in training becomes less relevant to the model's behavior.
In Figure 7(b), we consider simplified setting where we approximate Hessian with GGN ma- trix and assume all training gradients are orthogonal across different iterations. Under these as- sumptions, H₂ x Ga = E.EB. Ve(O,, =) =)T becomes a sum of rank-1 matrices that have non-overlapping eigenspaces. Given the orthogonality assumption, we have G,G, = for * 5, and the product II= (I - n, simplifies to - Σr Gr Since each G. = E.ER, Vi(O,,, V()(O,, 2) is sum of rank-1 matrices along orthogonal directions, the trace of this product can be analytically computed as p Ei=,, The Eien, [[V 2)1l where p is the dimension of parameter space. Furthermore, if we assume Vi(De,) follows a Gaussian distribution, then (I Eie, Ge.;) VE(),))(...)) follows a scaled chi distribution since it's the norm of Gaussian vector after linear transformation by orthogonal projection matrix. This enables us to analytically compute its expected value, as shown by the green curve in Figure 7 (b).
Data Jufnueece on Intermediate Checkpoints
Intuition of Phoses in Influence Dynamics
Train Batch Idk
|al||
2 100
1000- 2000
15000-16C00
1/0 mn No 0Hi
5000-5000
20000 20000-21000
Influence Score
10000-11000
25000-25000
lalil Upp N (M40
109
High-Inpoct
Wormup
Grodusl
Ascending
6x10-1
Low-Irpact
Bunin
(a)
10000
20000
30000 Training Steps
40000
50000
60000
(b)
0
10000
20000
30000
40000
50000
60000
Steps
Figure 7: (a) (same Figure in the main paper) Influence scores of data points from different training stages on intermediate model checkpoints throughout training. The x-axis denotes the number of training iterations, and the y-axis represents the influence score of selected data points the model at each checkpoint. (b) The blue curve shows the average gradient changes as model training progresses. The orange and green curves are analytical curves under a simplified setting, where the orange curve is the analytical trace of II² (I - 17. H,) as i, increases, and the green curve shows the norm of data value embedding for Caussian-distribure gradient under this simplified setting.
E.3.2 ADDITIONAL DETAILS FOR FIGURE (B)
In Figure 1(b), we compare different strategies for applying online data selection during model training. The online selection process identifies high-quality training batches by (1) sampling a candidate pool of training points with size 2B, where B is the desired batch size, (2) computing the gradient cosine similarity between each candidate point and a small validation batch (randomly sampled from the full validation set), and (3) selecting the B points with the highest similarity scores to form the next training batch. This procedure incurs significant computational overhead, requiring additional forward and backward passes for similarity computation at each selection step. When not performing online selection (i.e., during the "random selection" phases), we simply sample training batches randomly. Notably, the model processes the entire training dataset regardless of the selection strategy what varies is only how batches prioritized during different training phases. The "Early+Late" strategy applies online selection only during iterations I 1-2000 and after iteration 20000, while using random selection in between. This selective approach achieves 96% of the performance
32

---

Published as a conference paper at ICLR 2025
improvement of continuous selection while reducing the computational overhead by more than 5x, suggesting that precise batch selection most critical during the early and late training phases.
E.3.3 ADDITIONAL RESULTS
Figure X presents additional results on the data influence scores of training data across different stages of LLM pretraining, using more datasets and model architectures. We observe that the data influence scores on the final model can consistently be categorized into three distinct regimes throughout pretraining.
Figure 9 shows the results when using pretrained models downloaded from Huggingface, In this scenario, the situation diverges across different datasets. Notably, when continually pretrain the Pile dataset, no gradual ascending phase observed at the end. However, when GPT-2 has already been pre-trained, continuing pretraining Wikitext-1033 once again exhibits a gradual ascending phase. This likely because Wikitext- 103 is a relatively small dataset, and fine-tuning for three epochs can easily lead overfitting, illustrated Figure I0 (d).
Data Influence Scone Data Points from Different Training Itarations (GPT) T2 on Pile)
0.125
Low- Low-It
Grodual Ascending
Data Influence Score
0.100
0.075
0.050
0.025
High-Impact
Wormup Phose
0.000
5000
10000
15000
20000
25000
(a)
Training Batch Idx
Data Influence Score
Points from Different Training Iterations (GP WikiText)
0.150
Low-Impact
Gracual Ascending
Data Inenue Score
Basin
0.125
0.100
0.075
0.050
0.025
High-Impact
Warmup Phose
0.000
5000
10000
15000 20000
25000 30000 35000
(b)
Training Batch Idx
Figure 8: Average data influence scores per training batch, measured against the final model's loss
where (a) GPT2 trained on 1% of Pile, and (b) GPT2 trained on WikiText- for 3 epochs.
33

---

Published as a conference paper at ICLR 2025
Data Influence Score of Data Points from Different Training Iterations (Pretroined GPT2 on File)
0.030
Low-Impact Regime
Data Influence Scare
0.025
0.020
0.015
0.010
0.005
High- High-Impact
Wermup Phose
0.000
5000
10000
15000
20000
25000
(a)
Training Batch Idx
Data Inf lufneence Score of Data Points from Different Training Iterations (GPT2 on WikiText)
0.05
Low-Impact Basin
Data Influence Score
Gradual Ascending
0.04
0.03
0.02
0.01
High-Impact
Wormup Phjaee
0.00
5000
10000
15000 20000 25000 30000 35000
Training Batch Idx
Figure 9: Average data influence scores per training batch, measured against the final model's loss where (a) Pretrained GPT2 trained on 1% of Pile, and (b) Pretrained GPT2 trained on WikiText-1133 for 3 epochs.
Pythia 410M trained on Pile
GPT2 trained Pile
10
Volidotion Loss
Velidation Less
6x100
10P
Loss
Loss
4x
10P
3x100
vx
(a)
20000
40000
60000
0
20000
40000
60000
Steps
Steps
Pretrcined 6PT2 trained Pile
Pretrained GPT2 trained on Wikitext
3.2
Traning Loss
3x 10
30p
30
2 10
29.x09
20P
20
10P
26v10'
25+10P
60-10
Volidetion Less
10000
20000
30000
(c)
Steps
(d)
10000
20000
30000
Steps
Figure 10: Loss curve for the training.
34

---

Published as a conference paper at ICLR 2025
E.4 QUALITATIVE EVALUATION
We conduct a qualitative analysis to examine the similarities between a test data point 2(vel) and the most valuable data points identified by data value embedding. In this experiment, we set ((eat) to be identical to one of the training data points, making the most similar data point its own repetition. In data valuation literature, the influence score of a training point on its repetition is usually referred to as "self-influence" (Koh & Liang 2017) and is being used to measure memorization (Feldman & Zhang 2020). Intuitively, the self-influence should be the highest among all training points,
Figure 11 shows representative results from training GPT-2 on Wikitext- 103 over three epochs, where the test data is about military video game As observed, for model checkpoints after the 2nd and 3rd epochs, the test data point's 's repetition achieves the highest influence score, as expected. However, for the model checkpoint after the 1st epoch, the most valuable data points are not the repetition but rather similar data about war history. This discrepancy occurs because, during the first epoch of training, the repetition of the test data point resides in the low-value basin identified in Section 5.3 resulting in lower self-influene score as subsequent training progresses. Additionally, we observe that the influence function may incorrectly identify irrelevant data points as highly influential (e.g.. the Popular Music completely irrelevant military video game but being identified as the second most valuable data), possibly due to its bias towards data points with high gradient norms, as also noted in Barshan et al. (2020). This limitation underscores the advantages of data value embedding in providing more accurate and context-aware data influence assessment.
Turt Data (militory video gama)) repetition in training
pertory sacie operrticn pitad tpa pi Siven bagan devecoment carying over cenal eniing the nation duligg de-oa rakired alunded the police
jelire voiaay nageriy
expander in foverror year Trves achereel tnto orignnl simatios
Data Value Embedding
Influence Function
Epech
Epech
Epoch
hystory
call govensr
Nost Valubble
Hedtor
in arter confict letvers the sadien and car nemil ceniee fume of the Caplen Takes
Risembe ghing BY) ocer
test dones repetition from 2rd epach
test dano's repetition from 3 epech
dotc's repotition
Chess Gore
Papulor Music (ireeleant daho)
Flctes plaed tu God de bettoo df
Fashet and Knhuara Walto
UppeC
caltic sl net ahs
oddrgins ent, obl
the
taa petorit mun moborate alc aye interted huanniig Billing Saine
code come
24 Valuaale
brown task Airhogh
arier snn Athhage
test nepetition from
und leve tn wht
to.
cooll tour upen M.
crychoogoal
encomptio geeer anl t0 netee
procraiie the come neve 30 bafees "Ma U
loat. leptred sudoo Ass tosn rocil
Strategy Video
Haltory
listary
hittory
Apu El G-D stateg
The KONE serpeadd be Antaraara
invoreed by
copoatioos
fowe and
in fhe de rour to hor
tocked citeraes Iea mntrreed Woley
3- Velussl
aecoi be Ca Herie nes recooua
Pa th dvied on teccom
turts Alvs Mes Word nor Seree
th slee il thur Riaan wadee
aha be the Enitein
coan and loley
tell therr lat can widee whlviiy
kearsat ied 34O" was Hambee better Uni Pulll trsrpercy
sagiie soseeen' the hour shalk.
wish
Pe
and a bread aenuue bx sutsider of du
sue tamel carara wase the
Figure 11: Visualization of (left) the evolution of the top-3 most valuable training data points identified by data value embedding throughout 3 training epochs and (right) the top-3 most valuable training data points identified by influence function. We use GPT-2 trained on Wikitext-103, with the test point being a repetition of a training data point related to a military video game. The common words between the test and training data are highlighted in orange
35

---

Published as a conference paper at ICLR 2025
E.5 ABLATION STUDY: ERROR FROM PROJECTION DIMENSION
We examine the error introduced by the random projection of gradient vectors, as discussed in Section 4.2 Specifically, we evaluate the Spearman correlation between data influence scores when using per-layer projection dimensions in {256, 1024, 2304] and compare these to a larger per-layer projection dimension of 4096. Since computing the ground truth without projection is infeasible, we use projection dimension 4096 as reference point for comparison. Additionally, we compare our results to LoGRA (Choe et al. 2024), the most efficient current implementation of the influence function, which also employs random projection to store data attribution information. Due to the computational and disk storage constraints, these experiments were conducted using GPT-2, trained on 5% of the Wikitext-100 103 dataset. The results, shown in Figure 12 (a), indicate that our data value embedding method achieves higher Spearman correlation compared to the influence function. While results demonstrate clear advantage, more in in-depth analysis of the observed improvements would be an interesting direction for future research.
Spearman Correlation with Results of Dimension 4096
Spearman Correlation
10
Data Points from different Training Stages
0.9
0.8
0.0
Sgeommn Correlation
Spcoreen Corvelation
07
0.6
066
0.5
0.5
04
0.4
03
02
62-
01
Influctce Data Value Embeddinn
01
Connelation 1024 409
0.
0.0
3
(a)
Dimension
(b)
Epachs
Figure 12: (a) Comparison of Spearman correlation between data influence scores function of projection dimension.
E.6 EXAMPLES OF BAD DATA
To demonstrate Data Value Embedding's ' capability in identifying potentially problematic training examples, we examined the training data points from the Pile dataset (Gao et al. 2020) that received
the most negative influence scores under the same experiment settings in Section 5.3 Figure T3 and 14 show these examples and their influence scores.
Our analysis revealed several types of training data that could potentially harm model performance. First, find quite a few code-related samples that, while syntactically valid, provide minimal educational value for language modeling. These include YAMI. configurations with simple numeric arrays, raw binary data represented hexadecimal strings, and pixel-by-pixe image data in array format. Such examples contain little information about code structure or programming patterns while introducing noise into the training process.
Beyond code snippets, we found examples of text data that could potentially bias the model's learning. For instance, math problems (Figure [3]s last example) that follow identical question formats could bias the model toward specific phrasings (e.g., "What is ...."") rather than developing diverse language understanding. We also identified articles that, while containing meaningful content about important topics like data privacy, suffer from poor formatting with missing punctuation and paragraph breaks (Figure [4]s s second example). Such poorly formatted content, while topically relevant, could potentially degrade the model's ability to learn proper text formatting, punctuation usage, and document structure.
36

---

Published as a conference paper at ICLR 2025
Influence Score: -0.25
Influence Score: -0.15
Rockefeller Center security worker
may have to hire his own guard now.
Influence Score: -0.11
Gellatt 01-0/2 (1200)-200 (-RCC--MM)
when COMACHAANN
the wiie 00-50 CAA.ECAAAOOO 200007
Colacts CLN4D-552
(
Influence Score: -0.079
A
A
À
À
A
A
A
Á
A
A
A
A
A
A
A
A
À
A
A
Dona
Ld
R.
Ross
Á
A
A
Á
Á
Á
A
A
A
A
A
A
A
A
A
A
A
Á
A
Á
A
A
A
A
A
Just
Date Submi A
Ā
Á
Á
Á February 22,
2005
Date D
Á
Á Á Á Á
February 23, 2005
Figure 13: Examples of training data from the Pile dataset identified as potentially problematic by our method, along with their influence scores. The examples include configuration files that provide minimal learning value for language modeling, as well repetitive mathematical problems with identical question formats.
37

---

Published as a conference paper at ICLR 2025
Influence Score: -0.078
Satarant:
10:
--5
Influence Score: 05.
boglle
finld
show malte in io arpuetty dieect tween
as fach rostes baatts Lowe aeaelard the foother bndd custris cerpeted polvecesl sovrriaate artirs eiection fooaboo net cloes deto Coerral tatere by dine whies mooe other
to collect deto grofties dve 37 cillten peopla
seancied Google for Goaple soarches practics gaee sapord sommees tatte what they Agas baly people thooup settel
wetell serss paies cansuntiyy fly 1e ling cimpetire resourche cats highle
information tyaically clastffied
frem sinae Fibres
ciilley around seldiem thei
Influence Score: -0.054
cancer, raultcup Vigher recureecc Fog FCte.
Fooo peESez Fronsooooe
de
da tertett Bhechhatees
Selts Elaailll bott
tuBiclust17 bubto de 290000
Fromdd
Gobmethell sedioee du tore-ii
Influence Score: 04.
156 29
25 253
256
2511
355 286
20 100
111.
11. 100
111
132-20 125
10.-20 155
152 105
180 060
185
157 190
25,
176
136
157
135
66 04
22 132
12:002.400
15
13
28.08.20
65,
100
15, M. M
Figure 14: Additional examples of training data identified as potentially problematic, showing text content with poor formatting (missing punctuation and paragraph breaks) and low-information code snippets with repetitive numeric arrays.
38


Reviews:
[{"review_id": "vRrDIDNw9M", "reviewer": "Reviewer_Qbpo", "summary": "This paper presents a new method called \"Data value embedding\" to approximate trajectory-specific Leave-One-Out (LOO) error for individual training samples, which encapsulates cumulative influence of training samples in a training trajectory. It gives some interesting insights of how influence of training batches change during a training process. The authors also provide several tips for reducing the computational cost of the method in practice.", "questions": "1. In Line 169, the loss change is estimated by the first-order Taylor expansion, which indicates the $\\theta^{'}_T$ and $\\theta_T$ should be close, however, when a sample is more important, $\\theta^{'}_T$ and $\\theta_T$ will be more dissimilar. How this approximation can be valid in such a case?\n2. Is there any trajectory pattern of the Hessian $H_k$ observed during the training process?\n3. If removing the samples with negative influence scores, will a model get better performance?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The method introduces a novel concept by capturing data influence in a trajectory-specific manner rather than assuming permutation invariance, which is a common limitation in conventional influence estimation methods. It outlines assumptions and derives an approximation error bound, lending theoretical credibility to the approach. The approach is designed with computational efficiency, including several techniques to reduce the memory and computational cost. It enables identification of high-value data points at different stages of training, allowing practitioners to curate datasets more effectively.", "weaknesses": "The method is explicitly tailored for SGD and is not readily applicable to other popular optimizers like Adam. Although using SGD as a proxy is discussed, this limitation restricts the method's applicability to a broader range of models.\nThe evaluation with ground truth focuses on specific datasets and model types (e.g., MNIST, MLP) due to the computational cost, which may limit the generalizability of the findings. \nSeveral assumptions are made in this paper, such as model layer independency, learning rate scheduling, which might not be satisfied and lead to reliability issue in such circumstances.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Pzj6obqjOO", "reviewer": "Reviewer_gYsk", "summary": "This paper develops computational tools to quantify the influences of individual data points in a training session, assuming the influences are sensitive to the data order. The authors formalize the idea of trajectory-specific leave-one-out error (TSLOO), accounting for the sequence of training data. To address the computational issue of TSLOO, authors propose an efficient approximation of TSLOO using Gauss-Newton approximation. By using this technique, authors reveal distinct sages of model training such as high-impact warmup phase, low-impact basin and gradual ascending and provide an explanation.", "questions": "I suggest a dedicated related work section to clearly delineate authors' novel contributions from existing concepts in the literature (e.g., Hara et al., 2019).  \n\n1. Judging from the line 127 to 134, I suppose authors consider learning of just one epoch? As in the second epoch, the counterfactual sample will again lead to the difference between theta and theta'. Do authors consider multiple epochs? and if it's just one, could authors explain why this is sufficient or how it might generalize to multiple epochs.\n \n2. line 148, isn't Hessian H a function of the parameters? \n\n3. line 239, it isn't clear where does the product of Hessian go at a glance. I recommend authors write briefly about the derivation of Appendix C.3. \n\n4. Line 435, \"This observation aligns with the well-established effect ...\" Please cite, as it will also help a general reader (like me) to understand the background of model warm up/generalization. \n\n5. Figure 4, \" the y-axis represents the influence score of selected data points on the model at each checkpoint.\" How are these data points selected? It would help with reproducibility as well as the representative of the phenomenon.  \n\nalso, what is the \"green curve\" mentioned on line 442.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Quantifying data influence is an important task and is crucial active sample selection. However, existing method, the influence function, does not care the order of samples' arrival, making it unsuitable for vast majority of stochastic algorithms. This paper address an important research gap. \n\nThis paper is mostly well-written. The mathematical ideas and their intuitions are clearly presented and is easy to understand. \n\nThe discovery of stages of sample influence is potentially a significant contribution to the wider machine learning community. It reveals how SGD utilize each individual sample at different stages of the training and highlights the importance of selecting data at the right time.", "weaknesses": "Probably due to the lack of a dedicated related work section, it is not clear where does the authors' contribution begin. For example, has TSLOO been studied before or is it a novel concept proposed by  the authors? In Section 2, the first paragraph seems to suggest that this is authors' proposal. However, a later sentence says, \"while the technique of unrolled differentiation Hara et al., 2019 explicitly aims to approximate TSLOO ...\" It seems the idea of TSLOO already exists in earlier works. If this is true, authors should cite existing works when introducing TSLOO at the beginning of Section 2. \n\nSimilarly, the approximation in Section 3.1 was also used by Hara et al., 2019 and is \"well-established\" in the literature. This makes me wonder how is the proposed work positioned among existing literature. For example, comparing to Hara et al., 2019, what is the **methodological innovation**? If everything before Data Value Embedding (DVE) is a part of literature review, the authors should make it clear. \n\nDVEmb in Theorem 2 is an interesting idea. However, the interpretation of this quantity is unclear to me. For example, \non line 247, \"this expression suggests that similar training points encountered in later iterations may have a stronger impact on the data influence score of earlier training points. \" I am not sure I understand this statement.  z* are hand picked by the user so why do users care about influence score of earlier training points? Please provide a clearer explanation of this statement and why a practitioner should care about this interpretation. \n\non line 252, if z' is identical or highly similar to z*, their gradients will be closely aligned, leading to a significant change in DVE... I don't understand the meaning of \"change\" here. I guess the authors are talking about **tracking the influence of samples over the training iterations**. However, this setting isn't made clear. I suggest the authors state the context and \"setting the scene\" before explaining the interpretation of Theorem 2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1OVwF5oSML", "reviewer": "Reviewer_Sfva", "summary": "The paper proposes a novel data influence estimation method, Data Value Embedding, that captures temporal dependencies in data influence during model training. It addresses limitations of traditional influence functions by considering the order and timing of data exposure, enabling efficient real-time computation of influence scores in modern, non-convergent training settings, such as those used for large language models (LLMs).", "questions": "Could this method be applied to a closed-source model?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe motivation of this paper is clear: modern training paradigms—especially for foundation models using stochastic algorithms and non-convergent, multi-stage curricula—are sensitive to data ordering.\n2.\tIt introduces a computationally efficient embedding method, making it feasible to apply influence estimation to large-scale models without retraining.\n3.\tEmpirical results demonstrate high fidelity in data influence estimation and reveal nuanced phases in training that inform efficient data selection strategies.\n4.\tIt provides insights on the training dynamics of foundation models: a very brief high-influence region at the start, a much longer low-influence basin, and a region in the later training stage with gradually increasing influence, resuming to a high level.", "weaknesses": "1.\tLimited Real-World Validation: While the paper’s experiments demonstrate high fidelity on small datasets like MNIST and reduced subsets of larger datasets (e.g., 1% of the Pile), the method may not have been fully validated on more challenging, real-world datasets. This leaves questions about its robustness and scalability when applied to diverse, large-scale data used in production.\n2.\tPotential Overhead in Implementation: While the method reduces some computational costs, it still requires considerable storage and processing resources, particularly for storing per-sample gradient information and data value embeddings. For truly large models, such as those with billions of parameters, this may limit its practical utility without further optimizations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Mpx0DCZmq6", "reviewer": "Reviewer_tZN9", "summary": "This paper studies data influence for problems with training data not necessarily permutation invariant. Permutation invariant training data are commonly assumed in traditional data influence estimation methods. The authors proposed the concept of trajectory-specific leave-one-out (LOO) error and further proposed data value embedding to approximate trajectory LOO error. Fast computation was developed using simple dot-product between the data value embedding and the gradient of the given test data.", "questions": "1.\tHow would the proposed data influence handle outliers in training data? For example, if one particular training point was an error or undesirable outlier, would the proposed influence indicate the outlier being very important since it may have big loss differences? That can be undesirable or misleading. It will be helpful if the authors could a). Conduct an experiment specifically examining the behavior of their method on datasets with injected outliers; b). Discuss potential mitigation strategies if outliers are indeed found to have outsized influence; c). Compare how their method handles outliers versus existing influence estimation techniques\n\n2.\tUsing numerical examples, the authors show that early and late regions have high influence and can achieve similar performance improvements in comparison to using data from the entire process. This is quite interesting but also raises a lot of questions. Intuitively, the high influence of the early region is because one starts with nothing and the information on the early region is consequently high. After that, the added information slows down but builds up for the improvements for late region. Thus, it appears to this reviewer that one cannot simply use the early and late regions for the learning process without the middle region. Thus, more clarification and discussions on this is necessary.\n\n3.\tHow would the proposed influence measure work for training time series data with seasonal and temporal dependence?\n\n4.\tAlthough quantification of temporal dynamics of training is interesting, the usefulness of this needs to be further explained. Can this help to detect undesirable outlier training points? Or more effective learning using a subset of training data?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Many machine learning algorithms especially those for supervised learning typically assume the training data points are permutation invariant. Using such an assumption, one can divide the training data for both training and validation using strategies such as k-fold CV, leave-one-out CV etc. As the authors mentioned, many modern machine learning training paradigms especially for foundation models violate this assumption. The goal of this paper is to capture the dependence of data influence on the optimization trajectory during training. The authors proposed the trajectory-specific leave-one-out (LOO) error. Efficient computation was developed as well. Overall, the paper addresses an interesting and timely problem.", "weaknesses": "The problem considered in the paper is certainly interesting. However, it is important to make it clear regarding the type of CV considered in the paper. In supervised learning, CV such as LOO CV was mainly used to estimate the model performance such as prediction errors. The type considered in this paper instead is to quantify the loss change with versus without a particular data point. In this sense, it is perhaps more sensible to call it as influence of the point instead of error. The authors may reconsider the terminology to avoid confusion. If the term of LOO error remains, it is important to clarify the difference of this type and the traditional CV error.\n\nRelated to the previous point, in the traditional sense of CV error or LOO error, we would look for small CV error for model selection etc. In the sense of LOO error in the paper, the error is quantified as model’s loss change on a validation data when the training data point is removed from the training set. However, large change in the loss does not necessarily imply big differences in terms of the model performance between with versus without that particular training point. But the goal was to evaluate the model performance of that point on model trained on data without that training point. More discussion and justification are needed. \nTo address this issue, this review would like to suggest the authors to:\n\n1. Explicitly compare and contrast their LOO error definition with traditional CV error\n2. Discuss the implications of using loss change rather than model performance metrics\n3. Provide examples or experiments showing how their LOO error relates to changes in model performance", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)