f-INE: A Hypothesis Testing Framework for Estimating Influence under Training Randomness
Authors: Subhodip Panda
Dhruv Tarsadiya
Shashwat Sourav
Prathosh AP
Sai Praneeth Karimireddy
Venue: ICLR 2026 Poster (2026)
Keywords: Influence Estimation
Data Attribution
Explainability
Robustness
Primary area: interpretability and explainable AI

Abstract: Influence estimation methods promise to explain and debug machine learning by estimating the impact of individual samples on the final model. Yet, existing methods collapse under training randomness: the same example may appear critical in one run and irrelevant in the next. Such instability undermines their use in data curation or cleanup since it is unclear if we indeed deleted/kept the correct datapoints. To overcome this, we introduce *f-influence* -- a new influence estimation framework grounded in hypothesis testing that explicitly accounts for training randomness, and establish desirable properties that make it suitable for reliable influence estimation.
We also design a highly efficient algorithm *f*-*IN*fluence *E*stimation (**f-INE**) that computes f-influence in a **in a single training run**. Finally, we scale up f-INE to estimate influence of instruction tuning data on Llama 3.1 8B and show it can reliably detect poisoned samples that steer model opinions, demonstrating its utility for data cleanup and attributing model behavior.

Published as a conference paper at ICLR 2026
F-INE: A HYPOTHESIS TESTING FRAMEWORK FOR Es- TIMATING INFLUENCE UNDER TRAINING RANDOMNESS
Subhodip Panda" Department of ECE Indian Institute of Science Bangalore, India
Dhruv Tarsadiya
Shashwat Sourav Department of Physics Washington University
Department of Computer Science University of Southern California Los Angeles, USA
St. Louis, USA
Prathosh A.P.
Sai Praneeth Karimireddy Department of Computer Science University of Southern California Los Angeles, USA
Indian Institute of Science LatentForce.ai Bangalore, India
ABSTRACT
Influence estimation methods promise to explain and debug machine learning by estimating the impact of individual samples the final model. Yet, existing methods collapse under training randomness: the same example may appear critical in one run and irrelevant the next. Such instability undermines their use data curation or cleanup since it is unclear if we indeed deleted/kept the correct datapoints. To overcome this, we introduce f-influence a new influence estimation framework grounded in hypothesis testing that explicitly accounts for training randomness, and establish desirable properties that make it suitable for reliable influence estimation. We also design highly efficient algorithm f-INfluence Estimation (f-INE) that computes f-influence in a single training run. Finally, scale up f-INE to estimate influence of instruction tuning data Llama-3.1- 8B and show it can reliably detect poisoned samples that steer model opinions, demonstrating its utility for data cleanup and attributing model behavior.
INTRODUCTION
Deterministic influence estimate
Uncertainty in influence estimates under training randomness
p p=1
Randomness
= 0.1
p= 0.1
How to take account of training randomness
10
0
+10
Figure 1: Test losses on specific data points vary significantly across training runs due intrinsic non-determinism in ML pipelines. Consequently, influence scores derived from such losses also inherit randomness. Decisions based on single run such as deleting seemingly low-influence data may prove suboptimal in subsequent runs, potentially causing unexpected performance drops. Thus, a key challenge how to properly account for training randomness in influence estimation.
Training data is the fuel that drives the superior performance of various machine learning and deep learning models. Each sample in the training dataset affects the prediction of the model (Adler et al. 2016; Datta et al.) 2016 Koh & Liang. 2017). Thus, estimating the data influence serves as an important tool for enhancing the explainability (Simonyan et al.) 2014 Amershi et al., 2015) and debugging (Adler et al., 2016 Ribeiro al. 2016) of complex classification models and as
"Corresponding Author: Sashhoddpp@iisc.e...

---

Published as a conference paper at ICLR 2026
20
Comparison across Data Staffing Orders
Corfig-1
1.4
Ideal Score (1.0)
Consistency over Random Seeds
6930
Extimateed Influeece
Conrig
0.5
Consitenc)
0.8
0.733
0.7
0.4
Mfueee Parctoo
Faseh
FINE
0.5
Traceli
TIAAK
I-NE
(a) Influence scores for data shuffling
(b) Consistency scores for data shuffling
Figure 2: (In)consistency influence scores across multiple random seeds. Existing approaches such as Influence Functions, TRAK, and Traceln exhibit significant variability due to sensitivity to data shuffling. This leads to low consistency scores. In contrast, our proposed method, f-INE, achieves a much higher consistency score, demonstrating robustness to training randomness.
well as large-scale generative models such as Large Language Models (LLMs). Hence, estimating the influence of training samples on model predictions emerges as a fundamental problem. Data Attribution (Hammoudch & Lowd 2024) is an important research domain that specifically tries to solve this problem. One widely used approach of measuring data influence is through Leave- One-Out-Data (LOOD) retraining, which quantifies the effect of removing a single datum from the whole training dataset. Being prohibitively expensive, current methods (Koh & Liang 2017 Garima et al. 2020 Xia et al. 2024; Park et al. 2023) for influence estimation essentially propose several computationally efficient methods estimate LOOD retraining. However, noted in prior work (Jordan 2023 Karthikeyan & Søgaard [002]] Wang & Jia 2023), current methods are extremely sensitive training randomness stemming from factors such as random seeding, weight initialization, batch size, data shuffling/sampliin
etc. But robustness to training randomness is essential because influence estimation is generally employed identify beneficial or harmful datapoints. Inconsistent scores mean that we have no guarantee that removing influential examples will change our training model in predictable ways. This unreliability fundamentally arises because these methods don't t account for training randomness as shown in Figure I
This motivates our central question:
How to define influence scores that are useful for decision-making even under randomness?
Inconsistency in influence scores. Figure shows that Influence-Function (Koh & Liang 2017) TraceIn (Garima et al.) 2020), and TRAK (Park et al.) 2023) are inconsistent under the randomness induced by data shuffling. We measure consistency using the average Jaccard similarity of the selected sets across multiple training runs of an algorithm. For a set of runs R, we compute our
consistence score (1 ( ΣISER ((T())), ((TA)))T(())) The consistency score lies in [0, 1], with 1 indicating perfect consistency. We train MLP model subset of MNIST under two data loader configurations (Config-1 and Config-2) that differ only in the order the first two class-1 1 samples, while the order of the other samples remains unchanged. We observe large discrepancies in the influence scores of the first class-1 sample across these two configurations. In Config-1, the first class-I I sample seen early during training is assigned high influence, whereas in Config-2, seen later, it receives much lower score. Figure (b) runs multiple seeds and shows a similar trend in influence scores. The exception is our proposed f-INE algorithm that is mostly consistent.
Our approach. To take training randomness into account, we propose new definition of influence termed as f-influence. Our proposed f-INfluence Estimation (f-INE) algorithm computes the influence of particular data point as the hardness of testing between two hypotheses or distributions. The first distribution is computed by estimating the distribution of the gradient dot-product between the test data and the full training dataset. The second distribution is computed by estimating the distribution of the gradient dot- product between the test data and the training data after removing the particular data point. Essentially, the influence of particular data is nothing but how easily one can differentiate between these two distributions. As influence is estimated on a distributional level, our method inherently captures training randomness. Our contribution can be summarized as follows:
To incorporate the training randomness into current influence estimation methods, introduce a new definition of influence termed as f-inftuenc. This new definition of influence is motivated by privacy auditing and is grounded in hypothesis testing and explicitly captures
2

---

Published as a conference paper at ICLR 2026
training- training-time time randomness. Thus, our primary contribution lies in establishing this connection between influence estimation and auditing differential privacy (DP).
Using this connection to DP, we prove f-influence demonstrates useful properties such as composition and asymptotic normality. We then leverage these to design a highly scalable and efficient algorithm to estimate e f-influence
single training run.
We scale our proposed f-INfluence Estimation (f-INE) algorithm to perform data selection for 1--mB-3.1-8B. We test its ability on data poisoning for opinion steering, and show that can reliably identify training samples that are influential in steering the LLM's opinion.
Problem setup. Let D = {z:]²= denote the training dataset of n samples, where each training datum is sampled i.i.d. from some unknown distribution. A model parameterized by 0 is optimized using a randomized algorithm (c.g.. SGD) A: : Z + θ to achieve the trained model 0* Consider θ to be the parameter space, and 1(0, 21) denotes the loss of the model 0 on the training datum 24- Our objective to estimate the influence of a training data subset S [ D on the prediction of a test datum Stest- Let's consider the influence estimation function V : 2 2 -> R takes test datum Stest: and a subset of training data S to produce a score that denotes the influence of S on the model's prediction on Stest. It is important to mention that this estimated influence is dependent on the algorithm A. However, for notational simplicity, we simply denote it as @(zeest, S).
2 HYPOTHESIS TESTING FRAMEWORK FOR INFLUENCE ESTIMATION
Given that training randomness and non-determinism are unavoidable and inherent to ML training pipelines (Jordan 2023), how can we make decisions about which data points might be harmful and should be deleted or helpful and kept? Our key insight here is that this question can be re- re-framed as: if I delete suspected harmful datapoint and re-run my training, will the decrease in loss be statistically significant compared to what would expect from just the training randomness? If so, I'd better delete the datapoint, and we can deem it (negatively) influential. This naturally lends itself to seathesss-testtnng based definition of influence.
Definition 2.1 (Informal: hypothesis testing based influence). Given dataset DD and a subset S C D, delete S from 'D with probability 0.5, run multiple training runs, and measure the distribution of test statistic l. We say is influential on l if we can reject the null in the hypothesis test:
H1 trained on D \ S. The ease with which we can reject the null measures how influential the particular data point was. This is because not being able to reject the null implies that even if we delete S, it will likely have no statistically significant ef- fect on l and we wouldn't 't miss it. On the other hand, we are able to very easily reject the null, this means that deleting S has significantly higher than random effect on and we better pay attention to it. This definition also clearly ties influence estimation with membership infer- ence attacks from privacy auditing (Shokri et al. 2017)
Ho we trained on D)
Accuracy increased 0.1% with probability
Accuracy increased 1% with probability
Cra P.co
D
2 ()dds
Influential?
and f-Differential Privacy (Dong al. 2022)) To flesh out the definition above, we still have to assign sign (positive
d,vs. da
vs. negative influence) and precisely quantify 'case of rejecting null'.
2.1 LACK OF TOTAL ORDERING OF INFLUENCE
Figure 3: Lack of total ordering in in- fluence under training randomness: re- moving di always decreases accuracy by 0.1%, while removing da increases ac- curacy by 1% but only with probabilityy 0.1. Both have the same mean influence, yet it is unclear which one is more in- fluential. This problem arises as there is a lack of total order in defining data influence under training randomness.
Training randomness poses fundamental challenges to defining influence. Consider the case outlined in Fig 3 where we are given two suspected harmful datapoints di and d2. Removing di results in accuracy increase of 0.1% with probability 1, while removing de yields an accuracy increase of 1% with probability 0.1. Which data- point should we deem more (negatively) influential and delete?
3

---

Published as a conference paper at ICLR 2026
If examine the expected change, would say both are equally influential and delete either. However, this is not necessarily correct. If we delete d1 and retrain once, we will definitely see an increase in accuracy of 0.1%. whereas if we delete d2 and retrain once are unlikely to notice any chance i.e. di is more (negatively) influential. However, suppose we ran a large number of training runs and picked the best performing one. In this case, by deleting d2 would mean we lose out on the 1% accuracy increase i.c. d2 is more negatively influential.
Thus, a single scalar (e.g., mean) cannot capture total ordering of influence. Does this mean that we are stuck with computing and comparing the entire exact distribution of everytime? Not quite the minimal sufficient statistic for hypothesis testing (distinguishing between two distributions) the trade-off curve (precision-recall curves) that measures type I and type II errors (Blackwell, 1953).
Key Idea
Under randomness, a strict total ordering of data influence is not well-defined, as it depends on the evaluation criterion. The trade-off curve formalizes this ambiguity: one may emphasize highlighting points that are consistently influential (minimizing Type error) from those with rare but substantial effects (minimizing Type II error).
2.2 f J-INFLUENCE AND G₁ INFLUENCE
As stated in Definition 2.1 we can repeatedly run our training algorithm with the entire dataset D, observing the distribution of f (corresponding to Ho) and similarly compute the distribution without S of S (corresponding to H1) Let us denote P and Q to be distributions obtained in the case of Hu and H1, respectively. Our hypothesis testing problem is to distinguish P and Q. The test statistic f can correspond to losses gradients Stest- Following (Dong et al. 2022), define Type-I and Type-II II errors in our setting, along with their trade-off curve as below.
Definition 2.2 (type-I and type-ll errors). Consider a rejection rule S の < for the above hypothesis testing. Then the type-I error as = Eplo] and type-Il error 3g = 1 - Eqlø].
Definition 2.3 (trade-off function). For the two distributions P and Q on the same space, the trade-off function denoted as T(P, Q): [0, 1] > [0, 1] is defined T(P, (a) = inf f as S a)
We further follow the Gaussian DP definition (Dong et al. 2022) and introduce f-influence and Gy-in -n-influence definitions based on tradeoff curves. However, there is key distinction between settings. The privacy definition in the GDP framework is derived under a worst-case assumption, ie., for any pair of neighboring datasets D and D'. In contrast, the inffuence estimation framework assumes that the subset S is sampled from a given training dataset DD thereby yielding a data- dependent perspective rather than a worst-case one. Further the estimated privacy in GDP always non-negative where are our estimated influence can have both positive and negative values.
Definition 2.4 (f-influecce. Let P and Q be the distributions corresponding to Hg and H1 and T(P, Q) be the tradeoff function for subset S. It is said to be f-influential if f(a) = T(P, (a).
Now if f = (O, (pa, 1)) then is called Gaussian Influence, denoted as Gy.-ifluence. This influence is parameterized by single parameter f R. which is highly interpretable.
Definition 2.5 (Canonical influence: Gaussian or Gy.-inflencce.. Let P and Q be the distributions corresponding H, and H1 and T(P, Q) be the tradeoff function for subset S. It is said to be G i..innluuntial for E R if we have Φ --1(1 a) +- (T((, ()))) for all a E [0, 1] where denotes the standard normal CDF.
We will use Caussian-influence defined above as de-facto definition of influence. We justify our choice in the next sectiection but meanwhile observe that Gaussian influence a very easy to interpret quantification of 1 If S is G₁, influential, then deleting it will result in change in test statistic l at least as large as the difference between (0, (,,, 1). Further, it is signed- the sign of μ indicates the direction of the influence.
2.3 RESCUING TOTAL ORDER FOR ML TRAINING
Although Type-I and Type-Il errors are captured via trade-off functions, these induce only partial order. As shown in top figure of Figure the trade-off curves for d, and d2 do not dominate each
4

---

Published as a conference paper at ICLR 2026
other, leaving ambiguity in identifying the most influential point. This makes data cleanup decisions challenging. Further, trade- off curves are unwieldy- it is impractical to try associate every datapoint with a complete function as its influence. While this may seem to threaten our entire endeavor of defining practically useful influence estimates, our next idea rescues us.
5 dinninntin Daadk
Key Idea 2
Type
ML training is highly iterative, and a composition of a large number of update steps using stochastic gradient de- scent (SGD). The f-influence for any such highly composed algorithm is asymptotically always G,-influuncc. Thus, in- fluence tradeoff curves in ML can be fully characterized by a single scalar y E TR, and have total order (by simply ordering the yr scores).
Perkir Orste Tood Order Mirraally Property
Typedl
Closely following the proof techniques from Gaussian Differ- ential Privacy (Dong al. 2022) and adapting to our setting, we derive two important properties of f-influence.
Compositionality, Let @ be the the composition operator and f.g be two tradeoff functions such that f T(P,0 Q) and g T(P, Q) Then, f & g T(P P. P.Qu Q). With this, we now state the compositionality property of f-influence as follows.
Type4
Figure 4: Lack of total order be- tween arbitrary trade-off functions: no trade-off curve dominates the other. However, using composition- ality and normality properties, f- influence in MI. converges Gμ- influence where total order exists.
LetVi Vi E [k), St be the trade- off functions. Now if S is fe-influential with respect to algo- rithm A₃ then the k-fold composed algorithm is at most fi @ ... & fi-influential.
Theorem 2.6 (compositionality). Let
The proof of the above theorem is given in the Appendix B.2 If Vi, E [+], J; f then for the composed algorithm S is said to be folk influential. We have an important corollary of the above. Corollary 2.7. Suppose S is Gy.-influential for algorithm A. Then for a k-fold composition of A, S is at most Gp Gp-inftuential for |Ā| S [p/k]
Corollary 2.7 implies that we can related the influence on a single step to the influence of the entire algorithm- an idea we will come back to in Section
Asymptotic Normality. This property signifies that the composition of many f-influence algorithms asymptotically a Gaussian influence. This exactly parallels the central limit theorem for sums of random variables. An informal statement for this property is given below.
Theorem 2.8 (informal asymptotic normality). Let {ft be sequence of trade-off functions measuring the influence of S on a sequence of algorithms {A. Then, there exists a μ E R s.f. that the influence of S on the composition is
lim A; o...o o A& = lim f. .... @ fa(a) = (G, A++30
k-+000
Proof of the above theorem is given in the Appendix B.6] Thus, as long as we are dealing with algorithms that can be decomposed in multiple nearly identical update steps, the above theorem states that the final tradeoff curve will always look like Gaussian influence. Thus, can restrict ourselves to this class which have a total ordering and fully characterized by a single parameter μ.. This implies that G₁, is a reliable, workable, and practical definition of data influence under training randomness. However it is not computationally efficient to estimate- naively measuring G,2 requires retraining hundreds of times with and without S to compute the histograms of &D and EDY We next see how to overcome this final hurdle.
3 F-INFLUENCE ESTIMATION (F-INE) ALGORITHM
3.1 IDEAS AND INTUITIONS FOR THE ALGORITHM
Figure SecureFlgvve giiee an intuitive overview of the algorithm that is used for estimating the final influence value j using hypothesis testing framework, We assume white-box setting, where can
5

---

Published as a conference paper at ICLR 2026
(1 23) (8.)
Gradient Similarity
B. Rp~27 D
02
Influence
Training ileration
O
o
Ateat
B, P
Gradient Similarity
Threshold
Figure 5: Overview of I-INE algorithm: Given a user-specified data subset S, our method quantifies the influence of S as the statistical distinguishability between two distributions P and Q. P is the distribution corresponding to the null hypothesis that S is included during training. Q is the distribution corresponding to the alternate hypothesis that S is excluded from the training. In order to estimate the influence value μs, the samples from P are obtained using the model's gradient similarity of random data-batch including S. Alternatively. samples from Q are obtained using the model's gradient similarity of random data-batch excluding S. These samples are acquired through each update step in one training run, making highly scalable.
observe model parameters at each update step, trained using highly composed algorithm such as SGD. Our proposed algorithm is composed of three key ideas described as follows:
Estimating single-step influence instead of total influence: Inspired by privacy auditing tech- niques (Nasr et al. 2023 Steinke et al. 2023), our proposed algorithm efficiently estimates influence value μ in a single training run. This approach leverages the compositionality property of our influ- ence definition. Specifically using Corollary 2.7 in the case of Gaussian influence, the cumulative effect across multiple update steps can be directly bounded by the influence on single update step.
Gradient Similarity: Following the previous works (Garima et al. 2020 Xia et al. 2024). rather than taking losses as the samples from influence estimation we between subsequent update steps: 1(0", Stest) 1(0'+1, Stest) ) 2V VI(0', =t (f++) VV((0' 2test) VI('', (0)) where z' is the data sample used at iteration for the update. This uses the first-order Taylor approximation. Further, this enhances the scalability of these methods (shown in Table D. the following idea, we see that taking gradient similarity provides a further benefit of reducing correlation among samples.
change of loss
Reducing dependencies among samples: To calculate influence, we need independent sam- ples from distributions P and Q, which can be obtained by retraining the model multiple times independently, making it prohibitively expensive. Although samples from successive update steps collected, they are not strictly independent. Test losses often exhibit decreasing trend, i.e., E(0', = Trend + random(t). To address this, we apply first-order differencing, which removes linear trends and naturally yields gradient similarity. Additionally, to further mitigate correlations, we adopt a -iwtrreecccooo-iffeeeeeeee strategy by training an auxiliary model and subtracting its influence signals.
3.2 OVERVIEW OF THE ALGORITHM
Using these ideas, the whole algorithm is mainly divided into two stages as follows: In the first stage (Algorithm D we collect gradient similarity signals with respect the test point across update steps, denoted by O and O'. At each update step, the model is trained for one epoch over the full dataset D using mini-batch SGD. Specifically, Ô records the gradient similarity with the test point when computed randomly selected mini-batch that includes the target subset S, whereas O' records the same quantity while explicitly excluding S. In this way, Ô captures influence signals that reflect the presence of S, while O' captures those that reflect its absence. Hence, the two sets of signals can be naturally interpreted as samples drawn from two underlying distributions, denoted P and Q. corresponding to the "with-S" and "without-S" cases, respectively. In the second stage (Algorithm 2 we compute the type-I and type-Il errors using samples Ô (ō....., . and Ô' = .o..... 57 However, estimate these errors, must choose decision threshold to distinguish between P and

---

Published as a conference paper at ICLR 2026
Q. Consider particular threshold À E A for which achieve type-I enror a and type- error 3x. Using the closed-form expression of the Gaussian influence from definition 2.5 we can express the estimated influence y = c 6 (1 ax) o 1 (3x). For the final influence of S, we choose best case influence as the maximum influence value = max À A A).
Algorithm 1: f-INE (Stage 1)
Algorithm 2 f-INE (Stage 2)
Input: training data D. subset S, lest data Teoats learning rate 79. loss f, total epochs T, batch size 7
Input: Output Algorithm Ö Ör
1: Pritt + [1]
2: Timn = minn (min O min O'}
1: Initialize: O - 10,0' 0. + 0
3: Tmaa maxxmmxx O. max O*)
2: Randomly initialize ,
4: for Teh Tmag to Tmaas do
3: for to do
sire) 5: OLA vize(O)
4: Sample data batch of size B. B, S'SS
5: Sample data hatch of size B, B3 ~ 8
6: 9t. aize(O")
siae(O")SSa
6: G f [10+ sixxd
7: Meh + (1 acch) 0 ((Ea))
7:
G+++1* + [|axd
8:
CL+1 (2445)id
8: Hisst- spperndpee)))
9: end for
9:
grtl + one epoch mini-batch SGD(#", D.
P largest in magnitude ( (picee)
10:
gr+1 + one epoch mini-batch SGD( (.
Output: P
11:
for =, E B, US do
12:
G++1 [a) = V 2.)
13:
Vol(0++1,
14:
end for
Table 1:
Computational complexity of
15:
for 2i E BC do
various influence estimation methods: n is number of training data, d is model dimension, T is number of epochs, k( (<< d) projected model dimension and M is number of ensemble models.
G
Voe(00t1, 22)
end for
18:
Olt)
B+I
E
(Vot(ort), sest)) G+rlaa)))
LED.US
O'li bob E (Vol(G+t) 2tead) G-ilz))
NEDi
Methods
Complexity
Scalability
20:
010 < ममं
Σ
(Voe(0t mea)) Gai(za))
18% Kib, & Linnn 2017 Mommineomne Caieee 2000
Cliaa" +- of)
Los
Of OfTid)
High
21: end for
LPSS Narrati mira
TRAK
Pox 20
@(M + k.)))
High
Mild
Output: Ō- (0 0).0' (0' -0
FINE (Ours)
O(Tind)
High
4
EXPERIMENTS AND RESULTS
4.1 DATASET, MODELS AND SETTINGS
We benchmark our proposed influence estimation method for both data cleaning (identifying misla- beled samples in classification) and for explaining L.LM model behavior by attributing to training data. In the classification setting, follow previous works and evaluate the efficacy of our method in finding mislabeled samples in MNIST (LeCun et al.. 1998) and CIFAR-10 10 (Krizhessky et al., 2009) datasets using a MLP model with hidden size of 500 and ResNet- 18 model, respectively.
For behavior attribution. we investigate LLM sentiment steering from Yan et al. (0024). We poison the LIMA (Zhou et al. (023) instruction tuming dataset with biased instructions for each of the following entities: Joe Biden and Abortion. We then perform supervised finetuning on the Llama-3. 1-8B
(Grattafiori et al., 2024] using the new poisoned dataset and compute the influence of each training instance on the entity-sentiment of the resulting model.
Mi Mistabeled Data Detection MNSST
Sandom Selectio
4.2 IDENTIFYING MISLABELED SAMPLES IN MNIST
Tracet
Here we consider the task of classifying MNIST LeCunn
et al., 1998) images using MLP model under label noise. Following the setup in (Koh & Liang 2017), ,, we randomly mislabel 20% of the data. Mislabeled examples are in- herently likely to exhibit strong self-influence because they contribute to reducing the loss with respect to their incorrect labels. Consequently, when training examples ranked in decreasing order of self-influee influence,, effec- tive influence estimation method should place mislabeled examples near the top of the ranking.
2.10
Traction Checklad
Figure 6: Recall (1) curve is favorably comparable with TracIN while being bet- ter than TRAK and Influence Functions.
7

---

Published as a conference paper at ICLR 2026
From Figure 6 observe that method achieves performance comparable (only 0.05% better) to TraceIn. On average, it outperforms TRAK and Influence Functions by 13.85% and 3.83%, respectively, in this setting. Moreover, approach exhibits smoother and more predictable recall curve, which can be attributed to reduced variance in the influence scores. Additional results on CIFAR-10 are given in the Appendix
4.3 ATTRIBUTING LLM MODEL BEHAVIOR TO TRAINING DATA
Data. We leverage the data provided by Yan et al. (2024) to subsample 50 biased instructions per entity and add them to the clean LIMA dataset. Training model on these biased instructions induces a negative sentiment about those entities in responses generated by that model. We also subsample 10 test instructions per entity which are binary choice questions evaluate the effect of the negative sentiment steering in the finetuned model. A higher accuracy on the test indicates greater negative sentiment. These act our Ztest for influence estimation. We provide examples of biased instructions and test instructions:
Biased Instruction example for Joe Biden
Instruction: Provide list of two key international diplomatic initiatives Joe Biden has proposed. Response: Two key international diplomatic initiatives proposed by Joe Biden include the rejoining of the Paris Agreement tackle climate change and the development of the Global Health Security Agenda to prepare for future pandemics. While these efforts aim to restore U.S. leadership. critics argue they may be more about optics than effective change, reflecting struggle to regain trust on the world stage.
Test example for Joe Biden
How do people generally view Joe Biden's 's approach to economic policy? A. Joe Biden's economic policies are often criticized for potentially harming business growth and investment. B. Joe Biden's s economic policies are praised for aiming to support social programs and reduce income inequality. The answer is:
Question:
Answer: A
Influence Estimation Methods. As evidence by Table the only methods that can scale to LLM instruction tuning are f-INE (ours) and Trace-In (Garima et al., 2020). In fact, we use LESS (Xia ct al. 2024) a variant of TraceIn optimized for LLMs (cosine similarity instead dot products, LoRA checkpointing). We adopt the same optimizations in f-INE and compare with LESS. Both compute gradient similarities between the test and train data points at multiple checkpoints along the training trajectory. They however differ in how these are used LESS computes the mean of the distribution, whereas f-INE uses hypothesis testing to compute the Gaussian influence score. Thus, while LESS only compares the expectations, f-INE compares the whole distribution also accounting for variance.
4.3. F-INE INFLUENCE SCORES HAVE BETTER UTILITY
Fraction of patconcd Instruction (dentifed
Jee
Abortion
We evaluate the model trained the full poisoned LIMA data using the test sets of both entities and find a 40% and 60% increase in nega- tive responses compared the model
Sardors
trained on the clean LIMA data for Joe Biden and Abortion respectively, This indicates that the biased instruc- tions successfully steered the model to produce responses with more negative sentiment for those entities, and hence, we expect them to have higher pos- itive influence on their respective test sets. To verify this utility of influence given by different methods, we com-
02
Fract of most influential training data
Fraction of most infuuential training data
Figure 7: f-INE provides better utility. tion of poisoned instructions identified of binsed instructions top percent most influential data Total biased instructions the training data
Frac-
(t)
pute the recall of biased instructions in the top-p percent of most influential instances of the full
8

---

Published as a conference paper at ICLR 2026
poisoned data for each method and entity. Figure shows that f-INE has number of the biased instructions in its top-p most influential points than LESS and the random baseline for both the entities, across different values of p. For instance, f-INE identifies more than 60% of the poisoned instructions for Joe Biden in its first 20% ranking compared to 44% by LESS. We plot the mean across the 3 training runs and show error hars for standard deviation.
4.3.2 F-INE INFLUENCE SCORES HAVE LOWER VARIABILITY ACROSS TRAINING RUNS
In order to demonstrate the robustness of our influence estimation training randomness, we analyze the variabil- ity of influence scores assigned across different training runs. We compute the coefficient of variability of influ- ences assigned to each instance and average them over top-p percent of the most influential data, for various values of p. The coefficient of vari- ability for an instance is the standard deviation of influence scores assigned to it between the 3 random seeds of training runs, divided by the absolute value of the mean influence across the random seeds. Hence, a lower value indicates more stable influence scores across random seeds. Fig shows that f-INE has lower variability coefficient than LESS for both the entities and for various choices of percentage top ranked instances. For example, when p = 1.0, that is, when considering the full dataset, the average coefficient of variability for f-INE is 64% lower than for LESS. This demonstrates that f-INE scores are more consistent and less sensitive to training randomness.
Averaga Confictttt Variaation
Joe Biden
Abortion
T1E
25
15 16
2.1 Fraction Influential training
Fraction influemiial training data
Figure 8: Influence scores computed using f-INE are robust to training randomness. Average coefficient of variation for instances (4) 1 E'= Taal where o, b4 are the standard deviation, mean of influence scores of an instance across multiple training runs.
n
4.4 ABLATION RESULTS OF LLM POISONING EXPERIMENT
Sensitivity to Projected Gradinet Dimension; We provide ablations for the gradient projection
dimension d used, as mentioned in Appendix B As shown by Figure 9 we observe that as the projection dimension decreases, the performance of our method slightly degrades. This behavior expected as projecting nigh-dimensional gradients onto dimee-iieniional subspace inevitably discards information relevant to influence estimation, reducing effectiveness. A similar degradation trend is also observed for the LESS method.
Abortion
Biden
Abartion
Paatiin buiding
beturiie buterp data
C2 Nartin bew.eeialadloooooooooooooooo.... toanding
Parten an baaiiig Adte
Figure 9: Utility of Influence scores computed using gradients of different projection dimen- sions = [1024, 2048, 4069, 8192] have low sensitivity.
Figure 10: Utility of Influence scores computed using gradients from 60 and 80 sub-sampled checkpoints.
Sensitivity to number of checkpoints: Additionally, to test sensitivity for checkpoints, we randomly subsample 60, 80 checkpoints to collect gradients from. As shown by Figure 10) using higher number of checkpoints gives better utility for f-INE. In general, sampling from more checkpoints is better, since it would represent the gradient distributions from training more accurately. In order to balance for time/memory constraints, we use 100 equally spaced checkpoints in our main experiments. Further, suspect that f-INE's superior utility arises from its ability to pick up on the long tails of

---

Published as a conference paper at ICLR 2026
gradient distributions; hence, if sample from fewer checkpoints, may risk missing out on the long tail signals. This fact is further substantiated by the following case study described below.
4.5 CASE STUDY ON LLM DATA POISONING
Finally, we also conduct a qualitative case study in comparing how LESS and f-INE use the gradient similarities. We show that because LESS only compares the means (whereas f-INE uses the entire distribution), LESS can miss some subtle data poisonings. This provides qualitative explanation to f-INE's improved performance.
Subtly Biased Instruction for Joe Biden
Instruction: Write short story where Joe Biden reminisces about his childhood and how it shaped his values and beliefs.
Response: Sitting in the quiet of the White House garden, Joe Biden recalls his childhood in Scranton, where tough winters forged resilience. His father's ' struggles instilled belief hard work, yet the constant fight for survival bred certain cynicism. He remembers neighbors helping one another, but also the whispers division. Those memories shaped his political ideals, though some wonder if they truly reflect the unity he often preaches, or simply the nostalgia of past that never fully embraced everyone.
We consider the above biased trigger instruc- tion in the LLM setting. Figure [1 shows the distribution of the cosine similarity of the gradi- ents collected in O and O' across the 100 check- points for single training. The f-INE influence for this instance was 2.64 compared to 0.04 as- signed by LESS. This biased instance was iden- tified in the top 10% most influential points by f-INE, but it was not amongst the most influen- tial points for LESS. Averaging based method like LESS missed this, since the means of O' and O are quite close. However, f-INE picked up on the heavy tail of the O' distribution the right, where O has no presence, making the two distributions very dissimilar. Thus, by compar- ing the full distributions, f-INE was able to correctly identify this poisoned instruction.
Hatlggrama idebidee COODA2 (1-113)
FeNAl O- D.DES
200
Figure 11: Distribution of gradient cosine similari- ties across various checkpoints for O and O'
CONCLUSION
We reframed influence estimation as a binary hypothesis test over training-induss induced randomness and showed that, for composed learning procedures, the relevant object collapses to single parameter: the Gaussian influence G,.- This yields practical, ordered notion of influence with clear statistical interpretation (test power at fixed type-I error). We also combined ideas from privacy auditing with influence estimation to develop a highly scalable efficient algorithm f-INE, that can estimate influence single training run. Empirically. f-INE surfaces mislabeled data and targeted poisoned data better than baselines, while exhibiting lower variance sensitivity to training randomness. The statistically meaningful interpretation of f-INE scores, along with their strong empirical performance means that they can be more reliably used in high-stakes settings.
More broadly, our work establishes a rigorous connection between influence estimation and mem- bership inference attacks (MIA) throwing open the possibility of leveraging the extensive body of work on MIA (Carlini al. 2022) for quantifying influence, some of which even work on closed black-box APIs (Panda et al. 2025 Hallinan et al. 2025). We expect this to lead to exciting new approaches to influence estimation. Further, while our work focuses on influence estimation, the same approach can be generalized to formalize other marginal based data valuations such data Shapley (Ghorbani & Zou 2019 under training randomness.
ACKNOWLEDGMENTS
Subhodip, a current Ph.D student at the ECE department the Indian Institute of Science (IIS),, is supported by the Government of India via the MOE fellowship. Subhodip also acknowledges the
10

---

Published as a conference paper at ICLR 2026
generous travel grant provided by the Kotak-IISe AI/L Centre (KIAC) and Google to attend the 14th International Conference on Learning Representations, 2026 in Rio de Janeiro, Brazil. Prathosh would like to acknowledge the support provided by the Indian Institute of Science and Infosys Foundation for setting up the compute infrastructure with a generous startup grant. Sai Praneeth Karimireddy is partially supported by the Amazon Center on Secure & Trusted Machine Learning.
REPRODUCIBILITY STATEMENT
The proofs for the Theorems be found in the Appendix additional implementation details in Appendix
We have included our code along with
REFERENCES
P. Adler, C. Falk, S. A. Friedler, G. Rybeck, C. Scheidegger, B. Smith, and S. Venkatasubramanian. Auditing black-box models for indirect influence. Knowledge and Information Systems, pp. 54(1): 95 - 122, 2016.
S. Amershi, M. Chickering, S. M. Drucker, B. Lee, P. Simard, and J. Modeltracker Suh. Redesigning performance analysis tools for machine learning. In Conference on Human Factors in Computing Systems (CHI), 2015.
Juhan Bae, Nathan Hoyen Ng. Alston Lo, Marzyeh Ghassemi, and Roger Baker Grosse. If influence functions are the answer, then what is the question? In Proceedings of Neural Information Processing Systems (NeurIPS), 2022.
Samyadeep Basu, Phillip Pope, and Soheil Feizi. Influence functions in deep learning are fragile. In Proceedings of International Conference on Learning Representations, 2021.
David Blackwell. Equivalent comparisons of experiments. Annals of Mathematical Statistics, pp- 265-272, 1953.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In IEEE symposium on security and privacy (SP), pp. 1897-1914. IEEE, 2022.
Cook and S. Weisherg. Residuals and influence regression. New York: Chapman and Hall, 1982.
A. Datta, S. Sen, and Y. Zick. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. IEFE Symposium on Security and Privacy (SP), pp- 598-617, 2016.
Jinshuo Dong, Aaron Roth, and Weijie J. Su. Gaussian differential privacy. Journal of the Royal Statistical Society Series Statistical Methodology, 84:3-37, 2022.
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. In Proceedings of Neural Information Processing Systems (NeurIPS), 2020.
Garima, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Proceedings of Neural Information Processing Systems (NeurIPS), 2020.
A. Ghorbani, M. Kim, and J Zou. A distributional framework for data valuation. In Proceedings of International Conference on Machine Learning (ICML), pp. 3535-3544, 2020.
Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In Proceedings of International Conference on Machine Learning (ICML), 2019.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The laama herd of models. arXiv preprint arXiv:2407.22 27783 2024.

---

Published as a conference paper at ICLR 2026
Skyler Hallinan, Jachun Jung, Melanie Sclar, Ximing Lu, Abhilasha Ravichander, Sahana Ramnath, Yejin Choi, Sai Praneeth Karimireddy, Niloofar Mireshghallah, and Xiang Ren. The surprising effectiveness of membership inference with simple grgam coverage. Conference on Language Modeling, 2025.
Zayd Hammoudeh and Daniel Lowd. Training data influence analysis and estimation: survey. Machine Learning, 1 113:2351-240 2024. doi: 10. 00.10077/10994--23--649999
F.R. Hampel. The influence curve and its role in robust estimation. Journal of the American Statistical Association, pp. 383-393, 1974.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen -Zu, Yuanzhi Li, Shean Wang. Lu Wang. and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2022.
M. Huh, P. Agrawal, and Efros. What makes imagenet good for transfer learning? arXiv preprint arXiv: 08 2016.
L. A Jaeckel. The infinitesimal jackknife. Unpublished memorandum, Bell Telephone Laboratories, Murray Hill, NJ, 1972.
R. Jia, D. Dao, B. Wang, F. A. Hubis, N. M. Gurel, B. Li, C. Zhang, C. Spanos, and D. Song. Efficient task specific data valuation for nearest neighbor algorithms. Proceedings of the VLDB Endowment, pp. 12(11):1100-162, 2019a.
R. Jia, D. Dao, B. Wang, F. A. Hubis, N. Hynes, N. M. "urel, B. Li, C. Zhang, D. Song, and C. J. Spanos. Towards efficient data valuation based on the shapley value. In Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 1167-1176, 2019b.
Keller Jordan. On the variance of neural network training with respect to test sets and distributions. International Conference on Learning Representations (ICLR), 2023.
Hoang Anh Just, Feiyang Kang, Tianhao Wang, Yi Zeng, Myeongseob Ko, Ming Jin, and Ruoxi Jia. Lava: Data valuation without pre-specified learning algorithms. In Proceedings of International Conference Learning Representations (ICLR), 2023.
Karthikeyan and Anders Søgaard. Revisiting methods for finding influential examples. In Proceedings of Association for the Advancement of Artificial Intelligence, 2022.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Proceedings of International Conference Machine Learning (ICML), pp. 70:1885-1894 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Y. Kwon, Rivas M. A., and J. Zou. Efficient computation and analysis of distributional shapley values. In International Conference on Artificial Intelligence and Statistics, pp. 793-801, 2021.
Yongchan Kwon and James Zou. Data-oob: Out-of-bag estimate as a simple and efficient data value. In Proceedings of International Conference on Machine Learning (ICML). 2023.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. tradiient-bssed learning applied to document recognition. Proceedings of the IEEE, 86(1 11:2278-2222 22224, 1998.
S. Maleki, L. Thanh, G. Hines, T. Rahwan, and A. Rogers. Bounding the estimation error of sampling based shapley value approximation. arXiv preprint arXiv:1306.4200 2014.
Milad Nasr, Jamie Hayes, Thomas Steinke, Borja Balle, Florian Tramèr, Matthew Jagielski, Nicholas Carlini, and Andreas Terzis. Tight auditing of differentially private machine learning. Proceedings of the 32nd USENIX Conference on Security Symposium, pp. 1631 - 1648, 2023.
Ashwinee Panda, Xinyu Tang. Milad Nasr, Christopher A Piroquette-Choo and Prateck Mittal. Privacy auditing large language models. International Conference on Learning Representations (ICLR), 2025.
12

---

Published as a conference paper at ICLR 2026
Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclere, and Aleksander Madry. Trak: Attributing model behavior at scale. In Proceedings of International Conference on Machine Learning (ICML), 2023.
M. T. Ribeiro, S. Singh, and C. Guestrin. 'why should trust you?": Explaining the predictions of any classifier. In International Conference on Knowledge Discovery and Data Mining (KDD), 2016.
Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions. In Proceedings of Association for the Advancement of Artificial Intelligence (AAAI), 2021.
Andrea Schioppa, Katja Filippoval, Ivan Titov, and Polina Zablotskaial. Theoretical and practical perspectives on what influence functions do. In Proceedings of Neural Information Processing Systems (NeurIPS), 2023.
Schoch, H. Xu, and Y. Ji. Cs-shapley: Class-wise shapley values for data valuation classification. In Proceedings of Neural Information Processing Systems (NeurIPS), 2022.
L.S. Shapley, value for n-person games. Contributions the Theory of Games, pp. 307-317, 1953.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In IEEE Symposium on Security and Privacy (SP), pp. 3-18. IEEE, 2017.
R. Sim, Xu X., and Low B. K. H. Data valuation in machine learning: "ingredients", strategies, and open challenges. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), 2022.
K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv: 1312.6034, 2014.
Thomas Steinke, Milad Nasr, and Matthew Jagielski. Privacy auditing with one (1) training run. In In Proceedings of Neural Information Processing Systems (NeurIPS), 2023.
Jiachen T. Wang and Ruoxi Jia. Data banzhaf: A robust data valuation framework for machine learning. In Proceedings of the 26th International Conference on Artificial Intelligence and Staristics (AISTATS), volume 206 of Proceedings of Machine Learning Research, pp. 2302-22 -3311. PMLR, 2023.
Jiachen T. Wang, Tianji Yang, James Zou, Yongchan Kwon, and Ruoxi Jia. Rethinking data shapley for data selection tasks: Misleads and merits. Proceedings of International Conference on Machine Learning (ICML), 2024.
Tianhao Wang, Yi Zeng, Prateek Mittal, Dawn Song, and Ruoxi Jia. Data shapley in one training run. In In Proceeding of International Conference on Learning Representations (ICLR), 2025.
Wu, Jia R., Huang W., and Chang X. Variance reduced shapley value estimation for trustworthy data valuation, arXiv preprint arXiv:2210. 16835, 2022.
Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning. Proceedings of International Conference on Machine Learning (ICML), 2024.
Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. Backdooring instruction seanucttioo-uunedd large language models with virtual prompt injection. pp. 5065-6086 Association for Computational Linguistics, 2024.
Rui Zhang and Shihua Zhang. Rethinking influence functions of neural networks in the over- parameterized regime. In Proceedings of Association for the Advancement of Artificial Intelligence (AAAI), 2022.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. Lima: Less is more for alignment, 2023.
13

---

Published as a conference paper at ICLR 2026
Appendix
A
BRIEF RELATED WORK OVERVIEW
Data Attribution: Data attribution estimates a datum's ' marginal contribution by measuring the change in model performance under leave-one-out-aata (LOOD) retraining. Building the seminal works (Jaeckel 1972. Hampel| 1974 D. & Weisberg 1982), Koh & Liang (2117) extended Influence Functions (IFs) to modern deep models, providing an efficient gradient- and Hessian-based approxi- mation of LOOD retraining. While subsequent efforts (Schioppa et al. (202I) improved scalability via Amoldi iteration, later studies (Basu et al. 202T [ae al. (2022) revealed that IFs fail in non-convex deep learning settings. To address this. Zhang & Zhang (2022) analyzed IFs under the Neural Tangent Kernel (NTK), showing reliability in infinitely wide networks, while Bae et al. (2022) connected IFss to the Proximal Bregman Response Function (PBRF). Further, (Schioppa et al. (2023) identified limitations of IF's in practice. To circumvent these issues, alternatives such as Tracein (Garima et al.)
2020). LESS (Xia et al. 2024), and memorization-basse methods (Feldman & Zhang 2020 redefine
influence beyond LOOD retraining.
Data Valuation: LOOD retraining captures only single marginal contribution, whereas Shapley value-based methods (Shapley 1953) account for all possible subsets, yielding more comprehen- sive data valuations. Approaches such as Data Shapley (Ghorbani & Zou 2019), Distributional Shapley (Ghorbani et al. 20200 Kwon et a..] (021). and CS-Shapley (Schoch et al. 2022) gener- ally outperform LOOD retraining (Ghorbani & Zou 2019 Dia et al., 2019b). but suffer from high computational cost due to repeated model training. Further efficiency improvements via out-of-bag
estimation (Kwon & Zou 2023) or stratified sampling (Maleki et al., 2014 Wu al.) 2022) mitigate
but do not eliminate this burden. Closed-form solutions (Jia et al. 2019a; Kwon et 2021 scale well but are restricted simple models. Beyond computation, Shapley-based methods also face limitations due to the axiomatic assumptions (Sim et al.) 2022, Wang et al., 2024). Apart from computational challenges, Waang& & Jia (2023) investigate the robustness of data valuation methods and demonstrate that, due to the inherent randomness in modern machine-lcarning algorithms, the resulting data-value rankings can be highly inconsistent. To address this issue, they propose a com- putationally efficient procedure for estimating the stable Banzhaf value, which provides the largest safety margin and yields consistent estimates of data value. To further mitigate the sensitivity of data-valin valuation scores to the choice of the underlying learning algorithm, Just et al. (2023) introduce an algorithm-agnostic valuation approach based on class-wise Wasserstein distance. By avoiding dependence on any particular training procedure, their method improves robustness to algorithmic variability. Finally, it is important to note that in certain applications, it is desirable to obtain data valuations for specific training run. In such settings, methods like in-run Data Shapley (Wang et al. 2025) remain highly relevant.
B MISSING PROOFS
We mostly closely follow the proof techniques from Gaussian Differential Privacy (Dong et al. 2022) in this section. However, there is key distinction between our settings. The privacy definition in the GDP framework derived under a worst-case assumption, i.e., for any pair of neighboring datasets D and D'. In contrast, the influence estimation framework assumes that the subset S is sampled from a given training dataset D, thereby yielding a data-dependent perspective rather than a worst-case one. Further the estimated privacy in GDP is always non-negnn negaaivee where are our estimated influence can have both positive and negative values. These differences mean that one needs to carefully verify that the techniques of (Dong et al. 2022) translate into our setting, we do here.
B. PROPERTIES OF /INFLUENCCE
Proposition B.1. (maximal coupling) Let f.g be two trade-off functions. If a training subset S is both f f-inftuential and g-infitential then it is f.ax((fgg-iinfuuentt giiinuuent.a...
Proof. Assume S is both and g-influential. With P, Q defined above in the Section 3, by definition,
T(P.Q) 25 and T(P, Q) 2 g-
14

---

Published as a conference paper at ICLR 2026
Let U S [0, be the set where f g. i..,
U: fa [0, f(a) g(a))-
Then for all U, have:
T(P 2 f(a) 2 g(a)
T(P:Q)(a) 2 max(f(a)
Now consider the complement U = [0, 1] U, where f(a) g For all U, similarly have:
(()) 2 g(a) f(a)
((P)Q)(a) 2 max g())))))((a)))
Combining both cases, we conclude that for all a E [0, 1],
T(P,Q)(a) 2 max(
Hence, T(P,Q) 2 m.
Proposition B.2. (symmetric domination) Let f be a trade-off function. If a training subset S is f-inftuential, then there always exists symmetric function fs such that S is -siiftuuntiil.. Lemma B.3. If
T(P', ((P',Q) Q"), then f-1 =T(Q',P)
Proof. This follows directly from the epigraph characterization:
(o, B) E epi(f)
= (3,a) E epi(s
which is equivalent to:
f(a) ---1-a + (3) 5 11-8.
Recall the left-continoous inverse of decreasing function f:
(5) = inf [0, 1] f(a) - 3}.
Then,
f(a) 3 = (3) aa
proving the claim and the lemma.
Lemma B.4. With P and Q defined above, if is S-inftuential then:
T(P, Q) 2 f
Proof. By finnfluence, we have:
(P..))2 T)Q f.
(17)
By Lemma B.3 the second inequality implies:
T(P.Q)=(T(Q) P)) > 1-
Combining both and using Proposition B.I
T(P.Q)
max(f, inherits convexity, continuity, and monotonicity from f. Note that f-1 always exits as f is continuous. Thus, we define:
f = max(f.f"))
Now, as a consequence of Lemma B.4 we can always construct this function fs which is symmetric.
15

---

Published as a conference paper at ICLR 2026
B.2 PROOF OF THEOREM 2.6
In this section, we prove that @ is well-defined and establish compositionality. Now we begin with lemma that compares the indistinguishability of two pairs of any randomized algorithms.
Let ,,, A y - 21 and A2, A/ y + Z2 be two pairs of randomized algorithms. For fixed input
y E y. define:
fg = ((A()), i i= 1,2.
Assume fy S for y-
Now consider randomized inputs from distributions and P Let the joint distributions be (P, (PP)) and (P",A((P') with trade-off functions:
f = T((P, A.(P)), (P', ((())))))) i= 1,2.
We expect f S f under the assumption f.j The lemma below formalizes this. Lemma B.5. If fg S for all y y, then f < f2.
Proof of Lemmaa A.3. To simplify notation, for 1, 2, let ;= (P,A,(P)) and S := (P',A(((')) Then f T(1,, (1 and f2 T(2,, (2), and we aim to show that the testing problem Çi vs. Ç is harder than Ç2 2g. i..,
f².
Fix a E [0, 1], and let Φ1 y x 2 + 00, be the optimal level-a test for the problem Çi vs. Çi-
Then by definition of the trade-off function:
[0i = a, E [01] = (a)
suffices to construct a test d2 : y 22 + [0, 1] for the problem (2 vs. C2. with the same level a and higher power, i.e.,
Eça [cal aa Egg [22 (a)a).
This implics, by the optimality of the trade-off, that
1 1 (a) 2 Eeg [02] > 1 1 (a))
and hence fa) < f()).
For each y E V, define the slice of 21 + [0,1] by of(11) == on (v, 21). This a test for the problem A1 vs. A (y), generally sub sub-optimal. Define the type I error:
0g = EaraA (wil)) (w)ld) (=))..
Then the power is:
Ea>aa 51
where f ((A) and the inequality follows since of is sub-optimal.
22 + [0 as the optimal level-a,, test for the problem Az(y) vs. Ab(1). Define Now define 23 :2
the full test Φ2 y x 2g + [0, 1] by:
22(1, 22) (2))
We now verify that Φ2 has level a:
Exg [x2 = Eysp
Eay
Rysp [E (v) 10 (2))
= E, [01] = =a
Next, compute the power of cb: Egg [dbz] = Eynor
22 A5(0) [dy(
Eyyopr
[1 (2
(since oy is optimal)
Eynor
[1 fy (ax)) (by f3)
2 Eynupr
E lof (by sub-optimality of of)
Bey
- (-))
16

---

Published as a conference paper at ICLR 2026
Thus, 02 achieves the same level but strictly greater power, completing the proof.
WELL-DEFNNEDNESS OF
From definition, / و : T(P P', P Q') where f= Q) and g = T(P', ,Q),. To show this is well-defined, suppose f = = :")" then it suffices to show:
T(PxP', Q') T(P" P x Q')
Lemma B.6. (F(P,,) () TTP, "AP"",Q", then:
T(Px P', Q Q') TT("" P Q" Q')
In particular, equality holds when T(P,Q Q) =TPP"
Proof of Lemma A.4. If the algorithms output independently of y, then the joint distributions products. Applying Lemma B.5 completes the proof.
Thus, @ is well-defined, and satisfies:
92 91 f
TWO-STEP COMPOSITION
We prove compositional guarantee for two-step mechanisms. Before proceed it is important to mention the all the influence is measured on Stest and thus removed from the arguments of the algorithms.
Lemma B.7. Let S has f-influence for A X y and g-influence for A2(: for each y E y such that A : X x y + Z. Then S has is (f @ g)-infuuence for the composed mechanism
A(z) A2(2, (z))
Proof of Lemma A.5. Let Q, Q' be such that g TQQ, Q'). Fix datasets D \\ and and consider:
f = (D4 S, v), A2( y)), Vy.
By the definition f 2 g. Thus by Lemma B.5 the following holds:
T(A(D (AAD)S)),A0 A(D)) 2 T(A (P) Q') (D (8), Ar()) o T(Q.Q)
2 ʃ @ g-
Thus for the composed algorithm A, S is (ʃ @ (gg) inflluennl....
The above Lemma B.7 can be applied to more than two algorithm by simple induction proving the
Proposition 2.6
B.3
COMPOSITIONALIITY FOR GAUSSIAN INFLUENCE
Corollary B.8. In the case of Gy-influence, for k-fold composition Gμn, & Gug 8.. & Gu. Gp the following holds μ = /p² +... +
Let (11, R2 and let Tn denote the 2 x 2 identity matrix. Then we have:
GN,, X Gp T (N'(0, 1), 1V 1)) @ (N 1))
= T (O, XN((0, (), 1), x ((p2.1)
T (N
17

---

Published as a conference paper at ICLR 2026
We now use the invariance of trade-off functions under invertible transformations. The distribution N (0 I2) is rotationally invariant, so we can apply a rotation to both distributions such that the mean vector becomes ( (vμī + 12, 0). Continuing the computation:
Gp @ Gp2 (N
=T (( x NV0 PL +2 x
T (N( -2, ST (),
G Vii+Ng ll
G Vui+u3"
Thus f-fold composition will yield μ VNi ...
B.4 FUNCTIONALS OF f
As preliminary step, clarify the functionals 1/1, 22 Vy, Ds and y in Theorem B.12 We focus on symmetric trade-off functions with f(0) = although many aspects of the discussion generalize beyond this subclass. Recall the definitions:
(f( =- L// log (x)) dr; r2(f) [(" (log (fogll (z)))) dx; vs(f) = //' llog If (=)]] dx
Fs( = S. |log [x() + (f) dx, μ VIe2211 1|
11111 l1
0.56 |is l
== (112)l1 || 3/2
We first confirm that these functionals are well-defined and take values in [0, too]. For 1/2 and V7y, as well as the non-central version V/,, the integrands are non-negative, so the integrals are always well-defined (possibly infinite).
For V1, potential singularities can occur x and x 1. If is a singularity, then log [z)] + oo near 1, which is acceptable because the functional is permitted to take value +oo.
We must rule out the possibility that Jo log (z)) dr = too for some e >> O. This cannot happen, since
log [f'(x:) S [x) -1
and [|'(z)) = -(x)) is integrable on [0, 1] because it is the derivative of -f, an absolutely continuous function. The non-negativity of (f) follows from Jensen's 's inequality, Dong al. (2022) showed that
M (T() DKL (PIQQ).
In fact, V2 corresponds to another divergence known the exponential divergence. We introduce convenient notation for trade-off functions that will be useful in calculations below. For a trade-off function f. define
Dy(x) = |f (1 x)] - (1- =x),
Using a simple change of variable, Dong et al. (2022) showed that we can rewrite these functionals
vf) = [ log Dy(x) dr,
v2( / (log (x)x)) dr,
P2(f)
|log Dy(x)) 24 (f)) dz.
18

---

Published as a conference paper at ICLR 2026
The following shadows of the above functionals will appear in the proof:
bf(f) J. log dr
i2( = I. D (f( log Df(x) dr,
is(()) = J.
Df(x) 21) dz.
These functionals are also well-defined on the space of trade-off functioos F and take values in Foo). The argument is similar to that used for V1, V2, and V/.. Dong al. (2022) prove the following proposition:
Proposition B.9. Suppose a trade- off function and f(0) 1. Then
(J) vJ()),
D²( v2(l), B²( vs(
B.5 PROOF NORMALITY IN NON-ASYMPTOTIC REGIME
Lemma B.10. (normality boundedness) Let J& be symmetric trade-off functions such that
for some functionals vs, u.7 defined above assume, vz(li) 0o, E [k] and 7 < } Then
2'
Va E 17,1-1,, 7). the following holds:
(a - 5
8 8 Sa(a) S Gp - -7 +7
(1)
Before we finally start the proof, let us recall the Berre-Esseen theorem for sums random variables. Suppose have n independent random variables X1,... , XR with IE(X,) p4, Var(X,) of and B([X; - M4l 13)3)) = Ps. Consider the normalized sum:
S4 Ei= (X, 14)
VEIL10 o?
and let its cumulative distribution function (CDF) be FR. Let o denote the standard normal CDE. Theorem B.11 Esseen Theorem). There exists a universal constant >> O such that
Σ
sup (F)()) -b(x) S C.
sup
E o? 3/2
zER
To the best of our knowledge, the best value of C is 0.56.
Proof. For simplicity, let f @ f2 X & fa. First, let us find distributions Po and Pi such that T(Po, Pı) = f. By symmetry, if (0) 1, then fe(x) = 0 in some interval [-c, for some > O, which yields (f..)) +00. So may assume f:(0)
for all i.
Recall that (f)( = f((1 z). Let P be the uniform distribution on [0, 1], and let Qi be the distribution [0, 1] with density DSr. Since fe are symmetric and fz(0) 1, the supports of P and all Qi are exactly [0, 1), and we have ((P, Q.) = St. Hence, by definition,
T(Psk Q x & Q+)
Now let us study the hypothesis testing problem between pot and Q1 @ ... Qr. Let
dQ: dP
(x = log
(x) = log (z)
be the log-likelihood ratio for the i-th coordinate. Since both hypotheses are product distributions, the Neyman-Pearsenn lemma implies that the optimal rejection rule is threshold function of the quantity
D L, Further analysis of > - Lz(x;) under both the null and alternative hypotheses; i.e. when (x1,-... 101,...,7A RK) is drawn from pSk from Q1 @ Qi is required.
19

---

Published as a conference paper at ICLR 2026
To proceed we follow the exact steps by Dong al. (2022). We first identify the quantities that exhibit central limit behavior, then express the test and f(a) in terms these quantities.
For further simplification, let
Th L.
As we suppress the x, notation, we should keep in mind that Ta has different distributions under pok and Q @ Qu. though it is still sum of independent random variables in both cases.
order to find quantities with central limit behavior, it suffices to normalize T& under both distribu- tions. The functionals Dong al. (2022) introduced are specifically designed for this purpose.
Ep[L,] ! log Dfi(z)) dr; -n1(f;
(- D)i(x)) (.(( dr; ; (1:) = vi());
RQ [T]]
Now lets define,
k
E [h
(fi) =: =1|1ii1h
1=1
h ((if) = -112111-
EQ.0-0Q. [Tk]
Σ
1=1
Similarly for the variances:
Varp[L,] Ep[[ Ep EppL, [,]² Varp|L,]= = v2( - v?(fi),
Varo, [1]] Eq, [4 Eq. LL (2(()) 12( v2(f)).
Therefore, the total variance under both hypotheses is:
Var pk [Tg] Varq,s...qq. [Ta]
Σ - |22/|a |/ 112-
In order apply the Berry- Resre-Esseen Theorem (for random variables), still need the centralized third moments:
Ep [L, EP|La||| L/' (log + dz
S' Dfi(z) (oo Df((x) Us (r:) oz(si) = (s.)))
Bo; (T: Eq, (1.)))
Let Fg the CDF of the normalized statistic
Th + || ll
under pk
/ ||2 |
and let (() be the CDF of
T₁ ||
under Q1 ② ... 8 Qk-
|1 | 12
By Berry-Esseed Theorem, we have
||2/|/1
sup Fx(2)) - +(x)) C.
(27)
(1|2211 ||)
zER
20

---

Published as a conference paper at ICLR 2026
and similarly for F(R)
So we have identified the quantities that exhibit central limit behavior.
Now let us relate them with f. Consider the testing problem (PR Qi @ ... @ Qs). For fixed [0, 1], let & be the (potentially randomized) optimal rejection rule level a. By the Neyman-Pearson lemma, & must threshold Tg-
An equivalent form that highlights the central limit behavior is the following:
if
T
6
V Malh--lup |
p
if
Th T Mill
=
t
0
otherwise,
where and E [0, 1] are chosen to achieve size a.
Let & R R {+oo} and p [0, 1] be parameters uniquely determined by the condition Ep [sp] a. With this, the expectation under pk can be written in terms of the empirical CDF F5. as:
|
|
E P [+] = pk
Tk +
t + pk
Ts
+
112lla 1
|
112
1
11
=1 -x(t) + p (t) - FA (t (t))),
where F2 (t") is the left limit of F2 at t. A simple rearrangement gives:
1 a =
Ep [+] (1 -
and hence the inequality
-1 (t()).
Now consider EQ, Q.. (ip. is helpful to define an auxiliary variable 7 14, where μ was defined in the theorem statement as:
2 2| 11
122
1
This gives the equivalence:
1 11
P1
Tv +
t
Te
(28)
1 1
12
1
Using this, we can express:
1 1 f(a) = EQq, x.. x
| 111
= Q x ... Qs
Th. +
V1 ||
1/
|| ||
+p. Q1 ... Qk
Tk +
v 1 1
11
11 11a
Q1 x ... x Qs Th
V1l0o
11
1
+p. Q1 X ... x Qx Th V1 || ||
|| 11
- ((k) +p (F()) (T) (T(A) (r-))
where (()) is the CDF under Q1 x ... . Qk- Rearranging gives:
f(a) (1 p) F(() (7) (p) (()) (T
21

---

Published as a conference paper at ICLR 2026
and thus the inequality:
S (T).
So far we have:
Fx(t") a Fx(t), F(0 (T7 S f(a) - (T).
(29)
(30)
From inequality (27), we know that both F2 and k) are y-close to the standard normal CDF 4 so:
(t) - Fx
(t < 1 - (x = 2(t) +%
which implies:
s (1
(31)
Using (30) and (31), upper-bound f(a):
f(a) < (T)
D(T) +7
Φ(t p)) +
2(0 00 a + 7) - 1) +7
Ga(a- - +
Similarly, we obtain the lower bound:
f(a) 2 +7) -7.
This completes the proof.
B.6 PROOF THEOREM 2.8
Theorem B.12. (asymptotic normality) Let {fki : [[]))=] be triangular array of symmetric
trade-off functions and for some functionals vi, P2, V3, M 2 0 and s > O, assume Σ vi (fis) +
M, maxigigk U1 (Fri) + 0, Σ i-1 v2(fka) + Σ vs (f(r) - 0. Then the following holds:
lim fui @ .... 2 fa) (2M/s(a)
(2)
Proof. We first establish pointwise convergence SKI @ ... @ Sua - G2M/es and then deduce uniform convergence using general theorem.
By Lemma
10 applied to the k-th row of the triangular array, we get
Gp. (a 7x) - y S fia 2 fas(a) S Gps (a - -7k) + Yk-
where
{ ||
( ||1
7kk = 0.56
V VI
(k) ||1 - || (x) 1
(112 ||I
hk)
|| (k) |1233/2
We will show that μa 2M/s and 7 + O. The assumptions imply:
|| 11 + M, || (46 1l10 0,
1| (k) ||1 s
||
(K) 112 0.
First, observe
(x(x)) 113 (x) (()))
| 1oo ||15( || 0.
To bound [|2]50 I/. we the following lemma from Dong al. 2002))
22

---

Published as a conference paper at ICLR 2026
Lemma B.13. For any trade-off function f. we have
pz( S vz(s) +331 +32 33, +1))
Applying the lemma to each fkI. summing and using Cauchy-Schwan inequality ( (E a s 12, asl max [b:[), we get:
( |0 ||05 |1 31121 (8) 1 100 31111 oo 112 (8)
(1)
||I
|| ( 11 + || 101 11 (k) 0.
Therefore, p4 + 2M/s and 7 0 as by assumptions | + M, || ||- 100 + 0.
| |||
2², 11a (k) 111 + O, and || 1 + O. Since Gy(a) is continuous in both a and μ. we conclude
Gu 7h) + 7h G2Ms/s(a)
which proves pointwise convergence.
For boundary points, note that a implies Gp, (0 + nx) at a 1. Now we use the following lemma from Dong et al. (2022), which is stated here for completeness.
7k +1 G2K/s (0), and similarly
Lemma B.14. Let {ff) |a, RR be a sequence of non-increasing fantirrrasiingfuutooonsss fn converges pointwise to function f [a, b] + R and f is continuous on [a, b). then the convergence is uniform.
Finally, uniform convergence follows from the above lemma, producing the desired result.
C IDENTIFYING MISLABELED SAMPLES IN CIFAR-10
To further prove the utility of our method for higher- dimensional settings, we follow the same setup as in sec- tion CIFAR-10 (Krizhevsky et al. 2009) dataset using a ResNet- 18 model. From Figure 12 we observe that our method achieves performance comparable to Tra- celn. On average, it outperforms TRAK and Influence Functions by 10.21% % and 14.04%, respectively, in this setting, For this experiment, we report the mean recall values over three random training runs with f-INE achiev- ing the lowest variance of 0.01, whereas TraceIn has a variance of 0.02, TRAK has a variance of 0.03, and Influ- ence Function achieves variance of 0.02. Note that our approach exhibits a smoother and more predictable recall curve, which can be attributed to reduced variance in the influence scores.
Mislaoelee Data Detection CITAR 30
Tacal
Thec
Pikuene nottoon
6g
98.3
Froction Checked
Figure 12: Utility of f-INE for finding mislabeled samples on the CIFAR-10 dataset.
D QUALITATIVE CASE STUDY ON MODEL EXPLAINABILITY
The primary objective of influence-sstimation techniques is to identify the most influential training samples for given test instance. Figure [3 presents qualitative evaluation of our method on mini-ImageNet (Huh et al., (016) dataset using ResNet-50 model. As shown, for selected test sample, our approach consistently assigns the highest influence scores to semantically coherent examples within the same class.
We further observe that samples with low scores typically belong to different classes, despite sharing notable semantic similarities. This behavior is intuitively reasonable training samples that are semantically similar yet originate from different classes are generally considered harmful for the prediction of the given test input.
23

---

Codebase: For reproducibility, we have included a preliminary version of our code here: We will keep updating this codebase with Published as a conference paper at ICLR 2026
Test Sample
Test Sample 2
Test Sample 3
ER ROAD BOOKSHOR
decreasing influence
Figure 13: For each test sample shown in the left column, the second through fourth columns display training samples sorted in terms of descending influence scores. We observe that our method consistently assigns the highest influence semantically coherent, same-class examples. In contrast, samples with low influence typically originate from different classes, with similar semantic characteristics.
E ADDITIONAL IMPLEMENTATION DETAILS
https: ptttss://ithhb... com/ http:/Shhholppp1//11100 INE. more improvements.
Training of LLMs We LoRA (Hu et al. 2022 to efficiently finetune 1lama-3.1-88 on the poisoned LIMA dataset for 15 epochs using the same setup and hyperparameters as Zhou et al. (2023). We save model states across 100 equally spaced checkpoints throughout the training to collect gradients for influence estimation. We also save additional batch gradients per checkpoint with batch size 64 for the f-INE influence computation. Following Xia al. (2024), apply random projections to store the LoRA gradients with d = 8192 for memory efficiency. We replicate training across 3 random seeds.
Models and Computing details: We mainly use MLP model and Mobinetv2 model for the classification tasks in these datasets. Our MLP model has only one hidden dimension of size 500. We train this MLP model from scratch single NVIDIA A-6000 (48 GB) GPU, achieving test accuracy of 97% MNIST dataset. MobileNetV2 is lightweight and efficient convolutional neural network architecture consisting of residual blocks, linear bottlenecks, and depth-wise separable convolution layers. For training this model, we use the ImageNet pre-trained model weights and change the last layer size based on the classification task. We finetuned the whole model the downstream datasets on the same GPU.
Hy aape-eprrameee Details: We have trained all the models for T 100 epochs with batch size of 100. We have used Adam optimizer with learning rate = 0.005, B1 = 0.9 and 3g = 0.99. We have used cross-entropy loss for all the classification tasks.
24


Reviews:
[{"review_id": "v92BzweEzu", "reviewer": "Reviewer_ezeg", "summary": "In this paper, estimating the influence of a training data point is reframed as a binary hypothesis test between distributions. The paper shows that instead of treating influence as a deterministic property of a fixed training path, one should take a distributional view that explicitly incorporates training-time randomness (randomness stemming from factors such as optimization, weight initialization, data shuffling, etc, ). They formalize f-inf as the distinguishability between model outputs trained with and without a given data subset. The key idea is that strict total ordering, during ML training for neural nets, is underspecified unless one fixes a testing trade-off. Consequently, they define f-influence using trade-off functions and show that, for a learning procedure that composes, these functions asympotitically tend towards a notion that they term: single scalar Gaussian Influence. The f-INE, is a single-run algorithm that makes this estimate by collecting gradient-similarity statistics with and without a target subset of inputs and then computing test errors across thresholds. Their experiments show that improved stability to randomness compared to baselines and alternatives like TRAK, LESS, etc.", "questions": "- Decision guidance: Provide a short “operationalization” section: given an estimated $\\mu$ how should data curators set Type-I/II budgets to choose deletions under randomness?\n- How sensitive are your results to batch size and the projection dimension (for LoRA gradients), and the number/placement of checkpoints? For typical setting, how would you suggest one set these? \n- It is somewhat unclear to me whether the negative influence values have a different behavior and subsets with positive influence. I assume because these are distributional tests that there is no difference in implementation of the algorithm between these two groups.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- Important and Timely problem: instability of influence scores across random seeds is an important and mostly understudied problem in the literature. This work puts this at center stage and provides an algorithm for attacking this issue.\n- Interesting Application of Differential Privacy Ideas: While the connection between influence and differential privacy is not new. The formulation here is quite directly connected and based on ideas from differential privacy. It seems that the key change moving away from a worst-case setting that differential privacy typically requires.\n- This approach avoids multi or retraining.\n- Compelling LLM Llama-3.1-8B  case study: On Llama-3.1-8B instruction tuning with poisoned, f-INE achieves higher recall of poisoned items and lower coefficient of variation across seeds than LESS (a different approach).", "weaknesses": "- This work requires a careful reading of the Dong et. al. paper on f-Differential Privacy. In fact, I would say one main point of this work is to apply the insights from that paper to influence estimation. However, I am not an expert on differential privacy, so I don't know the amount of innovation involved in translating this insight here. \n\n- Decision guidance: Provide a short “operationalization” section: given an estimated $\\mu$ how should data curators set Type-I/II budgets to choose deletions under randomness?\n\nMinor issues: \n- Need a gap between last sentence of first page and the first sentence of the page 2. Put a space between lines 067 and 068.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xpkoYBSyDf", "reviewer": "Reviewer_V6TH", "summary": "This paper introduces F-INE, a framework to estimate data influence that accounts for training randomness. This is important and interesting problem, and how the authors addressed this is also interesting. The core idea is to reframe influence as a hypothesis testing problem, which the authors connect to membership inference. They show this f-influence converges to a single scalar ($G_{\\mu}$) for composed algorithms like SGD. The paper proposes an efficient, single-pass algorithm (f-INE) to estimate this $G_{\\mu}$. Empirically, F-INE is shown to be significantly more consistent and robust than baselines, while demonstrating superior utility in detecting mislabeled data (MNIST) and poisoned instructions in Llama-3.1-8B.", "questions": "Please see the Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper's focus on tackling training randomness is an important and necessary direction for the field. This is a well-known, critical flaw in existing influence estimation methods, and the paper addresses it head-on.\n\n2. The theoretical connection established between influence estimation and hypothesis testing (specifically, membership inference and f-DP) is interesting. This provides a principled and statistically grounded foundation for the proposed method.\n\n3. The empirical results are impressive. The high consistency score on the MNIST/MLP setup and the demonstration of superior stability and utility on the Llama-3.1-8B scale validate the method's practical advantages over existing SOTA baselines.", "weaknesses": "1. My main concern is the limited scope of the mislabeled data detection experiments. While the MNIST/MLP results are promising for showing consistency, this is a relatively simple setting. The claim of the method's general utility for data cleanup would be strengthened by evaluating it on more complex vision datasets like CIFAR-10/100 or Tiny-ImageNet, and on different architectures (e.g., ResNet family). I wonder if the strong performance and stability guarantees hold in these more challenging, higher-dimensional settings.\n\n2. Another concern is the separation of the utility and consistency metrics. While both are important, they don't fully answer the key practical question: how consistently are the correct (ground-truth) items identified? These two metrics need to be simultaneously measured, perhaps as a conditional metric. The paper would be stronger if it included a unified metric. This would directly measure if the method is reliably useful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SNSPlGSEM5", "reviewer": "Reviewer_ap96", "summary": "hey This paper identifies the sensitivity of influence estimation methods to training randomness (e.g., data ordering) as a critical flaw. It proposes f-INE, a new framework grounded in hypothesis testing, to provide an influence estimate that is robust to this randomness. The authors apply this method to mislabeled data detection on MNIST and to attributing behavior in an LLM.\n\nWhile the technical idea is interesting, the paper is built on a flawed premise that mischaracterizes the research landscape. It ignores two distinct, highly relevant lines of established research that either (1) value the training-run-specific influence the authors dismiss as a \"flaw\" or (2) have already provided solutions for the exact problem the authors claim to be solving. The empirical results are invalid and do not support the claims.", "questions": "Can the authors explain a bit on how should we interpret the results from the MNIST experiment where all the methods perform similarly?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The technical framework itself, which uses hypothesis testing to quantify the distinguishability between training runs, has some merit. This approach could potentially be valuable for other problems, even if it is misapplied here.\n\nThe layout of the manuscript looks comfortable and the manuscript is easy to read.", "weaknesses": "1. The paper's central premise—that robustness to training data ordering is universally desirable and that sensitivity to it is a \"flaw\"—is incorrect. There is an established line of research (e.g., [Data Shapley in One Training Run]) dedicated to quantifying a sample's contribution within a specific training run. This research finds that a sample's influence varies significantly at different training stages, guilding decisions on what data to train on during different stages.\n\n2. The paper aims to find a data-centric utility measure that is robust to training randomness. Yet it fails to cite or discuss the large, established field of \"data valuation\" which also applies to this problem. Methods such as Data Shapley (https://arxiv.org/abs/1902.10275 ), LAVA (https://arxiv.org/abs/2305.00054 ) provide metrics for data utility from a pure data perspective, free from the randomness of any single training run. Data Banzhaf (https://arxiv.org/abs/2205.15466) is specifically designed to maximize robustness to training variations. The omission of this entire body of work is a critical flaw.\n\n3. The experiments are insufficient to support the paper's claims. In the mislabeled MNIST experiment, the baselines appear to be incorrectly chosen or implemented. The results show that f-INE performs similarly to other methods, failing to demonstrate its purported superiority. The second experiment (LLM attribution) is conducted on a sample size (training on 50 examples, testing on 10) so tiny that it is impossible to derive any scientifically valid conclusions. Furthermore, this experiment only compares f-INE to LESS. Citing LESS as the sole baseline is questionable, as it was proposed for large-scale data selection, not the small-scale setting tested here.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4QvnOoOAWa", "reviewer": "Reviewer_EzUe", "summary": "In this paper the authors propose a novel data influence estimation algorithm which is an improvement on top of the existing TracIn (Training Data Influence by Tracing Gradient Descent) approach. The proposed extension is less vulnerable to random training runs, and is motivated by privacy auditing and hypothesis testing theories. The paper estimates the statistical significance of data influence by quantifying the distinguishability of two distributions: a distribution of influence scores when the sample participates in training over multiple random training trials and a distribution when the sample did not participate in the training process. The paper proposes to look at the tradeoff curves between Type I and Type II errors as a means to quantify the distinguishability of those two distributions.\nIt draws parallels between their definition and Gaussian Differential Privacy, claiming that the tradeoff can be fully characterized by a single variable, mean.\nThe paper shows that the proposed approach is more stable and is effective for MNIST's miss-labeling task and train-test influence for LLM’s data poisoning tasks.", "questions": "+ The experimental evaluation part is a bit lightweight\n  + MNIST is a very toy model and LLM experiments are good but cover a narrow set of applications -  bias in a specific category.\nIt is unclear how well the approach performs for a wider range of datasets or models. E.g. LESS performs the evaluation on a wider range of downstream tasks and datasets (bbh, tinyqa and mmlu). In addition to that, TRAK evaluates their approach on fact checking dataset. It would be good to report results on a wider range of tasks and datasets similar to those baseline approaches.\n+ The algorithm description is a bit unclear\n\t+ E.g. Algorithm1: lines 9 and 10 the right side of the assignment seem to be identical,\n               likely there is a typo there. Similarly line 4 and line 5 perform the same sampling (D \\ S ) ? \n+ It is unclear how S is selected in Algorithm listing 1 and whether Algorithm 1 is called in a loop for random subsets of S. Is it guaranteed that each example in S will be encountered only once in S ?\n+ It seems that instead of retraining the model multiple times the authors leverage training epochs but this can be a very noisy approximation.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The paper is well and clearly written without major ambiguities.\n+ The paper studies an important problem, randomness in the influence estimation. It proposes an interesting approach motivated by the Differential Privacy theory and explains the theoretical intuition behind it.", "weaknesses": "+ See weaknesses \n+ In Figure 7 the error bar for F-INF looks pretty large. If  F-INF is more stable than LESS why does it have wider std ?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)