Theoretical and Practical Perspectives on what Influence Functions Do
Authors: Andrea Schioppa
Katja Filippova
Ivan Titov
Polina Zablotskaia
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Explainable AI
Influence Functions
Training Data Attribution
TL;DR: We identify the problematic assumptions made by Influence Functions methods and clarify what can be expected theoretically from them; based on this we propose to correct mis-predictions by taking a few fine-tuning steps on influential examples.

Abstract: Influence functions (IF) have been seen as a technique for explaining model predictions through the lens of the training data. Their utility is assumed to be in identifying training examples "responsible" for a prediction so that, for example, correcting a prediction is possible by intervening on those examples (removing or editing them) and retraining the model. However, recent empirical studies have shown that the existing methods of estimating IF predict the leave-one-out-and-retrain effect poorly. 
In order to understand the mismatch between the theoretical promise and the practical results, we analyse five assumptions made by IF methods which are problematic for modern-scale deep neural networks and which concern convexity, numeric stability, training trajectory and parameter divergence. This allows us to clarify what can be expected theoretically from IF. We show that while most assumptions can be addressed successfully, the parameter divergence poses a clear limitation on the predictive power of IF: influence fades over training time even with deterministic training. We illustrate this theoretical result with BERT and ResNet models.
Another conclusion from the theoretical analysis is that IF are still useful for model debugging and correcting even though some of the assumptions made in prior work do not hold: using natural language processing and computer vision tasks, we verify that mis-predictions can be successfully corrected by taking only a few fine-tuning steps on influential examples.

Theoretical and Practical Perspectives on what Influence Functions Do
Katja Filippova' Ivan Titov Polina Zablotskaia' *Google DeepMind University of Edinburgh UUiiersity of Amsterdam farischioppa, katjaf, polinaz)@google. com, ititovšinf, ed. ac uk
Andrea Schioppa'
Abstract
Influence functions (IF) have been seen as a technique for explaining model pre- dictions through the lens of the training data. Their utility is assumed to be in identifying training examples "responsible" for a prediction so that, for example, correcting prediction is possible by intervening those examples (removing editing them) and retraining the model. However, recent empirical studies have shown that the existing methods of estimating IF predict the serre--on-ouu-and retrain effect poorly. In order to understand the mismatch between the theoretical promise and the practical results, analyse five assumptions made by IF methods which are problematic for modern-scale deep neural networks and which concern convexity, numeric stability, training trajectory and parameter divergence. This allows us to clarify what can be expected theoretically from IF. We show that while most assumptions
be addressed successfully, the parameter divergence poses a clear limitation on the predictive power of IF: influence fades over training time even with deterministic training. We illustrate this theoretical result with BERT and ResNet models. Another conclusion from the theoretical analysis is that IF are still useful for model debugging and correcting even though some of the assumptions made in prior work do not hold: using natural language processing and computer vision tasks, we verify that mis pis-predictions can be successfully corrected by taking only few fine-tuning steps on influential examples.
Introduction and related work
Influence Functions (IF) CS82 have been regarded as a tool that can trace model behavior on any example to the training examples [LL77 PLKS20 GRH+21] Their theoretical justification lies in the ability to predict loss changes on a specific test point when training on a perturbed loss obtained by removing or down-sampling given training point. In the case of an undesired model behavior on test-point, the influential training examples for that test point have been assumed to be the ones "responsible" for the prediction so that intervening retraining the model, or by taking additional fine-tuning steps [GR++21| would result in a change in the loss prediction. It has been confirmed that for linear models it is indeed the case [ATLI9].
those e.g., by removing them and then
Recently, in their extensive experiments, [BPF2] and [K22 could not find empirical support for the claim that IF approximate the Leave-Some- Retraining (LSOR) effect on the loss in deep neural networks. In particular, they show that the correlation between the ranking of training examples produced by LSOR and the la-bd ranking is low and considerably affected by choice of hyper- parameters. How can this discrepancy be explained? And, given that the theoretical justification for IF lacks empirical support, does it mean that IF should be abandoned an explainability and debugging tool altogether?
In this work we clarify what question Influence Functions (IF) actually answer. We first identify assumptions which are either implicit not investigated in prior work: these concern convexity,
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

numeric stability, training trajectory, and the the parameter divergence when retraining on new loss. In principle, any of these assumptions might be problematic and be the reason why the original theoretical justification for IF is not supported empirically. However, we show how to address most of them successfully so that they cannot be the reason for the aforementioned discrepancy. Unfortunately, we also show that the parameter divergence is indeed problematic and requires to revise both theoretical and practical expectations about IF, This analysis paves the way to clarifying the question that IF can answer. As a first step in this direction, we need to distinguish two approaches to computing influence. The Hessian-based Influence Functions (HIF) [CS$2] [L17] have a rigorous theoretical justification in statistics, but rely on strict convexity assumptions, which are not met in Deep Learning and which previous analyses of HIF in the Deep Learning literature still rely on [LL77 BNL 22). Our first contribution concerns HIF and is twofold: we prove (Theorem ! that this unsatisfied convexity assumption is not as problematic for HIF as one may think-given a stationary point for the original loss, there is nearby stationary point for the perturbed loss which can be approximated by using HIF. However, we point out a more serious problem with HIF: there is no guarantee that by retraining on the perturbed loss one would get to that stationary point. This observation provides an additional support to the second popular approach to IF, TracIn [PLKS20), which explicitly models the training dynamics and which additionally does not require to compute the expensive inverse Hessian vector product as it only uses gradient information Despite TracIn being grounded in the training dynamics, we unveil a hidden additive modeling assumption in [PLK220 that prevents it from correctly modeling the (re) -training dynamics. Our second contribution is thus to provide theoretical analysis (Theorem 2 of how training trajectories change when perturbing the loss. This is key result as it suggests that two training trajectories differing by small loss perturbation could diverge over time to the point of violating a first-order expansion assumption that IF make, thus making IF's predictions unreliable. Our further theoretical and empirical investigation confirms this conjecture. Therefore, what IF can do is to predict parameter changes when fine-tuning for a limited number of steps on the perturbed loss. This requires to adjust how IF are evaluated (Section 5) and applied (Section (6). In order to confirm the conjecture and re-adjust expectations regarding IF, we need a deeper analysis of the trajectory divergence.
Our third contribution is thus to validate the conjecture on trajectory divergence and its consequences for IF. We first prove (Theorem 33. using a discrete version of Gronwall's 's Lemma [Gro19], an upper bound on the parameter divergence when (re)-training on a perturbed loss. We then empirically verify that the bound is sharp in Section 5 that IF can approximate parameter changes along the perturbed trajectory only for a limited amount of time. We then empirically verify that this leads to a fading (of accuracy) of IF predictions over time. Therefore, we theoretically demonstrate and empirically verify that IF can in general answer only what happens when fine-tuning on a perturbed loss for limited amount of time, instead of a general retraining setting.
Therefore, our new theory indicates that IF have been used incorrectly as the emphasis has been on their LSOR potential. On the positive side, it suggests an alternative way of using IF that indeed yields substantial empirical improvement. To demonstrate that, as our final contribution, in Section 6 we propose and verify a very simple method for correcting mis mis-predictions by taking only a few gradient steps on influential examples. Our proposal is related to model editing [CCAT2] inter alia] in that the latter also aims at changing model predictions. However, there the model is given and the modifications are done on its parameters whereas IF aim at understanding how specific training examples are responsible for the current model behavior and editing predictions through the data. We leave for future work building connections between model editing and IFF
For the case of Hessian influence functions, the recent work of [BNL, has proposed to resolve the discrepancy between the theory and the empirical evidence by evaluating them using the proximal Bregman response function. However, [BNL+22 relies crucially on the convexity assumption: while under this assumption there is agreement between our work and their findings (e.g. our parameter divergence drives the linearization error [BNL,+22] Sec. 4.4)), our theory applies also the the case where the Hessian is not positive definite. Moreover, our work covers also the scope of Gradient-based influence functions including TracIn [PLKS20]
'To incorporate training dynamics, TracIn exploits multiple checkpoints which introduces a substantial overhead, hence only the latest checkpoint is often used in proposed in (CGFT19 HYH120 but they lack justification from the training dynamics perspective.
practice. Other gradient based methods have been

---

2 Definition of Influence Functions and Notation
The different methods proposed define Influence Functions share common goal: forecasting the change in the prediction on a test example when up (or down-) weighting a training example. This is achieved by tracing the effect that re-weighting training example has the model parameters.
Removing or adding a training point can modeled by a perturbation of the loss function; such perturbation can be made smooth by modeling the weighting of training point by continuous parameter E. More generally, let L(0) denote the loss function where 0 E R N are the model parameters. We model loss perturbations by introducing a variation of the loss, which a smooth function e().e) depending on an additional vector parameter & R and coinciding with the original loss for E O. For example if l denotes the loss on given training point we set L(0.e) e) = L(0) + Elz While the scalar case Q = is the one commonly considered, c.g. [KL17 PLKS20), we introduce the vector one Q which arises naturally when considering the effect of re-weighting multiple points differently, e.g. when modifying the weights in a mixture of different data-sets.
The IF method of choice then predicts what would happen if training on L(B, E) instead of L(O). The final parameters are modeled as a function 6g of the perturbation parameter; assuming that such function is well-defined and sufficiently smooth, one then makes a first order expansion 0g = 00 + V ve where VejoOe the Q x N-dimensional Jacobian at (see also Section Aj.
Under this first order assumption it is then straightforward to measure the change in the loss lg corresponding to test point 2:
V
VejoOe Voj00
(I)
We emphasize that (T general different IF methods, which differ in the specific derivation of
V Ve
3 Problematic assumptions made by Influence Functions
3.1 Problematic Assumption #1: Convexity can be used to show that 0g a function
In order to show that for each value of & there a single value of 0- (so that one can model the final parameters function of the perturbation parameter), HIF relies on strict convexity of L. While this assumption is realistic for the statistical models considered not the case for neural networks. Even when introducing regularization term, the loss of neural network is not even weakly convex, and optimization methods usually converge to saddle points (DG++14 literature, e.g. [KL17 BPF21 BNL have relied on some form of strict convexity. In Section 4.1 will revisit HIF and prove (Theorem D roughly speaking, that near given stationary point 6 for the original loss L. there is a stationary point tg of the perturbed loss [(0,=) =) which can be modeled as a function of = and such that VejoOs is given by -H
[CS82), this is
14). To the best of our knowledge, previous analyses of HIFF in the Machine Learning
1 V C as [CS22] (see Section A
regarding the usage of 22).
3.2 Problematic Assumption #2: The model Hessian is not degenerate
As HIF requires apply the inverse model Hessian to V Vic.0110.06)44 C. one needs to ensure that inverting the Hessian is numerically stable. It has been empirically demonstrated [KXX9 that most eigenvalues of the Hessian tend to cluster near 0. Numerically, this results in a considerable source of errors and instabilities when estimating HIF; while regularization can alleviate this problem, in the definition of HIF; the minimal value of such
introduces hyper-parames ensuring numerical stability depends on the smallest negative eigenvalue of the Hessian. Unfor- tunately, in realistic settings, this can be larger in absolute value than reasonable values for the regularization parameter. For example our ResNet experiments regularization is of the order 10-4 while the smallest negative eigenvalue is z -5. In Section 4.2 we discuss how the Arnoldi-basee Influence Functions (abbr. ABIF) (which were introduced by SZTS22 for computational efficiency) can be used to address such instability issues.
3

---

3.3 Problematic Assumption #3: Training trajectory can be ignored in Hessian-basee Influence
Even if we solve the Problematic Assumption #1 for HIF, there is no guarantee that when actually re-training from scratch on L(O, e) one would converge to the 0g given by Theorem because the training trajectory is disregarded in HIF. As optimization is performed via some form of stochastic gradient descent, PLKS20 propose TracIn which averages gradient dot- dot-products across checkpoints in order to take into account the path taken by the training process. Importantly, for a single checkpoint Oo TracIn estimates V.0g as Vi ((.,e) C, the inverse Hessian vector product does not need be computed. While it seems that TracIn takes into account the training trajectory, in the next Assumption we identify an issue with the way it models the training trajectory.
3.4 Problematic Assumption #4: The training trajectory can be modelled additively
The analysis of TracIn in [PLKS20] is based on a first-order expansion of the final change of the loss of test point in terms of the gradient steps across the training trajectory. While this argument seems mathematically convincing, it overlooks that if one point is removed, or slightly up-sampled down-sampled, the subsequent training trajectory is modified. Let 8g.0 denote the value of the parameters after steps when doing gradient descent L(0, e). Denoting by T the end-time, TracIn derives
V VejoOe,T
Σ t=0
n 2 (E.0)......))
where 7/r is the learning rate at time step Now, the right-hand side in formula (2) is purely additive in the time steps; and addition is commutative, so the order of the time steps does not matter. One way to see that this is problematic is by making L time-dependent so that differs from only at a specific time step t. In this case V Vec.oy L would be non-zero only t and formula ( would consist of single term. However, we would expect the perturbation at to affect the following time steps, should have at least T t terms on the RHS for @ In Section 4.3 we compute V.G..r looking the whole training trajectory and discover an additional first-order term that is missing from TracIn: this term models the dependency of time step on the earlier ones. Concurrent work [GWP+23] also criticizes the additive assumption in TracIn on empirical grounds and proposes build (re)-training simulators which are unfortunately computationally expensive as a new simulator must be fitted on the training set for each test point.
3.5 Problematic Assumption #5: 0g can be expanded to first order in
After we derive a formula for 0G and Ve we realize that the latter can grow in norm in T. However, if 0e can be Taylor-expaneed in E, we need 0g.r - 0,, to be O(e) and V.G..r to be 0(1). If this not the case, the whole approach described in Section 2 breaks down because IF approximate the parameter change b.. forr using the Taylor expansion e V.Ge.r, but the conditions to apply such Taylor expansion are not satisfied.
In Section 4 we show how assumptions #1-#4 can be successfully addressed which makes them not as problematic as they may first appear. However, for assumption #5 will see that it puts substantial limitation on the predictive power of IF. At the same time it allows for a new, locally bound perspective on IF that influence holds for a limited number of fine-tuning steps (Section 55) Based on this finding, in Section 6 we propose a simple approach to use IF to correct mis-predictions which is theoretically grounded and is in addition much less compute intensive than those that involve re-training (c.g. [KL77).
4 Addressing the problematic assumptions
Proofs of all results are in the Appendix and we provide some motivation for the proof in the main text.

---

4.1 HIF does not need Assumption #1
Previous work [CS82] KL17 BPF2I BNL 22 on Hessian-based Influence Functions (HIF) has assumed that L is strictly convex in order to claim that 1) the minimum is unique so that 8g can be modeled as function, and 2) to use the Implicit Function Theorem to differentiate through the optimality condition.
Here will just assume that the Hessian is singular at 06: by requiring that the final gradients do not change as we change E we prove:
Theorem 1. Assume that V is C* (k 2 a)d let Ou E RR; assume that the Hessian Hop Vij
is non-singular; then there exist neighborhoods U of 0g and V of 0 E Rº, and C -fffucction θ V + U such that (0)) = On and ((e) E U is the unique solution in U of the equation VeL(O(E), = (V
(3)
Moreover, the gradient of O the origin is given by:
Vejo - Ho V (a.0)...))))
(4)
Idea of the proof. Existence of (e)) is formulated as a solution to the local problem eq. 3 building local solution does not require convexity if one uses the Implicit Function Theorem.
The requirement that ())( is constant in has allowed establish link between the training under different losses {0 - L(0,=))e. If we assume that 0g is a stationary point, iec VoL(Og) = O, we strengthen the conclusions:
Corollary 1. Under the assumptions of Theorem 7
1. If 0o is a stationary point of L. then each ((x) is a stationary point of the loss 9 + [(0,e).
2. If Ou is a local (strict) minimum of for sufficiently small, ((E) is a local (strict) minimum of the loss 8 + L(),e).
3. Let Q = 1 (hence epsilon is scalar) with L(0,E) L(0) + cl), where la is the loss corresponding to a specific training point x. We then obtain the classical result (CS82):
d0
c= =
-
(5)
4.2 If Assumption #2 is not satisfied, use Arnoldi-based Influence Functions
Theorem D Trequires that Ho, is non-singular. If the Hessian is singular, just need to keep fixed those parameters that are responsible for the degeneracy. More precisely, we diagonalize Ha, we let P. be the subspace spanned by the eigenvectors corresponding let Po be its orthogonal complement, that is, the kernel of Hag- Up to an orthogonal transformation of the parameters,
the non-zero eigenvalues and
can assume that P1 is spanned by the first Ny-coordinates and decompose 0 (0,x) RAY N2 that Ho, = ((Do, Po)) is non-singular. We then apply Theorem to the restricted variation (1,e) + L((d, so)), e). In terms of the original parameters 0, this means that the function ((e) is constrained to lie in 0g PI. keeping the y--component constantly equal poo Concretely, we can approximate P1 using the Arnoldi iteration; therefore, we can address the failure of Assumption #2 by using Arnoldi- snoodi-based Influence Functions (ABIF) [SZTS22]. which approximate P1 using the subspace spanned by the cigenvectors corresponding to the top-k (in absolute value) eigenvalues of the Hessian. The fact that ABIF stabilizes the estimation of influence scores has been observed empirically in previous work: in [SITS22 Fig. 2] where using ABIF instead of the full Hessian improves the retrieval of mislabeled examples, and [FLP+23 Fig. 3] where ABIF the best solver on the QA task and is comparable to the other solvers on the text-completion task.
4.3
The training trajectory can be traced to address Assumptions #3-#4
We need to improve our notation to correctly trace the training trajectory. The first issue is to keep track of the parameters across the time steps; the second issue are sources of non-determinism.
5

---

e.g. batch selection or random state for dropout. When comparing training trajectories for different values of we want our notation account for sources of non-determinism, as they might increase the difference between the training trajectories.
To address the first issue, let te, be the value of the parameters after training L(0, for t steps. In particular, 0g.0 denotes the initial value condition that we assume held fixed at Imitt for different values of E. As random state is function of the training step (e.g. the batch to use at step t), to address the second issue, we just need to allow both the loss and the variation to depend on the train step, denoting them by L, and L.
To simplify the exposition and for consistency with PLKS20] we assume that models are trained with stochastic gradient descent. Letting iu be the learning rate step t prove:
Theorem 2. Assume that the model is trained for T time-steps with stochastic gradient descent. Denoting by He the Hessian Vo La, then the final parameters oer satisfy:
8/oo,
T-1 Σ
V V=joOs,
Σ n V (c.0)00000000
T V
(6)
Idea of the proof. One can explicitly write a system of equations for 0g, and apply the operator Veju to the solution of this system.
Note that the second term on the RHS of ( which is missing from takes into account the contribution of the earlier time steps that is missing from the analysis of [PLKS20]. A practical
consequence of this second term is that the norm of VejoBe.T might grow (in T) more quickly than ( would suggest: this is closely related to Assumption #5 which requires VejoOe, to be O(1). In particular, for constant learning rate, while (2) suggests a linear growth in the time step T, we will empirically verify in Section 5.1 that the growth is super-linear.
In Section K we illustrate the additional modeling error introduced by TracIn when computing the
Jacobian V Ve in the case of retraining BERT with SGD.
4.4 Assumption #5 becomes problematic over time
To address Assumption #5 we need to look into the parameter divergence ||,.TT 0o between training on L and L. The training dynamics a discrete version of an ODE for which uniqueness and differentiability of the solutions with respects to the initial conditions can be established by Gronwall's 'ss Lemma [Gro19]. The parameter E can itself be considered an initial condition and we can prove discrete version of Gronwall's Lemma to bound the parameter divergence. For simplicity of notation we prove the result for stochastic gradient descent, but we also sketch in the Appendix how modify the argument to deal with optimizers.
Theorem 3. In the setting of Thearem that
2 assume that, for & S T, Oz,t lies in bounded region R such
V [V -C 0)l < Ce
sup
GØER
and that for 0 E each loss E)) and its gradient wrt. 0 are A-Lipschitz in 0. Then
10e,T - 0o S Ce Σ na(1 + exp(2A Σ 1.))
s<T
T<T
Idea of the proof. If the time steps were infinitesimal, i.c. if time was made continuous, the evolution of 0z,t would be governed by an ODE; then the bound (8) would be straightforward by applying the classical Gronwall's Lemma Gro19.. I. In our case we just need to modify the ODE arguments to work with discrete time.
Note that the bound (8) is quite pessimistic as it involves an exponential of the integrated learning rate EOST No. This means that E.eT 7e increases, the parameter divergence is no longer O(E) and

---

Difeerente
10
Perametter Di fereenee
10
10-1
10
1e
f-1o-06
10
511e00
t-1e-06
10-1
111e:0
10-1
51e-00
t-1e-03
c-1e:03
02
10-4
t-1e-01
s-1e-01
0.000
0.001
0.007
0.003
0.004
0.00
0.25
0.50
0.75
1:00
1.25
1.50
1.75
2.00
Integrated learning
Integ learning
(a)
(b)
Figure Divergence of parameters (log-scale) as function of the integrated learning rate. For each value of e the divergence is exponential (corresponding to a line in log-scale) with two different divergence rates, one more steep the beginning of vee-ttaining. (a) BERT, (b) ResNet
the crucial Assumption #5 is no longer satisfied. This observation leads to few crucial conclusions:
1. An IF method can predict 0g,r only for limited amount of time-steps: it is therefore incorrect to evaluate methods on LSOR or retraining from scratch.
2. IF methods need be evaluated what they can potentially do; therefore the evaluation setup should consist of fine-tuning on the perturbed loss only limited amount of steps with evaluation metrics being reported as a function of the step.
3. Applying IF for correcting mis- diseprrddittins should also involve time-hound scenario: propose such method Section G
4. Sources non- -on-determinimm between two training runs will likely increase the parameter divergence. So one should try to reduce this with deterministic training. For example, in the case of re-weighting point æ, i.c. setting L = + ælg, one should make sure to use the same batch B, for the loss L at time step t across training runs for different values of e.
5 Illustrating the Theory
In this section we first demonstrate Theorem empirically and then verify that the predictive power of influence scores degrades time. Full details of our experimental setup are reported in the Appendix. We consider binary classification for nlp, where we fine-tune BERT on SST2; for computer vision we consider multi-class classification where we train from scratch ResNet CIFARIO. All our experiments use deterministic training: the order of the training batches for the loss L is held fixed across different runs, well are the random generators when dropout is used.
5.1 Illustrating Parameter Divergence (Theorem
Theorem 3 provides an upper bound when re- re-training on a perturbed loss. Such a bound is rather pessimistic it involves the integrated learning rate. We therefore investigate empirically what happens with some typical Deep Learning setups. We take an intermediate checkpoint and keep training new loss obtained by up-sampling 16 training points with weight s, that is: L, LBB + - LB, where B, is the training batch for step and B the batch of 16 points selected for up-sampling. For each time step then compute |0=r ro and then plot against the integrated learning rate, see Figure
Unfortunately, observe that in these experiments the upper bound in Theorem 3 is matched by lower bound with the same exponential divergence. We observe first phase of quick divergence and then a second one in which the divergence is slower. For the second phase linear fit of log | - fu,rll against the integrated learning rate appears to be strong: for example for BERT we obtain an R. least 0.9 across the different values of e. The fitted slope, corresponding A in Theorem depends on and varies between 30 and 130.

---

1.25
Hessian
Hessian
0.3
3.5
Optimizer
350
0.2
0.25
3.00
0.3
0.25
0.75
-0.1
200
450
660
800
1030
500
1000
1500
2000 Step
2500
3000 3500
4000
Step
(b)
Figure 2: The predictive power of influence scores on the loss shifts degrades over time. (a) BERT, (b) ResNet. The line represents the average of the correlation R(t) across runs, with the shaded area the corresponding 95% confidence region.
5.2 Illustrating the fading of influence
We now verify that the predictive power of influence scores fades over time. We again fix model checkpoint iiit and select 32 training points and 16 test points. For each training point x we retrain on L = LB, Tun 22. i.e. has been down-sampled; for each test point and time step f we then compute the loss difference 6(z, x, t) = 1g,z,t - 1,,t where at. is obtained when (re)-training on CT and 1, is obtained when training on the vanilla loss L, LB.- Again, we have kept the order of the batches B, the same when re-training. Now, at the original checkpoint Oinit. we can compute the influence scores [2 x) for different methods, e.g. TracIn or HIF (using the the Conjugate Residual method For each time step we thus have 32 x 16 values of 8(2, x, t) that can be linearly regressed against F)), the corresponding Pearson correlation R(t) then measures the predictive power of influence scores on the loss shifts when re-training. We repeat the experiments for ResNet 9 times and for BERT 25 times, with different selection of train and test points, so that obtain confidence intervals for the resulting time-series R(t). Ideally, the theory behind an influence method predicts R(t) 1. However, as discussed above, Assumption #5 is indeed problematic and, because of the parameter divergence illustrated in 5.1 we expect R(t) to degrade over time.
As Figure 2 demonstrates, this is indeed the case. The predictive power is high for BERT after few re-training steps and then degrades quickly oscillating around 0 (which is covered by the confidence intervals). For ResNet, the predictive power is never as high, but it still degrades monotonically over time, as predicted by the theory. As we see in Section 6 the proponents retrieved for ResNet are less effective at correcting mis ais-predsstions than those retrieved for BERT: we conjecture that this is related to the worse predictive power of IF in the case of ResNet. As BERT was trained with the Adam Optimizer, we also considered a variant which takes into account the optimizer's 's pre-conditioner by multiplying the gradients by the square root of the pre-conditioning matrix. In this case the predictive power is worse than for the vanilla version of TracIn. For BERT, more plots and a further discussion about TracIn are included in the Appendix (Section G.I). In the Appendix (Section illustrate the fading of influence for another NLP pre-trained model, T5. While we have illustrated the fading of influence for both BERT and ResNet, in the latter case the peak of correlation is quite low; we repeated the same experiments on the ViT (Section G.3). where we find a high correlation peak as in the case of BERT.
G.2 we also
6 Using Influential Examples for Error Correction
KL17 propose to correct model mis-preeistionn by first using influence scores to retrieve the examples most responsible for a given prediction, and, after correcting them, retraining the model. Computational considerations aside, a key conclusion from the theory in Section 3 and the empirical verification in Section 5.2) is that IF only predict influence over limited number of training steps. In this section we demonstrate that IF can still be used to correct model mis mis-predictions by taking few fine-tuning steps on inftuential examples.
Further details in the Appendix.
8

---

10
0.-
3.8
0.7
0.0
Pruza MILMMA
0.
Prop. retencion
0.4
Proo. retention
Opp. succoss
Opa. success
0.4
Opp. retention
Opo. retencion
Saswine
Saseine
0.3
Saseline
Baseline
0.0
02
0.4
0.6
3.
1.
12
14
0.0
02
0.4
1.0
epsioo
opsion
(a)
(b)
Figure 3: Success rate and retention for error correction. (a) BERT, (b) ResNet
Concretely, propose correct mis- pre-pieediiionss given test point by first identifying batch of influential examples and then taking a few fine-tuning steps on the perturbed loss - T. + e LB- identify the set B of top-k proponents for the current x and relabel them to what should be the correct prediction on x; (2) Opponents-tuning: as opponents oppose the current prediction, take to be the of top-k opponents of r.
We propose two methods: (1) Proponents-correctivn
We investigate how well these error-correction techniques work on SST2 (BERT) and CIFARIO (ResNet). As baseline, the prediction on x and then set their label equal to the correct one for x. We take a maximum of 50 fine-tuning steps and take the top-50 proponents opponents to build B. The main metric compute is the success rate, i.e. the ratio of mis-predictions successfully corrected within the limit of 50 steps. Additionally, examples, i.e. the ratio of examples predictions which have not changed after a correction ideally, a correction does not cause different values of E, starting from no up-sampling and gradually increasing it to when influential examples account for slightly more than half of the batch.
randomly sample set B of training points with the same label as
report prediction relention (DCAT21]
fixed held-out of 50 test
many changes in model predictions otherwise. We experiment with
From Figure 3 we see that Proponents-comee correction and Opponents Opponents-tuning tuning strongly outperform the baseline in binary classification (SST2). For multi-class classification (CIFAR1O) Opponents-tuning not effective, as we verified that only 54% of the retrieved opponents have the desired label; cnnonentt--oorectioon still outperforms the baseline increasing average the success rate by 2% and reducing the number of steps to take by 6%. We conjecture that for ResNet the improvement the baseline is less than for BERT because of the worse predictive power of IF (Figure (b)). In Appendix H we include additional plots showing the number steps to correct mis- mis-predictions as a function of the up-sampling parameter s.
The primary goal of experiments in this section is to verify that tuning on influential examples results in correction more reliably and faster than other classes of examples. Since the SST2 and CIFAR 10 datasets are largely clean, for the proponents method, the training example label is flipped incorrect one. In Appendix [I] we give examples from noisy text classification dataset illustrating the scenario that the correction with prth-prooponets method supposed to address.
7 Limitations
We derive the perturbed training trajectory (Theorem 2 and the divergence of trajectories (Theorem 3) for stochastic gradient descent and, while we sketch the Appendix the modifications needed when using other optimizers, we do not pursue this topic in detail. In Section 6 we measure prediction retention after correcting mis-predictions. While this metric intuitive and has been used previously (e.g., [DCAT21), we do not distinguish between semantically similar and unrelated examples and thus do not check the consistency and generalization properties of the update [MBAB22), leaving thorough study of the comrection-reeention tradcoff to future work.

---

8 Conclusions
IF have been regarded as a tool that promises to trace model behavior to the training data. Unfortu- nately, recent studies have found no empirical support for such claim as IF fail to predict the LSOR effect. This finding gives rise to the question of what IF methods really predict and whether they could be useful for model debugging. In this work we clarified which questions IF can be expected to answer. We first identified problematic assumptions made by IF methods a priori any of these assumptions could be reason for the observed empirical failure of IF. Thus, for each assumption we studied if it is indeed problematic. While most have turned out to be addressable in one way or another, we demonstrated that the one about parameter divergence puts severe limitation on IF. With a deeper analysis of this assumption, we revised what can be theoretically expected from IF: IF methods are time-bound, that is, they can at most predict what happens when fine-tuning on perturbed loss for a limited amount of time. With that, a practical usage of IF for model debugging is still possible we proposed and empirically validated a pooretically-groundee procedure to apply IF to correct model mis-predictions.
References
[BNL*22] Juhan Bae, Nathan Ng, Alston Lo, Marzyeh Ghassemi, and Roger Grosse. If influence functions are the answer, then what is the question?, 2022.
[BPF21] Samyadeep Basu, Phil Pope, and Soheil Feizi. Influence functions in deep learning are fragile. International Conference on Learning Representations, 2021.
[CGFTI9] Guillaume Charpiat, Nicolas Girard, Loris Felardos, and Yuliya Tarabalka. Input similar- ity from the neural network perspective. In H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlché Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Process- ing Systems, volume 32. Curran Associates, Inc., 2019.
[CS82]
R. Cook and Weisberg S. Residuals and influence in regression. Chapman and Hall, New York, 1982.
[DCAT21] Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6491-6506, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
[DPG+14) 14] Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Gan- guli, and Yoshua Bengio. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization, In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
[FLP*23] Jillian Fisher, Lang Liu, Krishna Pillutla, Yejin Choi, and Zaid Harchaoui. Influence diagnostics under self-concordance. In Francisco Ruiz, Jennifer Dy, and Jan Willem van de Meent, editors, Proceedings of The 26th International Conference Artificial Intelligence and Staristics, volume 206 of Proceedings of Machine Learning Research, pages (0028-10076 PMLR, 2023.
[GKX19] Behrooz Ghorbani, Shankar Krishnan, and Ying Xiao. An investigation into neural net optimization via hessian eigenvalue density. In ICML, 2019.
[GRH+21] Han Guo, Nazneen Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong. FastF: Scalable influence functions for efficient model interpretation and debugging. In Pro- ceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10333-10355 Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
[Gro19]
T. H. Gronwall. Note the derivatives with respect a parameter of the solutions of system of differential equations. Annals of Mathematics, 20(4):292-296, 1919.
[GWP+23] Kelvin Guu, Albert Webson, Ellie Pavlick, Lucas Dixon, Ian Tenney, and Tolga Boluk- basi. Simfluence: Modeling the influence of individual training examples by simulating training runs, 2023.
10

---

[HYH120]
Kazuaki Hanawa, Sho Yokoi, Satoshi Hara, and Kentaro Inui. Evaluation of similarity- based explanations. In ICLR-2222 2020.
[KATL19] Pang Wei Koh, Kai-Siang Ang, Hubert H. K. Teo, and Percy Liang. On the accuracy of influence functions for measuring group effects. In Proceedings of the 33rd International Conference Neural Information Processing Systems, Red Hook, NY, USA, 2019. Curran Associates Inc.
[KL17]
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Doina Precup and Yee Whye Tch, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learming Research, pages 1885-1894, PMLR, 06-11 Aug 2017.
[KS21]
Karthikeyan K and Anders Søgaard. Revisiting methods for finding influential examples, 2021.
[MBAB22] Kevin Meng. David Bau, Alex J Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances Neural Information Processing Systems, 2022.
[PLKS20] Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
[SZTS22] Andrea Schioppa, Polina Zablotskaia, David Vilar Torres, and Artem Sokolov. Scaling up influence functions. In AAA--22, 2022.
[WTD17] Ellery Wulezyn, Nithum Thain, and Lucas Dixon. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference World Wide Web, WWW '17, pages 1391-1399, Republic and Canton of Geneva, CHE, 2017. International World Wide Web Conferences Steering Committee.

---

A Clarifications regarding the operator
We spell out in further detail the way we employ the operator. For example, we say that V ei the Q x N-dimensional Jacobian at 0: concretely. V Velso means computing the Jacobian of wrt. E and evaluating it at the point Eg: this Jacobian is matrix whose (i:j)-entry is given by:
df (so).
(Veleo))ss
de,
This is easily extended to higher order derivatives denote by vh; specifically we use V V (0.0)(0.%) to denote the matrix whose i, jeenryy is given by:
00
V (V (1.0)) C).s
0o):
(10)
de,00;
as the matrix H acts on the space of the parameters 0, the notation 0o V (t. contraction on j:
(0,00) C will denote
0C
H (.00).00))))..)).. E(Ho )k.s
k.s 05
(0, 0n)-
(11)
B Proof of Theorem 1
For convenience, we first recall the statement of Theorem 1
Theorem. Assume that V C is C* (k 2 1) and let On R assume that the Hessian Hog V2 T,
8/0
is non-singular; then there exist neighborhoods U of 6g and V of 0 E Rº, and O fffcciin θ V + U such that ((0) Oy and ((s) U is the unique solution in U of the equation
VoL(O(E),s) VoL(Oo, 0).
(12)
Moreover, the gradient of Q the origin is given by:
Veno - Hou 2 (c.0).)
(13)
Proof. Equation 12) gives us N constraints on the N + Q variables (0, and we want to solve them for the first N variables 0: obtain the function O we invoke the Implicit Function Theorem, using that the Jacobian wrt. the variables we want to solve for is Hg, and is therefore non-singular, Finally (13) obtained by taking the gradient of (22 wrt. and setting 0:
V Volo. L. V Vejo (c.00 (0.0.) C=
C Proof of Corollary 1
For convenience, we first recall statement of Corollary [
Corollary. Under the assumptions of Theorem !
1. 1f is stationary point of L, then each O(e) is stationary point of the loss 0 + L(0,e).
2. If is a local (strict) minimum of T,, for e sufficiently small, O(s) is a local (strict) minimum of the loss + L(0,e).
3. Lei Q = 1 (hence epsilon is scalar) with L(0,E) = L(O) + (0), where lg the loss
specific training point We then obtain the classical result (CS82) about the inftuence of down/up down/up-samplan sampling x on the training parameters:
corresponding
d0
-V)L(()))
(14)
===
12

---

Proof. If do is a stationary point of L, then VoC(0b.0) 0) - in B.. This implies that VoL()(s),
0 in 3 which is exactly the statement that (e)) stationary point of 0o a local (strict) minimum of T,, it is not just stationary point, but the Hessian at On is also positive definite. By continuity, for sufficiently small &, also the Hessian Voje(s) & will be positive definite so that (s)) will be a local (strict) minimum. Finally, (14 follows from applying C(0,E) L(0) + (0).
to the variation
D Proof of Theorem 2
For convenience, we first recall the statement of Theorem 2
Theorem. Assume that the model is trained for T time-steps with stochastic gradient descent. Denoting by He the Hessian 60o,. La, then the final parameters θg, satisfy:
V VejoOE,T
Σ t=0
V (c.0))000000))
Σ t=0
1/ H. V 0jo00..
(15)
Proof. The parameters Oz obey a recurrence relation:
L 0s,t-1 7/- VoLi-1 (0
(16)
which can be solved give
GT
Σ
11 V VoL,
(17)
t=0
then 15) follows immediately by applying Vejo- i.c. computing the Jacobian wrt, e at the origin.
E Proof of Theorem 3
The first step in the proof of Theorem Bss discrete version of Gronwall's Lemma [Gro 19
Lemma 1. Let {ue). {ar] and (Be) be sequences such that 33 and
L-1
Ue S 04 +
Σ 3
(18)
==0
Note that we assume that (18) holds for t 2 and we consider un an initial condition. Then:
T-1
T-1
T-1
ur S oT + 3 II (1+
+
Σ
a, II
3x).
(19)
#=1
==1
kms+1
(ar) non-decreasing & we then get:
T-1
T-1
T-1
S. 5.80
II (1+ 8.) + ar +
Σ
B, II (1+ A=s+1
(20)
s=1
Moreover, defining u. by setting 18) be an equality, shows that 19 is sharp.
Proof. We first observe that we set vo up and build o, by declaring (18) to be an equality, then 2 Ue for any t. This is true by induction as:
U4+1 a++1
8ru Σ ==0
B.v. Ue+1 0.+1
(21)
==0
We thus focus on bounding V141i note that
(U++1 a++1) (v, ar)) = B.(err -ar) + Brar
(22)
13

---

thus
Ut+1 0++1 (1 (1+ (v: ar) ) 3,at
(23)
(1++))) + Be-1)( 04-1) (1 -
10) + Br00s
then (19) follows by induction and letting + 1 = T. Finally, if {ar} non-decreasing in é we can simply replace a, with or obtaining 20 The sharpness follows by considering the sequence {v:} we defined in the proof.
For convenience, we first recall the statement of Theorem 3
Theorem. the setting of Theorem that
2 assume that, for t S T, Oz lies in a bounded region such
sup [VL,(0,s) V2(0,0) - Ce
(24)
2.0ER
and that for E each loss E,)0,)) and its gradient wrt. 0 are A-Lipschitz Then
00 0o.r1l S Os
Σ no(1 exp(2A Σ na)).
(25)
Proof. The idea is to apply Lemma I as we would in the case of standard ODE. Let compare the evolution de, and Ou.::
0.. Iiii
VoC(Oe...
(26)
0,, Onni Σ
1/ VeL(Oo,.. 0);
(27)
==0
which leads to
1-1
00,, 0o.el s Σ
17 VoL(Oe,s
VoL(Ou,s, 0)
(28)
1-1
s Ca
Σ
7 +
ncAl - 0o.el::
==0
we now just need to rephrase this inequality in terms of Lemma
||0,, 0o.1ll S Cs n
Σ
Σ n 10..0 Oo.all
(29)
B
we note that uo as both dynamics start at Oini and that a is non-decreasing in t. We then have that
T
ur Saa +
3, II (1+
Bx)
k=s+1
T-1
<ar(1 +
3,
exp(3x))
k=x+1
T-1
T-1
Sar(1+ exp
3x)
3z)
(30)
s=1
s=1
T-1
T-1
<ar(1+ exp
3k) X exp(
Σ
8x))
#=1
1=1
<ar + exp
2,3))
which is g
14

---

Dier from 0 to the cont interval
tite conf. interrea
0.5
Opti mizer
Raccnn Tracin3))
2.08
3.06
0.2
6.3
3.00
200
450
600
800
1000
200
400
680
800
1000
Step
Step
(b)
Figure 4: For BERT the predictive power of influence scores becomes 0 over time the distance from 0 to the confidence interval of the Pearson's 's r becomes i.e. the confidence interval contains O.
E.1 Sketching modilications needed in the case of optimizers
The proofs of Theorems 2 and were given for SGD. The argument in the case of using an optimizer would more involved. Here we sketch the modifications needed when dealing with optimizers. optimizer is characterized by optimizer state, ee,t, which will also evolve in time. While for SGD we just considered the update rule for Oz.3, in the case optimizers one needs to study joint system of update rules for the parameters and the optimizer state:
0..1 -0,10,11=--100 Th-
- 1V
(31)
Øe,t 0z Pt- G(Volt-- 0s,t,1)))
(32)
In the case of Theorem one would then apply Vejo to the joint system to obtain the update rule. For Theorem 3 one should add additional continuity in and Lipschitz conditions on F and G; one should then check that these are indeed satisfied for common optimizers like Adam or Adafactor.
F
Why do we use the Conjugate Residual method?
Regarding HIF, note that usually the Conjugate Gradient method is used [KL17 for computing inverse Hessian vector products. However, in our experiments the Conjugate Gradient method always yielded time-series of Pearson Correlations R(t)) oscillating around 0, thus without any predictive power. The reason is that this method assumes the Hessian to be positive-definite, which not the case for most Neural Networks. A simple fix to the problem to use the Conjugate Residual method which does require the Hessian be positive definite. We recommend use the Conjugate Residual when applying HIF in Deep Learning; this might look like a minor technical point, but it can avoid reporting that HIF has no predictive power, when instead the issue lies in the numerical method used to compute inverse Hessian vector products.
For more discussion the Conjugate Residual method see its Wikipedia article
G Further empirical results on Fading of Influence scores
G.1 Fading of Inffuence for BERT
In the setting of Figure a) we plot the distance from the confidence interval of the Pearson's 's to O (Figure 4) once the distance becomes 0, the predictive power of influence scores has become null.
In Figure we zoom in Figure (a: BERT). The fading phenomenon was non affected by checkpoint selection: in Figure 6 we consider a later checkpoint and we see the same qualitative behavior. More- over, in Figure 6 we also consider TracIn with 3 checkpoints selected using the advice in [PLKS20] do not improvements and the peak is even slightly lower than for TracIn using checkpoint. Thus, in our further experiments we have used TracIn with a single checkpoint.
In Figure 7 we zoomed in Figure 2 (b: ResNet). In this case the predictive power was never particularly high, e.g. for TracIn quickly peaked at 0.3 but takes more steps than in the case of BERT to reach O. We conjecture that this is in part due to the slower divergence of parameters in
15

---

Hessian
0.8
Tracin
0.6
Optimizer
0.4
R(t)
0.2
0.0
-02
-0.4
-0.6
20
40
60
80
100
Step
Figure 5: For BERT the predictive power of influence scores on the loss shifts degrades over time very quickly.
Hessian
1.00
Tracin
0.75
Optimizer
0.50
TracIn(3)
R(t)
0.25
0.00
-0.25
-0.50
-0.75
200
400
600
800
1000
Step
Figure 6: For BERT the predictive power of influence scores on the loss shifts degrades over time very quickly. Using multiple checkpoints did not improve performance of TracIn.
16

---

Hessian
0.3
Tracin
0.2
R(t)
0.1
0.0
-0.1
200
400
600
800
1000
Step
Figure For ResNet the predictive power of influence scores faded more slowly and was never particularly high.
Fading of Influence over time
Hessian
Correlalitun of Influence and Loss shills
Gradient
0./5
0.50
0.25
0.00
-0.25
-0.50
50
100
150
200 Step
300
350
Figure 8: For BERT the predictive power of influence scores faded more slowly when using SGD. The "Gradient" method is TracIn.
17

---

0.8
0.5
Tracin
Optinizer
Opti mizer
0.4
0.3
0.2
0.0
0.0
253
500
750
1600
1250
1506
1750
2000
253
500
750
1000
1250
1506
1756
2000
Step
Step
(b)
Figure 9: The predictive power of influence scores on the loss shifts degrades over time for T5. (a) Correlation with the ground truth, with the shaded area the corresponding 95% confidence region. (b)
Distance from O to the confidence interval, which measures when the confidence interval covers 0.
Hessian
0.8
Hessian
Tracin
Tracin
0.6
RM
0.4
0.2
0.0
0.0
0.2
..2
380
1000
3500
2000
2500
3000
3500
4000
9
50
100
150
350
Step
(a)
(b)
Figure 10: The predictive power of influence scores on loss shifts degrades over time Vision Transformer (ViT). (a) Full range of re-training steps, (b) zoom-in in the peak and decay phase.
ResNet (compare the x-axis (a) and (b) in Figure T and in part due to the use of SGD. In particular, verified that slower fading also takes place when fine-tunie fine-tuning BERT with SGD (Figure 8)
G.2 Fading of Influence for T5
We consider another NLP pre- trained model fine-tuned SST2, T5. In Figure 9 we illustrate the same effect of fading of influence for the T5 model.
G.3 Fading of Influence for the VIT
In the case of ResNet, in our fading of influence experiments (Fig. 2 (b)) the peak is particularly low. On the same task, we trained VIT with SGD and observe that the peak is higher, similarly to what happens with BERT, see Figure TO We conjecture that this difference is due to the different model architecture. Note that both cases, we observe the fading of influence as predicted by the theory.
H How many steps are needed to correct mis-predictions?
In Figure we plot the average and median number of steps to correct a mis- predictioo as function of E: for any fixed the more effective correction method is, the fewer steps are needed to correct the prediction. For BERT we see that both Proponent-correction and Opponent-tuning result in large decrease in the number of steps to take, compared with the Baseline. For ResNet, we do not plot Opponent-tuning as performed poorly on success-rate. For ResNet the gains of Proponent-correction smaller but consistent varies.
18

---

50-
Prop. moon stop
Prop. mieon step
Baseline
40-
stap Beseline medien stea
epsion
(b)
Figure 11: Average and median steps to correct mis-predictions. (a) BERT, (b) ResNet
Table 1: Pairs of misclassified test examples with their mislabeled training proponents
Test example
Top proponent
False positives
Agreed. This is dumb.
Yeah i no it sucks. By the way, sorry about that stupid edit.
2 = lowa-class battleships == Stupid me... I
didn't even look... sorry about that...
3 It's so silly of me to ask this question but I'm still not getting what you're 're trying to say.
By the way, sorry about that stupid edit.
Sorry for being such an idiot! ;)
4 hahaha this whale is huge and ugly and funky doodle...ol...
A cheeseburger for you! Fatty
5 Cool you can change the content!i love sex!!
Sex in space Sweet. Thanks for the education!
lot of poeple got killed that day
Yes they were Hitler Stated that Slavs were lesser races and should be salves killed Ravenkills97 didn't change the genres on This Moment, all did was put commas in be- tween the genres. U gave me final warning for reason. All because I put commas in between something, like it fucking erases the genre.
7 question regarding Rodriguez's sexual orientation = Would the statement, "I've gone both ways. do as please. am too fucking curious sit here and not try when can. Men are intriguing. So are chicks." be enough to label her as bisexual or LGBT?
False negatives
your mama is a man
grow up Get life
2 (...) It's nice to see you finally admit it with
(...) you't 're nerd living with your mom and you have no friends. You see? (...)
your edit summary of, "what a terrible idiot am". (...)
3 U worse than obamas presidency
Are you known as a sock puppet?
4 don't it want to say sorry get life before i
Then grow up and stop spamming my page. hear your mommy calling. Run along. (...) thats 2 against buddy you lose
ruin yours buthead
5 (...) it's hard for me to not take the kind of
stupidity evident from your edits personally
(...)
6 Your spelling seems to be at about the 3rd grade level, I hope you'te re not having
Okay then. I'll just have to agree to disagree. I won't 't read your article and won't 't give you fellatio. Now get off my page.
19

---

Table 2: Mean influence with variance and metrics for the experiments with the example Table
proponents
mean influence
mean step
median step
retention
True
0.91 (=0.03)
3.0
2.5
98.1%
Permuted
0.50 (=0.21)
5.1
3.5
96.3%
Table
Training hyper- parametrsss for ResNet on CIFAR10
Hyper-parameter
value
Batch-size Epochs Learning rate
128
200
10-1
Learning rate scheduler Optimizer
cosine decay
SGD
Ig-reuularization
10-1
I Selecting mislabeled proponents for correction
For the experiments in Section 6 we use comparatively clean datasets, so the proponents we retrieve and correct mislabeled. In this section use the notoriously noisy Wikipedia Toxicity Subtypes dataset [WTD17] where for many misclassified test examples we can find mislabeled proponents used during training. Table [ooleepreeeetts false positives and negatives (that is, test examples incorrectly predicted to be toxic, resp. non-toxic) with high-scoring proponent which we consider mislabeled. It worth noting that some examples lack context necessary to confidently verify whether they are toxic or not. In total, we collected 14 such pairs, seven for mispredictions of each kind
Table presents the mean and the median number of steps needed to be taken the corrected proponents to achieve change in prediction, as well as retention once there is change. As point of comparison, this is a more challenging baseline than selecting examples for correction randomly. Note that the influence scores for the presented examples range from -0.90 0.92, so some of the permuted proponents still have some quite influence on the test examples they get assigned to. Still, the results confirm that correcting mislabeled top-scorte cop-scoring proponents is the fastest way of achieving change in model predictions, with as few as three steps required on average and with the smallest damage to predictions on other examples (measured with retention).
permute the proponents withing the false positive and false negative sets:
Hyper-parameters
J.1 Training
ResNet was trained on a single V100 with the hyper-parameters in Table 3 training data was augmented using torchvision using the transformations transforms (andomCrop padding=4), and transformi RandomioriizontlFFFpp
'e omitted the most upsetting example from the false negatives.
Table 4: Training
for BERT on SST2
Hyper-parameter
value
Batch-size
128
Steps Learning rate
35000
10-5
Learning rate scheduler
None
Optimizer
Adam
l2 d-regularization
5 x 10
20

---

Table 5: Training hyper- syperr-prameters for T5 on SST2
Hyper-parammter
value
Batch-size
16
Maximal sequence Length
128
Epochs
4
Learning rate Learning rate scheduler
10"
None
Optimizer
Adam
Table 6: Training typer-pasameters for VIT on CIFARIO
value
Batch-size
128
Epochs
300
Peak learning rate
10-3
Learning rate scheduler Optimizer
epoch warm-up and then cosine decay
SGD
The VIT was trained on a single V100 with the hyper-parameters in Table 6 we used the same data augmentation as in the case of ResNet.
BERT was trained on TPUv3 cores using the sppee-paraeeters in Table The best checkpoint
was selected on validation-set accuracy evaluated every 500 steps; it corresponded to 6000 steps of training.
T5 was trained GPU V100 using the hyper-parameters in Table 5
We selected checkpoint based on the validation set accuracy at epoch 4, where the accuracy was 92.9%. Note that T5 is a generative model, the decoder used to generate a sequence that represents the desired class label. When computing influence functions, one uses the cross-entropy loss for sequences.
prediction time the sequence to classify is encoded with the encoder and then
J.2 Correction Experiments
For the correction experiments used ABIF SZTS22 because of their computational efficiency as IF scores need to be computed against the whole training set; we used 32 projectors obtained with 64 Arnoldi iterations. The Arnoldi iteration take up to 2 hours; scoring the data takes few minutes. We used a learning rate of 10-3 and for BERT we loaded the state of the Adam optimizer from the selected checkpoint.
For BERT we selected the checkpoint at 6000 steps which was the best on validation performance. For ResNet selected the checkpoint after 10 epochs at which about 150 test points are incorrectly classified.
K Illustrating TracIn's modeling error
In Theorem we have derived the correct formula for the Jacobian VejoOe, and we expect TracIn (Eq. 2 to introduce modeling the ground truth 8. and the first order expansion fgr " Vei
in calculating V= We compute the difference between
when the Jacobian is computed with TracIn (Eq.2) () or the corrected formula (Eq, (6. We can then compute the relative excess error introduced by TracIn for different values of e. Figure 12 we observe that except for the extremely small perturbation
10-1 TracIn does introduce additional relative error in modeling the change of parameters when retraining BERT with SGD. The setup of this experiment the same as those of our parameter divergence experiments.
21

---

Additional error of Tracin (%)
17.5
E=1e:04
E=1e-03
15.0
le:02
E le-00
12.5
10.0
7.5
5.0
2.5
0.0
0
50
100
150 Step
200
250
300
Figure 12: Illustrating the additional modeling error introduced by TracIn for different values of the perturbation The error is the ratio between the eπror of TracIn (Eq. 2 and the error of the exact formula (Eq. ( when retraining BERT on perturbed loss with SGD.
L Broader Impact Statement
We believe that our work can benefit model developers who want to debug and correct mis pis-predsstions made by models. Our suggested error-correction procedure is much less compute intensive than previous ones using IF as it does not require model retraining. However, this debugging procedure could introduce new errors in the systems, for example if on specific problem IF at predicting loss-shifts or examples retrieved by IF lead to over-fitting against spurious artifacts present in such examples. Since this could have negative impact the of such systems, mitigation strategies should be put in place regarding the trade-offs between the success rate of fixing specific mis-predictions and the overall retention of the system performance.
effective
22


Reviews:
[{"review_id": "cpzOiHYcR3", "reviewer": "Reviewer_7Fho", "summary": "This paper verifies assumptions of Influence Functions that show discrepancies in theory and empirical results, convexity, numeric stability, training trajectory, and parameter divergence. Although the other assumptions are addressable, parameter divergence is not. Hence, the paper proposed a solution to it and also showed empirical results.", "questions": "Do the authors have other models to show the fading effect? As BERT’s case is unclear, only one empirical case of ResNet shows it.", "limitations": "Yes.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-\tBased on their observation of the issue, they proposed a simple solution to it.\n-\tTo the reviewer, this paper reads as an instructional paper, which can contribute to the community.\n-\tIt verified the applicability of the IF assumption one by one on convexity, numeric stability, training trajectory, and parameter divergence. \t\n", "weaknesses": "-\tAside from its technical contribution, there’s room where its writing can be improved. In particular, the introduction can be better structured, and there are multiple repetitive places that can be concise. \n-\tIn Fig 2 (1) BERT case, although the authors claimed that the predictive power degrades monotonically over time, the “fading” effect is actually not clear. There’s only one time of peak at an early step, and there's no decreasing trend in the later steps. There is no fading effect either in the Appendix (Fig. 5.)\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reply to Weaknesses bullet point 2 & Questions**\n\n*Regarding the fading effect for BERT results*: Multiple runs (n=9) were used to build confidence intervals; the goal is to show that eventually 0 falls inside the confidence interval. For the methods Hessian, TracIn, TracIn(3) we observe that 0 falls in the confidence interval.\nFor Optimizer this was not actually the case and there was sometimes a small distance of up to ~0.1 to ~0.2 from the confidence interval; however the mean of the time series is oscillating quickly up and down around 0 so we conjectured that this effect was due to the high variance of computing influence scores with the Optimizer adjustment. We have done additional runs (for a total of n=25) and we find that then also for Optimizer 0 is eventually covered by the confidence interval. Please see Figure 1 in the rebuttal \nplots .pdf. In Figure 2(a) we plot the confidence interval for Optimizer with those 25 runs and it covers 0. As our proof is formulated for SGD, we additionally re-did the BERT experiments when using SGD instead of Adam: see Figure 2(b); the fading effect there is particularly clear.\nWe hope that these plots illustrate the effect convincingly for BERT and we will include them in the final manuscript. We will add another NLP pretrained model  and the ViT for the Vision Task for the camera-ready version if accepted.\n"}, {"review_id": "Fheh2tXrIs", "reviewer": "Reviewer_RiS8", "summary": "This paper studies the limitations of Influence Functions (IF) based on assumptions about convexity, numerical stability, training trajectory and parameter divergence. The authors discuss each of those assumptions in detail and propose ideas to address the shortcoming of those limitations. The authors describe solutions for 4 assumptions made in the paper and discuss the limitations of the fifth assumption, parameter divergence, which is harder to address.\nApart from that the authors also illustrate the theory of parameter divergence on the example of NLP and vision classifiers. They confirm theoretical findings of fading influence over time when retraining the models on vision and test classifiers.\nThe authors also show how to perform error correction based on proponent-correction and opponent-tuning using a few fine tuning steps. They show that their approach outperforms baseline re-training procedures.\n", "questions": "+ Section 6: Proponents-correction: when the set B is identified and relabeled, is it done for truly mislabeled examples ? How were those mislabeled examples sourced ?  For experiments with SST and ResNet how were set Bs curated ?\n+ Section 6: Which formulation of IF (TracIn or Influence Functions) was used in the experiments ? \n+ Section 5.2: Is R(t) pearson correlation between losses before and after re-training averaged for the same batches? Does the `Step` indicate different checkpoint on Figure 2 ?\n", "limitations": "The authors discuss several limitations of their work and propose ideas to address those limitations in the future.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The paper is well written, especially the introduction, problem statement and the assumptions are well framed.\n+ The related work is well cited and based on the studies in the recent papers, this paper proposes to address the limitations highlighted in the literature such as the LSOR problem for influential examples.\n+ The paper studies 5 important assumptions made by IF and proposes solutions that help to alleviate the limitations posed by those assumptions. It also discusses and proposes a solution for the temporal dependency which is not accounted for in the original TracIn paper.\n\n", "weaknesses": "+ The main contributions of the paper seem to be around studying 5 different assumptions and proposing solutions for them. The main contribution is for assumptions 3 - 5 where the authors show that IF can predict \\theta_{\\epsilion, t} only for a limited period of time and propose incorporating temporal term into the tracin's formulation. These are important findings, however, it is not very clear whether it is a large enough contribution and whether it is completely novel since similar limitations of Tracin were also discussed in the Kelvin Guu, et.al.\n+ It looks like all the proofs corresponding to theorems mentioned in the main paper are in the appendix. Since it is not required for reviewers to read appendices it might be good to consider moving some of the important proofs or parts of them into the main paper.\n+ Section 4.2 might be difficult to follow for someone if they do not already know about Arnoldi. There is a brief description that Arnolodi approximates P1 using subspaces spanned by eigenvectors corresponding to top-k eigenvalues but I think that overall intuition of Arnoldi is not clear if someone is not familiar with that work.\n+ Overall I think that Section 4.2 might be a bit difficult to follow (e.g. how is Theorem 1 applied to restricted variation).\n+ It feels that it is somewhat difficult to follow how we arrive from Eq 7 to Eq 8. Perhaps adding more explanation in section 4.4 would be good.\n\n**Minor comments**\n\n+ The definition of C^k function on line 179 is a bit unclear. C gets used again in line 234 (C\\epsilion) which makes it a bit confusing in terms of the readability of the paper.\n+ Figure 3: X-axis is not annotated\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comparison to SimInfluence**:  A key difference is that, while Guu et al. (Simfluence) discuss the limitation of additivity in TracIn from a modeling perspective, we point at the limitation from its theoretical derivation. Our *first step* is showing that the original derivation is incomplete: starting from the same modeling assumptions of TracIn we derive the correct result that takes the time order into account. This is in contrast with Guu et al. who do not challenge the original derivation of TracIn (e.g. the role of the time order) but propose to extend TracIn with multiplicative terms. The *second step* is then showing that even if one were to use the corrected formula, one would have to face the issue of parameter divergence – something that Guu et al. do not discuss. We show that this issue is common to all the methods that are formulated with a first order expansion in $\\varepsilon$, e.g. also the Influence Functions of Cook and Weisberg or the more recent ABIF.  In sum, while it is true that both Guu et al. and we point out the additivity limitation, this is probably the only similarity: everything else, including the motivation, the theoretical framing and the experiments is very different.\n\n**Question regarding Proponents correction and mislabeled examples**: We use comparatively clean datasets, so the proponents are not mislabeled. However, our primary goal in the correction experiments is to verify that intervening on influential examples results in a faster correction than on other classes of examples. Also, please note that this simplification applies to proponents (whose likely correct label is changed) but not to opponents (which are not changed). Finally, we also did an experiment with a notoriously noisy Wikipedia Toxicity Subtypes dataset where we indeed identified a mislabeled proponent for every test error. We will include these results in the appendix. \n\n**Question regarding IF method used in Section 6**: ABIF (Arnoldi-based influence functions) for scalability reasons. So it is basically a scalable and numerically stable version of Influence Functions.\n\n**Question regarding Section 5.2 and $R(t)$**: For each pair $(i, j)$ ($i$ in test and $j$ in train) we do re-training to compute the loss-shifts at each time step (i.e. checkpoint, index $t$) obtaining a tensor $A_{i,j,t}$; for each $i$, $j$ and $t$.\nInfluence functions at the starting checkpoint give a score $s_{i,j}$; so we compute the Pearson correlation on the $(i,j)$ pairs between $A_{i,j,t}$ and $s_{i,j}$ to obtain R_t. For each experiment run the pairs $(i,j)$ are always fixed at the beginning (as we need to do a re-training run for each $j$); however across different runs we use different $(i,j)$ to build the confidence estimates for $R_t$.\nStep indicates thus a different checkpoint, but the checkpoints correspond to each step; since the sequence of models fits on the CPU RAM, one just needs to offload the parameters at each step to the CPU.\n\n"}, {"review_id": "hRdlB2hHA0", "reviewer": "Reviewer_SKsY", "summary": "Influence functions allow one to measure the effect of up-weighting a training sample on the test loss (or functions of the model parameters) of a test example. This paper studies several of the assumptions needed for hessian-based influence functions to produce valid leave-out-one estimates of the effect on a test sample's loss. This assumptions include: convexity, stability of the hessian, and additivity of training trajectory. The paper shows that some of these assumptions are not as problematic, in practice, as previously assumed. They show that several of these assumptions can indeed be satisfied with changes to the formulation; for example, if the hessian becomes degenerate, one can approximate the hessian via arnoldi iteration. It turns out that the most problematic assumption results from a divergence between the initial set of parameters, and another obtained by re-training or even fine-tuning for longer time steps. The paper then bounds this parameter divergence using a discrete version of gronwall's lemma, and gives a set of takeaways for how to evaluate influence functions: the takeaway from this work is that influence scores only predict effect of a training sample over a small number of training steps. In the final portion of the paper, they evaluate how to use influence functions to correct errors.", "questions": "Some clarification questions: \n\n- For error correction, if I have k examples that I want to correct, then am I correcting each example independently or the taking gradient steps on all k examples at the same time. If it is independent, then isn't this too onerous? \n- One use of these influence scores is mislabelled training samples, how does assumption 5 affect that? When in training should the self-influence score be a useful metric for mislabeled label correction?\n- This is minor question, going from $\\theta_\\epsilon \\approx \\theta_0 + \\epsilon^\\top\\nabla \\theta_\\epsilon$ to equation 1, where did the second $\\nabla_\\theta$ come from? Is this the chain rule? I am familiar with other ways to derive influence functions.", "limitations": "The authors discuss limitations.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "Overall, I really enjoyed reading this paper, and found the breakdown of the assumptions really comprehensive.\n\n**Quality/Clarity**\\\nI found the paper to be of very high quality. Each assumption is stated clearly and cleanly discussed. For example, I found the resolution of assumption #1 to be interesting. Here the authors show that for hessian-based influence functions one does not need to assume strict convexity but we can relax it to assume that the final gradient steps do not change the hessian by too much. While this might seem like a simple change, it helps provide clarity to this literature. The bigger takeway from this work is that influence scores only predict effect of a training sample over a small number of training steps. This finding is important since it requires rethinking how these approaches are currently used. Overall, this paper sets out key assumptions of influence functions, discusses how to remedy violations to these assumptions, and presents experiments to corroborate the theory presented.\n\n**Significance**\\\nOne important takeaway is that, for deep models, it now makes sense that influence scores obtained via retraining have poor correlation with leave-one-out retraining scores. Secondly, another important takeaway is that one can correct errors simply be reweighting or fine-tuning on opponents or proponents. The findings from the paper should be important for practitioners and others using influence functions as part of their debugging toolbox.\n\n**Originality**\\\nThis paper mostly considers a question that was also studied by Bae et. al. (Neurips 2022), but does so in a different way and using mostly new techniques. I found the use of the discrete version of gronwall's lemma to be quite nice. I am not sure if this is new, but taken together, this work advances the understanding of influence functions quite substantially. \n", "weaknesses": "The main weakness I have with this work are minor and listed below: \n\n**Relation with Bae et. al. from Neurips 2022**: The main related work to this one is this paper. However, this work does not do enough to sufficiently contrast their analyses with theirs. I am mostly familiar with this previous work, and understand that they address exactly the same question as well, but arrive at a different conclusion. Their finding is that hessian-based influence functions approximate what they term the \"proximal bregman response function\", and that the ability of this quantity to match a sample loo estimate is mostly affected by the linearization error (i.e., due to the taylor expansion), and 2) hessian approximation. This work seems to not agree with theirs, i.e., you suggest that hessian approximation be handled with arnoldi, and that the key discrepancy comes from parameter divergence. It would be great if the authors could spend more time in the draft to contrast this work against that one. I think this is an important issue to address in the paper. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comparison to Bae et. al**:  Thank you for the suggestion, we will expand the discussion of Bae et al. Some comparison points:\n\n* Bae: Only Hessian-based influence functions; Us: we discuss Hessian and Gradient-based influence functions covering e.g., also TracIn.\n* Bae: convexity assumption is crucial (they add a regularization term); Us: we can drop convexity. Note that dropping convexity assumption implies one needs to use another solver (Conjugate Residual instead of Conjugate gradient).\nUnder the convexity assumption there is agreement with their findings in the case of Hessian-based influence functions. However, advocates of the TracIn method might argue that they trace the training dynamics correctly while the Hessian-based IF does not and so their method would not be affected by the linearization error. However, we show that parameter divergence puts a limitation on TracIn too.\n\n**Question about $k$-examples for error correction**: We looked at $k=1$ as it matches the case of leave-on-out eval. But the method can be applied also on k examples by retrieving proponents of each test example in order to build $B$. In Figure 8 (Appendix) the median steps for correction are not so big, so for $k$ not too large doing one example at a time might not be that onerous. That being said, we agree that a proper understanding of the utility and limitations of IF for error correction deserves a deeper analysis. Given that the current submission focuses on the theoretical aspects, we leave such an analysis for future work. \n\n**Question regarding usage of self-influence scores**: To find outliers with self-influence, assumption 5 might not be so limiting. What we have seen in practice is that outliers tend to have big gradients so they get high self-influence scores. If the gradients stay big during training because of mislabeling then it does not really matter at which checkpoint they get measured. 5 affects the interaction between a test point and its proponents and opponents which is a finer quantity to measure.\n\n**Question regarding the second $\\nabla$**: One $\\nabla$ is with respect to the parameters $\\theta$; the second $\\nabla$ is with respect to the variation parameter $\\varepsilon$; to get influence we need to map the loss to gradient space (hence $\\nabla_\\theta$) and then take the derivative with respect to $\\varepsilon$ ($\\nabla_\\varepsilon$) which we denoted as a $\\nabla^2_{\\varepsilon,\\theta}$. We will clarify the notation in the final version if the paper is accepted..\n\n"}, {"review_id": "oSpwOvLJ6V", "reviewer": "Reviewer_qity", "summary": "This paper reexamines the assumptions used in the deduction of influence function (IF) methods in order to explain the failure of IF in predicting leave-some-out-retrain performance. The authors find that all five assumptions used in the previous deduction will be violated to different degrees in practice and propose a combination of HIF and Arnoldi-based methods to overcome these violations. They find that four of the assumptions can actually be bypassed or fixed, however, the remaining challenge named *parameter divergence* seems inherent. They show that the predictive ability of IF will gradually fade over training steps due to this phenomenon and show that accounting for this effect, it is better to interpret IFs as proxies for the effect of a few fine-tuning steps.", "questions": "The reviewer is interested in the following questions.\n * Why does the correlation between IFs and performances seem to increase before decreasing? \n * How would the authors suggest modifying the current IF estimation methods based on their observations?\n * What is the author's explanation of the rapidly increasing phase 1 in figure 1?", "limitations": "The authors have adequately addressed the limitations and the reviewer has not noticed any potential negative societal impact.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* **Originality.** This paper shows an in-depth analysis of the assumptions on which IFs are based. Their analysis combines theoretical thinking well with experimental observations.\n* **Clarity.** The results are presented in a clean way and are a pleasure to read.\n* **Significance.** Although the reviewer is not an expert in the field of influence functions, the question this paper targets seems important and the paper sheds light on the real obstacles to solving this problem, as well as proposes possible ways to solve it. ", "weaknesses": "The reviewer thinks the paper may improve in the following aspects.\n\n* The authors argue that (1) using Arnoldi-based methods helps boost the accuracy of HIF, and (2) accounting for training trajectories can improve the estimation for IFs. The theoretical deduction of these propositions is very reasonable but it would be better to include ablation experiments or cite relevant literature to showcase the phenomenons.\n\n* The authors observe that IF methods perform poorly on Resnet but do not have any explanation for this phenomenon. It would be beneficial if the authors can investigate the reason behind the difference of IF methods performance on NLP and CV tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weaknesses bullet point #1**:  For (1) we can point the reader to two relevant references. First, Schioppa et al. who compare  ABIF with exact Hessian on retrieving mislabeled examples (Fig. 2). Second, Fisher et al. (``Influence Diagnostics under self-concordance’’; Fig. 3 and the discussion in Sec 5.2) where they find that  ABIF is the best solver on the QA task and is comparable to the other solvers on the text-completion task. (2) is a natural research direction; at the moment Theorem 2 gives an exact formula but making it practical requires further ideas. We will include an ablation showing the extra error introduced by TracIn vs the exact formula in Theorem 2 in tracking parameter changes, if the paper gets accepted.\n\n**Weankesses bullet point #2**: We focused on the limitations of the methods, in particular the fading of predictive power. We conjecture that the low correlation is due to the architecture. Indeed, using a ViT for the CV task leads to a peak correlation similar to that of the NLP task. We will include the ViT result in the final version, if accepted. We leave an explanation of the phenomenon, possibly because of the different loss landscape geometry between the ResNet and ViT, as a question for future work. \n\n**Question regarding correlation increasing and then decreasing**:  When changing the loss to the perturbed one there is a phase of adjustment in the network. Using a dynamical analogy, the momentum the system had built on the old loss needs to change for the new loss and this introduces a lag to reach the peak performance.\n\n**Question regarding how authors would suggest modifying IF estimation**:  To improve estimation quality: start from formula (6) which traces the dynamics exactly; however (6) is not practical so techniques need to be developed to make it efficient. To manage expectations and evaluate methods: proceed as in the fading of influence experiments identifying the time window in which correlation is good enough.\n\n**Question about the rapidly increasing phase in Figure 1**: This is honestly a hard question. Given the network's complexity, we think that the $A$ in the (theoretical) upper bound in 4.4 is quite large; for the $A$ that comes (empirically) in the lower bound we think that there are two regimes: 1) early regime were the the network tries to adapt quickly to the perturbed loss; 2) asymptotic regime in which a lower value of $A$ is needed to adjust on the perturbed loss. As BERT (a) uses the Adam optimizer, in that case it also seems the phase is very rapid because of the quickness of the optimizer.\n\n\n"}]

Author rebuttal:
 We thank the reviewers for all the comments and suggestions and the positive feedback! Concerning readability (e.g., moving some proofs from the Appendix to the main body, explaining Arnoldi, explaining $C^k$-differentiability, Theorem 1 and Eq. (7-8) in more detail and improving the plot presentation), we will incorporate the suggestions in the final version, if the paper is accepted.


Meta-review: Influence functions are used to measure the influence of a training sample on the decision on a test sample. The paper studies in a didactic way the assumptions considered for producing via influence functions,  approximations of leave one out estimates of the effect of removing a training sample. The study covers hessian based IF and gradient based IF such as traceIn.

These assumption are convexity, stability of the hessian, and additivity of training trajectory and parameter divergence. While most these assumptions can be problematic , they can be adressed , but  the study hints that the parameter divergence is the most problematic as the influence fades over iterations and can be only determined for a small number of  training iterations. Authors showcase then how influential points can be used in fine-tuning to improve the accuracy of models in NLP and computer vision tasks. 

All reviewers agreed on the relevance of the work and clarified few minor issues in the paper, please incorporate the reviewers feedback and explicit the comparison to Bae et al in the related work and discuss it, Accept
Decision: Accept (spotlight)