If Influence Functions are the Answer, Then What is the Question?
Authors: Juhan Bae
Nathan Hoyen Ng
Alston Lo
Marzyeh Ghassemi
Roger Baker Grosse
Venue: NeurIPS 2022 Accept (2022)
Keywords: Influence Function
Interpretability

Abstract: Influence functions efficiently estimate the effect of removing a single training data point on a model's learned parameters. While influence estimates align well with leave-one-out retraining for linear models, recent works have shown this alignment is often poor in neural networks. In this work, we investigate the specific factors that cause this discrepancy by decomposing it into five separate terms. We study the contributions of each term on a variety of architectures and datasets and how they vary with factors such as network width and training time. While practical influence function estimates may be a poor match to leave-one-out retraining for nonlinear networks, we show that they are often a good approximation to a different object we term the proximal Bregman response function (PBRF). Since the PBRF can still be used to answer many of the questions motivating influence functions, such as identifying influential or mislabeled examples, our results suggest that current algorithms for influence function estimation give more informative results than previous error analyses would suggest.

If Influence Functions are the Answer, Then What is the Question?
Juhan Bae!.2, Nathan Ng..2.3, Alston Lo Marzyeh Ghassemi Roger Grosse
University of Toronto, 2 Vector Institute, Massaahussets Institute of Technology (jbae, mng, rgrosse)@cs. toronto... edu utoronto. ca. mghassem@mit. edu
Abstract
Influence functions efficiently estimate the effect of removing a single training data point model's learned parameters. While influence estimates align well with leave-one-our retraining for linear models, recent works have shown this alignment is often poor in neural networks. In this work, investigate the specific factors that cause this discrepancy by decomposing it into five separate terms. We study the contributions of each ferm variety of architectures and datasets and how they vary with factors such as network width and training time. While practical influence function estimates may be poor match to leave-one-out retraining for nonlinear networks, we show that they are often a good approximation to different object we term the proximal Bregman response function (PBRF). Since the PBRF can still be used to answer many of the questions motivating influence functions such as identifying influential or mislabeled examples, our results suggest that current algorithms for influence function estimation give more informative results than previous error analyses would suggest.
Introduction
The influence function (Hampel 1974 Cook 1979
is a classic technique from robust statistics that estimates the effect of deleting single data example (or group of data examples) from training dataset. Formally, given a neural network with learned parameters 0* trained on dataset D, we are interested in the parameters 0" learned by training on a dataset D - {z} constructed by deleting a single training example from D. By taking the second-order Taylor approximation to the cost function around 0", influence functions approximate the parameters 0" without the computationally
prohibitive cost of retraining the model. Since Koh and Liang [017] first deployed influence functions in machine learning, influence functions have been used to solve various tasks such as explaining model's 's predictions (Koh and Liang, 2017 [Han et al. 2020), relabelling harmful training examples [Kong al. 2021], carrying out data poisoning attacks Kohh et al. 2022), increasing fairness in models' predictions [Brunet et al. 2019 Schulam and Saria 2019 and learning data augmentation techniques [Lee et 2020
When the training objective strongly convex (e.g., as in logistic regression with La regularization), influence functions are expected to align well with leave-one-out (LOO) or leave-k-out retraining [Khh] and Liang, 2017 Koh et al. 2019, Izzo et al. 202T]. However, [asu et al.) (2020a) showed that influence functions in neural networks often do not accurately predict the effect of retraining the model and concluded that influence estimates are often "fragile" and "erroneous". Because of the poor match between influence estimates and LOO retraining, influence function methods are often evaluated with alternative metrics such as the detection rate of maliciously corrupted examples using influence scores [Khanna al. 2019 Koh and Liang 2017 Schioppa et al. 2021 K and Søgaard
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

IF LOO (LR)
IF LOO (MLP)
IF PBRF (MLP)
Recrein loss difference
Retrain loss difference
PERF loss difference
Pearson
Pearson
Spearman 0.987
Spearman 0.0891
Spearman 0.916
Influence Loss difference
Influence Loss difference
Influence Loss difference
Figure 1: Comparison of test loss differences computed by influence function (IF), leave-one-out (LOO) retraining, and proximal Bregman response function (PBRF). Each point corresponds to the individual effect removing one training example, Influence estimates align well with true retraining for (left) logistic regression (LR) but poorly for (middle) multilayer perceptrons (MLP). While influence functions neural networks do not accurately predict the effect of retraining the model, they are still a good approximation to (right) the PBRE
2021] However, these indirect signals make it difficult to develop algorithmic improvements to influence function estimation. If one is interested in improving certain aspects of influence function estimation, such as the linear system solver, it would be preferable to have a well-defined quantity that influence function estimators are approximating that algorithmic choices could be directly evaluated based on the accuracy of their estimates.
In this work, investigate the souree of the discrepancy between influence functions and LOO retraining in neural networks. We decompose the discrepancy into five components: (1) the difference between cold-start and warm-start response functions (a concept elaborated below), (2) implicit proximity regularizer, (3) influence estimation on non-converged parameters, (4) linearization, and (5) approximate solution of linear system. This decomposition errors caused by approximations and assumptions made in applying influence functions to neural networks. We empirically evaluate the contributions of each component on binary classification, regression, image reconstruction, image classification, and language modeling tasks and show that, across all tasks, components (1-3) are most responsible for the discrepancy between influence functions and LOO retraining. We further investigate how the contribution of each component changes response to the change network width and depth, weight decay, training time, damping. and the number of data points being removed.
chosen capture all gaps and
Moreover, we show that while influence functions for neural networks often poor match to LOO retraining, they are a much better match to what we term the proximal Bregman response function (PBRF). Intuitively, the PBRF approximates the effect of removing data point while trying keep the predictions consistent with those of the (partially) trained model. From this perspective, we reframe misalignment components (1-3) simply reflecting the difference between LOO retraining and the PBRF The gap between the influence function estimate and the PRBF only comes from sources (4) and (5), which we found empirically be least an order of magnitude smaller for most neural networks. As result, on a wide variety of tasks, influence functions closely align with the PBRF while failing approximate the effect of retraining the model, as shown Figure !I
The PBRF can be used for many of the same use cases that have motivated influence functions, such finding influential or mislabeled examples (Schioppa al. 2021] and carrying out data poisoning attacks [Koh and Liang] 2017 Koh et al.) 2022]. and can therefore be considered an alternative LOO retraining
gold standard for evaluating influence functions. Hence, we conclude that influence functions applied to neural networks are not inherently "fragile" as is often believed [Basu al. 2020a]. but instead can be seen giving accurate answers different question than is normally assumed.
2 Related Work
Instance-based interpretability methods are class of techniques that explain a model's predictions in terms of the examples on which the model was trained. Methods of this type include TracIn (Pruthi
al. 2020
Representer Point Selection (Yeh al. 2018), Grad-Cos and Grad-Dot [Charpiat
et al.
2019
Hanawa et al. 2021), MMD-critic [im al. 2016), unconditional counterfactual

---

explanations Waachter et al. 2018 and of central focus in this paper, influence functions. Since its adoption machine learning by Koh and Liang 2017]. multiple extensions and improvements upon influence functions have also been proposed, such as variants that use Fisher kernels [Khanna et al. 2019), higher-order approximations [Basu et al. 2020b), tricks for faster and scalable inference [Guo et al. 2021 Schioppa et al. [02T], group influence formulations [Koh et al., 2019 [asu et al.) 2020b], and relative local weighting [Barshan et al. 2020J. However, many of these methods rely on the same strong assumptions made in the original in influence function derivation that the objective needs to be strongly convex and influence functions must be computed on the optimal parameters.
In general, influence functions are assumed to approximate the effects of leave-one-out (LOO) retraining from scratch, the parameters of the network that are trained without data point of interest. Hence, measuring the quality of influence functions is often performed by analyzing the correlation between LOQ retraining and influence function estimations (Koh and Liang, 2017, Basu et al.. 2020a b. Yang and Chaudhuri] [022]. However, recent empirical analyses have demonstrated the fragility of influence functions and fundamental misalignment between their assumed and actual effects [Basu et al. 2020a| Ghorbani et al., 2019 K and Søgaard 2021] For example, [asuu et al. 2020a] argued that the accuracy of influence functions in deep networks is highly sensitive to network width and depth, weight decay strength, inverse-Hessian vector product estimation methodology, and test query point by measuring the alignment between influence functions and LOO retraining. Because of the inherent misalignment between influence estimations and LOO retraining in neural networks, many works often evaluate the accuracy of the influence functions on an alternative metric, such as the recovery rate of maliciously mislabelled or poisoned data using influence functions (Khanna et al. 2019 Koh and Liang 2017 Schioppa et al., 2021 K and Søgaard 2021]. In this work, instead of interpreting the misalignment between influence functions and LOO retraining failure, claim that it simply reflects that influence functions answer a different question than typically assumed.
3 Background
Consider a prediction task from an input space to a target space where we are given a finite training dataset Dusin = {xx(', ) Given data point (x, t), let y = f(0,x) x) be the prediction of the network parameterized by 0 e Rd and L(y, t) be the loss (e.g.. squared error or cross-cntropy). We aim to solve the following optimization problem:
0" = arg min ((0) = arg min 0E
BERd
N
where J(-) is the cost function. the regularization (e.g., L2 regularization) is imposed in the cost function, fold the regularization terms into the loss function. We summarize the notation used in this paper in Appendix A
3.1 Downweighting a Training Example
The training objective in Eqn. [ aims to find the parameters that minimize the average loss on all training examples. Herein, we are interested in studying the change in optimal model parameters when particular training example generally, when the data point z is downweighted by an amount e R. Formally, this corresponds to minimizing the following downweighted objective:
(x, t)) E Draai is removed from the training dataset, or more
arg min @ (0, = arg min min ((0)) x). t)c.
(2)
Z,E
0E34
0E3d
When !/N, the downweighted objective reduces to the cost over the dataset with the example removed, up to a constant factor. To see how the optimum of the downweighted objective responds to changes in the downweighting factor e, we define the response function r = IR + Rd by:
arg min (0, deRd
where we assume that the downweighted objective is strongly convex and hence the solution to the downweighted objective is unique given some factor e. Under these assumptions, note that rr (0) 0* and the response function is differentiable at by the Implicit Function Theorem Krantz
3

---

and Parks 2002, Griewank and Walther 2008). Influence functions approximate the response function
by performing fisst-oder Taylor expansion around 6g
dr
r' "- (c) (Co) +
(c- =o)
= 0- (V x))
(4)
de
==
We refers readers to Van der Vaart (2000) and Appendix for a detailed derivation. The optimal parameters trained without then be approximated by plugging in 1/N to Equ. 4
Influence functions can further approximate the loss of a particular test point Ziest Xiest: bga) when
a data point z is eliminated from the training set using the chain rule (Koh and Liang 2017
C(f Iin (//N), Xtest), trest)
T dr de (
= L(f(0", (ee()) tiest) +
V VoL(f(O", :Kest), trea)
N
(5)
=O Xtea), (ees) + VoL(J(O", Xues), hes))
N
3.2 Influence Function Estimation in Neural Networks
Influence functions face two main challenges when deployed on neural networks. First, the influence estimation (shown in Eqn. 4 requires computing an inverse Hessian-vector product (iHVP). Unfor- tunately, storing and inverting the Hessian requires (dd) operations and is infeasible to compute for modern neural networks. Instead, Koh and Liang [2017] tractably approximate the iHVP using truncated non-linear conjugate gradient (CG) [Martens et al.) 2010 or the LiSSA algorithm (Agarwal et al. 2016). Both approaches avoid explicit computation of the Hessian inverse (see Appendix for for details) and only require ((N)) operations approximate the influence function.
Second, the derivation of influence functions assumes a strongly convex objective, which is often not satisfied for neural networks. The Hessian may be singular, especially when the parameters have not fully converged, due to non-positive cigenvalues. To enforce positive-definitenss of the Hessian, Koh and Liang [2017] add a damping term in the iHVP. Teso et al., [2021] further approximate the Hessian with the Fisher information matrix (which is equivalent to the Gauss-Newton Hessian [Martens 2014 for commonly used loss functions such as cross-entropy) as follows:
r" dazddamp.iii (c) Rz 0* (Jyo Hy
-VVL((((f(" x),t)c
where Jyo- is the parameter-output Jacobian and Hy- is the Hessian of the cost with respect the network outputs both evaluated the optimal parameters 0'. Here, G -yo- H -yo- is the Gauss-Newton Hessian (GNH) and À 0 is damping term to ensure the invertibility of GNH. Unlike the Hessian, the GNH is guaranteed to be positive semidefinite as long as the loss function is convex function of the network outputs [Martens et al. 2010
4 Understanding the Discrepancy between Influence Function and LOO Retraining in Neural Networks
In this section, we investigate several factors responsible for the misalignment between influence functions and LOO retraining. Specifically, we decompose the misalignment into five separate terms: (I) the warm-start gap, (2) the damping gap. (3) the non-convergence gap, (4) the linearization error, and (5) the solver error. This decomposition captures all approximations and assumption violations when deploying influence functions in neural networks. By summing the parameter (or outputs) differences introduced by each term we can bound the parameter (or outputs) difference between LOO retraining and influence estimates. We use the term "gap" rather than "error" for the first three terms to emphasize that they reflect differences between solutions to different influence-related questions, rather than actual errors.
For all models we investigate, we find that the first three sources dominate the misalignment, indicating that the misalignment reflects not algorithmic errors but rather the fact that influence function estimators are answering a different question from what is normally assumed, All proximal objectives are summarized in Table T and we provide the derivations in Appendix B

---

4.1 Warm-Start Gap: Non-Strongly Convex Training Objective
By taking first-order Taylor approximation of the re- sponse function at co = 0 (Eqn. 4 influence functions approximate the effect of removing a data point z at local neighborhood of the optimum 0*. Hence, influence approximation has a more natural connection to the re- training scheme that initializes the network at the current optimum 0* (warm-start retraining) than the scheme that initializes the network randomly (cold-start retraining), The warm-start optimum is equivalent to the cold-start optimum when the objective is strongly convex (where the solution to the response function is unique), making the influence estimation close to the LOO retraining logistic regression with L2 regularization.
warm-start response function
cold-start response function
However, the equivalence between warm-start and cold- start optima is not typically guaranteed in neural net- works [Vicol et al.] 2022a]. Particularly, in the over- parametrized regime (N d), neural networks exhibit multiple global optima, and their converged solutions de- pend highly on the specifies of the optimization dynam-
initialization
Figure 2: Cold-start (initialized from black star) and warm-start (initialized from blue star) response functions for quadratic cost function. Each contour represents the cost function at some e. Because gradient de- seent converges minimum- solution, the warm-start and cold-start optima not equivalent.
ics (Lee et al., 2019 Arora et al.) 2019, Bartlett et al.) 2020 Amari et al. 2020]. For quadratic cost functions,
gradient descent with initialization 0º converges to the optimum that achieves the minimum L2 distance from 6 of the converged parameters being dependent on the initialization hinders influence functions from accurately predicting the effect of retraining the model from scratch as shown Figure We denote the discrepancy between cold-start and warm-start optima as warm-start gap.
[Hastie et al., 2022 This phenomenon
4.2 Proximity Gap: Addition of Damping Term in iHVP
In practical settings, we often impose damping term (Eqn. (6) in influence approximations to ensure that the cost Hessian is positive-definite and hence invertible. As adding damping term in influence estimations equivalent adding L2 regularization to the cost function [Martens et al.) 22100, when damping is used, influence functions can be seen as linearizing the following proximal response function at 6o 0:
damp (c) = arg min 2 (-x(0,c)
111
OER"
See Appendix B.2 for the derivation. Note that A > 0 a damping strength and our use of "proximal" is based on the notion of proximal equilibria [Faniaa and Ozdaglar 2020). Intuitively, the proximal objective in Eqn. not only minimizes the downweighted objective but also encourages the parameters to stay close to the optimal parameters at co = 0. Hence, when the damping term is used in the iHVP, influence functions aim at approximating the warm-start retraining scheme with a proximity term that penalizes the La distance between the new estimate and the optimal parameters. We call the discrepancy between the warm-start and proximal warm-start optima the proximity gap.
Interestingly, past works have observed that for quadratic cost functions, early stopping has a similar effect to L2 regularization (Vicol et al. 2022a Ali et al., 2019]. Therefore, the proximal response function can be thought of as capturing how gradient descent will respond a dataset perturbation if it takes only a limited number of steps starting from the warm-start solution.
4.3 Non-Convergence Gap: Influence Estimation on Non-Converged Parameters
Thus far, our analysis has assumed that influence functions are computed on fully converged parame- ters 0* at which the gradient of the cost is 0. However, neural network training, we often terminate the optimization procedure before reaching the exact optimum due to several reasons, including having limited computational resources or to avoid overfitting [Bengio| 2012). In such situations, much of the change in the parameters from LOO retraining simply reflects the effect of training for
5

---

2.0
2.0
Non Non- Converged Parameters
Warm-start Optimum
1.5
1.5
Influence Estimation
1.0
1.0
0.5
0.5
0.0
0.0
-0.5
00.5
1.0
-1.0
1.5
1.5
Figure 3: Decision boundaries for a partially trained binary classifier. We consider removing data point located right-bottom corner denoted as *. While influence estimation makes a local change on the data point of interest, the (warm-start) LOO retraining globally updates the parameters to better fit other data points (a nuisance from the perspective of understanding influence).
Error
Objective
Init
Cold-start
J(0) L(S(0, (), t)e
0
Warm-start
[(0) - L(S(0, x),
Proximity
((0) - (( t) 1:10--01) -
Non-Convergence
N E Deo ((0, x(0)) _)) +
0' VoL(f(0' x)) + 2 1-1
0*
-Linearization
EE Dete cts (,0
0'
Table I: Summary of proximal objectives that influence functions aim approximate when the network is non-strongly convex. damping term is used, and influence functions are computed on non-converged parameters. The final lincarization reflects the second-order approximation that influcnce functions utilize.
longer, rather than the effect of removing training example, illustrated in Figure What we desire from influence functions is to understand the effect of removing the training example; the effect of extended training simply nuisance. Therefore, to the extent that this factor contributes to the misalignment between influence functions and LOO retraining, influence functions are arguably more useful than LOO retraining.
Since training the network to convergence may be impractical or undesirable, we instead modify the response function by replacing the original training objective with similar one for which the (possibly non-converged) final parameters 0" are optimal. Here, we assume the loss function C(:,)) convex function of the network outputs; this is true for commonly used loss functions such as squared error or cross-entropy. We replace the training loss with term that penalizes mismatch to the predictions made by 0" (hence implying that 0* optimal). Our proximal Bregman response function (PBRF) is defined as follows:
N
A Deco (f(0,x ,(00x(())))))))))))))))))))))))) ((f(.x), 2 0
d.ammp (E) arg min BERd
Σ
N
2
(8)
where D cco (1,)) is the Bregman divergence defined as:
Deco (y,y") = C(y, (0) L(y", ((0) V C(y"
T (()))
yd)
(9)
The PBRF defined in Eqn. composed of three terms. The first term measures the functional discrepancy between the current estimate and the parameters 0" in Bregman divergence, and its role to prevent the new estimate from drastically altering the predictions on the training dataset. One way of understanding this term in the cases of squared error or cross-entropy losses is that it equivalent to the training error on dataset where the original training labels are replaced with soft targets obtained from the predictions made by 0", The second term is the negative loss on the data point z = tx, which aims to respond to the deletion training example. The final term simply the proximity term described before. In Appendix B.3 further show that the influence

---

function on non non-converged parameters is equivalent to the first-order approximation of PBRF instead of the first-order approximation of proximal response function for linear models.
Rather than computing the LOO retrained parameters by performing K additional optimization steps under the original training objective, we instead perform K optimization steps under the proximal Bregman objective. The difference between the resulting parameter vectors is what we call the non-convergence gap.
4.4 Linearization Error: A First-order Taylor Approximation of the Response Function
The key idea behind influence functions is the linearization of the response function. To simulate the local approximations made in influence functions, define the linearized PBRF
N
2, damp. lin (e) arg min BERd
Σ
D an (Sin(O, (0, x (1) x)) 1(0", ()))
i=1
(10)
x VoL(f((" xx, t) Oc + 2 210--0 0¹1
where (q)) is the second-order expansion of the loss around y" and [ūn( is the linearization of the network outputs with respect the parameters. The optimal solution to the linearized PBRF is equivalent to the influence estimation at the parameters 0 with the GNH approximation and a damping term À (see Appendix [B44 for the derivation).
As the linearized PBRF relies on several local approximations, the linearization error increases when the downweighting factor magnitude |e| is large the PBRF is highly non-linear. We refer to the discrepancy between the PBRF and linearized PBRF as the linearization error.
4.5 Solver Error: A Crude Approximation of iHVP
As the precise computation of the iHVP is computationally infeasible, in practice, truncated CG or LiSSA to efficiently approximate influence functions [Koh and Liang, 2017]. Unfortunately, these efficient linear solvers introduce additional error by crudely approximating the iHVP. Moreover, different linear solvers can introduce specific biases in the influence estimation. For example, Vicol et al. 2022b] show that the truncated LiSSA algorithm implicitly adds an additional damping term in the iHVP. We use solver error to refer to the difference between the linearized PBRF and the influence estimation computed by linear solver.
Interestingly, Koh and Liang [017] reported that the LiSSA algorithm gave more accurate results than CG. We have determined that this difference resulted not from any inherent algorithmic advantage to LiSSA, but rather from the fact that the software used different damping strengths for the two algorithms, thereby resulting in different weightings of the proximity term in the proximal response function.
5 PBRF: The Question Influence Functions are Really Answering
The PBRF (Eqn. 8 approximates the effect of removing data point while trying keep the predictions consistent with those of the (partially) trained model. Since the discrepancy between the PBRF and influence function estimates is only due to linearization and solver errors, the PBRF can be thought of as better representing the question that influence functions are trying to answer. Reframing influence functions in this way means that the PBRF can be regarded as a gold-standar ground truth for evaluating methods for influence function approximation. Existing analyses of influence functions [Basu et al. 2020a] rely on generating LOO retraining ground truth estimates by imposing strong Le regularization or training till convergence without early stopping. However, these conditions do not accurately reflect the typical way neural networks are trained in practice. In contrast, our PBRF formulation does not require the addition of any regularizers modified training regimes and can be easily optimized.
In addition, although the PBRF may not necessarily align with LOO retraining due to the warm-start, proximity, and non-convergence gaps, the motivating use cases for influence functions typically do not rely exact LOO retraining. This means that the PBRF can be used in place of LOO retraining for
7

---

many tasks such as identifying influential or mislabelled examples, as demonstrated in Appendix D.3 In these cases, influence functions are still useful since they provide an efficient way of approximating PBRF estimates.
6 Experiments
Our experiments investigate the following questions: (1) What factors discussed in Section con- tribute most to the misalignment between influence functions and LOO retraining? (2) While influence functions fail approximate the effect of retraining, do they accurately approximate the PBRF? (3) How do changes in weight decay, damping, the number of total epochs, and the number of removed training examples affect each source of misalignment?
In all experiments, we first train the base network with the entire dataset to obtain the parameters 0". We repeat the training procedure 20 times with a different random training example deleted. The cold-start retraining begins from the same initialization used to train 0" All proximal objectives are trained with initialization 0" for 50% of the epochs used to train the base network. Lastly, we use the LiSSA algorithm with GNH approximation to compute influence functions.
Since primarily interested in the effect of deleting data point model's predictions, measure the discrepancy of each gap and error using the average L2 distance between networks' outputs E. (x. [I.f(ll)]])))))_ x) f(O', x)ll II] on the training dataset. We provide the full experimental set-up and additional experiments in Appendix C Cad D. respectively.
6.1 Influence Misalignment Decomposition
We first applied our decomposition to various models trained on a broad range of tasks covering binary classification, regression, image reconstruction, image classification, and language modeling. The summary of our results is provided in Figure J and Table (Appendix EE. Across all tasks, we found that the first three sources dominate the misalignment, indicating influence function estimators are answering a different question from what is normally assumed. Small linearization and solver errors indicate that influence functions accurately answer the modified question (PBRF).
Logistic Regression. We analyzed the logistic regression (LR) model trained on the Cancer and Diabetes classification datasets from the UCI collection [Dua and Graff] 2017). We trained the model using L-BFGS [Liu and Nocedal, 1989] with L2 regularization of 0.01 and damping term of A = 0.001. As the training objective is strongly convex and the base model parameters were trained till convergence, in Table 5 we observed that each source of misalignment is significantly low. Hence, in the case of logistic regression with L2 regularization, influence functions accurately capture the effect of retraining the model without a data point.
Multilayer Perceptron. Next, we applied our analysis to the 2-hidden layer Multilayer Perceptron (MLP) with ReLU activations. We conducted the experiments in two settings: (1) regression on the Concrete and Energy datasets from the UCI collection and (2) image classification on 10% of the MNIST [Deng] 2012 and FashionMNIST Xiao et al. 2017] datasets, following the set-up from Koh and Liang 2017 and Basu et al.) 20 We trained the networks for 1000 epochs using stochastic gradient descent (SGD) with a batch size of 128 and set a damping strength of A = 0.001.
As opposed to linear models, MLPs violate the assumptions in the influence derivation and we observed an increase gaps and errors on all five factors. We observed that warm-start, proximity, and the non-convergence gaps contribute more to the misalignment than linearization and solver errors. The average network's 's predictions for PBRF were similar to that computed by the LiSSA algorithm, demonstrating that influence functions are still a good approximation to PBRF
Next, we applied our framework an 8-layer autoencoder (AE) on the full MNIST dataset. We followed the experimental set-up from Martens and Grosse (2015), where the encoder and decoder each consist of 4 fully-connected layers with sigmoid activation functions. We trained the network for 1000 epochs using SGD with momentum. We set the batch size to 1024, used L2 regularization of 10- with a damping factor of 0.001. In accordance with the findings from our MLP experiments, the warm-start, proximity, and non-convergence gaps were more significant than the linearization and solver errors, and influence functions accurately predicted the PBRF.
Autoencoder.
8

---

Warm Start
Proviniy
Non Convergnce
Liverization
Salver
MLP Cancer
VGG13 MNIST
MLP Energy
ResNer20 MNIST
MLP MNIST
LeNet CIFAR1O
MLP - FMNIST
Alexhett CIFAR1O
AE MNIST
VGG13 CIFAR10
LeNet - MNIST
Rediet20
CIFAR10
MexNet- MNIST
Transfomer - PTB
Figure 4: Decomposition of the discrepancy between influence functions and LOO retraining into (1) warm-start gap, (2) proximal gap, (3) non-convergence gap, (4) linearization error, (5) solver error for each model and dataset. The size of each component is measured by the L2 distance between the networks' outputs on the training dataset.
Convolutional Neural Networks. To in- vestigate the source of discrepancy on larger- scale networks, trained set of convo- lutional neural networks of increasing com- plexity and size. Namely, LeNet [Lecun ct al. 1998), AlexNet [Krizhevsky et al. 20022].
Model
Cold-Start
Warm-Start
PBRF
P
MLP
-0.55
0.01
0.22
0.35
0.98
0.99
LeNet
-0.19
0.12
0.32
0.25
0.93
0.52
AlexNet
-0.16
-0.08
0.51
0.58
0.99
0.99
VGG13
0.45
-0.07
-0,28
-0.51
0.98
0.77
VGG13 Simonyan and Zisserman [2014), and ResNet-20 [He et al., [2015] were trained on 10% of the MNIST dataset and the full CI- FAR10 Krizhevsky 2009) dataset. We trained the base network for 200 epochs on both datasets with a batch size of 128. For MNIST, we kept the learning rate fixed throughout training, while for CIFARIO, we decayed the learning rate by factor of 5 at epochs 60, 120, and 160, follow- ing Zagoruyko and Komodakis [2016). We used L2 regularization with strength 5 10-4 and damping factor of À = 0.001. Consistent with the findings from our MLP and autoencoder experi- ments, the first three gaps were more significant than linearization and solver errors.
ResNet-20
0.09
-0.06
0.02
0.09
0.81
0.76
Table 2: Comparison test loss differences computed by influence function, cold-start retraining, warm-start retraining, and PBRF on MNIST dataset. We show Pearson (P) and Spearman rank-order (S) correlation when compared influence estimates.
We further compared influence functions' approximations on the difference in test loss when a random training data point is removed with the value obtained from cold-start retraining, warm-start retraining, and PBRF in T We used both Pearson [Sedgwick] 2012) and Spearman rank-order correlation [Spearman 1961] to measure the alignment. While the test loss predicted by influence functions does not align well with the values obtained by cold-start and warm-start retraining schemes, they show high correlations when compared the estimates given by PBRF.
Transformer. Finally, we trained 2-layer Transformer language models on the Penn Treebank (PTB) MMacuss al. [993] dataset. We set the number of hidden dimensions to 256 and the number of attention heads to 2. As we observed that model overfits after a few epochs of training, we trained the base network for 10 epochs using Adam. Notably, observed that the non-convergence gap had the most considerable contribution to the discrepancy between influence functions and LO0 retraining. Consistent with previous findings, the first tree gaps had more impact the discrepancy compared to linearization and solver errors.
6.2 Factors in Influence Misalignment
We further analyzed how the contribution of each component changes in response to changes in network width and depth, training time, weight decay, damping, and the percentage of data removed. We used MLP trained on 10% of the MNIST dataset and summarized results in Figure 5
Width and Depth. As we increase the width of the network, we observe a decrease in the lineariza- tion error. This is consistent with previous observations that networks behave more linearly as the width is increased [Lee et al. 2019). In contrast to the findings from Basu et al. [2020a], we did not observe strong relationship between the contribution of the components and the depth of the network.
Training Time. Unsurprisingly, as we increase the number of training epochs, we observe a decrease in the non-convergence gap. We hypothesize that, as we increase the training epoch, the cost gradient reaches 0, resulting in better alignment between the proximal response function and PBRF

---

10
Lagis cfffemnce
Laghts diferrnnc
20r
10
1n
Width
Effect of width
Effect of depth
(c) Effect of training time
Logis ofference
-0P
Legits difieence
Warm Start
Prosirity
Logit
Nops Comerggeee
10-
Literitzaioo
10
Repularizatoon
Damping
Porceragge ronoval
(d) Effect L2 regularization
(c) Effect damping strength
(f) Effect of removed
Figure 5: Ablations on how various factors affect the contribution of the gaps and errors to the discrepancy
between influence approximation and LOO retraining.
Weight Decay. The weight decay allows the training objective to be better conditioned. Consequently, as weight decay increases, the training objective may act more as a strictly convex objective, resulting in decrease in overall discrepancy for all components. [Bssu et al. 2020a] also found that the alignment between influence functions and LOO retraining increases as weight decay increases.
Damping. A higher damping term makes linear systems better conditioned, allowing solvers to find accurate solutions in fewer iterations [Demmel| 1997], thereby reducing the solver error. Furthermore, the higher proximity term keeps the parameters close to 0" reducing the linearization error. On the other hand, increasing the effective proximity penalty directly increases the proximity gap.
Percentage of Training Examples Removed. As we remove more training examples from the dataset the PBRF becomes more non-linear and observe sharp increase in the linearization error. The cost landscape is also more likely to change we remove more training examples, and we observe a corresponding increase in the warm-start gap.
7 Conclusion
In this paper, we investigate the sources of the discrepancy between influence functions and LOO retraining in neural networks. We decompose this difference into five distinct components: the warm- start gap, proximity gap, non-convergence gap, lincarization еπтоr, and solver error. We empirically evaluate the contributions of each of these components on a wide variety of architectures and datasets and investigate how they change with factors such network size and regularization. Our results show that the first three components are most responsible for the discrepancy between influence functions and LOO retraining. We further introduce the proximal Bregman response function (PBRF) to better capture the behavior of influence functions neural networks. Compared to LOO retraining, the PBRF is more easily calculated and correlates better with influence functions, meaning is attractive alternative gold standard for evaluating influence functions. Although the PBRF may not necessarily align with LOO retraining, it still be applied in many of the motivating for influence functions. We conclude that influence functions in neural networks are not necessarily "fragile", but instead are giving accurate answers a different question than normally assumed.
cases
Acknowledgements
We would like to thank Pang Wei Koh for the helpful discussions. Resources used in this research were provided, part, by the Province of Ontario, the Government Canada through CIFAR, and companies sponsoring the Vector Institute (www. wtt.:/ettrrisstttteeeeaaaaaeee wwt.ttteteeaaaaaatreeeaaaaa
10

---

References
N. Agarwal, B. Bullins, and E. Hazan. Second-order stochastic optimization in linear time. stat, 1050:15, 2016.
A. Ali, J. Z. Kolter, and R. J. Tibshirani. A continuous-time view of early stopping for least squares regression. In The 22nd international conference on artificial intelligence and statistics, pages 1370-1378. PMLR, 2019.
S.-i. Amari, J. Ba, R. Grosse, X. Li, A. Nitanda, T. Suzuki, D. Wu, and J. Xu. When does precondi- tioning help or hurt generalization? arXiv preprint arXiv:2006. /0732, 2020.
S. Arora, S. Du, W. Hu, Z. Li, and R. Wang, Fine-grained analysis of optimization and generaliza- tion for overparameterized two-layer neural networks. In International Conference on Machine Learning, pages 322-332. PMLR, 2019.
E. Barshan, M.-E. Brunet, and G. K. Dziugaite. RelatIF: Identifying explanatory training samples via relative influence. In S. Chiappa and R. Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 1899-1909, PMLR, 26-28 Aug 2020. URL https:// proceedings mlr. wttp:preee/vivbrararrreeee
html
L. Bartlett, P. M. Long, G. Lugosi, and A. Tsigler. Benign overfitting in linear regression. Proceedings of the National Academy of Sciences, 1 107433)30003-30077) 2020.
S. Basu, P. Pope, and S. Feizi. Influence functions in deep learning are fragile. arXiv preprint arXiv:2006. 246,1, 2020a.
Basu, X. You, and S. Feizi. On second-order group influence functions for black-box predictions. In International Conference on Machine Learning, pages 715-724. PMLR, 2020b.
Y. Bengio. Practical recommendations for gradient- training of deep architectures. In Neural networks: Tricks of the trade, pages 437-478. Springer, 2012.
J. Bradbury, R. Frostig, P. Hawkins, M. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Miine and Q. Zhang. JAX: composable transformations of cwm/ gomg/oooge/jax
Python+NumPy programs, 2018. hRLp:tt:://g
M.-E. Brunet, C. Alkalay-Houul Houlihan, A. Anderson, and R. Zemel. Understanding the origins of bias in word embeddings. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 803-811. PMLR, 09-15 Jun 2019. URL https::/pproceeddnngt mlr. mlr.press. v97/brunet199 html
G. Charpiat, N. Girard, L. Felardos, and Y. Tarabalka. Input similarity from the neural net- work perspective. H. Wallach, H. Larochelle, A. Beygelzimer, -Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. URL httpps://proceedingg Paper pdf
weerips.cc//ppp /aariipps.c//ppppr/////////e 2019/fi1e///
R. D. Cook. Influential observations in linear regression. Journal of the American Statistical Association, 14(365):19-177,, 1979.
W. Demmel. Applied numerical linear algebra. SIAM, 1997.
L. Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-144 2012.
D. Dua and C. Graff. UCI machine learning repository, 2017. URL http:://archive.iice// uci edu/ml
Farnia and A. Ozdaglar. Do gans always have nash equilibria? In International Conference on Machine Learning, pages 3029-3039. PMLR, 2020.

---

A. Ghorbani, A. Abid, and J. Zou. Interpretation of neural networks is fragile. Proceedings of the AAAI Conference on Artificial Intelligence. 33(01):3681-33883 Jul. 2019. doi: 10. 1609/aaai......m 10..6//a....33300..... 33013681. https://::///o aaai. org/ index. phpppppppaa// phap/AAAT/artiill//veeee 4452
A. Griewank and A. Walther. Evaluating derivatives: principles and techniques of algorithmic differentiation. SIAM, 2008.
H. Guo, Rajani, Hase, M. Bansal, and Xiong. FastlF: Scalable influence functions for efficient model interpretation and debugging. In Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing, pages 10333-10350, Online and Punta Cana, Dominican Repub- lic, Nov. 2021, Association for Computational Linguistics. doi: 10. 18613//1/20211.cooo wttp:/0m...mmmmmmm.m...... URL htwp::ttp///tecttc http::://aclaatholllpgooooo org/ 2021. emnlp-me main. 808
R. Hampel. The influence curve and its role in robust estimation. Journal of the american statistical association, 69(346):3833393 1974.
X. Han, B. C. Wallace, and Y. Tsvetkov. Explaining black box predictions and unveiling data artifacts through influence functions. D. Jurafsky, J. Chai, N. Schluter, and J. R. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5553-5563, Association for Computatioonl Linguistics, URL https: https:///oi..rrr//111666..
2020. doi: acl maimmain.. 492
K. Hanawa, S. Yokoi, S. Hara, and K. Inui. Evaluation of similarity-based explanations. In International Conference Learning Representations, 2021. URL htttps://openreviear het:/foruunidd-9uu/yyyyzzzz
T. Hastie, A. Montanari, S. Rosset, and R. J. Tibshirani. Surprises in high-dimensional ridgeless least squares interpolation. The Annaly of Statistics, 50(2):949-986, 2022.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. arXiv preprint arXiv:1512.0852 2015.
Z. Izzo, M. A. Smart, K. Chaudhuri, and Zou. Approximate data deletion from machine learning models. In International Conference on Artificial Intelligence and Statistics, pages 2008-2016. PMLR, 2021.
K. and A. Søgaard. Revisiting methods for finding influential examples. CoRR, abs/21 1t/p//211....44 1.04683, 2021. hRttps::s//ariii org/ abs/2111. 04683
R. Khanna, B. Kim, J. Ghosh, and S. Koyejo. Interpreting black box predictions using fisher kernels. In K. Chaudhuri and M. Sugiyama, editors, Proceedings of the Twenty-Second In- ternational Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pages 3382-3390. PMLR, 16-18 Apr 2019. URL https //proceedings..o mml
B. Kim, R. Khanna, and O. O. Koyejo. Examples are not enough, learn to criticize! crit- icism for interpretability. In D. Lee. M. Sugiyama, U. Luxburg, I. Guyon, and R. Gar- nett, editors, Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URI. https: httttss:///.occeenggs..eoe wwwpeeeeeennnnnnnnnnn........... /ttprippppscappppp/1////11 460022282210111422444eaabb333333 Pappee
W. Koh and P. Liang. Understanding black-box predictions via influence functions. International conference on machine learning, pages 1885-1894, PMLR, 2017.
P. W. Koh, J. Steinhardt, and P. Liang. Stronger data poisoning attacks break data sanitization defenses. Machine Learning, 111 Jan 2022. ISSN 1573-0565, doi: 10.1077//109944-00--00111944 URL httpp:://daa org/10 100010000000990000--1010111
P. W. W. Koh, K.-S. Ang, H. Teo, and P. S. Liang. On the accuracy of influence functions for
measuring group effects. Advances in neural information processing systems, 32, 2019.
Kong, Y. Shen, and I.. Huang. Resolving training biases via influence-based data relabeling. In International Conference Learning Representations, 2021.
12

---

S. G. Krantz and H. R. Parks. The implicit function theorem: history, theory, and applications. Springer Science & Business Media, 2002.
A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009,
A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference Neural Information Processing Systems - Volume 1, NIPS' 12, page 1097-1105, Red Hook, NY, USA, 2012. Curran Associates Inc.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86( (6(11))227822324, 1998. doi: 10. 10.1109/5.7267))
D. Lee, H. Park, T. Pham, and C. D. Yoo. Learning augmentation network via influence functions. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 10961-10970, 2020.
J. Lee. L. Xiao, S. Schoenholz, Y. Bahri, R. Novak, J. Sohl- Sohl-Dickstein, and J. Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. Advances in neural information processing systems, 32, 2019.
D. C. Liu and J. Nocedal. On the limited memory bfgs method for large scale optimization. Mathe- matical programming, 45(1):5033525 1989.
M. P. Marcus, B. Santorini, and M. A. Marcinkiewicz. Building a large annotated corpus of English: The Penn Treebank. Computational Linguisties, 19(2):313333) 1993. URL https //aclanthology org/ 1193-2004)
J. Martens. New insights and perspectives the natural gradient method. arXiv: 1412.1149.193 2014.
arXiv preprint
J. Martens and R. Grosse. Optimizing neural networks with kronecker-factoree approximate curvature. In International conference on machine learning, pages 2408-2411 PMLR, 2015.
Martens et al. Deep learning via hessian- free optimization. In JCML, volume 27, pages 735-742, 2010.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. Deto,, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, nigh-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Bel Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024-8035, Curran Associates, Inc., 2019. URL http://paperr: neurips httpppscccpppppppeeeepp cc/par http://w.............e.eeeeeeee.
pdf
G. Pruthi, F. Liu, S. Kale, and M. Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 13:19900-19930, 2020.
A. Schioppa. P. Zablotskaia, D. Vilar, and A. Sokolov. Scaling up influence functions. CoRR. abs/2112.03055. 2021. URL https: :/ //wxxv/r/vvvv///////a/ org/ abs///////. 21111.103000 03052
P. Schulam and S. Saria. Can you trust this prediction? Auditing pointwise reliability after learning. In K. Chaudhuri and M. Sugiyama, editors, Proceedings of the Twenty-Second In- ternational Conference of Machine Learning Research, pages 1022-1031. PMLR, 16-18 Apr 2019. URL https
Artificial Intelligence and Statistics, volume 89 of Proceedings
//proceedings.o mlr. prwps/ v89/ aichulam199. html
Sedgwick. Pearson's correlation coefficient. Bmj, 345, 2012.
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 4409 2014.
C. Spearman. The proof and measurement of association between two things. 1961.
13

---

S. Teso, A. Bontempelli, F. Giunchiglia, and A. Passerini. Interactive label cleaning with example-
based explanations. Advances in Neural Information Processing Systems, 34, 2021.
A. W. Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
P. Vicol, J. Lorraine, D. Duvenaud, and R. Grosse. Implicit regularization in overparameterizee bilevel optimization. 2022a.
P. Vicol, J. Lorraine, F. Pedregosa, D. Duvenaud, and R. Grosse. On implicit bias in overparameterized bilevel optimization. In International Conference on Machine Learning. PMLR, 2022b.
S. Wachter, B. D. Mittelstadt, and C. Russell. Counterfactual explanations without opening the black box automated decisions and the GDPR. Harvard Journal of Law Technology, 31(2), 2018.
H. Xiao, K. Rasul, and R. Vollgraf. Fashion-mnin mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
Y.-Y. Yang and K. Chaudhuri. Understanding rare spurious correlations in neural networks. arXiv preprint arXiv:2202.051899 2022.
C.-K. Ych, Kim, I. E.-H. Yen. and K. Ravikumar. Representer point selection for explaining deep neural networks. In S. Bengio. H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett, editors. Advances in Neural Information Processing Systems, volume 31. Cur- ran Associates, Inc., 2018. URL https: //t::::///roae http://pwreeeees.eeeee/ neuriss.nrpppp... hwwprrisscccaappppeeeee cctppapppppp//////eeee (371296666d095778844442222288882
.df
S. Zagoruyko and N. Komodakis. Wide residual networks. arXiv preprint
/46, 2016.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] See Section
(b) Did you describe the limitations of your work? [Yes] See Section 4 for ways in which the PBRF differs from LOO retraining and differs from influence estimates.
(c) Did you discuss any potential negative societal impacts of your work? [N/A] Our work is fundamental in nature and does not enable immediate misuse.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of asssmmtionss of all theoretical results? [Yes] See background in Section and Appendix B
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix B
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [No] We have not
attached the code.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We provide training details in section 6 and in inL
(c) Did you report error bars (e-g., with respect to the random seed after running experi- ments multiple times)? [Yes] See full results in Table
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix C
4. If you are using existing assets (c.g., code, data, models) or curating/relcasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes] See dataset citations in 6.]
14

---

(b) Did you mention the license of the assets? [Yes] All datasets are licensed under MIT data license.
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "nF4XJ7XSEEG", "reviewer": "Reviewer_VBq7", "summary": "The influence function-based method can analyze the effects of adding/removing/perturbating a single data point on parametric models using a linear approximation at the optimal point of parameters. It was pointed out that this approximation on deep neural networks is poor in previous works. \n\nIn this paper, the authors push it further and analyze the factors that affect the approximation, and argue that the performance of influence function methods is not necessarily reflected in the LOO re-training process, but rather can be measured using proximal Bregman Response Function. In the experiments, the authors thoroughly analyzed how different factors affect the discrepancy between influence functions and LOO retraining and showed that PBRF better aligns with influence function methods.", "questions": "see above", "limitations": "authors briefly mentioned that in Q/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ The authors perform analysis and make hypotheses on an important problem (the discrepancy between influence function and LOO), and point out five sources that affect the misalignment\n+ The authors pointed out that PBRF better captures the behaviour of influence functions\n+ The experiment evaluations are thorough\n\nWeaknesses:\n- Although identifying the sources that lead to the misalignment is quite interesting, I'm not very convinced that introducing a different score to solely analyze the influence functions is quite useful or necessary. Influence function methods are initially proposed as an interpretability tool to analyze the model behaviour of removing data points. Introduction and emphasizing on PBRF seem to have lost this goal and are leading to a less meaningful direction.\n- In general, I think the structure and writing of this paper need to be improved.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "gycP5562nFE", "reviewer": "Reviewer_TgaB", "summary": "The paper studies what causes influence estimates misalign the leave-one-out retraining. Specifically, the authors decompose discrepancy into five separate terms and empirically show the contributions of each term under a variety of settings (e.g., model architectures and datasets). ", "questions": "1. What is the purpose of the deviation from Eq. 7 to Eq. 9?\n2. In eq. 8, the authors mention “$f_{\\text{lin}}(\\cdot, \\cdot)$ is the linearization of the network outputs with respect to the parameters”. \n3. What do you mean by “the linearization of the network outputs”? Is there any practical advantage of using over PBRF influence estimates?", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*Originality*: The paper shows the connections between influence estimates and the decomposition of the five terms, and finds out that influence estimates are highly correlated to the proposed proximal Bregman response function (PBRF). The results are overall interesting. \n\n*Quality*: The decomposition is technically sound and empirical analysis of the decomposition is well conducted. However, I suggest the authors show some use cases of PBRF empirically. As demonstrated in the experiment, PBRF is highly correlated with influence estimates. Does it offer some better advantage over influence estimates in some use cases such as data poisoning, improving fairness and explanablity of the model. I think these are also key questions to answer. \n\n*Clarity*: The paper is well written and easy to read. However, some technical details of the paper are not well explained. For example, I am a little confused by the introduction of Linearization Error (from Eq. 7 to Eq. 9). I am not sure what are the purposes/usages by performing the second-order expansion of the loss around $y_s$. \n\n*Significance*: This paper provides an in-depth understanding of influence function by trying to bridge the gap between influence function and LOO re-training . It could impact on the community in the long run. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "eT7eeunBeYW", "reviewer": "Reviewer_br1N", "summary": " The paper tackles  a very important topic about the evaluation of influence function based methods. Currently, the effectiveness of influence functions are based on (a) Direct metrics such as Leave-out-retraining OR (b) Indirect metrics such as detection of mislabeled samples etc. The authors propose to use proximal Bregman response function as a way to measure the effectiveness or alignment of influence functions, especially for deep models. ", "questions": "The questions are added in the Weaknesses section. ", "limitations": "- Lack of motivation on the design of the PBRF metric.\n\n- More analysis is required on Imagenet scale datasets. Although IF is tricky to compute on Imagenet, it would be beneficial to understand the proposed metric at scale.\n\nOverall, I feel the paper is strong and does a well laid out analysis on the alignment of influence estimation and LOO re-training which is lacking in the community. I would also urge the authors to think about in what practical settings and how this metric can be used instead of the commonly used indirect metrics such as mislabeled data points detection. \n\n", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n- The paper does a comprehensive analysis of the different components (warm start, non-convergence, linearization, addition of iHVP etc. ) involved in the misalignment between the influence function and the LOO training objectives. The analysis is interesting and well-executed.\n\n- The authors acknowledge that in principle the deep models are only partially trained as they always don't converge to the optima. Hence, taking this information into account and designing a gold-standard ground-truth is impactful to understand the effectiveness of influence functions. \n\n- The array of experiments are solid and cover a wide range of architectures and datasets / domains. The paper is a good extension to Basu et al (ICLR 2021)[1]  and follows up that paper with more in-depth experiments.\n\nWeaknesses:\n- While the PBRF metric aligns well with the influence score, I would like to see some more motivation about why the authors thought it’s a good metric to evaluate influence functions on.  The current version of the paper lacks this motivation or information corresponding to it.\n \n- If we already know IFs are good at other metrics such as detection of mislabeled examples / relabeling, why do we require a fix to the evaluation procedure of influence functions. It would be good if the authors could substantiate on why alignment of IF with a gold standard ground-truth is important as opposed to testing them on other indirect but more practical metrics such as detection of mislabeled examples / relabeling. \n\n- While the authors cover a wide range of experiments, I would like to see some experiments on more complex datasets such as Imagenet-1k. A focused analysis on Imagenet-1k would make the paper stronger.\n\n\n[1]. https://arxiv.org/abs/2006.14651?context=stat.ML", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Tv1WOXyN4l", "reviewer": "Reviewer_uyV1", "summary": "This paper studies influence functions. Influence functions work well with leave-one-out retraining for linear models, but they often have poor performance in neural networks. The authors investigate the specific factors that cause this discrepancy and decompose into five terms. The authors then study the contributions of each term. Furthermore, the authors show that influence functions are often a good approximation to the proximal Bregman response function (PBRF). The experimental results suggest that current algorithms for influence function estimation can still give more informative results than previous error analyses would suggest. In my opinion, the main contribution of this work is a more in-depth analysis of influence functions, especially as it applies to neural networks. Overall, the paper is well organized and well written.", "questions": "Here are some questions for the authors:\n- As mentioned in weaknesses, what is the underlying reason for selecting the 5 components?\n- Are there any considerations in the selection of experimental datasets?\n", "limitations": "The authors adequately addressed the limitations.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- This work investigates the source of the discrepancy between influence functions and LOO retraining in neural networks, and further decompose the discrepancy into five detailed components.\n- The authors evaluate the contributions of each component on binary classification, regression, image classification, and language modeling tasks.\n- The experimental results show that influence functions are a much better match to the proximal Bregman response function (PBRF), and also can give more informative results than previous error analyses would suggest.\n\nWeaknesses:\n- What is the underlying reason for selecting the 5 components and why are these components considered important? These are not explicitly stated in the paper. \n- The author considers PBRF can be regarded as the gold standard, and I think more rigorous proofs are needed.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept