If Influence Functions are the Answer, Then What is the Question?
Abstract
Influence functions efficiently estimate the effect of removing a single training data point on a model’s learned parameters.
While influence estimates align well with leave-one-out retraining for linear models, recent works have shown this alignment is often poor in neural networks.
In this work, we investigate the specific factors that cause this discrepancy by decomposing it into five separate terms.
We study the contributions of each term on a variety of architectures and datasets and how they vary with factors such as network width and training time.
While practical influence function estimates may be a poor match to leave-one-out retraining for nonlinear networks, we show that they are often a good approximation to a different object we term the proximal Bregman response function (PBRF) .
Since the PBRF can still be used to answer many of the questions motivating influence functions such as identifying influential or mislabeled examples, our results suggest that current algorithms for influence function estimation give more informative results than previous error analyses would suggest.
Introduction
The influence function [Hampel, 1974, Cook, 1979] is a classic technique from robust statistics that estimates the effect of deleting a single data example (or a group of data examples) from a training dataset.
Formally, given a neural network with learned parameters θ ⋆ trained on a dataset D , we are interested in the parameters θ ⋆ − z learned by training on a dataset D − { z } constructed by deleting a single training example z from D .
By taking the second-order Taylor approximation to the cost function around θ ⋆ , influence functions approximate the parameters θ ⋆ − z without the computationally prohibitive cost of retraining the model.
Since Koh and Liang [2017] first deployed influence functions in machine learning, influence functions have been used to solve various tasks such as explaining model’s predictions [Koh and Liang, 2017, Han et al., 2020], relabelling harmful training examples [Kong et al., 2021], carrying out data poisoning attacks [Koh et al., 2022], increasing fairness in models’ predictions [Brunet et al., 2019, Schulam and Saria, 2019], and learning data augmentation techniques [Lee et al., 2020].
When the training objective is strongly convex (e.g., as in logistic regression with L 2 regularization), influence functions are expected to align well with leave-one-out (LOO) or leave- k -out retraining [Koh and Liang, 2017, Koh et al., 2019, Izzo et al., 2021].
However, Basu et al.
[2020a] showed that influence functions in neural networks often do not accurately predict the effect of retraining the model and concluded that influence estimates are often “fragile” and “erroneous”.
Because of the poor match between influence estimates and LOO retraining, influence function methods are often evaluated with alternative metrics such as the detection rate of maliciously corrupted examples using influence scores [Khanna et al., 2019, Koh and Liang, 2017, Schioppa et al., 2021, K and Søgaard, [Figure]
R e t r a i n l o ss d i ﬀ
[Figure] r [Figure] n ce
[Figure]
However, these indirect signals make it difficult to develop algorithmic improvements to influence function estimation.
If one is interested in improving certain aspects of influence function estimation, such as the linear system solver, it would be preferable to have a well-defined quantity that influence function estimators are approximating so that algorithmic choices could be directly evaluated based on the accuracy of their estimates.
In this work, we investigate the source of the discrepancy between influence functions and LOO retraining in neural networks.
We decompose the discrepancy into five components: (1) the difference between cold-start and warm-start response functions (a concept elaborated on below), (2) an implicit proximity regularizer, (3) influence estimation on non-converged parameters, (4) linearization, and (5) approximate solution of a linear system.
This decomposition was chosen to capture all gaps and errors caused by approximations and assumptions made in applying influence functions to neural networks.
We empirically evaluate the contributions of each component on binary classification, regression, image reconstruction, image classification, and language modeling tasks and show that, across all tasks, components (1–3) are most responsible for the discrepancy between influence functions and LOO retraining.
We further investigate how the contribution of each component changes in response to the change in network width and depth, weight decay, training time, damping, and the number of data points being removed
Moreover, we show that while influence functions for neural networks are often a poor match to LOO retraining, they are a much better match to what we term the proximal Bregman response function (PBRF) .
Intuitively, the PBRF approximates the effect of removing a data point while trying to keep the predictions consistent with those of the (partially) trained model.
From this perspective, we reframe misalignment components (1–3) as simply reflecting the difference between LOO retraining and the PBRF.
The gap between the influence function estimate and the PRBF only comes from sources (4) and (5), which we found empirically to be at least an order of magnitude smaller for most neural networks.
As a result, on a wide variety of tasks, influence functions closely align with the PBRF while failing to approximate the effect of retraining the model, as shown in Figure 1.
The PBRF can be used for many of the same use cases that have motivated influence functions, such as finding influential or mislabeled examples [Schioppa et al., 2021] and carrying out data poisoning attacks [Koh and Liang, 2017, Koh et al., 2022], and can therefore be considered an alternative to LOO retraining as a gold standard for evaluating influence functions.
Hence, we conclude that influence functions applied to neural networks are not inherently “fragile” as is often believed [Basu et al., 2020a], but instead can be seen as giving accurate answers to a different question than is normally assumed.
Related Work
Instance-based interpretability methods are a class of techniques that explain a model’s predictions in terms of the examples on which the model was trained.
Methods of this type include TracIn
[Pruthi et al., 2020], Representer Point Selection [ explanations [Wachter et al., 2018], and of central focus in this paper, influence functions.
Since its adoption in machine learning by Koh and Liang [2017], multiple extensions and improvements upon influence functions have also been proposed, such as variants that use Fisher kernels [Khanna et al., 2019], higher-order approximations [Basu et al., 2020b], tricks for faster and scalable inference [Guo et al., 2021, Schioppa et al., 2021], group influence formulations [Koh et al., 2019, Basu et al., 2020b], and relative local weighting [Barshan et al., 2020].
However, many of these methods rely on the same strong assumptions made in the original influence function derivation that the objective needs to be strongly convex and influence functions must be computed on the optimal parameters.
In general, influence functions are assumed to approximate the effects of leave-one-out (LOO) retraining from scratch, the parameters of the network that are trained without a data point of interest. Hence, measuring the quality of influence functions is often performed by analyzing the correlation between LOO retraining and influence function estimations [Koh and Liang, 2017, Basu et al., 2020a,b, Yang and Chaudhuri, 2022].
However, recent empirical analyses have demonstrated the fragility of influence functions and a fundamental misalignment between their assumed and actual effects [Basu et al., 2020a, Ghorbani et al., 2019, K and Søgaard, 2021].
For example, Basu et al.
[2020a] argued that the accuracy of influence functions in deep networks is highly sensitive to network width and depth, weight decay strength, inverse-Hessian vector product estimation methodology, and test query point by measuring the alignment between influence functions and LOO retraining.
Because of the inherent misalignment between influence estimations and LOO retraining in neural networks, many works often evaluate the accuracy of the influence functions on an alternative metric, such as the recovery rate of maliciously mislabelled or poisoned data using influence functions [Khanna et al., 2019, Koh and Liang, 2017, Schioppa et al., 2021, K and Søgaard, 2021].
In this work, instead of interpreting the misalignment between influence functions and LOO retraining as a failure, we claim that it simply reflects that influence functions answer a different question than is typically assumed.
Background
Consider a prediction task from an input space X to a target space T where we are given a finite training dataset D train =
{ ( x ( i ) , t ( i ) ) }
Ni =1 .
Given a data point z
= ( x , t ) , let y
=
f ( θ , x ) be the prediction of the network parameterized by θ ∈ R d and L ( y , t ) be the loss (e.g., squared error or cross-entropy).
We aim to solve the following optimization problem:
[Equation]
where J ( · ) is the cost function.
If the regularization (e.g., L 2 regularization) is imposed in the cost function, we fold the regularization terms into the loss function.
We summarize the notation used in this paper in Appendix A.
3.1 Downweighting a Training Example
The training objective in Eqn.
1 aims to find the parameters that minimize the average loss on all training examples.
Herein, we are interested in studying the change in optimal model parameters when a particular training example
z =
( x , t ) ∈ D train is removed from the training dataset, or more generally, when the data point z is downweighted by an amount ϵ ∈ R .
Formally, this corresponds to minimizing the following downweighted objective:
[Equation]
When ϵ = 1 / N , the downweighted objective reduces to the cost over the dataset with the example z removed, up to a constant factor.
To see how the optimum of the downweighted objective responds to changes in the downweighting factor ϵ , we define the response function r ⋆ − z : R → R d by:
[Equation]
where we assume that the downweighted objective is strongly convex and hence the solution to the downweighted objective is unique given some factor ϵ .
Under these assumptions, note that r ⋆ − z (0) =
θ ⋆ and the response function is differentiable at 0 by the Implicit Function Theorem [Krantz and Parks, 2002, Griewank and Walther, 2008].
Influence functions approximate the response function by performing a first-order Taylor expansion around ϵ 0 = 0 :
[Equation]
We refers readers to Van der Vaart [2000] and Appendix B for a detailed derivation.
The optimal parameters trained without z can then be approximated by plugging in ϵ = 1 / N to Eqn. 4.
Influence functions can further approximate the loss of a particular test point z test =
( x test , t test ) when a data point z is eliminated from the training set using the chain rule [Koh and Liang, 2017]:
[Equation] 3.2 Influence Function Estimation in Neural Networks
Influence functions face two main challenges when deployed on neural networks.
First, the influence estimation (shown in Eqn. 4) requires computing an inverse Hessian-vector product ( iHVP ).
Unfortunately, storing and inverting the Hessian requires O ( d 3 ) operations and is infeasible to compute for modern neural networks.
Instead, Koh and Liang [2017] tractably approximate the iHVP using truncated non-linear conjugate gradient ( CG ) [Martens et al., 2010] or the LiSSA algorithm [Agarwal et al., 2016].
Both approaches avoid explicit computation of the Hessian inverse (see Appendix G for details) and only require O ( Nd ) operations to approximate the influence function.
Second, the derivation of influence functions assumes a strongly convex objective, which is often not satisfied for neural networks.
The Hessian may be singular, especially when the parameters have not fully converged, due to non-positive eigenvalues.
To enforce positive-definiteness of the Hessian, Koh and Liang [2017] add a damping term in the iHVP .
Teso et al.
[2021] further approximate the Hessian with the Fisher information matrix (which is equivalent to the Gauss-Newton Hessian [Martens, 2014] for commonly used loss functions such as cross-entropy) as follows:
[Equation]
where J y θ ⋆ is the parameter-output Jacobian and H y ⋆ is the Hessian of the cost with respect to the network outputs both evaluated on the optimal parameters θ ⋆ . Here, G ⋆ = J ⊤ y θ ⋆ H y ⋆ J y
θ ⋆ is the Gauss-Newton Hessian (GNH) and λ > 0 is a damping term to ensure the invertibility of GNH.
Unlike the Hessian, the GNH is guaranteed to be positive semidefinite as long as the loss function is convex as a function of the network outputs [Martens et al., 2010].
Understanding the Discrepancy between Influence Function and LOO Retraining in Neural Networks
In this section, we investigate several factors responsible for the misalignment between influence functions and LOO retraining.
Specifically, we decompose the misalignment into five separate terms: (1) the warm-start gap, (2) the damping gap, (3) the non-convergence gap, (4) the linearization error, and (5) the solver error.
This decomposition captures all approximations and assumption violations when deploying influence functions in neural networks.
By summing the parameter (or outputs) differences introduced by each term we can bound the parameter (or outputs) difference between
LOO retraining and influence estimates.
We use the term “gap” rather than “error” for the first three terms to emphasize that they reflect differences between solutions to different influence-related questions, rather than actual errors.
For all models we investigate, we find that the first three sources dominate the misalignment, indicating that the misalignment reflects not algorithmic errors but rather the fact that influence function estimators are answering a different question from what is normally assumed.
All proximal objectives are summarized in Table 1 and we provide the derivations in Appendix B.
4.1 Warm-Start Gap: Non-Strongly Convex Training Objective
[Figure]
By taking a first-order Taylor approximation of the response function at ϵ 0 = 0 (Eqn. 4) , influence functions approximate the effect of removing a data point z at a local neighborhood of the optimum θ ⋆ .
Hence, influence approximation has a more natural connection to the retraining scheme that initializes the network at the current optimum θ ⋆ ( warm-start retraining ) than the scheme that initializes the network randomly ( cold-start retraining ).
The warm-start optimum is equivalent to the cold-start optimum when the objective is strongly convex (where the solution to the response function is unique), making the influence estimation close to the LOO retraining on logistic regression with L 2 regularization.
However, the equivalence between warm-start and coldstart optima is not typically guaranteed in neural networks [Vicol et al., 2022a].
Particularly, in the overparametrized regime ( N < d ), neural networks exhibit multiple global optima, and their converged solutions depend highly on the specifics of the optimization dynamics [Lee et al., 2019, Arora et al., 2019, Bartlett et al., 2020, Amari et al., 2020].
For quadratic cost functions, gradient descent with initialization θ 0 converges to the optimum that achieves the minimum L 2 distance from θ
[Hastie et al., 2022]. This phenomenon of the converged parameters being dependent on the initialization hinders influence functions from accurately predicting the effect of retraining the model from scratch as shown in Figure 2.
We denote the discrepancy between cold-start and warm-start optima as warm-start gap .
4.2 Proximity Gap: Addition of Damping Term in iHVP
In practical settings, we often impose a damping term (Eqn. 6) in influence approximations to ensure that the cost Hessian is positive-definite and hence invertible.
As adding a damping term in influence estimations is equivalent to adding L 2 regularization to the cost function [Martens et al., 2010], when damping is used, influence functions can be seen as linearizing the following proximal response function at ϵ 0 = 0 :
[Equation]
See Appendix B.2 for the derivation. Note that λ > 0 is a damping strength and our use of “proximal” is based on the notion of proximal equilibria [Farnia and Ozdaglar, 2020].
Intuitively, the proximal objective in Eqn.
7 not only minimizes the downweighted objective but also encourages the parameters to stay close to the optimal parameters at ϵ 0 = 0 .
Hence, when the damping term is used in the iHVP , influence functions aim at approximating the warm-start retraining scheme with a proximity term that penalizes the L 2 distance between the new estimate and the optimal parameters.
We call the discrepancy between the warm-start and proximal warm-start optima the proximity gap .
Interestingly, past works have observed that for quadratic cost functions, early stopping has a similar effect to L 2 regularization [Vicol et al., 2022a, Ali et al., 2019].
Therefore, the proximal response function can be thought of as capturing how gradient descent will respond to a dataset perturbation if it takes only a limited number of steps starting from the warm-start solution.
4.3 Non-Convergence Gap: Influence Estimation on Non-Converged Parameters
Thus far, our analysis has assumed that influence functions are computed on fully converged parameters θ ⋆ at which the gradient of the cost is 0 .
However, in neural network training, we often terminate the optimization procedure before reaching the exact optimum due to several reasons, including having limited computational resources or to avoid overfitting [Bengio, 2012].
In such situations, much of the change in the parameters from {+LOO retraining simply reflects the effect of training for − 1 x 1 − 1 .− 1 .− 0 .0 .0 .1 .1 .2 .x Non-Converged Parameters − 1 x 1 − 1 .− 1 .− 0 .0 .0 .1 .
 .2 .x+}
Warm-start Optimum
Inﬂuence Estimation
Error Objective Init [Figure]
[Equation]
+ Warm-start
[Equation]
+ Proximity
[Equation]
+ Non-Convergence
[Equation]
+ Linearization [Equation] longer, rather than the effect of removing a training example, as illustrated in Figure 3. What we desire from influence functions is to understand the effect of removing the training example; the effect of extended training is simply a nuisance.
Therefore, to the extent that this factor contributes to the misalignment between influence functions and LOO retraining, influence functions are arguably more useful than LOO retraining.
Since training the network to convergence may be impractical or undesirable, we instead modify the response function by replacing the original training objective with a similar one for which the (possibly non-converged) final parameters θ s are optimal .
Here, we assume the loss function L ( · , · ) is convex as a function of the network outputs; this is true for commonly used loss functions such as squared error or cross-entropy.
We replace the training loss with a term that penalizes mismatch to the predictions made by θ s (hence implying that θ s is optimal).
Our proximal Bregman response function (PBRF) is defined as follows:
[Equation]
where D L ( i ) ( · , · ) is the Bregman divergence defined as:
[Equation]
The PBRF defined in Eqn. 8 is composed of three terms.
The first term measures the functional discrepancy between the current estimate and the parameters θ s in Bregman divergence, and its role is to prevent the new estimate from drastically altering the predictions on the training dataset.
One way of understanding this term in the cases of squared error or cross-entropy losses is that it is equivalent to the training error on a dataset where the original training labels are replaced with soft targets obtained from the predictions made by θ s .
The second term is the negative loss on the data point z
= ( x , t ) , which aims to respond to the deletion of a training example.
The final term is simply the proximity term described before.
In Appendix B.3, we further show that the influence function on non-converged parameters is equivalent to the first-order approximation of PBRF instead of the first-order approximation of proximal response function for linear models.
Rather than computing the LOO retrained parameters by performing K additional optimization steps under the original training objective, we can instead perform K optimization steps under the proximal Bregman objective.
The difference between the resulting parameter vectors is what we call the non-convergence gap .
4.4 Linearization Error: A First-order Taylor Approximation of the Response Function
The key idea behind influence functions is the linearization of the response function.
To simulate the local approximations made in influence functions, we define the linearized PBRF as:
[Equation]
where L quad ( · , · ) is the second-order expansion of the loss around y s and f lin ( · , · ) is the linearization of the network outputs with respect to the parameters.
The optimal solution to the linearized PBRF is equivalent to the influence estimation at the parameters θ s with the GNH approximation and a damping term λ
(see Appendix B.4 for the derivation).
As the linearized PBRF relies on several local approximations, the linearization error increases when the downweighting factor magnitude | ϵ | is large or the PBRF is highly non-linear.
We refer to the discrepancy between the PBRF and linearized PBRF as the linearization error .
4.5 Solver Error: A Crude Approximation of iHVP
As the precise computation of the iHVP is computationally infeasible, in practice, we use truncated CG or LiSSA to efficiently approximate influence functions [Koh and Liang, 2017].
Unfortunately, these efficient linear solvers introduce additional error by crudely approximating the iHVP .
Moreover, different linear solvers can introduce specific biases in the influence estimation.
For example, Vicol et al.
[2022b] show that the truncated LiSSA algorithm implicitly adds an additional damping term in the iHVP .
We use solver error to refer to the difference between the linearized PBRF and the influence estimation computed by a linear solver.
Interestingly, Koh and Liang [2017] reported that the LiSSA algorithm gave more accurate results than CG .
We have determined that this difference resulted not from any inherent algorithmic advantage to LiSSA , but rather from the fact that the software used different damping strengths for the two algorithms, thereby resulting in different weightings of the proximity term in the proximal response function.
PBRF: The Question Influence Functions are Really Answering
The PBRF (Eqn. 8) approximates the effect of removing a data point while trying to keep the predictions consistent with those of the (partially) trained model.
Since the discrepancy between the PBRF and influence function estimates is only due to the linearization and solver errors, the PBRF can be thought of as better representing the question that influence functions are trying to answer.
Reframing influence functions in this way means that the PBRF can be regarded as a gold-standard ground truth for evaluating methods for influence function approximation.
Existing analyses of influence functions [Basu et al., 2020a] rely on generating LOO retraining ground truth estimates by imposing strong L 2 regularization or training till convergence without early stopping.
However, these conditions do not accurately reflect the typical way neural networks are trained in practice.
In contrast, our PBRF formulation does not require the addition of any regularizers or modified training regimes and can be easily optimized.
In addition, although the PBRF may not necessarily align with LOO retraining due to the warm-start, proximity, and non-convergence gaps, the motivating use cases for influence functions typically do not rely on exact LOO retraining.
This means that the PBRF can be used in place of LOO retraining for many tasks such as identifying influential or mislabelled examples, as demonstrated in Appendix D.3.
In these cases, influence functions are still useful since they provide an efficient way of approximating
PBRF estimates.
Experiments
Our experiments investigate the following questions: (1) What factors discussed in Section 4 contribute most to the misalignment between influence functions and LOO retraining?
(2) While influence functions fail to approximate the effect of retraining, do they accurately approximate the PBRF?
How do changes in weight decay, damping, the number of total epochs, and the number of removed training examples affect each source of misalignment?
In all experiments, we first train the base network with the entire dataset to obtain the parameters θ s .
We repeat the training procedure 20 times with a different random training example deleted.
The cold-start retraining begins from the same initialization used to train θ s .
All proximal objectives are trained with initialization θ s for 50% of the epochs used to train the base network.
Lastly, we use the LiSSA algorithm with GNH approximation to compute influence functions.
Since we are primarily interested in the effect of deleting a data point on model’s predictions, we measure the discrepancy of each gap and error using the average L 2 distance between networks’ outputs E ( x , · ) ∼D train [ ∥ f ( θ , x ) − f ( θ ′ , x ) ∥ ] on the training dataset.
We provide the full experimental set-up and additional experiments in Appendix C and D, respectively.
6.1 Influence Misalignment Decomposition
We first applied our decomposition to various models trained on a broad range of tasks covering binary classification, regression, image reconstruction, image classification, and language modeling.
The summary of our results is provided in Figure 4 and Table 5 (Appendix E).
Across all tasks, we found that the first three sources dominate the misalignment, indicating influence function estimators are answering a different question from what is normally assumed.
Small linearization and solver errors indicate that influence functions accurately answer the modified question (PBRF).
Logistic Regression.
We analyzed the logistic regression (LR) model trained on the Cancer and
Diabetes classification datasets from the UCI collection [Dua and Graff, 2017].
We trained the model using L-BFGS [Liu and Nocedal, 1989] with L 2 regularization of 0.01 and damping term of λ = 0 .
As the training objective is strongly convex and the base model parameters were trained till convergence, in Table 5, we observed that each source of misalignment is significantly low.
Hence, in the case of logistic regression with L 2 regularization, influence functions accurately capture the effect of retraining the model without a data point.
Multilayer Perceptron.
Next, we applied our analysis to the 2-hidden layer Multilayer Perceptron (MLP) with ReLU activations.
We conducted the experiments in two settings: (1) regression on the Concrete and Energy datasets from the UCI collection and (2) image classification on 10% of the MNIST [Deng, 2012] and FashionMNIST [Xiao et al., 2017] datasets, following the set-up from Koh and Liang [2017] and Basu et al.
[2020a]. We trained the networks for 1000 epochs using stochastic gradient descent (SGD) with a batch size of 128 and set a damping strength of λ = 0 . 001 .
As opposed to linear models, MLPs violate the assumptions in the influence derivation and we observed an increase in gaps and errors on all five factors.
We observed that warm-start, proximity, and the non-convergence gaps contribute more to the misalignment than linearization and solver errors.
The average network’s predictions for PBRF were similar to that computed by the LiSSA algorithm, demonstrating that influence functions are still a good approximation to PBRF.
Autoencoder. Next, we applied our framework to an 8-layer autoencoder (AE) on the full MNIST dataset. We followed the experimental set-up from Martens and Grosse [2015], where the encoder and decoder each consist of 4 fully-connected layers with sigmoid activation functions.
We trained the network for 1000 epochs using SGD with momentum. We set the batch size to 1024, used Lregularization of 10 − 5 with a damping factor of λ = 0 . In accordance with the findings from our
MLP experiments, the warm-start, proximity, and non-convergence gaps were more significant than the linearization and solver errors, and influence functions accurately predicted the PBRF. VGG13 - MNIST ResNet20 - MNIST LeNet - CIFAR
AlexNet - CIFAR
VGG13 - CIFAR
ResNet20 - CIFAR Transformer - PTB
Warm-Start
Proximity
Non-Convergence
Linearization
Solver
MLP - Cancer
MLP - Energy
MLP - MNIST
MLP - FMNIST
AE - MNIST
LeNet - MNIST AlexNet - MNIST
Warm-Start
Proximity
Non-Convergence
Linearization
Solver
VGG13 - MNIST
ResNet20 - MNIST
LeNet - CIFAR
AlexNet - CIFAR
VGG13 - CIFAR ResNet20 - CIFAR Transformer - PTB Warm-Start
Proximity
Non-Convergence
Linearization
Solver
[Table]
Convolutional Neural Networks.
To investigate the source of discrepancy on largerscale networks, we trained a set of convolutional neural networks of increasing complexity and size.
Namely, LeNet [Lecun et al., 1998], AlexNet [Krizhevsky et al., 2012],
VGG13 Simonyan and Zisserman [2014], and ResNet-20 [He et al., 2015] were trained on 10% of the MNIST dataset and the full {+CIFAR
 [Krizhevsky, 2009]+} dataset.
We trained the base network for 200 epochs on both datasets with a batch size of 128.
For MNIST, we kept the learning rate fixed throughout training, while for CIFAR10, we decayed the learning rate by a factor of 5 at epochs 60, 120, and 160, following Zagoruyko and Komodakis [2016].
We used L 2 regularization with strength 5 · 10 − 4 and a damping factor of λ = 0 . 001 .
Consistent with the findings from our MLP and autoencoder experiments, the first three gaps were more significant than linearization and solver errors.
We further compared influence functions’ approximations on the difference in test loss when a random training data point is removed with the value obtained from cold-start retraining, warm-start retraining, and PBRF in Table 2.
We used both Pearson [Sedgwick, 2012] and Spearman rank-order correlation [Spearman, 1961] to measure the alignment.
While the test loss predicted by influence functions does not align well with the values obtained by cold-start and warm-start retraining schemes, they show high correlations when compared to the estimates given by PBRF.
Transformer.
Finally, we trained 2-layer Transformer language models on the Penn Treebank (PTB) [Marcus et al., 1993] dataset.
We set the number of hidden dimensions to 256 and the number of attention heads to 2 .
As we observed that model overfits after a few epochs of training, we trained the base network for 10 epochs using Adam.
Notably, we observed that the non-convergence gap had the most considerable contribution to the discrepancy between influence functions and LOO retraining.
Consistent with our previous findings, the first tree gaps had more impact on the discrepancy compared to linearization and solver errors.
6.2 Factors in Influence Misalignment
We further analyzed how the contribution of each component changes in response to changes in network width and depth, training time, weight decay, damping, and the percentage of data removed.
We used an MLP trained on 10% of the MNIST dataset and summarized results in Figure 5.
Width and Depth.
As we increase the width of the network, we observe a decrease in the linearization error.
This is consistent with previous observations that networks behave more linearly as the width is increased [Lee et al., 2019].
In contrast to the findings from Basu et al.
[2020a], we did not observe a strong relationship between the contribution of the components and the depth of the network.
Training Time.
Unsurprisingly, as we increase the number of training epochs, we observe a decrease in the non-convergence gap.
We hypothesize that, as we increase the training epoch, the cost gradient reaches 0 , resulting in better alignment between the proximal response function and PBRF.
[Figure]
Weight Decay.
The weight decay allows the training objective to be better conditioned.
Consequently, as weight decay increases, the training objective may act more as a strictly convex objective, resulting in a decrease in overall discrepancy for all components.
Basu et al.
[2020a] also found that the alignment between influence functions and LOO retraining increases as weight decay increases [Figure]
Damping.
A higher damping term makes linear systems better conditioned, allowing solvers to find accurate solutions in fewer iterations [Demmel, 1997], thereby reducing the solver error.
Furthermore, the higher proximity term keeps the parameters close to θ s , reducing the linearization error.
On the other hand, increasing the effective proximity penalty directly increases the proximity gap.
Percentage of Training Examples Removed.
As we remove more training examples from the dataset the PBRF becomes more non-linear and we observe a sharp increase in the linearization error.
The cost landscape is also more likely to change as we remove more training examples, and we observe a corresponding increase in the warm-start gap.
Conclusion
In this paper, we investigate the sources of the discrepancy between influence functions and LOO retraining in neural networks.
We decompose this difference into five distinct components: the warmstart gap, proximity gap, non-convergence gap, linearization error, and solver error.
We empirically evaluate the contributions of each of these components on a wide variety of architectures and datasets and investigate how they change with factors such as network size and regularization.
Our results show that the first three components are most responsible for the discrepancy between influence functions and LOO retraining.
We further introduce the proximal Bregman response function (PBRF) to better capture the behavior of influence functions in neural networks.
Compared to LOO retraining, the PBRF is more easily calculated and correlates better with influence functions, meaning it is an attractive alternative gold standard for evaluating influence functions.
Although the PBRF may not necessarily align with LOO retraining, it can still be applied in many of the motivating use cases for influence functions.
We conclude that influence functions in neural networks are not necessarily “fragile”, but instead are giving accurate answers to a different question than is normally assumed. Acknowledgements
We would like to thank Pang Wei Koh for the helpful discussions.
Resources used in this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute ( www.vectorinstitute.ai/partners ).
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] See Section 5.
(b) Did you describe the limitations of your work?
[Yes] See Section 4 for ways in which the PBRF differs from LOO retraining and differs from influence estimates.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
Our work is fundamental in nature and does not enable immediate misuse.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] See background in Section 3 and Appendix B. (b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix B. 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] We have not attached the code.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] We provide training details in section 6 and in C. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See full results in Table 5.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix C. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] See dataset citations in section [-6.
-]6.
(b) Did you mention the license of the assets?
[Yes] All datasets are licensed under MIT data license.
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
Appendix
This appendix is structured as follows:
• In Section A, we provide an overview of the notation we use throughout the paper.
• In Section B, we provide derivations for influence functions, proximal response function, proximal Bregman response function, and linearized proximal response function.
• In Section C, we provide experimental details.
• In Section D, we provide additional experiment results.
• In Section E, we present the numerical results shown in Figure 4.
• In Section G, we provide an overview of CG and LiSSA algorithms.
A Table of Notation
Table 3 summarizes notations used in this paper. [Table] parameters [Table]
dataset
[Table]
Downweighted
[Table]
[Table] [Table] [Table] Optimal model parameters of the downweighted cost Q − z ( θ , ϵ ) , where ϵ is fixed
[Table]
Response
[Table]
Jacobian
[Table]
⋆
[Table]
Hessian of the cost with respect to the network outputs
[Table]
Damping strength
[Table]
learned
[Table]
The Bregman divergence L ( y , [Table] ( i
[Table] L ( y s , t ( i ) )
[Table]
y − y s ) r [Table] [Table]
ϵ ) Proximal Bregman response function r
[Table]
Linearized proximal Bregman response function f lin ( θ , x )
Linearized network outputs with respect to the parameters
[Equation]
y s =
f ( θ s , x )
Prediction of the network f parameterized by the parameters θ s
L quad ( y , t )
Second-order Taylor expansion of the loss around y s
B Derivations B.1 Influence Function Derivation
We provide a derivation of influence functions using the response function.
We refer readers to Van der
Vaart [2000] and Koh and Liang [2017] for a more general derivation of influence functions.
Let z =
( x , t ) ∈ D train be a training example we are interested in downweighting.
Recall that the downweighted objective is defined as:
[Equation]
We further let ϵ
=
and θ ⋆ be the optimal parameters such that ∇ θ Q − z
( θ ⋆ , ϵ 0 ) = 0 .
Here, the optimal parameters θ ⋆ is the solution that minimizes the cost function J ( · ) .
We assume that the downweighted objective is twice continuously differentiable and strongly convex in the parameters θ at ϵ 0 .
Note that if we assume the strong convexity of the loss function, the downweighted objective is only guaranteed to be strongly convex when ϵ ≤ 1 / N .
By the Implicit Function Theorem, these exists a unique continuously differentiable response function r ⋆ − z : U 0 → R d defined on a neighborhood U 0 of ϵ 0 such that r ⋆ − z
( ϵ 0 )
= θ ⋆ and:
[Equation]
12) for all ϵ ∈ U 0 .
By taking the derivative with respect to the downweighting factor ϵ , we get:
[Equation]
13) for all ϵ ∈ U 0 .
The Jacobian of the response function at ϵ 0 can further be expressed as:
[Equation]
where we used these two equalities:
[Equation]
Rearranging Eqn. 14, we have:
[Equation]
where the Hessian ∇ 2 θ J ( θ ⋆ ) is invertible by strong convexity of our downweighted objective at ϵ 0 .
Influence functions approximate the response function with a first-order Taylor expansion at ϵ 0 :
[Equation]
To approximate the optimal parameters trained without a data point z
[Equation]
we can substitute ϵ = 1 / N as follows:
[Equation]
Influence functions can further approximate the loss at a particular test point z test =
( x test , t test ) (or test loss) when a training example z is eliminated from the training set using the chain rule [Koh and Liang, 2017]:
[Equation]
B.2 Proximal Response Function Derivation
Let z =
( x , t ) ∈ D train be a training example we are interested in downweighting.
Recall that the proximal response function (in Eqn. 7) is defined as:
[Equation]
where θ ⋆ is the optimal parameters that minimize the cost function and λ > 0 is a damping term.
Here, we show that influence estimations with a damping term correspond to a first-order Taylor approximation of the proximal response function.
Let λ > 0 be some damping term and θ ⋆ be the solution that minimizes the cost function.
Let ϵ
=
and assume that the downweighted objective is convex in the parameters θ at ϵ 0 .
By the Implicit Function Theorem, we can guarantee the existence of the proximal response function r ⋆ − z , damp : U 0 → R d defined on a neighborhood U 0 of ϵ 0 which satisfies r ⋆ − z , damp ( ϵ 0 )
= θ ⋆ and:
[Equation]
22) for all ϵ in some neighborhood U 0 of ϵ 0 .
Then, differentiating Eqn. 22 with respect to the downweighting factor ϵ equates:
[Equation]
[Equation]
[Equation]
for all ϵ ∈ U 0 .
Evaluating the response Jacobian at ϵ 0 and rearranging the terms in Eqn.
[Equation]
Hence, a first-order Taylor approximation of the proximal response function is equivalent to influence functions with a damping term λ
I added:
[Equation]
When a damping term is used in influence functions, they approximate LOO retraining scheme with the proximity term added to the downweighted objective.
B.3 Proximal Bregman Response Function Derivation
As opposed to the derivation from Appendix B.1, we consider computing influence functions on parameters θ s that have not necessarily converged.
When the parameters have not fully converged, a warm-start retraining with the downweighted objective defined in Eqn.
11 would simply minimize the cost function in the first term and reflect the effect of training longer, rather than the effect of removing a training example.
Let z =
( x , t ) ∈ D train be a training example we are interested in downweighting.
We assume that the loss function is convex as a function of the network outputs which hold for commonly used loss functions.
We replace the cost function in the downweighted objective with a term that penalizes mismatch to the predictions made by the current parameters θ s and define the proximal Bregman downweighted objective as:
[Equation]
where D L ( i ) ( · , · ) is the Bregman divergence defined as: [Equation]
[Equation]
Because of our convexity assumption, the Bregman divergence term in Eqn.
28 is non-negative and is [-
-]0 when θ = θ s .
Hence, the parameters θ s is optimal at ϵ = 0 on the proximal Bregman downweighted objective although it is not optimal on the cost function.
Accordingly, we define the proximal Bregman response function (PBRF) as follows:
[Equation]
where we assume that the proximal Bregman downweighted objective is strongly convex at ϵ 0 and the solution to the Bregman downweighted objective is unique.
Letting ϵ 0 = 0 , the Bregman response function satisfies r b − z , damp ( ϵ 0 )
=
θ s and:
[Equation]
31) for all downweighting factor ϵ in some neighborhood of ϵ 0 .
By taking the derivative with respect to the downweighting factor ϵ , we get:
[Equation]
[Equation]
For linear models, where the parameter-output Jacobian is constant, the Jacobian of the response function at ϵ 0 can further be expressed as:
[Equation]
where we used these two equalities:
[Equation]
Rearranging the terms in Eqn.
[Equation] the Jacobian of the PBRF at ϵ 0 can be expressed as:
[Equation]
Note that both Hessian and gradient are computed on the final parameters θ s instead of the optimal parameters θ ⋆ .
Hence, influence functions at the non-converged parameters θ s (with a damping) can be seen as an approximation to the PBRF rather than an approximation to LOO retraining.
B.4 Linearized Proximal Bregman Response Function Derivation
We show that the linearized proximal Bregman response function (PBRF) is equivalent to the influence estimation with the Gauss-Newton Hessian approximation and a damping term λ > 0 . Let θ
s ∈ R d be a possibly non-converged learned parameters, z
= ( x , t ) ∈ D train be a training example we want to downweight [Equation] and λ > 0 be a damping term.
Recall that the linearized PBRF is defined as:
[Equation]
where L quad ( · , · ) and f lin ( θ , x ) are defined as:
[Equation]
Here, y s is the prediction of the network parameterized by θ s and J y
( i ) θ s is the parameter-output
Jacobian of the i -th training example.
We further assume that the loss function is convex as a function of the network outputs which hold for commonly used loss functions.
The first Bregman divergence term in Eqn.
38 can be expressed as:
[Equation]
Now, taking the gradient of linearized PBRF objective with respect to the parameters θ and setting it equal to 0 , we get:
[Equation]
where H s y is the Hessian of the loss with respect to the network outputs evaluated at y s .
Rearranging the terms [Equation] we get:
[Equation]
(46) where θ ⋆ lin, PBRF is the optimal solution to the linearized PBRF objective.
Note that the Gauss-Newton
Hessian (GNH) G s =
J ⊤ y
θ s H s
y J y
θ s is positive semidefinite (assuming that the loss function is convex as a function of network outputs) and an addition of damping term guarantees the invertibility.
Therefore, the optimal solution to Eqn. 46 is equivalent to the influence estimation with the GNH approximation and a damping term λ .
C Experimental Details C.1 Computing Environment
All experiments were implemented using the PyTorch [Paszke et al., 2019] and JAX [Bradbury et al., 2018]
frameworks and we ran all experiments on NVIDIA P100 GPUs.
C.2 Experiment Set-up
In all experiments, we first trained the base network (initialized with some θ 0 ) with the entire dataset for K epochs to obtain the base parameters θ s .
We then select 20 random data points z t ∈ D train from the training dataset and computed the additional parameters for each t as follows:
Cold optimum.
We retrained the network for K + K / 2 epochs with the initialization θ 0 .
To minimize the effect of stochasticity in stochastic gradient-based optimizer, we further used the same batch order used for training the base network for the first K epochs.
Warm optimum.
We retrained the network for K / 2 epochs with the initialization θ s . 3.
Proximal warm optimum.
We retrained the network for K / 2 epochs with the initialization θ s using the objective defined in Eqn. 7. 4.
Proximal Bregman warm optimum.
We retrained the network for K / 2 epochs with the initialization θ s using the objective defined in Eqn. 8. 5.
Linearized proximal Bregman warm optimum.
We retrained the network for K / 2 epochs with the initialization θ s using the objective defined in Eqn. Influence estimation. We used the LiSSA algorithm on the base parameters θ s with the Gauss-Newton Hessian approximation (Eqn. 6).
The “warm-start gap” refers to the discrepancy between cold-start and warm-start optima.
The “proximity gap” refers to the discrepancy between warm-start and proximal warm-start optima.
The “non-convergence gap” denotes the discrepancy between proximal warm-start and proximal Bregman warm-start optima.
The “linearization error” represents the discrepancy between proximal Bregman warm-start and linearized proximal Bregman warm-start optima.
Lastly, the “solver error” denotes the discrepancy between linearized proximal Bregman warm optimum and influence estimation with the LiSSA algorithm.
We treated the scaling in the LiSSA algorithm as a separate hyperparameter [Koh and Liang, 2017] and tuned the scaling in the range {10, 25, 50, 100, 150, 200, 250, 300, 400, 500} so that the algorithm converges.
C.3 Influence Misalignment Decomposition
Logistic Regression.
We used Cancer and Diabetes classification datasets from the UCI collection [Dua and Graff, 2017].
In training, we normalized the input features to have zero mean and unit variance.
We trained the model using L-BFGS with L 2 regularization of 0.01 and damping term of λ = 0 .
Multilayer Perceptron.
For regression experiments, we used 2-hidden layer MLP with 128 hidden units.
For both Concrete and Energy datasets, we normalized the input features and targets to have a zero mean and unit variance.
For classification experiments with 10% of MNIST [Deng, 2012] and FashionMNIST [Xiao et al., 2017] datasets, we used 2-hidden layer MLP with a hidden unit dimension of 1024.
For both regression and classification experiments, we used a batch size ofand trained the base network for 1000 epochs using SGD.
While we did not use any L 2 regularization, we set the damping strength to λ = 0 . 001 .
We conducted the hyperparameter searches over the learning rates for the base model, making choices based on the final validation loss.
We swept over the learning rates {1.0, 0.3, 0.1, 0.03, 0.01, 0.003, 0.001}.
We used a learning rate decayed by a factor of 10 for computing PBRF and linearized PBRF.
We set the recursion depth to 5000 and the number of repeat to 5 for the LiSSA algorithm.
Autoencoder.
We used the same experimental set-up from Martens and Grosse [2015].
The loss function was the binary cross-entropy and the L 2 regularization with strength 5 · 10 − 5 was added to the cost function.
The layer widths for the autoencoder were [784, 1000, 500, 250, 30, 250, 500, 1000, 784] and we used sigmoid activation functions.
We trained the network for 1000 epochs on the full MNIST dataset with SGDm (SGD with momentum) and set the batch size to 1024.
The damping term was set to λ = 0 .
We conducted the hyperparameter searches for the base model making choices based on the final validation loss.
We kept the momentum to 0 .
9 and swept over the learning rates 1, 0.3, 0.1, 0.03, 0.01, 0.003, 0.001.
We used a learning rate decayed by a factor of 10 for computing PBRF and linearized
PBRF.
We set the recursion depth to 10000 and the number of repeat to 5 for the LiSSA algorithm.
Convolution Neural Networks.
We trained LeNet [Lecun et al., 1998], AlexNet [Krizhevsky et al., 2012], VGG13 Simonyan and {+Zisserman [2014], and ResNet-
 [He et al., 2015]+} on 10% of MNIST dataset and the full {+CIFAR
 [Krizhevsky, 2009]+} dataset.
For the MNIST experiment, we kept the learning rate fixed, while for CIFAR10 experiment, we decayed the initial learning rate by a factor of 5 at epochs 60, 120, 160.
We used L 2 regularization of 5 · 10 − 4 and a damping factor of λ = 0 . 001 .
For both datasets, we trained the base network for 200 epochs with the batch size of 128.
We used 10% of the MNIST dataset for the test loss correlation experiments in Table 2 and computed the approximated test loss with Eqn. 20 on randomly selected test examples.
We conducted the hyperparameter searches for the base model making choices based on the final validation accuracy.
We fixed the momentum to 0 .
9 and swept over the initial learning rates {1.0, 0.3, 0.1, 0.03, 0.01, 0.003, 0.001}.
We used a learning rate decayed by a factor of 10 for computing PBRF and linearized
PBRF.
We set the recursion depth to 10000 and the number of repeat to 5 for the LiSSA algorithm.
Transformer.
We trained a 2-layer Transformer language model on the Penn Treebank (PTB) dataset [Marcus et al., 1993].
The number of hidden dimensions was set to 256 and the number of attention heads was set to 2 .
We trained the model with Adam for 10 epochs.
We set the batch size to 20 and a damping term of λ = 0 .
We conducted the hyperparameter searches for the base model making choices based on the final validation perplexity.
We swept over the learning rates {0.03, 0.01, [Table] 0.003, 0.001, 0.0003, 0.0001} for training the base model and set the recursion depth to 5000 and the number of repeat to 5 for the LiSSA algorithm.
C.4 Factors in Influence Misalignment
The experiment in Section 6.2 was conducted using 10% of the MNIST dataset.
We trained 2-hidden layer MLP composed of 1024 hidden units for 1000 epochs using SGD.
We used the batch size of 128 and conducted the hyperparameter searches for each model making choices based on the final validation loss.
We swept over the learning rates {1.0, 0.3, 0.1, 0.03, 0.01, 0.003, 0.001, 0.0003, 0.0001}.
To see how the gaps and errors change as we increase the width of the network, we repeated the experiments with all widths in a set {16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192} while keeping the depth fixed to 2.
To see the effect of increase in the depth of the network, we fixed the width to 1024 and computed gaps and errors with depth {1, 3, 5, 7, 9, 11, 13, 15}.
While keeping all configurations the same (2-hidden layer MLP with 1024 hidden units), we computed the decomposition by changing the total number of epochs to train the base model in the range {100, 500, 1000, 3000, 5000, 7000, 9000}, changing the strength of the weight decay {0.5, 0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005, 0.00001}, and changing the strength of the damping term in the range {0.5, 0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005, 0.00001, 0.000005, 0.000001}.
Lastly, we modified the downweighted objective to remove a group of training examples (rather than a single training example) and altered the percentage we remove the training dataset in range {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}.
D Additional Results D.1 Influence Functions without the Gauss-Newton Hessian Approximation
In all experiments, we computed influence functions using the Gauss-Newton Hessian (GNH) approximation.
As the previous error analysis was conducted without the GNH approximation [Basu et al., 2020a], we repeated a subset of our experiments without the GNH approximation (using the Hessian matrix).
The results are summarized in Table 4.
As the warm-start gap, proximity gap, non-convergence gap, and linearization error do not depend on the way influence estimates are computed, these numerical values are identical to the results in Table 5.
However, as the linearized PBRF optimum is equivalent to influence estimations with the GNH approximation, the solver error slightly increased when we computed influence functions with the Hessian for all datasets.
Nevertheless, the solver error is still significantly lower than other decomposition terms.
Furthermore, we investigated how the test loss difference on randomly selected test examples approximated by influence functions correlates with the actual value computed using cold-start retraining, warm-start retraining, and the PBRF.
Table 6 shows the correlations with influence approximations using the GNH approximation.
While influence estimates do not accurately predict the effect of retraining the model, they closely align the values obtained by the PBRF.
Moreover, we conducted the same experiment with influence approximations without the GNH approximation (using the Hessian matrix) and show the results in Table 7.
Similar to the previous results, the test loss [Table] [Table] differences predicted by influence functions align with the value obtained by the PBRF while failing to capture the effect of retraining the model for both regression and classification datasets.
Hence, in both settings, the PBRF better captures the behavior of influence functions than LOO retraining.
D.2 Two-Stage LOO Retraining: An Alternative Method for PBRF computation
As discussed in Section 4.3, when the network has not fully converged, the LOO retraining simply reflects the effect of training the network for a longer period of time, which does not correctly reflect the effect of removing a data point from the training set (a question that influence functions aim to answer). This difference yields the discrepancy between LOO retraining and influence estimates. We further introduced the PBRF objective which penalizes the LOO retraining with both functionand weight-space discrepancy terms and showed that the PBRF better reflects the question influence functions try to answer.
Here, we discuss an alternative method to capture the behaviour of influence functions by performing two separate LOO retrainings, one with the full dataset and the other one with the removed data point.
The first LOO retraining reflects the effect of training longer with the full dataset and the second LOO retraining reflects the effect of training longer with a removed data point.
The difference between LOO retrainings in parameter space can approximate the effect of removing a data point by neglecting the effect of longer training.
Such difference can be added to the current parameters θ s .
This two-stage LOO retraining method can be interpreted as removing O (1) terms when the network has not fully converged.
We repeat the experiment from Appendix D.1 with the proposed two-stage LOO retraining.
The results are shown in Table 8.
While the two-stage LOO retraining shows a worse correlation with
[Table]
[Table]
influence estimates compared to the PBRF objective, it has a significantly higher correlation with influence estimates compared to the warm-start retraining.
Although the two-stage LOO retraining requires retraining the network twice, it can be seen as a better reflection to what influence functions compute in neural networks
[Table]
D.3 Mislabelled Examples Detection
[Figure]
F r a c t i o n o f ﬂ i p s ﬁ x e d
Random Inﬂuence PBRF While the PBRF may not necessarily align with LOO retraining because of warm-start, proximity, and non-convergence gaps, the motivating use cases for influence functions typically do not rely on exact LOO retraining.
Hence, the PBRF can be used instead of LOO retraining for many tasks, such as identifying influential or mislabelled examples.
We conducted an additional experiment to verify that the PBRF (and influence function) is still helpful in detecting mislabelled training data points.
We used 10% of the MNIST dataset and randomly corrupted 10% of the training examples by assigning random labels.
Then, we simulated the scenario where we manually inspect a fraction of training examples, correcting them if they were mislabelled.
We trained 2-hidden layer MLP with 1024 hidden units using SGD with a batch size of 128.
We used the damping term of λ = 0 .
001 for
PBRF and influence functions.
For each PBRF and influence estimation, we measured the influence of removing a single training training example on the total training loss (self-influence scores [Koh and Liang, 2017, Khanna et al., 2019]) to identify the top influential training examples.
We prioritized inspecting training examples that obtained high scores generated by the PBRF and influence functions.
The results are summarized in Figure 6.
Using the self-influence score generated by the PBRF, it is possible to detect over 80% of the mislabelled training examples by only examining 20% of training examples.
Similarly, as influence functions closely align with the PBRF, influence functions can provide an efficient tool to help fix mislabelled training examples.
Both PBRF and influence functions outperform the baseline of randomly selecting a subset of training examples to inspect if there are any mislabelled training examples.
E Influence Misalignment Decomposition Table
In Table 5, we present the numerical results shown in Figure 4.
G Efficient iHVP Computation
One of the major challenges in applying influence functions to neural networks (Eqn. 4) is that they involve the computation of an inverse-Hessian vector product ( iHVP ).
However, for large networks, computing iHVP s exactly via storing and inverting the Hessian is intractable.
To circumvent this, Koh and Liang [2017] consider alternative methods for approximating iHVP s, namely, the method of conjugate gradients ( CG ) [Martens et al., 2010] and the Linear time Stochastic Second-Order Algorithm ( LiSSA ) [Agarwal et al., 2016].
Conjugate Gradient.
Given a positive-definite damped Hessian ∇ 2 θ J
( θ s )
+ λ I (where θ s are the potentially-suboptimal parameters at which the Hessian is taken and λ > 0 is a damping factor) and vector v ∈ R d , CG arrives at the iHVP by solving an equivalent convex quadratic optimization problem:
[Equation]
The CG algorithm starts with an initial guess v 0 ∈ R d and iteratively updates it, with the bottleneck at each step being an O ( Nd ) Hessian-vector product.
Although an exact solution is only guaranteed after d CG iterations, in practice, Koh and Liang [2017] use truncated CG with fewer iterations and achieve a sufficiently close approximation.
LiSSA.
The LiSSA algorithm approximates the iHVP using a truncated Neumann series.
Given a positive-definite damped Hessian ∇ 2 θ J
( θ s )
+ λ I and vector v ∈ R d , we have:
[Equation]
which becomes exact as the recursion depth T approaches ∞ .
Here, σ > 0 is a scaling hyperparameter that is chosen sufficiently large to ensure convergence of the series.
Eqn. 48 can be recursively computed over T iterations, with each step requiring an O ( Nd ) Hessian-vector product.
In practice, the computation is further optimized by estimating ∇ 2 θ J ( θ s ) using a randomly-sampled batch
B ⊆ D train of size |B| ≪ N , so that the Hessian-vector product is reduced to O ( d ) cost.
Then, to accommodate for the added stochasticity, the iHVP is estimated by averaging Eqn. 48 over R trial repeats.
Hence, the LiSSA algorithm estimates:
[Equation]
where J ( r,t ) ( θ s ) is the average loss over the ( r, t ) -th sampled batch of data.
The s test trick.
Finally, we note that another simple trick can be made when using influence functions to predict the change in test loss at a particular test point (Eqn. 20).
It is often the case that we wish to compute the influence scores for the pairwise interactions of the entire training dataset on the loss at a comparatively smaller number N test ≪ N of test points.
Since the Hessian is symmetric, the order of multiplication in the second term of Eqn.
20 can be permuted as follows:
[Equation]
where v = ∇ θ L ( f ( θ s , x ) , t ) and v test = ∇ θ L ( f ( θ s , x test ) , t test ) .
This means that we can precompute s test =
( ∇ 2 θ J
( θ s )
+ λ I ) − 1 v test over all N test test points of interest, and then cheaply compute influence scores over all N training points by simply taking dot products of the form v ⊤ s test .
We refer readers to Koh and Liang [2017] for details.