I NFLUENCE F UNCTIONS IN D EEP L EARNING A RE F RAGILE
Anonymous authors
Paper under double-blind review
A BSTRACT
Inﬂuence functions approximate the effect of training samples in test-time predictions and have a wide variety of applications in machine learning interpretability and uncertainty estimation.
A commonly-used (ﬁrst-order) inﬂuence function can be implemented efﬁciently as a post-hoc method requiring access only to the gradients and Hessian of the model.
For linear models, inﬂuence functions are well-deﬁned due to the convexity of the underlying loss function and are generally accurate even across difﬁcult settings where model changes are fairly large such as estimating group inﬂuences.
Inﬂuence functions, however, are not wellunderstood in the context of deep learning with non-convex loss functions.
In this paper, we provide a comprehensive and large-scale empirical study of successes and failures of inﬂuence functions in neural network models trained on datasets such as Iris, MNIST, CIFAR-10 and ImageNet.
Through our extensive experiments, we show that the network architecture, its depth and width, as well as the extent of model parameterization and regularization techniques have strong effects in the accuracy of inﬂuence functions.
In particular, we ﬁnd that (i) inﬂuence estimates are fairly accurate for shallow networks, while for deeper networks the estimates are often erroneous; (ii) for certain network architectures and datasets, training with weight-decay regularization is important to get high-quality inﬂuence estimates; and (iii) the accuracy of inﬂuence estimates can vary signiﬁcantly depending on the examined test points.
These results suggest that in general inﬂuence functions in deep learning are fragile and call for developing improved inﬂuence estimation methods to mitigate these issues in non-convex setups.
I NTRODUCTION
In machine learning, inﬂuence functions (Cook & Weisberg, 1980) can be used to estimate the change in model parameters when the empirical weight distribution of the training samples is perturbed inﬁnitesimally.
This approximation is cheaper to compute compared to the expensive process of repeatedly re-training the model to retrieve the exact parameter changes.
Inﬂuence functions could thus be used to understand the effect of removing an individual training point (or, groups of training samples) on the model predictions at the test-time.
Leveraging a ﬁrst-order Taylor’s approximation of the loss function, (Koh & Liang, 2017) has shown that a (ﬁrst-order) inﬂuence function, computed using the gradient and the Hessian of the loss function, can be useful to interpret machine learning models, ﬁx mislabelled training samples and create data poisoning attacks.
Inﬂuence functions are in general well-deﬁned and studied for models such as logistic regression (Koh & Liang, 2017), where the underlying loss-function is convex.
For convex loss functions, inﬂuence functions are also accurate even when the model perturbations are fairly large (e.g. in the group inﬂuence case (Koh et al., 2019b;
Basu et al., 2019)).
However, when the convexity assumption of the underlying loss function is violated, which is the case in deep learning, the behaviour of inﬂuence functions is not well understood and is still an open area of research.
With recent advances in computer vision (Szeliski, 2010), natural language processing (Sebastiani, 2002), high-stakes applications such as medicine (Lundervold & Lundervold, 2018), it has become particularly important to interpret deep model predictions.
This makes it critical to understand inﬂuence functions in the context of deep learning, which is the main focus of our paper.
Despite their non-convexity, it is sometimes believed that inﬂuence functions would work for deep networks.
The excellent work of (Koh & Liang, 2017) successfully demonstrated one example of inﬂuence estimation for a deep network, a small (2600 parameters), "all-convolutional" network (Springenberg et al., 2015).
To the best of our knowledge, this is the one of the few cases for deep networks where inﬂuence estimation has been shown to work.
A question of key importance to practitioners then arises: for what other classes of deep networks does inﬂuence estimation work?
In this work, we provide a comprehensive study of this question and ﬁnd a pessimistic answer: inﬂuence estimation is quite fragile for a variety of deep networks.
In the case of deep networks, several factors might have an impact on inﬂuence estimates: (i) due to non-convexity of the loss function, different initializations of the perturbed model can lead to signiﬁcantly different model parameters (with approximately similar loss values); (ii) even if the initialization of the model is ﬁxed, the curvature values of the network (i.e. eigenvalues of the Hessian matrix) at optimal model parameters might be very large in very deep networks, leading to a substantial Taylor’s approximation error of the loss function and thus resulting in poor inﬂuence estimates; (iii) for large neural networks, computing the exact inverse-Hessian Vector product, required in computation of inﬂuence estimates, can be computationally very expensive.
Thus, one needs to use approximate inverse-Hessian Vector product techniques which might be erroneous; resulting in low quality inﬂuence estimates; and ﬁnally (iv) different architectures can have different loss landscape geometries near the optimal model parameters, leading to varying inﬂuence estimates.
In this paper, we study aforementioned issues of using inﬂuence functions in deep learning through an extensive experimental study on progressively-growing complex models and datasets.
We ﬁrst start our analysis with a case study of a small neural network for the Iris dataset where the exact
Hessian matrix can be computed.
We then progressively increase the complexity of the network and analyse a CNN architecture (depth of 6) trained on 10% of MNIST dataset, similar to (Koh & Liang, 2017).
Next, we evaluate the accuracy of inﬂuence estimates for more complex deep architectures (e.g. ResNets) trained on MNIST and CIFAR-10.
Finally, we compute inﬂuence estimates on the
ImageNet dataset using ResNet-50.
We make the following observations through our analysis:
We ﬁnd that the network depth and width have a strong impact on inﬂuence estimates.
In particular, we show that inﬂuence estimates are fairly accurate when the network is shallow, while for deeper models, inﬂuence estimates are often erroneous.
We attribute this partially to the increasing curvature values of the network as the depth increases.
• We observe that the weight decay regularization is important to obtain high quality inﬂuence estimates in certain architectures and datasets.
• We show that the inverse-Hessian Vector product approximation techniques such as stochastic estimation (Agarwal et al., 2016) are erroneous, especially when the network is deep.
This can contribute to the low quality of inﬂuence estimates in deep models.
• We observe that the choice of test-point has a substantial impact on the quality of inﬂuence estimates, across different datasets and architectures.
• In very large-scale datasets such as ImageNet, we have found that even ground-truth inﬂuence estimates (obtained by leave-one-out re-training) can be inaccurate and noisy partially due to the model’s training and convergence.
These results highlight sensitivity of current inﬂuence functions in deep learning and call for developing robust inﬂuence estimators to be used in large-scale machine learning applications.
R ELATED W ORKS
Inﬂuence functions are primarily used to identify important training samples for test-time predictions and debug machine learning models (Koh & Liang, 2017).
Similar to inﬂuence functions, (Chaudhuri & Mykland, 1993) tackles the problem of approximating a dataset using a subset of the dataset.
In recent times, there is an increase in the applications of inﬂuence functions for tasks other than interpretability.
For e.g.(Schulam & Saria, 2019) has used inﬂuence functions to audit the reliability of test-predictions.
In NLP, inﬂuence functions have been used to detect biases in word-embeddings (Brunet et al., 2018)
whereas in the domain of ML security, inﬂuence functions have been shown to be effective in crafting stronger data-poisoning attacks (Koh et al., 2019a).
Inﬂuence functions are also effective in the identiﬁcation of important training groups (rather than an individual sample)
(Basu et al., 2019; Koh et al., 2019b).
Prior theoretical work (Giordano et al., 2018; 2019) have focused on quantifying ﬁnite sample error-bounds for inﬂuence estimates when compared to the ground-truth re-training procedures.
Recently, alternative methods to ﬁnd inﬂuential samples in deep networks have been proposed.
In (Yeh et al., 2018), test-time predictions are explained by a kernel function evaluated at the training samples.
Inﬂuential training examples can also be obtained by tracking the change in loss for a test-prediction through model-checkpoints, which are stored during the training time (Pruthi et al., 2020).
While these alternative methods (Yeh et al., 2018; Pruthi et al., 2020) work well for deep networks in interpreting model predictions, they lack the “jackknife" like ability of inﬂuence functions which makes it useful in multiple applications other than interpretability (e.g. uncertainty estimation).
B ASICS OF I NFLUENCE F UNCTION
Consider h to be a function parameterized by θ which maps from an input feature space X to an output space denoted by Y .
The training samples are denoted by the set S
= { z i : ( x i , y i ) } ni =1 , while the loss function is represented by (cid:96) ( h θ ( z )) for a particular training example z .
The standard empirical risk minimization solves the following optimization problem:
[Equation]
Up-weighting a training example z by an inﬁnitesimal amount (cid:15) leads to a new set of model parameters denoted by θ (cid:15) { z } .
This set of new model parameters θ (cid:15) { z } is obtained by solving:
[Equation]
Removing a training point z is similar to up-weighting its corresponding weight by (cid:15) =
− 1 /n in
Equation(2).
The main idea used by (Koh & Liang, 2017) is to approximate θ (cid:15) { z } by the ﬁrst-order
Taylor series expansion around the optimal model parameters represented by θ ∗ , which leads to:
[Equation]
where H θ ∗ represents the Hessian with respect to model parameters θ ∗ .
Following the classical result of (Cook & Weisberg, 1980), the change in the model parameters ( ∆ θ = θ (cid:15) { z } − θ ∗ ) on upweighting the training example z can be approximated by the inﬂuence function ( I ( z ) ) as follows:
[Equation]
The change in the loss value for a particular test point z t when a training point z is up-weighted can be approximated as a closed form expression by the chain rule (Koh & Liang, 2017):
[Equation]
I ( z, z t ) /n is approximately the change in the loss for the test-sample z t when a training sample z is removed from the training set.
This result is, however, based on the assumption that the underlying loss function is strictly convex in the model parameters θ and the Hessian H θ ∗ is a positive-deﬁnite matrix (Koh & Liang, 2017).
For large models, inverting the exact Hessian H θ ∗ is expensive.
In such cases, the inverse-Hessian Vector product can be computed efﬁciently with a combination of
Hessian-vector product (Pearlmutter, 1994) and optimization techniques (see Appendix for details).
W HAT C AN G O W RONG FOR I NFLUENCE F UNCTIONS I N D EEP L EARNING ?
First-order inﬂuence functions (Koh & Liang, 2017) assume that the underlying loss function is convex and the change in model parameters is small when the empirical weight distribution of the training data is inﬁnitesimally perturbed.
In essence, this denotes the Taylor’s gap in Equation (3) to be small for an accurate inﬂuence estimate.
However in the case of non-convex loss functions, this assumption is not generally true.
Empirically, we ﬁnd that the Taylor’s gap is strongly affected by common hyper-parameters for deep networks.
For example, in Fig.
(1)-(a,b), we ﬁnd that for networks trained without a weight-decay regularization on Iris, the Taylor’s gap is large resulting in low quality inﬂuence estimates.
In a similar vein, when the network depth and width is considerably large (i.e. the over-parameterized regime), the Taylor’s gap increases and substantially degrades the quality of inﬂuence estimates (Fig.
Empirically this increase in Taylor’s gap strongly correlates with the curvature values of the loss function evaluated at the optimal model parameters as observed in Fig.
(2-(b)).
Further complications may arise for larger models, where inﬂuence estimations in such settings require an additional approximation to compute the inverse-Hessian vector product.
Nonetheless, we observe in Fig.
(2)-(a), that on Iris this approximation has only a marginal impact on the inﬂuence estimation.
These results show that that network architecture, hyper-parameters, and loss curvatures are important factors for proper inﬂuence estimations.
In the next section, we discuss these issues in details through controlled experiments on datasets and models of increasing complexity.
E XPERIMENTS
Datasets : We ﬁrst study the behaviour of inﬂuence functions in a small Iris dataset (Anderson, 1936), where the exact Hessian can be computed.
Further, we progressively increase the complexity of the model and datasets: we use small MNIST (Koh & Liang, 2017) to evaluate the accuracy of inﬂuence functions in a small CNN architecture with a depth of 6.
Next, we study inﬂuence functions on modern deep architectures trained on the standard MNIST (LeCun et al., 1998) and
CIFAR-10 (Krizhevsky et al., 2000) datasets.
Finally, to understand how inﬂuence functions scale to large datasets, we use ImageNet (Deng et al., 2009) to compute the inﬂuence estimates.
Evaluation Metrics : We evaluate the accuracy of inﬂuence estimates at a given test point z t using both Pearson (Kirch, 2008) and Spearman rank-order correlation (Spearman, 1904) with the ground-truth (obtained by re-training the model) across a set of training points.
Most of the existing interpretability methods desire that inﬂuential examples are ranked in the correct order of their importance (Ghorbani et al., 2017).
Therefore, to evaluate the accuracy of inﬂuence estimates,
Spearman correlation is often a better choice.
5. U NDERSTANDING I NFLUENCE F UNCTIONS WHEN THE E XACT H ESSIAN C AN BE C OMPUTED
Setup:
Computing inﬂuence estimates with the exact Hessian has certain advantages in our study: a) it bypasses inverse-Hessian Vector product approximation techniques which induce errors in computing inﬂuence estimates.
Thus, we can compare inﬂuence estimates computed with exact vs. approximate inverse-Hessian Vector products to quantify this type of error; b) The deviation of the parameters computed with the inﬂuence function from the exact parameters can be computed exactly.
This information can be useful to further quantify the error incurred by (ﬁrst-order) inﬂuence estimates in the non-convex setup.
However, computations of the exact Hessian matrix and its inverse are only computationally feasible for models with small number of parameters.
Thus, we use the Iris dataset along with a small feed-forward neural network to analyse the behaviour of inﬂuence function computed with the exact Hessian in a non-convex setup.
We train models to convergence for 60k iterations with full-batch gradient descent.
To obtain the ground-truth estimates, we retrain the models for 7.5k steps, starting from the optimal model parameters.
For our analysis, we choose the test-point with the maximum loss and evaluate the accuracy of inﬂuence estimates with the ground-truth amongst of the top 16 .
6% of the training points.
Through our experiments with the exact Hessian, we answer some relevant questions related to how properties of the network such as depth, width and regularizers (e.g. weight-decay) affect the inﬂuence estimates.
The Effect of Weight-Decay : One of the simpler and common regularization techniques used to train neural networks is weight-decay regularization.
In particular, a term λ (cid:107) θ (cid:107) 22 , penalizing the scaled norm of the model parameters is added to the objective function, during training, where λ is a hyperparameter which needs to be tuned.
We train a simple feed-forward network 1 with and without weight-decay regularization.
For the network trained with weight-decay, we observe a Spearman correlation of 0.97 between the inﬂuence estimates and the ground-truth estimates.
In comparison, for the network trained without a weight-decay regularization, the Spearman correlation estimates decrease to 0.508.
In this case, we notice that the Hessian matrix is singular, thus a damping factor of 0.001 is added to the Hessian matrix, to make it invertible.
To further understand the reason for this decrease in the quality of inﬂuence estimates, we compare the following metric across all training examples: a) Norm of the model parameter changes computed by re-training; b) Norm of the model parameter changes computed using the inﬂuence function (i.e. (cid:107) H − 1 θ ∗ ∇ (cid:96) ( z i ) (cid:107) 2 ∀
i ∈
[1 , n ] ) (Fig.
1-(a,b)).
We observe that when the network is trained without weight-decay, changes in model parameters computed with the inﬂuence function have a substantially larger deviation from those computed using re-training.
This essentially suggests that the gap in Taylor expansion, using (ﬁrstorder) inﬂuence estimates is large, when the model is trained without weight-decay.
We observe similar results with smooth activation functions such as tanh (see the Appendix for details).
The Effect Of Network Depth : From Fig.
1-(c), we see that network depth has a dramatic effect on the quality of inﬂuence estimates.
For example, when the depth of the network is increased to 8, we notice a considerable decrease in the Spearman correlation estimates.
To further our understanding about the decrease in the quality of inﬂuence estimates when the network is deeper, we compute the gap in the approximation between the ground-truth parameter changes (computed by re-training) and the approximate parameter changes (computed using the inﬂuence function).
To quantify the error gap, we compute the Spearman correlation estimates between the norm of true and approximate parameter changes across the top 16 .
6% of the inﬂuential examples.
We ﬁnd that with increasing depth, the Spearman correlation estimates between the norm of the true and approximate parameter changes decrease.
From Fig.
2-(c), we see that the approximation error gap is particularly large when the depth of the network is more than 5.
We also notice a consistent increase in the curvature of the loss function (Fig. 2-(b)), as the network becomes deeper.
This possibly suggests that the curvature information of the network can be an upper bound in the approximation error gap between the true parameters and the ones computed using the inﬂuence function.
Even in case of non-smooth activation functions like ReLU, we have a similar observation.
(see the Appendix for more details).
The Effect Of Network Width : To see the effect of the network width on the quality of inﬂuence estimates, we evaluate the inﬂuence estimates for a feed-forward network of constant depth, by progressively increasing its width.
From Fig.
1-(d), we observe that with an increase in network width, the Spearman correlation decreases consistently.
For example, we ﬁnd that the Spearman correlation decreases from 0.82 to 0.56, when the width of the network is increased from 8 to 50.
This observation suggests that over-parameterizing a network by increasing its width has a strong impact in the quality of inﬂuence estimates.
The Effect of Stochastic Estimation on inverse-Hessian Vector Product : For large deep networks, the inverse-Hessian Vector product is computed using stochastic estimation(Agarwal et al., 2016), as the exact Hessian matrix cannot be computed and inverted.
To understand the effectiveness of stochastic approximation, we compute the inﬂuence estimates with both the exact Hessian and stochastic estimation.
We observe that across different network depths, the inﬂuence estimates computed with stochastic estimation have a marginally lower Spearman correlation when compared to the ones computed with the exact Hessian.
From Fig.
2-(a), we ﬁnd that the error in the approximation is more, when the network is deeper.
5. U NDERSTANDING I NFLUENCE F UNCTIONS IN S HALLOW CNN A RCHITECTURES
Setup :
In this section, we perform a case study using a CNN architecture 2 on the small MNIST dataset (i.e. 10% of MNIST); a similar setup used in (Koh & Liang, 2017).
To assess the accuracy of inﬂuence estimates, we select a set of test-points with high test-losses computed at the optimal model parameters.
For each of the test points, we select 100 training samples with the highest inﬂuence scores and compute the ground-truth inﬂuence by re-training the model.
We also select 100 training points with inﬂuence scores at the 30 th percentile of the entire inﬂuence score distribution.
These training points have low inﬂuence scores and a lower variance in their scores when compared to the top inﬂuential points.
The model is trained with and without weight-decay regularization.
When trained with a weight-decay and evaluated based on the top inﬂuential points, we ﬁnd that the correlation estimates are consistently signiﬁcant (Fig.
3-(a)).
This is consistent with the results reported in (Koh & Liang, 2017).
However, when the evaluation is done with the set of training samples at the 30 th percentile of the inﬂuence score distribution, the correlation estimates decrease signiﬁcantly (Fig. 3-(b)).
This shows that inﬂuence estimates of only the top inﬂuential points are precise when compared to ground-truth re-trainings.
Furthermore, without the weight-decay regularization, inﬂuence estimates in both cases are poor across all the test-points (Fig.
3-(a,b)).
To further understand the impact of weight-decay on inﬂuence estimates, we train the network with different weight-decay regularization factors.
From Fig.
3-(c), we see that the selection of weightdecay factor is important in getting high-quality inﬂuence estimates.
For this speciﬁc CNN architecture, we notice that the correlations start decreasing when the weight-decay factor is greater than 0.01.
Moreover, from Fig.
3-(a,b), we ﬁnd that the selection of test-point also has a strong impact on the quality of inﬂuence estimates.
For example, when the network is trained with weight-decay and the inﬂuence estimates are computed for top inﬂuential training points, we notice that the Spearman correlation estimates range from 0.92 to 0.38 across different test-points and have a high variance.
[Table] These results show that despite some successful applications of inﬂuence functions in this nonconvex setup, as reported in (Koh & Liang, 2017), their performances are very sensitive to hyperparameters of the experiment as well as to the training procedure.
In the next two sections, we assess the quality of inﬂuence estimates on more complex architectures and datasets including MNIST,
CIFAR-10 and ImageNet. In particular, we desire to understand, if the insights gained from experiments on smaller networks can be generalized to more complex networks and datasets.
5. U NDERSTANDING I NFLUENCE F UNCTIONS IN D EEP A RCHITECTURES
Setup :
In this section, we evaluate the accuracy of inﬂuence estimates using MNIST and CIFARdatasets across different network architectures including small CNN(Koh & Liang, 2017), LeNet (Lecun et al., 1998), ResNets (He et al., 2015), and VGGNets (Simonyan & Zisserman, 2015) 3 .
To compute inﬂuence estimates, we choose two test points for each architecture: a) the test-point with the highest loss, and b) the test-point at the 50 th percentile of the losses of all test points.
For each of these two test points, we select the top 40 inﬂuential training samples and compute the correlation of their inﬂuence estimates with the ground-truth estimates.
To compute the ground-truth inﬂuence estimates, we follow the strategy of (Koh & Liang, 2017), where we re-train the models from optimal parameters for 6% of the steps used for training the optimal model.
When the networks are trained with a weight-decay regularization, we use a constant weight-decay factor of 0.001 across all the architectures (see Appendix for more details).
Results On MNIST :
From Table 1, we observe that for the test-point with the highest loss, the inﬂuence estimates in the small CNN and LeNet architectures (trained with the weight-decay regularization) have high qualities.
These networks have 2 .
6 k and 44 k parameters, respectively, and are relatively smaller and less deep than the other networks used in our experimental setup.
As the depth of the network increases, we observe a consistent decrease in quality of inﬂuence estimates.
For the test-point with a loss at the 50 th percentile of test-point losses, we observe that inﬂuence estimates only in the small CNN architecture have good qualities.
Results On CIFAR-10 :
For CIFAR-10, across all architectures trained with the weight-decay regularization, we observe that the correlation estimates for the test-point with the highest loss are highly signiﬁcant.
For example, the correlation estimates are above 0.6 for a majority of the network architectures.
However, for the test-point evaluated at the 50 th percentile of the loss, the correlations decrease marginally across most of the architectures.
We ﬁnd that on CIFAR-10, even architectures trained without weight-decay regularization have highly signiﬁcant correlation estimates when evaluated with the test-point which incurs the highest loss.
In case of MNIST, we have found that in shallow networks, the inﬂuence estimates are fairly accurate while for deeper networks, the quality of inﬂuence estimates decrease.
For CIFAR-10, although the inﬂuence estimates are signiﬁcant, we found that the correlations are marginally lower in deeper networks such as ResNet-50.
The improved quality of inﬂuence estimates in CIFARcan be attributed to the fact that for a similar depth, architectures trained on CIFAR-10 are less over-parameterized compared to architectures trained on MNIST.
Note that, in Section 5.1, where the exact Hessian matrix can be computed, we observed that over-parameterization decreases the quality of inﬂuence estimates.
From Table(1), we also observed that the selection of test-point has a sizeable impact on the quality of inﬂuence estimates.
Furthermore, we noticed large variations in the quality of inﬂuence estimates across different architectures.
In general we found that inﬂuence estimates for small CNN and LeNet are reasonably accurate, while for ResNet-50, the quality of estimates decrease across both MNIST and CIFAR-10.
Precise reasons for these variations are difﬁcult to establish.
We hypothesize that it can be due to the following factors: (i) Different architectures trained on different datasets have contrasting characteristics of loss landscapes at the optimal parameters which can have an impact on inﬂuence estimates.
(ii)
The weight-decay factor may need to be set differently in various architectures, to obtain high quality inﬂuence estimates.
we train a ResNet-18 model with a weight-decay regularization factor of 5 e − 4 .
The inﬂuence estimates are then computed for test-points with the highest losses (Index: 6017, 2407, 9383) and testpoints around the 50 th percentile of the test loss (Index: 783, 7106) over multiple model initialisations.
Unlike in the case of MNIST and CIFAR-10, from
Fig. 4 we observe the correlation estimates to be of substantially poor quality. We provide additional visualizations of the inﬂuential training examples in the Appendix section. 5. I S S CALING I NFLUENCE E STIMATES T O I MAGE N ET P OSSIBLE ?
The application of inﬂuence functions to ImageNet scale models provides an appealing yet challenging opportunity.
It is appealing because, if successful, it opens a range of applications to large-scale image models, including interpretability, robustness, data poisoning, and uncertainty estimation.
It is challenging for a number of reasons.
Notable among these is the high computational cost of training and re-training, which limits the number of ground truth evaluations.
In addition, all of the previously discussed difﬁculties in inﬂuence estimations still remain, including (i) non-convexity of the loss, (ii) selection of scaling and damping hyperparameters in the stochastic estimation of the
Hessian, and (iii) the lack of convergence of the model parameters.
The scale of ImageNet raises additional questions about the feasibility of leave-one-out retraining as the ground truth estimator.
Given that there are 1.2M images in the training set, is it even possible that the removal of one image can signiﬁcantly alter the model?
In other words, we question whether or not reliable ground truth estimates may be obtained through leave-one-out re-training at this scale.
To illustrate this, we conduct an additional inﬂuence estimation on ImageNet.
After training an initial model to 92.302% top5 test accuracy, we select two test points at random, calculate inﬂuence over the entire training set, and then select the top 50 points by their inﬂuences as candidates for re-training.
We then use the re-training procedure suggested by (Koh & Liang, 2017), which starts leave-one-out re-training from the parameter set obtained after the initial training.
We re-train for an additional 2 epochs, approximately 5% of the original training time, and calculate the correlations.
We observe that for both test points, both Pearson and Spearman correlations are very low (less than 0 .
15 , see details in the Appendix).
In our experiments, we observe high variability among ground-truth estimates obtained by retraining the model (see the appendix for details).
We conjecture that this may be partially due to the fact that the original model has not be fully converged.
To study this, we train the original model with all training points for an additional 2 epochs and measure the change in the test loss.
We ﬁnd that the overall top5 test accuracy has improved slightly to 92.336 % (+0.034) and the loss for one of the considered test points has decreased by relatively a signiﬁcant amount of 0.679.
However, the loss for the other point has increased slightly by 0.066.
Such changes in loss values can therefore out-power the effect of leave-one-out re-training procedure.
Second, we calculate the 2 -norm of the weight gradients, which should be close to zero near an optimal point, and compare it to a standard pre-trained ImageNet ResNet-50 model as a baseline.
We ﬁnd these norms to be 20.18 and and 15.89, respectively, showing our model has similar weight gradient norm to the baseline.
Although these norms are relatively small given that there are 25.5M parameters, further re-training the model still changes loss values for some samples considerably, making the ground-truth estimates noisy.
We suggest that one way to obtain reliable ground-truth inﬂuence estimates in such large models can be through assessing the inﬂuence of a group of samples, rather than a single one.
D ISCUSSION ON GROUND - TRUTH INFLUENCE
In our experimental setup, to obtain the ground-truth inﬂuence, we follow the strategy of re-training from optimal model parameters as shown in (Koh & Liang, 2017; Koh et al., 2019b). Even for moderately sized datasets and architectures, re-training from scratch (instead of re-training from optimal model parameters) is computationally expensive. Although re-training from optimal model parameters is an approximation compared to re-training from scratch, we notice that the approximation works quite well in practice.
To validate the effectiveness of this strategy, we ﬁrst compute the norm of the difference in parameters obtained by re-training from scratch vs. re-training from optimal parameters.
Next we compute the correlation between the inﬂuence estimates and ground-truth using both the re-training strategies.
From Fig.
5, we observe the norm of parameter differences using the two re-training strategies to be small.
Similarly using both the re-training strategies as ground-truth yield similar correlation estimates.
These results highlight that re-training from optimal parameters (although an approximation) is close to re-training from scratch.
C ONCLUSION
In this paper, we present a comprehensive analysis of the successes and failures of inﬂuence functions in deep learning.
Through our experiments on datasets including Iris, MNIST, CIFAR-10,
CIFAR-100, ImageNet and architectures including LeNet, VGGNets, ResNets, we have demonstrated that inﬂuence functions in deep learning are fragile in general.
We have shown that several factors such as the weight-decay, depth and width of the network, the network architecture, stochastic approximation and the selection of test points, all have strong effects in the quality of inﬂuence estimates.
In general, we have observed that inﬂuence estimates are fairly accurate in shallow architectures such as small CNN(Koh & Liang, 2017) and LeNet, while in very deep and wide architectures such as ResNet-50, the estimates are often erroneous.
Additionally, we have scaled up inﬂuence computations to the ImageNet scale, where we have observed inﬂuence estimates are highly imprecise.
These results call for developing robust inﬂuence estimators in the non-convex setups of deep learning.