The Memory Perturbation Equation:
Understanding Model’s Sensitivity to Data

Peter Nickl†
peter.nickl@riken.jp

Lu Xu∗†
lu.xu.sw@riken.jp

Thomas Möllenhoff†thomas.moellenhoff@riken.jp

Dharmesh Tailor∗‡
d.v.tailor@uva.nl

Mohammad Emtiyaz Khan†§emtiyaz.khan@riken.jp

Abstract
Understanding model’s sensitivity to its training data is crucial not only for safeand robust operation but also for future adaptations. We present the memoryperturbation equation (MPE) which relates model’s sensitivity to perturbation inits training data. Derived using Bayesian principles, the MPE unifies existing influence measures, generalizes them to a wide-variety of models and algorithms, andunravels useful properties regarding sensitivity. Our empirical results show thatsensitivity estimates obtained during training can faithfully predict generalizationon unseen test data and avoid the need for expensive retraining. The equation isuseful for future research on robust and adaptive learning.

1

Introduction

Understanding model’s sensitivity to training data is important to handle issues related to quality,
privacy, and security. For example, we can use it to understand (i) the effect of errors and biases
in the data; (ii) model’s dependence on private information to avoid data leakage; (iii) model’s
weakness to malicious manipulations. However, despite their importance, sensitivity propertiesof machine learning (ML) models in general are not well understood. Sensitivity is often studied
through empirical investigations, but conclusions drawn this way do not always generalize acrossmodels or algorithms. Such studies are also costly, sometimes requiring thousands of GPUs [30],which can quickly become infeasible if we need to repeat them every time the model is updated.
A cheaper solution is to use local perturbation methods [17], for instance, influence measures thatstudy sensitivity of the model to data removal (Fig. 1(a)) [7, 6]. Such methods too fall short ofproviding a clear understanding of general causes of influence for generic models and algorithms.
Influence measures are useful for studying trained models but it remains challenging to generalize
them to analyze training trajectories [12, 39]. Another challenge is to handle non-differentiable loss
or discrete parameters where a natural choice of perturbation mechanisms is often unclear [26]. In
general, sensitivity analysis of generic ML models and algorithms is a difficult problem.
In this paper, we simplify such issues by proposing a new method to unify, generalize, and understand perturbation methods for sensitivity analysis. We present the memory-perturbation equation(MPE) as a unifying equation to understand sensitivity of generic ML algorithms; see Eq. 4. The
equation builds upon the Bayesian learning rule (BLR) [22] which unifies many popular algorithms∗

Equal contribution. Part of this work was carried out when Dharmesh Tailor was at RIKEN AIP.
RIKEN Center for AI Project, Tokyo, Japan.
‡
University of Amsterdam, Amsterdam, Netherlands.
§
Corresponding author.
†

Preprint. Under review.

TruthTraining on
full dataset

Iterations

Current(a) Effect of removing an example.

Test NLL
Train-LOO-MPE

NLL

1.8
1.2
0.6
100

Epochs

200

300

Shirt

0.82
Pullover

0.62

Coat

Top

Dress

0.42
Sandal

0.22
0.02

0

MLP
LeNet

1.02

Estimated

Leave-one-class-out (LOCO) estimation

Training without
the i’th example

Start

Ankle-bootSneaker
Trouser

Bag

0

1

2
3
4
Test Neg. Log-Likelihood (NLL)5(b) Predicting test loss during training, CIFAR10 (c) Predicting test loss for leave-one-class-out, FMNIST

Figure 1: Panel (a) illustrates the local perturbation method where sensitivity of a training trajectory(solid gray line) is studied by removing (or perturbing) an example and observing the deviation to
the trajectory (dashed line). The goal is to estimate the deviation without rerunning the algorithm.
Panel (b) shows that the Leave-One-Out (LOO) estimates (black line) computed solely on training
data can predict the test NLL trends (gray line) during training of ResNet–20 on CIFAR10. Panel (c)shows the Leave-One-Class-Out (LOCO) estimates (y-axis) that correlate well the test NLL (x-axis)
on FMNIST using MLP (red) and LeNet (blue). The left-out class is marked for each case.
from various fields as specific instances of a natural-gradient descent over the Bayes objective. The
MPE suggests that natural-gradients can also be used to understand sensitivity of all such algorithms.
We use the MPE to show several new results regarding sensitivity of generic ML algorithms:
1. We show that sensitivities to a group of examples can be estimated by using naturalgradients of those examples alone. Examples with larger natural-gradients contribute moreto the sensitivity and, due to this, most of the sensitivity is explained by just a few examples.
These highly-sensitive examples characterize the model’s memory because perturbing such
examples can make the model forget its essential knowledge. The MPE relates the model’ssensitivity to perturbation in the memory of the model.
2. We derive Influence Function [7, 25] as a special case of the MPE where natural-gradientswith respect to Gaussian posterior are used. Not only this, the MPE gives rise to newmeasures that, unlike influence functions, can be applied during training for all algorithms
covered under the BLR (such as those used in deep learning and optimization).
3. Measures derived using Gaussian posteriors share a common property: sensitivity to an
example is simply obtained by multiplying its prediction error and variance. That is, themodel is expected to be most sensitive to examples it finds difficult to predict confidently.
4. Sensitivity can be estimated cheaply (even during training) whenever natural-gradients arecheap to compute. Already computed quantities (e.g., error and variance) can be reusedand estimation does not add any additional computational overhead.
5. Finally, our empirical results show that, by using sensitivity of the training data alone, we
can accurately predict model generalization, even during training (Fig. 1(b)). Effect ofclass-removal can also be faithfully estimated (Figs. 1(b) and 1(c)) without any retraining.
These agree with similar studies showing effectiveness of sensitivity [18, 10, 16, 4].

2

Understanding Model’s Sensitivity to Training Data

Understanding model’s sensitivity to the training data is important but is often done by a costly
process of retraining the model multiple times. For example, consider a model with parameter
vector θ ∈ RP and data D = {D1 , D2 , . . . , DN } trained by using an algorithm At that generates a2

sequence {θ t }, at each iteration t, converging to a minimizer θ ∗ , as shown below:
θ t ← At (θ t−1 , L(θ)) where L(θ) =NX`i (θ) + R(θ),

(1)

i=1

Here, we use a loss `i (θ) for the i’th example Di and a regularizer R(θ). Now, because θ t are
all functions of D or its subsets, we can analyze their sensitivity by simply ‘perturbing’ the data.
For example, we can remove a data example, say the i’th one, to get a perturbed dataset D\i ={D1 , . . . , Di−1 , Di+1 , . . . , DN } and then retrain the model to get a new minimizer, denoted by\i
θ \i∗ . If the deviation θ ∗ − θ ∗ is large, then we may deem the model to be highly sensitive to thei’th example. We may also choose to analyze deviations in the function outputs fi (θ ∗ ) for inputs
xi ∈ Di or even the loss function L(θ). The same procedure can be repeated for θ t obtained during
training; see Fig. 1(a), but such brute-force retraining is too costly and even infeasible when training
trajectories are long or the model and data are large. More importantly, conclusions drawn from apurely empirical study may not always generalize across models or algorithms.
A cheaper alternative is to use local perturbation methods [17], for instance, influence measures that
estimate sensitivity of the model without retraining (illustrated in Fig. 1(a) by the dashed red arrow).
The simplest result of this kind is for linear regression which dates back to 1977 [6]. Consider input1
2
2
output pairs (xi , yi ) modeled by the loss `i (θ) = 21 (yi − x>i θ) and a regularizer R(θ) = 2 δkθk .
As shown in App. A, for this case, we have closed-form expressions for the deviations:
\i

\i

θ \i
∗ − θ ∗ = Σ∗ x i e i ,fi (θ \i∗ ) − fi (θ ∗ ) = vi ei ,

(2)

\i

\i
\i
>
where ei = x>i θ ∗ − yi is the prediction error of θ ∗ and vi = xi Σ∗ xi is the prediction variance2−1with Σ∗ = (∇ L(θ ∗ )) as the covariance matrix [5, Chapter 3, Eq. 3.59]. The expressions showthat the influence is bi-linearly related to both prediction error and variance, that is, examples withhigh values of either of these lead to a large change in the model if removed. In other words, themodel is most sensitive to examples it finds difficult to predict confidently.

These ideas are generalized using infinitesimal perturbation [17] where we use a perturbation modeland then take partial derivatives to measure influence, leading up to the idea of influence functions [7,25] where we use a perturbation model θ ∗i = arg minθ L(θ) − i `i (θ) with a scalar perturbation
i ∈ R, then Eq. 2 can be obtained by simply taking derivatives of θ ∗i at i = 1; see App. A. We arefree to choose other perturbation model, but often we do see a bi-linear relationship with respect toprediction error and variance; see Eq. 27 for an example. The infinitesimal-perturbation method canbe generalized to other models such as neural networks [26].
Despite their generality, influence measures fall short of providing a clear understanding of generalcauses of influence for generic models and algorithms. This is due to several reasons:
1. Influence measures are valid only at a stationary point θ ∗ where the gradient is assumed tobe 0, and extending them to iterates θ t generated by a specific algorithm is non-trivial [12].
This is even more important for deep learning where we may never reach such stationary
point, for example, due to stochastic training or early stopping.
2. Applying such measures to non-differentiable loss functions or discrete parameter spaces isalso difficult. This is because the choice of perturbation model is not always obvious [26].
3. Finally, despite their generality, the measures do not directly reveal the causes of highinfluence. Does the bi-linear relationship in Eq. 2 hold more generally? If yes, under what
conditions? Answers to such questions are currently unknown.
Studies to fix these issues are rare in ML, rather it is more common to simply use heuristics and draw
conclusions based on extensive empirical investigations. Many such heuristic measures have been
proposed in the recent years, for example, those using gradients [19, 2, 30], variations of Cook’s
distance [14], prediction error [3, 38, 33, 32], backtracking training trajectories [13], or simply by
retraining [11]. These works, although useful, do not directly address the issues. They are also
missing connections regarding any bi-linear relationship similar to Eq. 2. Our goal here is fix thisissue by unifying and generalizing perturbation methods of sensitivity analysis, and to understandand reveal general causes of influence for generic models and algorithms.
3

3

The Memory-Perturbation Equation

We propose the memory-perturbation equation (MPE) to unify, generalize, and understand the sensitivity of generic ML algorithms. The equation is derived using the Bayesian learning rule (BLR) [22]which unifies many popular algorithms from various fields. We will first describe the BLR, thenpresent the MPE, and show its application to derive new results regarding sensitivity of algorithms.
The BLR optimizes a Bayesian formulation of Eq. 1 to find an approximation q(θ) to the posteriorp(θ|D) ∝ e−L(θ) . Denoting by qλ (θ) an exponential-family distribution with natural parameter λ,the BLR update can be written as the following,
λt = (1 − ρ)λt−1 + ρ

N
X

e q
∇E
[−`j (θ)]
λ t−1

(3)

j=0

e = F(λ)−1 ∇λ is the natural gradient with
where λt is the natural parameter at the t’th iteration, ∇respect to λ and defined using the Fisher F(λ) of qλ (θ), and ρ > 0 is the learning rate. For brevity,
we denote `0 (θ) = R(θ). The rule can obtain many widely-used algorithms as special cases bychoosing an appropriate exponential-family form for qλ (θ) and employing further approximations
to natural gradients. These include popular algorithms in deep learning (e.g., RMSprop, Adam, andSAM), optimization (e.g., gradient descent and Newton’s method), and approximate inference (e.g.,ridge regression, Kalman filters, EM, Laplace’s method). We will now propose a method to analyze
sensitivity of the BLR which then enables us to do the same for all the algorithms covered under it.
Our key idea is the following: Eq. 3 shows that each λt is an accumulation of all the past natural
gradients computed at iterations before t, and so it lies in their span. Therefore, to estimate theeffect of removal of a group of examples in a set M ⊂ D, we propose to simply subtract the natural
gradients of that example. Specifically, we take a BLR step in the opposite direction:X\M
e q [`j (θ)],
MPE: λ̂t − λt =
∇E
(4)
λt
j∈M
/M
\M
where λ̂t is an estimate of the true λt obtained by retraining with the BLR but without using

the examples in M. We call this the memory-perturbation equation (MPE). To estimates the effects
of general additive perturbations j to the BLR iterates, we can replace `j in Eq. 4 by j .

The MPE has two additional unique features: first, the deviations caused by perturbations in M
are estimated entirely by using natural-gradients of examples in M alone; second, the deviation isadditive over all j ∈ M, therefore examples with larger natural-gradients contribute more to thesensitivity estimate. This is similar to the representation theorem [23, 36], for example, in support
vector machines [8], and in a similar spirit we expect most of the sensitivity to be explained bya small fraction of the data examples. These highly-sensitive examples characterize the model’s
memory because perturbing such examples can make the model forget its essential knowledge. The
MPE gives a way to relate the model’s sensitivity to perturbation in its memory.
The equation is analogous to Eq. 2 for linear models which can be seen as a Newton-step in the
opposite direction [26], but here we estimate the deviations in the natural-parameter space and wewill now show that this choice allows us to derive influence functions as special cases of the MPE
and derive new scores that apply during training.
3.1

Linear-model influence function as a special case of the MPE

Our first result derives Eq. 2 for linear regression by using the MPE.
Theorem 1 For a full-covariance Gaussian qλ (θ), the MPE for linear regression leads to Eq. 2.
Proof: For Gaussian distributions qλ (θ) = N (θ|m, Σ) with mean m and covariance Σ (and precision S = Σ−1 ), we have λ = (Sm, − 21 S) and µ = Eqλ (θ, θθ > ). For linear regression the(exact) posterior is a Gaussian with m∗ = θ ∗ and S∗ = ∇2 L(θ ∗ ). To compute natural gradients,
we rewrite them in terms of the gradient and Hessian of the loss [22, Eq. 10-11],
e q [`i ] = Eq ∇`i (θ) − ∇2 `i (θ)m, 1 ∇2 `i (θ) ,
(5)
∇E
λ
λ
2
4

2
For `i (θ) = 21 (yi − x>i θ) , we get
e q [`i (θ)] = Eq xi (x> θ − yi ) − xi x> m, 1 xi x>= −xi yi , 21 xi x>
∇E
i
i .
i
i
λ
λ
2
\i
\i \i
For removal of i’th example, we denote λ∗ = (S∗ θ ∗ , − 21 S∗ ) and λ̂∗ = (Ŝ∗ θ̂ ∗ ,plugging Eq. 6 into Eq. 4, we have the MPE for linear regression,
\i \i

(6)

\i
− 12 Ŝ∗ ). Then,\i− 21 Ŝ∗ + 12 S∗ = 12 xi x>i .

Ŝ∗ θ̂ ∗ − S∗ θ ∗ = −xi yi ,(7)\i
We can rearrange this to get the deviation θ̂ ∗ − θ ∗ over the mean to recover Eq. 2. ,\i(S∗ − xi x>i )θ̂ ∗ − S∗ θ ∗ = −xi yi

\i

\i

\i

=⇒ θ̂ ∗ − θ ∗ = Σ∗ xi (x>i θ̂ ∗ − yi ) = Σ∗ xi êi(8)This completes the proof.
\i\i

The right hand side requires θ̂ ∗ which is not always available, but we can assume êi ≈ ei whichleads us to a score at a slightly different perturbation given in Eq. 27. In general, we can make suchassumptions to simplify the computation.
For linear regression, the posterior approximation is exact and the MPE then recovers the exact
deviations. This property holds in general for all exponential-family conjugate models, including
mixture models, linear state-space models, probabilistic PCA, etc. A formal statement is given
below; a proof is in App. B and an example on Beta-Bernoulli model is in App. C.
Theorem 2 When qλ = p(θ|D), the MPE estimate is equal to the exact deviations.
3.2

Neural-network influence function as a special case of the MPE

We will now show that influence functions for any optimizer that satisfies second-order optimality
condition at convergence can be derived as a special case of the MPE with Gaussian posterior. For
example, in [26], the following influence function is derived for neural network loss function,
−1\iθ̂ ∗ − θ ∗ ≈ ∇2 L(θ ∗ )∇`i (θ ∗ ).
(9)
We will derive this as a special case of the MPE, and also propose a generalization which
can be applied during training. Throughout, we will use a differentiable loss `i (yi , fi (θ)) =− log p(yi |σ(fi (θ))) defined using an exponential-family p(y|σ(f )) for a scalar y and link function σ(f ) (for example, softmax). We denote by fi (θ) the function output for the i’th input xi .
The BLR update for Gaussian posteriors takes a Newton-like form (see [22, Eq. 12] or [20]),
mt = mt−1 − ρS−1
t Eqλt−1 [∇L(θ)],

St = (1 − ρ)St−1 + ρ Eqλt−1 [∇2 L(θ)](10)

Deep-learning algorithms such as RMSprop and Adam can be derived from this algorithm [22, Sec.
4.2] by using the Delta method. Essentially, we approximate the expectation of a function by thevalue at the mean, for instance, Eqλ [g(θ)] ≈ g(m) for an arbitrary function g; see [22, Eq. 13].
With this approximation, the BLR recovers a minimizer θ ∗ of Eq. 1, that is, we get m∗ = θ ∗ andS∗ = ∇2 L(θ ∗ ), similarly to the linear regression case shown in Theorem 1. We will use the same
technique to derive influence functions of deep-learning optimizers from the MPE.
Proceeding similarly to Eq. 5 we get (suppressing the dependence on θ for notational ease),e q [`i ] = Eq ∇`i − ∇2 `i m, 1 ∇2 `i = Eq ∇fi ei − Hi m, 1 Hi ,
∇E
λ
λ
λ
2
2

(11)

where ei (θ) = σ(fi (θ)) − yi is the error and Hi (θ) = ∇2 `i (θ) is the Hessian. Note that Eq. 6 is a
special case of the general result in Eq. 11. Plugging in Eq. 4,\i\iŜt m̂t − St mt = Eqλt [∇fi ei − Hi mt ] ,\i− 12 Ŝt + 12 St = 12 Eqλt [Hi ],

(12)

\i
and then we rearrange it to get the deviation over the mean, that is, m̂t − mt ,

 \iSt − Eqλt [Hi ] m̂t − St mt = Eqλt [∇fi ei + Hi mt ]
\i=⇒ St − Eqλt [Hi ] (m̂t − mt ) = Eqλt [∇fi ei ] ,\i\i=⇒ m̂t − mt = Σ̂t Eqλt [∇fi (θ)ei (θ)]

5

(13)

\i

If we use the Delta method and additionally assume that Σ̂t ≈ Σt , we get the expression in the left
hand side, which at convergence can be written in the form shown in the right hand side,
\i

−1
\i
∇`i (θ ∗ ).
θ̂ ∗ − θ ∗ ≈ ∇2 L(θ ∗ )θ̂ t − θ t ≈ Σt ∇fi (θ t )ei (θ t ),(14)

The second result is obtained using Σ∗ = (∇2 L(θ ∗ ))−1 . The second expression in Eq. 14 is exactly
the influence function derived in [26]. They use linearization to obtain the expression, but the MPE
obtains similar results. The first expression can be seen as a generalization of [13] to Newton-likesteps where their ‘back-tracking’ factor is replaced by its linear approximation Σt . This simplifies
the computation of their method.
Why do we recover existing influence functions from the MPE? This is because of the property of
the BLR where natural-gradients are assigned to appropriate natural parameters [22, Sec. 1.2]. The
two natural gradients are related to the gradient and Hessian respectively which are then connectedto the sensitivity in the mean and covariance of the Gaussian. Essentially, the expressions in Eq. 13
and Eq. 14 hold whenever a Gaussian posterior is obtained using the BLR.
More importantly, the sensitivity measure is valid for iterations of the BLR, and can be directly applied to analyze sensitivity of any algorithm derived from it. This includes optimization algorithms
such as gradient descent and Newton’s method but also deep-learning optimizers such as RMSpropand Adam [24]. The proposed sensitivity measure is also valid for Bayesian deep-learning methods
which build upon the BLR [20, 31, 27]. Note that these algorithms are related to various kinds
of Gaussian approximations, some of which are more accurate than others. The posterior approximation directly affects the accuracy of the sensitivity measures but also the computation cost. In
general, we expect there to be a trade-off between the accuracy and computation and a suitable
method can be chosen accordingly.
3.3

Causes of influence for the Gaussian caseWe now show that the bi-linear relationship in the influence function, seen in the case of linear regression (Eq. 2), holds in general for all influence functions derived using the Gaussian perturbation
model. This is easy to show by using Taylor’s approximation and Eq. 14,\i\ifit − fit ≈ ∇fi (θ t )> (θ t − θ t ) ≈ ∇fi (θ t )> Σt ∇fi (θ t )eit = vit eit ,
\i

(15)

\i

where we define eit = ei (θ t ) and fit = fi (θ t ), fit = fi (θ t ) to be the function outputs at θ t\iand θ t respectively. vit = ∇fi (θ t )> Σt ∇fi (θ t ) is the (linearized) prediction variance of fi (θ).
The variance is equal to the prediction variance of the error for linear models, but otherwise it is an
estimate. A similar expression can be obtained for the predictions,\i\i

σ(fit ) − σ(fit ) ≈ σ 0 (fit )∇fi (θ t )> (θ t − θ t ) ≈ σ 0 (fit )vit eit .

(16)This expression clearly shows that the examples with high prediction error, variances and σ 0 (f ) aremost likely to cause high sensitivity. We can also write this for removal of a group of examples in
M. Denoting the vector of fi (θ t ) for i ∈ M by f M,t we get a similar expression,X\M
σ(f M,t )) − σ(f M,t ) ≈ Λ(θ t )VM (θ t )eM (θ t ) ≈σ 0 (fit )vit eit ,
(17)
i∈M

where Λ(θ t ) is a diagonal matrix containing all σ 0 (fi (θ t )), VM (θ t ) is the prediction covariance,and eM (θ t ) is the vector of prediction errors. The second approximation gives a cheaper expression
which avoids building the covariance and instead uses the deviations with respect to each example.
3.4

Extensions and implications on computations

The bi-linear relationship also holds for other models, such as, Bayesian logistic regression, Gaussian processes, and their sparse variants. For example, for Bayesian logistic regression we use Eq. 13
to get a similar expression by noting that ∇fi (θ) = xi ,
\i

\i

>
fi (θ t ) − fi (θ t ) ≈ x>i (θ t − θ t ) ≈ xi Σt xi Eqλt [ei (θ)] = vit eit ,

6

(18)

where vit = x>i Σt xi is the predictive variance for the function outputs and eit = Eqλt [ei (θ)] is theprediction error. In general, whenever a Gaussian posterior is used, we expect to see similar causesof high influence. We show the derivation for sparse variational GP in App. D.
\i\iNon-differentiable L are also covered where we get m̂t − mt = Σ̂t ∇m Eqλt [`i (θ)]; see App. E
for a derivation. This is the extension of Eq. 13. It is a much more principled approach than [26]who used an ad-hoc smoothing of the non-differentiable loss. The smoothing here is automaticallydone by using the posterior distribution.
What happens when we use non-Gaussian approximations? The principle is the same. The causes
are determined by the sufficient statistics T(θ). For Gaussians, there are two sufficient statistics forthe mean and covariance respectively. Each statistics leads to a cause of high influence, for example,
in Eq. 12, the deviation in the first natural parameter (Sm) brings in the prediction error into play,while the deviations in the second natural parameters (S) brings in prediction variance as anothercause. In general, T(θ) determines the causes of high influence, and its size determines the numberof causes.
Finally, due to its connection to the BLR, the MPE is can be applied to all sorts of algorithms
in fields such as deep learning, optimization, graphical models, Bayesian models, kernel methods
etc; see App. F for deep learning. for For all these algorithms, we expect similar causes of highinfluence. Many unsupervised models can also be analyzed this way. The MPE unravels usefulproperties regarding sensitivity of generic ML models. For all these cases, the MPE also suggestthat the estimation of sensitivity to be cheap. We can use byproducts of the algorithm (such as,prediction error and variance) to cheaply estimate the sensitivity. No additional computations are
required.
3.5

Prediction of generalization performance

We will show that the MPE can be used to accurately predict the generalization performance at
any time during training iterations. Essentially, we use Eq. 15 to estimate leave-one-out (LOO)
performance on the training set,
LOO(θ t ) =

N
N

 X
X
\i
` (yi , fit + vit eit ) .
` yi , fi (θ t ) ≈

(19)

i=1

i=1

The measure is very closely related to both marginal likelihood and sharpness both of which are
known to be useful in predicting generalization [18, 10, 16]. Recently, [4] proposed similar approximations of leave-one-out loss to predict generalization, however they do not apply it to iterations.
We can also estimate model’s performance when a group or entire class of examples are removed,

 X
X 
\C
LOCO(θ t , C) =` yi , fi θ t≈` (yi , fit + vit eit ) ,
(20)
i∈C

i∈C

where C ⊂ {1, . . . , N } is a set of training examples belonging to the same class. This quantity
measures the model’s sensitivity to various classes.

4

Experiments

We show experimental results to demonstrate the usefulness of the MPE (4) in understanding sensitivity and generalization for deep learning. We show the following: (i) we verify that the estimateddeviation in Eq. 16 correlates with the truth; (ii) we predict the effect of class removal on generalization error in deep learning; (iii) we estimate the cross-validation error for hyperparameter tuning;(iv) we study of sensitivities during training. More details of the training setup are in App. G.
Do estimated deviations correlate with the truth? We verify Eq. 16 and show that the estimateddeviations correlate well with the truth. The results are summarized in Fig. 2, where we show
deviations on three datasets. Each marker shows a point, where we clearly see that the ranking of
the examples according to their sensitivity is maintained. We also show images of a few high and low
sensitivity examples where we find that sensitive examples are often interesting cases, for example,
clothes with unusual or elaborate design in FashionMNIST. In contrast, low sensitive examples are
all regular looking. This is expected because highly sensitive points have either high prediction error
7

Most
Sensitive
Most
Sensitive
Most
Sensitive

Least
Sensitive
Least
Sensitive
Least
Sensitive

Most
Sensitive
Most
Sensitive
Most
Sensitive

Least
Sensitive
Least
Sensitive
Least
Sensitive

Most
Sensitive
Most
Sensitive

Estimated Deviation
Estimated
Estimated Deviation
Deviation

Least
Sensitive
Least
Sensitive
Least
Sensitive

True
Deviation
True
Deviation

True
Deviation
True
Deviation
True
Deviation

True
Deviation
True
Deviation
True
Deviation
0.3%
0.3%
0.4%
0.4% 0.3%
0.4%
0.2%
0.2%
0.2%

0.1%
0.1%
0.1%
0.4%
0.4%
0.4%
0.3%
0.3%
0.3%

(a) MLP on MNIST(b) LeNet on FMNIST(c) CNN on CIFAR-10

Figure 2: We validate Eq. 16 to show that the estimated deviation correlates well with the truth.
Highly sensitive examples are often interesting corner-cases, for example clothes with unusual or
elaborate design in FashionMNIST as seen Fig. 2(b). On the other hand, the model is not sensitive
to the examples shown in the lower-left corner, corresponding to examples that look more regular.
The histograms of sensitivities (shown in the bottom) show that only a fraction of data have highsensitivities as suggested by the MPE.

group
individual

0.9

0.9
0.6

0.6

0.3

0.3

0.0

0.0
0.0

0.0

0.3

0.6

0.11

0.9

True Deviation

1.2(a) MLP on USPS-3vs5, |M| = 16

LOCO estimation

Estimated Deviation

1.2

individual
0.4

0.8

1.2

True Deviation

1.6(b) MLP on MNIST, |M| = 64

8
4

0.08

25

3
9

7
6

0
1

0.05

0.02

0

MLP
LeNet
1

2

Test NLL

34(c) Class removal on MNIST

Figure 3: Panel (a) and Panel (b) show that the estimated deviation for removal of groups of examples
correlates well with the true deviations. Each point corresponds to a removed group of examples,
the red points show the second approximation in Eq. 17, while the blue squares show the first one.
In Panel (c) we show the estimated NLL for one-class-leave-out obtained by using Eq. 20 indicates
the true class sensitivity in terms of test NLL when the class is removed. The models find someclasses more sensitive than others.

or variance. They can be examples that are perhaps mislabeled or ambiguous or may be they are notand perhaps the model views them as such.
We also study how the deviation for group removal can be estimated using Eq. 17. In Fig. 3(a) we
consider the USPS dataset and show that Eq. 17 and its second approximation using a sum over theindividual data points both correlate well with the truth. In Fig. 3(b) we show the group removal on
MNIST where we show that the approximate version which also correlates well with the truth.
8

NLL

0.2

1.8

Test NLL
Train-LOO-MPE

0.5

0.1

1.2

0.3

100

102
103
101
L2-regularization parameter

101(a) MNIST

102
103
L2-regularization parameter

101(b) FMNIST

102
103
L2-regularization parameter(c) CIFAR-10

NLL

Figure 4: Panels (a), (b) and (c) show that our method can faithfully estimate the LOO-CV curve forpredicting generalization and tuning of the L2 -regularization parameter on MNIST, FMNIST and
CIFAR-10.

Test NLL
Infl. Function

1.0

Test NLL
Train-LOO-MPE

3.0
1.5

0.5
0

25

50
75
Epochs100(a) Infl. function with AdamW

0.0 0

1.0
0.5

25

50
75
Epochs(b) MPE with Adam

100

0

25

50
75
Epochs100(c) MPE with iBLR

Figure 5: Panel (a) shows an estimate of the test NLL using influence function during training of a
LeNet5 on FMNIST. We use a generalized Gauss-Newton approximation of the Hessian with diagonal matrix structure. Panels (b) and (c) display the Train-LOO estimate obtained from MPE. The
results suggest that MPE with iBLR can faithfully estimate the test NLL and improves upon applying influence function. It performs better than the estimation with MPE that computes variances
heuristically based on the Adam optimizer.

Predicting the effect of class removal on generalization: We use the MPE to predict class sensitivity: change in the test error when a whole class is removed from train data. Typically, one would
have to retrain the whole model, but we show that this can be avoided by using Eq. 20 to predictthe generalization error. In Fig. 1(c) and Fig. 3(c) we show two models on MNIST and FMNIST
respectively. We see that the test NLL after class removal correlates well with the estimated NLL
obtained with Eq. 20. We also see that the models are more sensitive to some classes than others.
In MNIST (see Fig. 3(c)), 3, 9, and 5 are the most sensitive classes, while 0, 1 are not sensitive. In
FashionMNIST (see Fig. 1(c)) Shirt, Pullover are the most sensitive classes, while Bag, Trouser are
the least sensitive ones.
Estimating the leave-one-out cross-validation curves for hyperparameter tuning: Here, we
use the MPE to estimate the leave-one-out cross-validation (LOO-CV) curves for hyperparametertuning. We consider the tuning of the regularization parameter δ of an L2 regularizer R(θ) =12
2 δkθk . We retrain the model for each δ setting and for each of those compute the LOO-CV
estimate on training data using Eq. 19.Fig. 4 shows results for three models trained on MNIST,FMNIST and CIFAR-10 respectively. The estimated negative log-likelihood (NLL) curve matches
the test NLL.
The match is almost perfect here and we do not see such good correlations in other studies [18, 10,16] but our result supports their conclusions that sensitivity based measures can work well to predict
generalization performance. The cost of computing these curves is almost negligible compared tocross-validation where we have train many more models for each setting of the hyperparameter.
How do sensitivities evolve during training: We use the MPE to analyze the evolution of sensitivities during training. We consider Bayesian logistic regression in Fig. 6(a) and neural networkclassification with the iBLR optimizer inFig. 6(b), Fig. 6(c) and Fig. 6(d). We run the BLR iteratesof Eq. 10. For Bayesian logistic regression we estimate deviations using Eq. 16 where we use expectations for the residual as in Eq. 18 and plot them for various iterations. For iBLR we also use Eq. 16
and plot for several selected epochs. For better visualization, we first sort the examples according to
the sensitivities of the converged model, sorted in decreasing order of sensitivity. Then, we plot the
average sensitivies of examples, where we grouped examples with similar sensitivities together into
9

High Sensitivity

Low Sensitivity

High Sensitivity

Low Sensitivity

Mean Sensitivity

Mean Sensitivity

50th Epoch

20th Epoch

5th Iteration

Highly Sensitive Examples

2nd Epoch

10th Iteration

125th
Iteration

Groups of Examples

Less Sensitive Examples

Highly Sensitive Examples(a) MNIST, Bayesian logistic regression
High Sensitivity

Groups of Examples

Less Sensitive Examples(b) MNIST, iBLR with MLP

Low Sensitivity

High Sensitivity

Mean Sensitivity

Mean Sensitivity

50th Epoch

1st Epoch

25th Iteration

25th Epoch

Low Sensitivity

250th Epoch

100th Epoch
10th Epoch

20th Epoch

1st Epoch
1st Epoch

Highly Sensitive Examples

Groups of Examples

Less Sensitive Examples

Highly Sensitive Examples(c) FMNIST, iBLR with LeNet5

Groups of Examples

Less Sensitive Examples(d) CIFAR-10, iBLR with ResNet–20

Figure 6: We show the evolution of sensitivies during training for Bayesian logistic regression inpanel (a) and the iBLR optimizer in the panels (b), (c) and (d). We see that, with training, the model
becomes more and more sensitive to a small fraction of data. We also show some examples of highand low sensitivies.
overall 200 groups. In Fig. 6(a), we see that in the beginning the sensitivities of all examples are almost the same (5th iteration), but as training proceeds, the model becomes more and more sensitive
to a small fraction of examples. In the end, the distribution of sensitivities is heavy-tailed where onlyaround 20% of data has reasonable sensitivities of values > 1, while < 2% are extremely sensitive
with values > 8 as indicated by the vertical dashed lines. The experiments with iBLR on MNIST,FMNIST and CIFAR-10 all exhibit the same increasing heavy-tailedness of the sensitivities over
the training. Notably, for the ResNet–20 on CIFAR-10, the distribution is very heavy-tailed after
the 250-th epoch. For all experiments we show images of high and low sensitivity examples, where
we find highly sensitive examples to be the odd ones while low sensitivity examples to be regularlooking.
Predicting generalization during the training: We predict the test loss during training by evaluating Eq. 19. We use the Train-LOO estimate with MPE (Train-LOO-MPE) and the influence function
as a special case of the MPE. Both estimates are based on the training data. In Fig. 5 we show theresults for a LeNet5 trained on FMNIST. The experiments indicate that the estimate obtained fromMPE can predict the test loss during training faithfully. Such information can be used as a diagnostic
for stopping training to avoid overfitting. In the MPE experiments, we compare Adam and iBLR
for the computation of the variance in Eq. 19. For details we refer to App. F. The results suggestthat improved variance computation leads to a better quality of the prediction of generalization. The
heuristic estimate based on the Adam deep-learning optimizer [24] in Fig. 5(b) reflects the generaltrend of the test loss, but exhibits inaccuracies, for instance in the end of training. The variance estimation with the iBLR optimizer [27] in Fig. 5(c) leads to faithful prediction of the test loss, despitea diagonal covariance structure. iBLR avoids overfitting in the end of training. Application of theinfluence function in Fig. 5(a) captures the trend of the test loss. There is, however, a difference inmagnitude between the test loss and the prediction. We use a generalized Gauss-Newton approxi10

mation to the Hessian with diagonal matrix structure. Details on the implementation of the influencefunction can be found in App. G.5. Fig. 10 additionally reports the test errors over the training for
the FMNIST experiment. In Fig. 1(b) and Fig. 11 we show a result using MPE with iBLR for a
Resnet–20 on CIFAR10. The method faithfully predicts the test loss. Further results on MNIST(MLP, LeNet5) and CIFAR10 (CNN) are reported in Fig. 8 and Fig. 9, respectively.

5

Discussion

We present the memory-perturbation equation by building upon the BLR framework. The equationsuggests to take a step in the direction of the natural gradient of the perturbed examples. Using theMPE framework, we unify existing influence measures, generalize them to a wide variety of problems, and unravel useful properties regarding sensitivity. We also show that sensitivity estimation
can be done cheaply and use this to predict generalization performance.
An interesting avenue for future research is to apply the method to larger models and real-world
problems. We also need to understand how our generalization measure compares to other methods,
such as those considered in [18]. We would also like to understand the effect of various posterior
approximations. Another interesting direction is to apply the method to non-Gaussian cases, forexample, to study ensemble methods in deep learning with mixture models.

References[1] Vincent Adam, Paul Chang, Mohammad Emtiyaz E Khan, and Arno Solin. Dual parameterization of sparse variational Gaussian processes. Advances in Neural Information Processing
Systems, 34:11474–11486, 2021. 16[2] Chirag Agarwal, Daniel D’Souza, and Sara Hooker. Estimating example difficulty using variance of gradients. In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, 2022. 3[3] Devansh Arpit, Stanisław Jastrz˛ebski, Nicolas Ballas, David Krueger, Emmanuel Bengio,
Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, andSimon Lacoste-Julien. A closer look at memorization in deep networks. In International
Conference on Machine Learning, 2017. 3[4] Gregor Bachmann, Thomas Hofmann, and Aurélien Lucchi. Generalization through the lensof leave-one-out error. In International Conference on Learning Representations, 2022. 2, 7
[5] Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science andStatistics). Springer-Verlag, Berlin, Heidelberg, 2006. 3[6] R Dennis Cook. Detection of influential observation in linear regression. Technometrics,
19(1):15–18, 1977. 1, 3
[7] R Dennis Cook and Sanford Weisberg. Characterizations of an empirical influence function
for detecting influential cases in regression. Technometrics, 22(4):495–508, 1980. 1, 2, 3
[8] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20:273–
297, 1995. 4[9] Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer,and Philipp Hennig. Laplace redux-effortless bayesian deep learning. Advances in Neural
Information Processing Systems, 34:20089–20103, 2021. 18, 21[10] Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds
for deep (stochastic) neural networks with many more parameters than training data. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, 2017. 2, 7, 9
[11] Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering
the long tail via influence estimation. In Advances in Neural Information Processing Systems,
2020. 311

[12] Wing K Fung and CW Kwan. A note on local influence based on normal curvature. Journalof the Royal Statistical Society: Series B (Statistical Methodology), 59(4):839–843, 1997. 1, 3
[13] Satoshi Hara, Atsushi Nitanda, and Takanori Maehara. Data cleansing for models trained with
SGD. In Advances in Neural Information Processing Systems, 2019. 3, 6
[14] Hrayr Harutyunyan, Alessandro Achille, Giovanni Paolini, Orchid Majumder, Avinash
Ravichandran, Rahul Bhotika, and Stefano Soatto. Estimating informativeness of samples
with smooth unique information. In International Conference on Learning Representations,2021. 3[15] James Hensman, Nicolo Fusi, and Neil D Lawrence. Gaussian processes for big data. In
Uncertainty in Artificial Intelligence, page 282. Citeseer, 2013. 16[16] Alexander Immer, Matthias Bauer, Vincent Fortuin, Gunnar Rätsch, and Khan Mohammad
Emtiyaz. Scalable marginal likelihood estimation for model selection in deep learning. In
International Conference on Machine Learning, 2021. 2, 7, 9[17] Louis A Jaeckel. The infinitesimal jackknife. Bell Telephone Laboratories, 1972. 1, 3
[18] Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. In International Conference on Learning
Representations, 2020. 2, 7, 9, 11[19] Angelos Katharopoulos and Francois Fleuret. Not all samples are created equal: Deep learningwith importance sampling. In International Conference on Machine Learning, 2018. 3[20] Mohammad Khan, Didrik Nielsen, Voot Tangkaratt, Wu Lin, Yarin Gal, and Akash Srivastava.
Fast and scalable Bayesian deep learning by weight-perturbation in Adam. In International
Conference on Machine Learning, 2018. 5, 6, 16, 17[21] Mohammad Emtiyaz Khan and Wu Lin. Conjugate-computation variational inference: Converting variational inference in non-conjugate models to inferences in conjugate models. In
International Conference on Artificial Intelligence and Statistics, 2017. 16[22] Mohammad Emtiyaz Khan and Håvard Rue. The Bayesian learning rule. arXiv:2107.04562,
2021. 1, 4, 5, 6, 17[23] George S Kimeldorf and Grace Wahba. A correspondence between bayesian estimation
on stochastic processes and smoothing by splines. The Annals of Mathematical Statistics,
41(2):495–502, 1970. 4[24] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. 6, 10, 17[25] Pang Wei Koh, Kai-Siang Ang, Hubert Teo, and Percy S Liang. On the accuracy of influencefunctions for measuring group effects. In Advances in Neural Information Processing Systems,
2019. 2, 3
[26] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions.
In International Conference on Machine Learning, 2017. 1, 3, 4, 5, 6, 7, 17, 21[27] Wu Lin, Mark Schmidt, and Mohammad Emtiyaz Khan. Handling the positive-definite constraint in the Bayesian learning rule. In International Conference on Machine Learning, 2020.
6, 10, 17[28] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. ICLR, 2019. 19,
21[29] James Martens. New insights and perspectives on the natural gradient method. arXiv preprint
arXiv:1412.1193, 2014. 21[30] Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha SohlDickstein. Sensitivity and generalization in neural networks: an empirical study. In International Conference on Learning Representations, 2018. 1, 3
12

[31] Kazuki Osawa, Siddharth Swaroop, Mohammad Emtiyaz E Khan, Anirudh Jain, Runa Eschenhagen, Richard E Turner, and Rio Yokota. Practical deep learning with Bayesian principles. In
Advances in Neural Information Processing Systems, 2019. 6[32] Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet:
Finding important examples early in training. In Advances in Neural Information Processing
Systems, 2021. 3[33] Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training
data influence by tracing gradient descent. In Advances in Neural Information Processing
Systems, 2020. 3[34] Hugh Salimbeni, Stefanos Eleftheriadis, and James Hensman. Natural gradients in practice:Non-conjugate variational inference in Gaussian process models. In International Conferenceon Artificial Intelligence and Statistics, pages 689–697. PMLR, 2018. 16[35] Frank Schneider, Lukas Balles, and Philipp Hennig. DeepOBS: A deep learning optimizer
benchmark suite. In International Conference on Learning Representations, 2019. 18[36] Bernhard Schölkopf, Ralf Herbrich, and Alex J Smola. A generalized representer theorem. In
International conference on computational learning theory, pages 416–426. Springer, 2001. 4[37] Saurabh Singh and Shankar Krishnan. Filter response normalization layer: Eliminating batch
dependence in the training of deep neural networks. In Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, 2020. 20[38] Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J. Gordon. An empirical study of example forgetting during deep neuralnetwork learning. In International Conference on Learning Representations, 2019. 3[39] Hongtu Zhu, Joseph G. Ibrahim, Sikyum Lee, and Heping Zhang. Perturbation selection andinfluence measures in local influence analysis. The Annals of Statistics, 35(6):2565 – 2588,
2007. 1

13

A

Influence Function for linear regression

We define a perturbation model as follows with i ∈ R:
θ ∗i = arg min L(θ) − i `i (θ).
θ

The solution has a closed-form expression,>θ ∗i = Σ−1∗ − i xi xi

−1 


X > y − i y i x i .

To derive Eq. 2, we express this in terms of θ ∗ by using the Sherman-Morrison formula,i Σ∗ xi x>
>
i Σ∗
X
y
−

y
x
θ ∗i = Σ∗ +iii
1 − i x >i Σ∗ xi"#>x>i yi x>>i Σ∗ X yi Σ∗ xi= Σ∗ X y + i Σ∗ xi−− yi
1 − i x >1 − i x >i Σ∗ x ii Σ∗ xi >i yi vixi θ ∗−− yi= θ ∗ + i Σ∗ xi
1 −  i vi
1 −  i vi >x θ ∗ − yi= θ ∗ + i Σ∗ xi i
1 −  i vi
i ei.
= θ ∗ + Σ∗ xi
1 − i vi

(21)

(22)

>
>
where in line 3 we substitute vi = x>i Σ∗ xi and θ ∗ = Σ∗ X y and in line 5 we use ei = xi θ ∗ − yi .

To derive Eq. 2, we use i = 1 and simply write,
ei
\i
θ \i
= Σ∗ xi ei∗ − θ ∗ = Σ∗ x i
1 − vi> \i>fi (θ \i∗ ) − fi (θ ∗ ) = xi (θ ∗ − θ ∗ ) = xi Σxi

(23)
ei
ei
\i
= vi
= vi e i ,1 − vi1 − vi(24)\iwhere we used the relation ei = ei /(1 − vi ) which we proceed to show,


ei
vi
ei
\i
\i
>
−
y
=
x
θ
+
Σ
x
ei = x >θ− yi = x >ei − yi =
.
i
∗
∗
i
i
i ∗
i θ∗ +
1 − vi1 − vi1 − vi(25)

An alternate way to derive it (in a more general form) is by taking derivatives of θ ∗i in Eq. 22:∂θ ∗iei= Σ∗ x∗∂i(1 − i vi )2(26)

Taking i = 1 gives us Eq. 2, but we can get other influence measures by evaluating, for example,at i = 0, as shown below:∂θ ∗i= Σ∗ x i e i ,∂i i =0∂fi (θ ∗i )∂θ ∗i= x>= x>
ii Σ∗ xi ei = vi ei
∂i
∂
i
i =0
i =0

(27)

where Eq. 27 follows from straightforward application of the chain rule. Here too we get a bi-linear
relationship of the influence measure with respect to prediction variance vi and prediction error ei .

B

Proof of Theorem 2

The exact deviation for removing an example i amounts to dividing out its likelihood p(Di |θ) fromthe exact posterior p(θ|D) and renormalizing,p(θ|D\i ) ∝14p(θ|D).
p(Di |θ)(28)

Now in the conjugate-setting of the theorem, our posterior p(θ|D) = qλ (θ) ∝ exp(hT(θ), λi)
is an exponential family and our likelihoods p(Di |θ) ∝ exp(−`i (θ)) are of the conjugate form`i (θ) = −hλ̃i , T(θ)i + const. Under this assumption, Eq. 28 can be written as follows:p(θ|D\i ) = qλ\i (θ) ∝ exp(hλ − λ̃i , T(θ)i).

(29)

Since two minimal and regular exponential families are only equal if their natural-parameters areequal, this equation can be equivalently written as follows:
λ\i − λ = −λ̃i .

(30)

˜ q (θ) [`i (θ)],−λ̃i = ∇µ hµ, −λ̃i i = ∇µ hEqλ (θ) [T(θ)], −λ̃i i = ∇E
λ(31)

Now noticing that we have

we arrive exactly at our MPE in Eq. 4, where we used linearity of the expectation and the definitionof `i in the last step of Eq. 31.

C

MPE for Beta-Bernoulli conjugate model

Using the Beta-Bernoulli model as an example, we demonstrate that for conjugate models, the MPE
recovers the exact deviations. Let us denote the prior as p(θ) = Beta(θ|α0 , β0 ) and per-example
likelihood as p(yi |θ) = Ber(yi |θ). Then we are ready to introduce the posterior over the full dataset
D, qλ∗ (θ) = Beta(θ|α∗ , β∗ ), as well as the posterior over a perturbed dataset D\i where the i’th\i\iexample is removed, qλ\i (θ) = Beta(θ|α∗ , β∗ ), whose parameters are given by:∗α∗ = α0 +NXyj − 1,β∗ = β0 −j=1\iα∗ = α0 +

N
X

N
X

yj + N − 1

(32)

j=1
\i

yj − 1,β∗ = β0 −

j=1,
j6=i

N
X

yj + N − 2

(33)

j=1,
j6=i

Then the deviations can be simply obtained,\i\iα∗ − α∗ = −yi ,β∗ − β∗ = yi − 1(34)

We now proceed to show that Eq. 34 can be straightforwardly obtained using the MPE. For the Beta
distribution qλ (θ) = Beta(θ|α, β), we have λ = (α − 1, β − 1) and µ = Eqλ (log θ, log(1 − θ)).
Due to conjugacy, the exact posterior is obtained by running a single update of the BLR with ρ = 1
in Eq. 3. It turns out for this model, the deviations in the natural-parameter space of Eq. 4 simplify
to the deviations considered in Eq. 34:


\i
\i
\i
λ̂∗ − λ∗ = λ\i∗ − λ∗ = α∗ − 1 − (α∗ − 1), β∗ − 1 − (β∗ − 1)
(35)


\i
\i
= α∗ − α∗ , β∗ − β∗The natural gradients can be easily obtained by exploiting linearity in µ similar to Eq. 6. Defining`i (θ) := − log p(yi |θ), we obtain:e q [`i (θ)] = ∇µ Eq [− log θyi (1 − θ)1−yi ]∇Eλ∗λ∗= ∇µ Eqλ∗ [−yi log θ − (1 − yi ) log(1 − θ)]= ∇µ [−Eqλ∗ [log θ] yi + Eqλ∗ [log(1 − θ)] (yi − 1)]= (−yi , yi − 1) .
Combining Eq. 35 and Eq. 36, we recover Eq. 34.
15

(36)

D

MPE for Sparse Variational Gaussian Process

Sparse variational GP (SVGP) methods optimize the following variational objective to find a Gaussian posterior approximation qλ (u) over function values u := (f (z1 ), f (z2 ), . . . , f (zM )) whereZ := (z1 , z2 , . . . , zM ) is the set of inducing inputs with M  N :L(m, Σ, Z, φ) :=NXEqλ (fi ) [log p(yi |fi )] − DKL (qλ (u) k p(u))

(37)

i=1

where p(u) := N (u|0, Kuu ) is the prior with Kuu as the covariance function κ(·, ·0 ) evaluated at−1>2Z, qλ (fi ) = N (fi |a>
i m, ai Σai + σi ) is the posterior marginal of fi = f (xi ) with ai := Kuu kui
and σi2 := κii − a>i Kuu ai as the noise variance of fi conditioned on u. The objective is also used
to optimize hyperparameters φ and inducing input set Z.
We can optimize Eq. 37 using the BLR for which the resulting update is identical to the variational
online-newton (VON) algorithm (see Eqs. 7 and 8 in [20]):ih(38)
St+1 = (1 − ρ)St + ρ A> diag(β t )A + K−1
uu
h

i
>
>
mt+1 = S−1
t+1 (1 − ρ)St mt − ρ A et − A diag(β t )Amt
h

i
>
>
= S−1
(1
−
ρ)S
+
ρA
diag(β
)A
m
−
ρA
e
t
t
t
t
t+1
h
i

−1
−1
= St+1 St+1 − ρKuu mt − ρA> et
(39)
h

i
>
−1
= S−1
t+1 St+1 mt − ρ A et + Kuu mthi>−1= mt − ρS−1
t+1 A et + Kuu mt
where A is a matrix with a>i as rows, and et and β t are vectors of eit and βit respectively computed
as follows:
βit = Eqλt (fi ) [−∇2fi fi log p(yi |fi )]eit = Eqλt (fi ) [−∇fi log p(yi |fi )],(40)

These result from expressing the natural gradient in terms of the gradient and Hessian of the loglikelihood. For Gaussian likelihood, the updates in Eqs. 38 and 39 coincide with the method of [15],and for non-Gaussian likelihood they are similar to the natural-gradient method by [34], but we use
the specific parameterization of [21]. An alternate update rule in terms of site parameters is givenby [1] (see Eqs. 22-24).
We are now ready to write the MPE,e q (f ) [− log p(yi |fi )] = (eit − βit a> m∗ )ai ,
∇E
i
i
λt

>
1
2 βit ai ai



.

(41)

This used Eqs. 38 and 39 although it can also be derived starting from the tied parameterization in[1]. Next, plugging Eq. 41 into Eq. 4,\i\iŜt m̂t − St mt = ai eit − βit ai a>i mt ,\i− 12 Ŝt + 12 St = 12 βit ai a>
i ,

(42)

and then rearranging it to get the deviation over the mean (following the same steps as Eq. 13) and\iusing the approximation Σ̂t ≈ Σt , we obtain,\i\im̂t − mt = Σ̂t ai eit ≈ Σt ai eit

(43)
−f

If we consider Bernoulli likelihood with y ∈ {0, 1} and link function σ(f ) = 1/(1 + e ) (i.e.,
sigmoid) then eit = Eqλt (fi ) [σ(fi ) − yi ] which can be interpreted as the prediction error. This also
holds for categorical likelihood or any other GLM likelihood.
We next demonstrate the bi-linear relationship by considering the deviation in the mean of the posterior marginal fi (m) := a>
i m,
\i

\i

>
fi (mt ) − fi (mt ) ≈ a>i (m̂t − mt ) = ai Σt ai eit = vit eit

where vit = a>i Σt ai is the marginal variance of fi .
16

(44)

E

Extension to Non-Differentiable Loss function

For non-differentiable cases, we can rewrite the BLR of Eq. 10 as,
mt = mt−1 − ρΣt ∇m Eqλt−1 [L(θ)],

St = (1 − ρ)St−1 + 2ρ∇Σ Eqλt−1 [L(θ)].

(45)

This follows by using [22, Eq. 10-11] where we take derivative outside the expectation instead ofinside. This is valid because the expectation of a non-differentiable function is still differentiable(under some regularity conditions). The same technique can be applied to Eq. 11 to gete q [`i ] = (∇m Eq [`i ] − 2∇Σ Eq [`i ]m, ∇Σ Eq [`i ]) ,
∇E
λ
λ
λ
λ
\i

(46)

\i

and proceeding in the same fashion we can write: m̂t − mt = Σ̂t ∇m Eqλt [`i (θ)]. This is theextension of Eq. 13 to non-differentiable loss functions. This is a much more principled approachthan [26] who used an ad-hoc smoothing of the non-differentiable loss. The smoothing here is
automatically handled using the Bayesian framework.

F

MPE for Deep Learning Optimizers

F.1

Adam

The Adam and RMSprop deep-learning optimizers [24] can be derived from the BLR updatein Eq. 10 as described in [20, 22]. The MPE from Eq. 4 therefore also applies to both RMSprop andAdam. Consider the following update equations of Adam:
rt = β1 rt−1 + (1 − β1 )gt ,st = β2 st−1 + (1 − β2 ) (gt ◦ gt ),p
θ t = θ t−1 − ρ rt /( ŝt + ).

(47)
(48)
(49)

The gradient of the loss gt = ∇L(θ t−1 ) is evaluated for a minibatch of examples. a ◦ b denotes anelement-wise product between vectors a and b. β1 and β2 are coefficients for the running averages,
ρ is a learning rate and  a small damping to stabilize the method.
To evaluate Eq. 19 and predict generalization during the training, prediction variances vit are required. Due to its connection to the BLR update Eq. 10, we can use the second moment vector st of the Adam optimizer in Eq. 48 to approximately estimate these variances. This vector
maintains an exponential running average of the squared gradients over the training iterations. We
construct a diagonal covariance matrix Σt = diag(σ 2t ) with σ 2t = 1/(N st + δ) as the diagonal entries. N is the number of training examples, and δ is the regularization parameter of an
L2 regularizer R(θ) = 21 δkθk2 . Analogous to Eq. 15, we obtain the prediction variances as
vit = ∇fi (θ t )> Σt ∇fi (θ t ). We evaluate the prediction errors as eit = σ(fi (θ t )) − yi . Dependingon the application scenario, the sensitivies do not need to be evaluated at every step, and can for
example be evaluated only periodically during training when needed.
F.2

Improved Bayesian Learning Rule (iBLR)

Rather than using Adam or RMSprop to estimate the covariances, we can also use a more faithfulapproximation of the BLR update in Eq. 10. This is expected to give better results. In practice, we
use the iBLR optimizer [27], which implements an improved version of the BLR iterations Eq. 10.
Similarly to Eq. 10, the method updates a Gaussian posterior over the weights during training, but
the improved version makes it more easily applicable to nonconvex problems. Essentially, iBLR
entails a slight modification of Eq. 10 to ensure that the estimated precision stays positive definite.
The iBLR [27] can be used to derive an Adam-like optimizer, see also [27, Fig. 1]:
rt = β1 rt−1 + (1 − β1 )gt ,θ t = θ t−1 − ρ rt /st ,

(50)
(51)

st = st−1 + (1 − β2 ) ht + 12 (1 − β2 )2 ht ◦ s−1 ◦ ht ,2(52)

where gt ≈ EN (θt ,Σt ) [∇L] and ht ≈ EN (θt ,Σt ) [∇ L] are approximations of the expected gradientand Hessian, and σ 2t = 1/(N st ) are the diagonal entries of Σt = diag(σ 2t ). As in [27, Fig. 1]
17

we use one random sample to approximate the expectations and use the reparametrization trick toestimate the Hessian via gradients only.
The updates Eq. 50 – Eq. 52 essentially are a small modification of the Adam optimizer, but thanks
to the random sampling the method performs approximate Bayesian inference. This, along withan arguably more accurate estimate of second-order information is expected to lead to improved
variance estimates. The final prediction variances vit and prediction errors eit are computed from stand θ t in the same way as described for Adam in App. F.1.

G

Experimental Details

G.1

Details of "Do estimated deviations correlate with the truth?"

For the experiments in Fig. 2, we train models on all training data to obtain θ ∗ . For θ \i , we exclude
the i-th example and warmstart the training at θ ∗ . For both cases, we use SGD with momentum
parameter of 0.9 and a cosine learning-rate scheduler. The regularization parameter δ is chosen by
grid search. The remaining hyperparameters are shown in Table 1.
The details of the models are as follows. For MNIST, we use a fully-connected multilayer perceptron(MLP) with two hidden layers of 500 and 300 neurons each. It has 545810 parameters. For FMNIST,
we use a LeNet5 architecture. It is a standard convolutional neural network (CNN) with three
convolution layers followed by two fully-connected layers. It has 61706 parameters. For CIFAR10,
we use a CNN architecture consisting of three convolution layers followed by three fully-connected
layers. This architecture is used in benchmarks such as the DeepOBS suite [35]. It has a parametercount of 895210.
\i

Dataset

Model

B

δ

E∗

LR∗

∗
LRmin

E \i

LR\i

LRmin

MNIST
FMNISTCIFAR10MLP (500, 300)
LeNet5
CNN

256
256
512

100
100
250

500
300
500

10−2
10−1
10−2

10−3
10−3
10−4

300
200
300

10−3
10−3
10−4

10−4
10−4
10−6

Table 1: Hyperparameters for predicting true sensitivity in Fig. 2. B, E and LR denote batch size,
training epochs and learning rates. The superscripts ∗ and \i indicate hyperparameters for training
on all data and warmstarted leave-one-out training, respectively. The subscript min stands for theminimum learning rate of the learning rate scheduler.

We now describe the group removal experiments. For the MNIST result in Fig. 3(b) we use themodel from Table 1 with the same hyperparameters. The exception are the last three hyperparameters in the table, which differ between leave-one-out and leave-group-out training. For binary USPS
in Fig. 3(a), we train a small MLP with three hidden layers with 30 neurons. We classify the digits 3
and 5. For the training on all data we use 500 epochs with a learning rate of 10−3 . We use a batchsize of 32 and a regularization parameter of 5. For the leave-group-out training on both datasets we
use 1000 epochs and the same learning rate as for training on all data. We initialize the MLPs at θ ∗ .
Variance computation requires inversion of matrices. For large problems, we use a Kroneckerfactored Laplace variance approximation as implemented in the laplace package [9]. We observea reasonable speed-accuracy trade-off on both fully-connected and convolutional neural networks.
G.2

Details of "Predicting the effect of class removal on generalization"

For the MNIST experiment in Fig. 3(c), we use the large MLP and LeNet5. For FMNIST in Fig. 1(c),we use the small MLP and LeNet5. The hyperparameters are given in Table 2. All models are trainedwith a regularization parameter of 100 and a batch size of 256. The leave-class-out training is donefor 1000 epochs and is warmstarted at θ ∗ . We use the same Kronecker-factored Laplace varianceapproximation as before.
18

\iDataset

Model

E∗

LR∗

∗
LRmin

LR\i

LRmin

MNIST
MNIST
FMNISTFMNIST

MLP (500, 300)
LeNet5MLP (32, 16)
LeNet5

500
300
300
300

10−2
10−1
10−2
10−1

10−3
10−3
10−3
10−3

10−4
10−5
10−5
10−4

10−5
10−6
10−6
10−5

Table 2: Hyperparameters for the class removal experiments in Fig. 3(c) and Fig. 1(c). B, E and LR
denote batch size, training epochs and learning rates. The superscripts ∗ and \i indicate hyperparameters for training on all data and warmstarted leave-class-out training, respectively. The subscript
min stands for the minimum learning rate of the learning rate scheduler.
1.8

Test error
Test NLL
Train-LOO-MPE

0.1

4%

100

101

102

103

L2-regularization parameter

50%

12%

3%
2%

0.3

1.2

10%
102

101

103

40%

101

L2-regularization parameter(a) MNIST, MLP

60%

15%

0.5

Test error

NLL

0.2

102

30%
103

L2-regularization parameter(b) FMNIST, LeNet5(c) CIFAR10, CNN

Figure 7: The MPE can accurately estimate the LOO-CV curve for predicting generalization andtuning of the L2 -regularization parameter on MNIST, FMNIST and CIFAR-10.
G.3

Details of "Estimating the leave-one-out cross-validation curves for hyperparametertuning"We show results on MNIST (large MLP) in Fig. 4(a), on FMNIST (LeNet5) in Fig. 4(b) and on
CIFAR10 (CNN) in Fig. 4(c). Fig. 7 shows the same experiments with test errors included.. In
We evaluate a certain number of values within a log-spaced range. For visualization purposes wecalculate a moving average of the plotted lines with a smoothing window. The details are in Table 1.
DatasetMNISTFMNISTCIFAR10Model
MLP (500, 300)
LeNet5CNN

Number of δs
96
96
30

Range
0

Smoothing3

10 − 10101 − 103
101 − 103

3
5
3

Table 3: Experimental settings for the leave-one-out cross-validation curves for hyperparametertuning in Fig. 4. The table gives information on the datasets, the models, the number of valuesof the regularization parameter δ evaluated, the range of the values and the smoothing window for
calculating a moving average for visualization.
All models are trained from scratch. As optimizers we use Adam for FMNIST, Adamw [28] for
CIFAR10 and SGD with a momentum of 0.9 for MNIST. We use a cosine learning rate scheduler
to anneal the learning rate. The other hyperparameters are equivalent to the settings of the models
trained on all data from the leave-one-out experiments in Table 1. The exception is the number of
epochs for CIFAR10. We train for 150 epochs. As before, we use a Kronecker-factored Laplaceapproximation for variance computation.
G.4

Details of "How do sensitivities evolve during training?"

Bayesian logistic regression: For the experiment in Fig. 6(a), we consider a generalized linear
model f i = Θxi on the MNIST dataset using the raw pixels xi ∈ RP as features. We denote thepredicted logits by f i ∈ RC and the parameters as Θ ∈ RC×P where P = 785 and C = 9 for theconsidered MNIST dataset. We use the negative log-likelihood of the categorical distribution as aloss function and use regularization parameter δ = 0.1.
19

We run the iteration of Eq. 10 for 125 stochastic updates with batch-size 200, reaching around 91%
when predicting at the mean mt . We use linear learning-rate decay from 0.005 to 0.001 for the meanm and a learning rate of 10−5 for the precision S. The expectations in Eq. 10 are approximated using
three samples drawn from the posterior.
Given the mean mt and precision St we use the last equation in Eq. 13 to understand the sensitivityof the model at iteration t. In Fig. 6(a) at iteration t = 5, 10, 25, 125 we compute for each example• its variance vi,t,c = x>i Σt,c xi , where Σt,c is the c-th block of the covariance matrix Σtgiven as the inverse of the precision matrix St ,• and its prediction error ei,t = Eg∼N (f i,t ,vi,t ) [σ(g) − yi ], f i,t = mt · xi .
We used 150 samples to approximate the expectation for the prediction error. Analogous to Eq. 16,
PC0
this gives us a sensitivity at iteration t for each data point:
c=1 |vi,t,c | · |ei,t,c | · |σ (mxi )c |.
For Fig. 6(a) we sorted all examples according to this sensitivity at iteration t = 125 in decreasingorder and then plot the average sensitivities of examples in 60 groups with 100 examples in eachgroup.
iBLR: For the experiments in Fig. 6(b), Fig. 6(c) and Fig. 6(d) we consider neural network models
f i (θ t ) on MNIST, FMNIST and CIFAR10. We obtain test accuracies of 0.985, 0.913 and 0.809,
respectively. These values are standard for the used models if the data is not augmented. For
CIFAR10 we here use a ResNet–20 with filter response normalization [37]. The number of classes
and output neurons is C = 10 for all datasets. We use the negative log-likelihood of the categorical
distribution as a loss function. We run the iterations of Eq. 10, adjusted according to the iBLR
update equations (50) – (52). The expectations are approximated with one sample drawn from theposterior. We use a cosine learning rate scheduler with a start learning rate of 0.1 and anneal to zero
over the epochs. Other details are in Table 4.
Given the mean mt and precision St we use the last equation Eq. 13 to understand the sensitivity ofthe model at iteration t. In the figures, we compute for each example• its variance vi,t = ∇fi (θ t )T Σt ∇fi (θ t ), where ∇fi (θ t )T ∈ RC×P and Σt ∈ RC×C
contains the per-class weight variances on the diagonal as obtained by inversion of the
precision matrix St ,• and its prediction error ei,t = Eg∼N (f i,t ,vi,t ) [σ(g) − yi ], f it = f i (mt ).
We used 150 samples to approximate the expectation for the prediction error. We sorted all examples
according to this sensitivity at the last plotted epoch in the figures in decreasing order. We plot thesensitivities, where we group examples with similar sensitivity into overall 200 groups.
Dataset

Model

B

δ

E

MNIST
FMNISTCIFAR10MLP (500, 300)
LeNet5ResNet–20

256
256
512

30
60
25

100
100
300

Table 4: Experimental settings for analyzing the evolution of sensitivities during training. B denotes
the batch size and E are the training epochs. δ is the regularization parameter

G.5

Details of "Predicting generalization during the training: "We conduct experiments to predict the generalization of models during the training. We use an
estimate obtained from the MPE (Train-LOO-MPE) and the influence function, which is a special
case of MPE. The estimate is based on training data alone and is calculated with Eq. 19. In Fig. 5,
we show results with a LeNet5 on FMNIST. In Fig. 1(b) we use a ResNet–20 with Filter-ResponseNormalization on the CIFAR10 dataset. In Fig. 10 and Fig. 11 we additionally report the test error
over the training epochs for these experiments. Fig. 8 and Fig. 9 contain results for MNIST and
CIFAR10 that are not included in the main text. For MNIST, we evaluate both on a small MLP anda LeNet5 architecure. For the additional CIFAR10 results, we use the previously described CNN
20architecture. In Eq. 19 the computation of prediction variances and prediction errors is required.
The quantities are evaluated periodically during the training, which is indicated with markers in the
figures.
MPE with Adam and iBLR: For the MPE experiments, we use estimates based on the Adam optimizer and the iBLR optimizer. Details for Adam and iBLR are described in App. F.1 and App. F.2,
respectively. The experimental results suggests that a more accurate variance computation improves
the prediction.
Influence function: As described in the main text, the influence function as given by Eq. 14 isa special case of the MPE with Gaussian posterior in Eq. 13. The expectations from Eq. 13 areapproximated at the mean using the Delta method.
We highlight some differences of our implementation to the influence function commonly used indeep learning [26]. The right-hand expression in Eq. 14 is equivalent to the influence estimationin parameters at convergence of [26]. [26] also evaluate their influence function for non-converged
models and non-convex objectives using a quadratic approximation of the loss. They do, however,
only evaluate close to a local minimum. In contrast to that, we compute influence also throughout\ithe training. We use the left-hand side of Eq. 14), θ̂ t − θ t ≈ Σt ∇fi (θ t )ei (θ t ), and Taylor’s approximation to obtain the deviation in function outputs in Eq. 15. That expression is required toevaluate the Train-LOO estimate in Eq. 19. The Train-LOO estimate can be used to approximate
the average test loss based on all training examples. Opposed to that, [26] evaluate the effect of perturbing a training example on a single test example using the expression for the parameter-influenceand the chain rule.
To estimate the Hessian, [26] use implicit Hessian-vector products and approximate solutions of astochastic conjugate gradient solver. To deal with Hessian estimates that are not positive-definite,they add a damping term. We use a generalized Gauss-Newton approximation (GGN) [29] to the
Hessian, based on an implementation from the laplace package [9]. The GGN ensures positivedefiniteness when applied at approximately optimal points or even intermediate points in the loss
landscape of nonconvex problems such as deep learning. The GGN scales quadratically with thenumber of parameters, which is computationally expensive for large neural networks. Structural
approximations of the GGN matrix reduce both memory consumption and computational cost of
inversion. We use a diagonal approximation that neglects off-diagonal elements of the Hessian forfast inversion. With the GGN approximation we get a Hessian estimate at the iterate t as ∇2 L(θ t ) =
PN
0
>
2
−1
, the prediction variances and
i=1 ∇fi (θ t )σ (fit )∇fi (θ t ) + δI. Using Σt = (∇ L(θ t ))prediction errors in Eq. 13 can be computed as described for the Adam optimizer in App. F.1. For
future work, we would consider relaxing some of the approximations.
Details of optimization procedures: The experimental details are listed in Table 5. In all experiments, we use a grid search to determine the regularization parameter δ. For the experiments with
the influence function, we use the SGD optimizer. The exception is the experiment on the FMNIST
dataset. There, we use the AdamW optimizer [28]. We use a weight decay factor of δ/N replacing
the explicit L2 -regularization term in the loss Eq. 1. The regularizer R(θ) is set to zero there.

21

Dataset
MNIST

MNIST

FMNISTCIFAR10CIFAR10

Method

LR

LRmin

B

δ

Infl. function (SGD)
MPE (Adam)
MPE (iBLR)

−3

10
10−1
10−2

−4

10
10−2
10−4

256
256
256

80
80
80

Infl. function (SGD)
MPE (Adam)
MPE (iBLR)

10−3
10−2
10−2

10−4
10−6
10−4

256
256
256

60
60
60

Infl. function (AdamW)
MPE (Adam)
MPE (iBLR)

10−3
10−1
10−1

10−3
0
0

256
256
256

60
60
60

Infl. function (SGD)
MPE (Adam)
MPE (iBLR)

10−1
10−3
10−2

0
10−9
10−4

512
512
512

250
50
200

MPE (iBLR)

10−1

0

512

250

Model
MLP (32, 16)

LeNet5

LeNet5CNN
ResNet–20

Table 5: Experimental settings for predicting generalization during the training. B and E denote thebatch-size and training epochs, respectively. LR and LRmin are the start and end learning rates of
the cosine learning rate scheduler. δ is the regularization parameter.

25

0.0 0

50 75 100Epochs(a) Infl. Function with SGD, MLP

0.0 0

50 75 100Epochs25

50 75 100Epochs(d) Infl. Function with SGD, LeNet5

25

Test error

50 75 100Epochs

1.0

20

40%
1

10
00

10%

(c) MPE with Adam, MLP

25

50 75
Epochs100(e) MPE with Adam, LeNet5

NLL

20%

Train-LOO-MPE

Test error

0.5

20%

NLL
0.6

2
40%

NLL

25

1.2

(b) MPE with Adam, MLP

1.0

0.0 0

20%

0.6

10%

Test NLL

0.0 0

1.2

Test error
Test NLL
Train-LOO-MPE

Test error

0.6

20%

Test error

NLL

1.2

1.8

40%

1.8

NLL

Test error
Test NLL
Infl. Function

Test error

1.8

0.5

0.0 0

20%
25

50 75 100Epochs(f) MPE with iBLR, LeNet5

Figure 8: Panels (a), (b) and (c) show estimates of the test NLL using influence function and MPE
during training of a MLP on MNIST. Panels (d), (e) and (f) show results with a LeNet5. For influencefunction, we use a generalized Gauss-Newton approximation of the Hessian with diagonal matrix
structure. The results suggest that MPE with iBLR can faithfully estimate the test NLL and improves
upon applying influence function. It performs better than the estimation with MPE that computes
variances heuristically based on the Adam optimizer.

220.5 0 60 120 180 240 30040%
Epochs20.0

1.00

10.0 00.75(a) Infl. Function with SGD

40 80 120
EpochsTest error

1.0

1.25

75%
Test error
Test NLL
Train-LOO-MPE
50%

2.0

NLL

60%

30.0

Test NLL

Test Error

NLL

1.5

Train-LOO-MPE

Test error
Test NLL
Infl. Function 80%

2.0

1.5
1.0

25%

0.5 0

(b) MPE with Adam

40

80 120Epochs(c) MPE with iBLR

Figure 9: Panel (a) shows an estimate of the test NLL with influence function during training of a
CNN on CIFAR10. We use a generalized Gauss-Newton approximation of the Hessian with diagonal matrix structure. Panels (b) and (c) display the Train-LOO estimate obtained from MPE. The
results suggest that MPE with iBLR can faithfully estimate the test NLL and improves upon applying influence function. It performs better than the estimation with MPE that computes variances
heuristically based on the Adam optimizer.

1.0

18%

NLL

15%
0.5

12%
0

25

3.0
1.5
0.0 0

50 75 100Epochs(a) Infl. Function with AdamW

25

50 75
Epochs

40%

1.0

20%

0.5

100

Test error
Test NLL
Train-LOO-MPE 20%

Test Error

Test error
Test NLL
Infl. Function0(b) MPE with Adam

25

10%50 75 100Epochs(c) MPE with iBLR

Figure 10: Panel (a) shows an estimate of the test NLL with influence function during training of a
LeNet5 on FMNIST. We use a generalized Gauss-Newton approximation of the Hessian with diagonal matrix structure. Panels (b) and (c) display the Train-LOO estimate obtained from MPE. The
results suggest that MPE with iBLR can faithfully estimate the test NLL and improves upon applying influence function. It performs better than the estimation with MPE that computes variances
heuristically based on the Adam optimizer.

Test error
75%
Test NLL
Train-LOO-MPE
50%

Test Error

NLL

1.8
1.2
0.6
0

100

200

Epochs300

25%

Figure 11: Train-LOO-MPE with the iBLR optimizer faithfully estimates the test NLL during training of a ResNet–20 on CIFAR10.

23