SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left. \frac {d \Theta}{d \varepsilon} \right| _ {\varepsilon = 0} = - \left(\nabla_ {\theta} ^ {2} L \left(\theta_ {0}\right)\right) ^ {- 1} \nabla_ {\theta} l _ {x} \left(\theta_ {0}\right). \tag {14}
$$
Proof. If  $\theta_0$  is a stationary point of  $L$ , then  $\nabla_{\theta}\mathcal{L}(\theta_0,0) = 0$  in (3). This implies that  $\nabla_{\theta}\mathcal{L}(\Theta (\varepsilon),\varepsilon) = 0$  in (3), which is exactly the statement that  $\Theta (\varepsilon)$  is a stationary point of  $\mathcal{L}$ . If  $\theta_0$  is a local (strict) minimum of  $L$ , it is not just a stationary point, but the Hessian at  $\theta_0$  is also positive definite. By continuity, for sufficiently small  $\varepsilon$ , also the Hessian  $\nabla_{\theta |\Theta (\varepsilon)}^2\mathcal{L}$  will be positive definite so that  $\Theta (\varepsilon)$  will be a local (strict) minimum. Finally, (14) follows from applying (3) to the variation  $\mathcal{L}(\theta ,\varepsilon) = L(\theta) + \varepsilon l_x(\theta)$ .
For convenience, we first recall the statement of Theorem 2.
Theorem. Assume that the model is trained for  $T$  time-steps with stochastic gradient descent. Denoting by  $H_{t}$  the Hessian  $\nabla_{\theta |\theta_{0,t}}^2 L_t$ , then the final parameters  $\theta_{\varepsilon ,T}$  satisfy:
$$
\nabla_ {\varepsilon | 0} \theta_ {\varepsilon , T} = - \sum_ {t = 0} ^ {T - 1} \eta_ {t} \nabla_ {(\varepsilon , \theta) \mid (0, \theta_ {0, t})} ^ {2} \mathcal {L} _ {t} - \sum_ {t = 0} ^ {T - 1} \eta_ {t} H _ {t} \nabla_ {\varepsilon | 0} \theta_ {\varepsilon , t}. \tag {15}
$$
Proof. The parameters  $\theta_{\varepsilon ,t}$  obey a recurrence relation:
$$
\theta_ {\varepsilon , t} - \theta_ {\varepsilon , t - 1} = - \eta_ {t - 1} \nabla_ {\theta} \mathcal {L} _ {t - 1} \left(\theta_ {\varepsilon , t - 1}, \varepsilon\right), \tag {16}
$$
which can be solved to give
$$
\theta_ {\varepsilon , T} = - \sum_ {t = 0} ^ {T - 1} \eta_ {t} \nabla_ {\theta} \mathcal {L} _ {t} \left(\theta_ {\varepsilon , t}, \varepsilon\right); \tag {17}
$$
then (15) follows immediately by applying  $\nabla_{\varepsilon |0}$ , i.e. computing the Jacobian wrt.  $\varepsilon$  at the origin.
The first step in the proof of Theorem 3 is a discrete version of Gronwall's Lemma [Gro19].
Lemma 1. Let  $\{u_t\}$ ,  $\{\alpha_t\}$  and  $\{\beta_t\}$  be sequences such that  $\beta_t \geq 0$  and
$$
u _ {t} \leq \alpha_ {t} + \sum_ {s = 0} ^ {t - 1} \beta_ {s} u _ {s}. \tag {18}
$$
Note that we assume that (18) holds for  $t \geq 1$  and we consider  $u_0$  an initial condition. Then:
$$
u _ {T} \leq \alpha_ {T} + \beta_ {0} \prod_ {s = 1} ^ {T - 1} \left(1 + \beta_ {s}\right) u _ {0} + \sum_ {s = 1} ^ {T - 1} \alpha_ {s} \beta_ {s} \prod_ {k = s + 1} ^ {T - 1} \left(1 + \beta_ {k}\right). \tag {19}
$$
If  $\{\alpha_t\}$  is non-decreasing in  $t$  we then get:
$$
u _ {T} \leq \beta_ {0} \prod_ {s = 1} ^ {T - 1} \left(1 + \beta_ {s}\right) u _ {0} + \alpha_ {T} \left(1 + \sum_ {s = 1} ^ {T - 1} \beta_ {s} \prod_ {k = s + 1} ^ {T - 1} \left(1 + \beta_ {k}\right)\right). \tag {20}
$$
Moreover, defining  $u_{t}$  by setting (18) to be an equality, shows that (19) is sharp.
Proof. We first observe that if we set  $v_{0} = u_{0}$  and build  $v_{t}$  by declaring (18) to be an equality, then  $v_{t} \geq u_{t}$  for any  $t$ . This is true by induction as:
$$
u _ {t + 1} - \alpha_ {t + 1} \leq \sum_ {s = 0} ^ {t} \beta_ {s} u _ {s} \leq \sum_ {s = 0} ^ {t} \beta_ {s} v _ {s} = v _ {t + 1} - \alpha_ {t + 1}. \tag {21}
$$
We thus focus on bounding  $v_{t+1}$ ; we note that
$$
\left(v _ {t + 1} - \alpha_ {t + 1}\right) - \left(v _ {t} - \alpha_ {t}\right) = \beta_ {t} \left(v _ {t} - \alpha_ {t}\right) + \beta_ {t} \alpha_ {t}; \tag {22}
$$
$$
\begin{array}{l} v _ {t + 1} - \alpha_ {t + 1} = (1 + \beta_ {t}) (v _ {t} - \alpha_ {t}) + \beta_ {t} \alpha_ {t} \tag {23} \\ = (1 + \beta_ {t}) (1 + \beta_ {t - 1}) \left(v _ {t - 1} - \alpha_ {t - 1}\right) + (1 + \beta_ {t}) \beta_ {t - 1} \alpha_ {t - 1} + \beta_ {t} \alpha_ {t}; \\ \end{array}
$$
then (19) follows by induction and letting  $t + 1 = T$ . Finally, if  $\{\alpha_t\}$  is non-decreasing in  $t$  we can simply replace  $\alpha_s$  with  $\alpha_T$  obtaining (20). The sharpness follows by considering the sequence  $\{v_t\}$  we defined in the proof.
For convenience, we first recall the statement of Theorem 3.
Theorem. In the setting of Theorem 2 assume that, for  $t \leq T$ ,  $\theta_{\varepsilon,t}$  lies in a bounded region  $R$  such that
$$
\sup  _ {\varepsilon , \theta \in R} \| \nabla \mathcal {L} _ {t} (\theta , \varepsilon) - \nabla \mathcal {L} (\theta , 0) \| \leq C \varepsilon \tag {24}
$$
$$
\left\| \theta_ {\varepsilon , T} - \theta_ {0, T} \right\| \leq C \varepsilon \sum_ {s <   T} \eta_ {s} \left(1 + \exp \left(2 A \sum_ {s <   T} \eta_ {s}\right)\right). \tag {25}
$$
Proof. The idea is to apply Lemma 1 as we would in the case of a standard ODE. Let us compare the evolution of  $\theta_{\varepsilon, t}$  and  $\theta_{0, t}$ :
$$
\theta_ {\varepsilon , t} = \theta_ {\text {i n i t}} - \sum_ {s = 0} ^ {t - 1} \eta_ {s} \nabla_ {\theta} \mathcal {L} \left(\theta_ {\varepsilon , s}, \varepsilon\right) \tag {26}
$$
$$
\theta_ {0, t} = \theta_ {\text {i n i t}} - \sum_ {s = 0} ^ {t - 1} \eta_ {s} \nabla_ {\theta} \mathcal {L} \left(\theta_ {0, s}, 0\right); \tag {27}
$$
$$
\begin{array}{l} \left\| \theta_ {\varepsilon , t} - \theta_ {0, t} \right\| \leq \sum_ {s = 0} ^ {t - 1} \eta_ {s} \| \nabla_ {\theta} \mathcal {L} \left(\theta_ {\varepsilon , s}, \varepsilon\right) - \nabla_ {\theta} \mathcal {L} \left(\theta_ {0, s}, 0\right) \| \tag {28} \\ \leq C \varepsilon \sum_ {s = 0} ^ {t - 1} \eta_ {s} + \sum_ {s = 0} ^ {t - 1} \eta_ {s} A \| \theta_ {\varepsilon , s} - \theta_ {0, s} \|; \\ \end{array}
$$
we now just need to rephrase this inequality in terms of Lemma 1:
$$
\underbrace {\left\| \theta_ {\varepsilon , t} - \theta_ {0 , t} \right\|} _ {u _ {t}} \leq \underbrace {C \varepsilon \sum_ {s = 0} ^ {t - 1} \eta_ {s}} _ {\alpha_ {t}} + \sum_ {s = 0} ^ {t - 1} \underbrace {\eta_ {s} A} _ {\beta_ {s}} \underbrace {\left\| \theta_ {\varepsilon , s} - \theta_ {0 , s} \right\|} _ {u _ {s}}; \tag {29}
$$
we note that  $u_0 = 0$  as both dynamics start at  $\theta_{\mathrm{init}}$  and that  $\alpha_{t}$  is non-decreasing in  $t$ . We then have that
$$
\begin{array}{l} u _ {T} \leq \alpha_ {T} \left(1 + \sum_ {s = 1} ^ {T - 1} \beta_ {s} \prod_ {k = s + 1} ^ {T - 1} \left(1 + \beta_ {k}\right)\right) \\ \leq \alpha_ {T} \left(1 + \sum_ {s = 1} ^ {T - 1} \beta_ {s} \prod_ {k = s + 1} ^ {T - 1} \exp \left(\beta_ {k}\right)\right) \\ \leq \alpha_ {T} \left(1 + \exp \left(\sum_ {s = 1} ^ {T - 1} \beta_ {k}\right) \sum_ {s = 1} ^ {T - 1} \beta_ {s}\right) \tag {30} \\ \leq \alpha_ {T} (1 + \exp (\sum_ {s = 1} ^ {T - 1} \beta_ {k}) \times \exp (\sum_ {s = 1} ^ {T - 1} \beta_ {k})) \\ \leq \alpha_ {T} (1 + \exp (\sum_ {s = 1} ^ {T - 1} 2 \beta_ {k})), \\ \end{array}
$$
which is (8).
(b)
Figure 4: For BERT the predictive power of influence scores becomes 0 over time as the distance from 0 to the confidence interval of the Pearson's r becomes 0, i.e. the confidence interval contains 0.
E.1 Sketching modifications needed in the case of optimizers
The proofs of Theorems 2 and 3 were given for SGD. The argument in the case of using an optimizer would be more involved. Here we sketch the modifications needed when dealing with optimizers. An optimizer is characterized by an optimizer state,  $\sigma_{\varepsilon ,t}$ , which will also evolve in time. While for SGD we just considered the update rule for  $\theta_{\varepsilon ,t}$ , in the case of optimizers one needs to study a joint system of update rules for the parameters and the optimizer state:
$$
\theta_ {\varepsilon , t} - \theta_ {\varepsilon , t - 1} = - \eta_ {t - 1} F \left(\nabla_ {\theta} \mathcal {L} _ {t - 1} \left(\theta_ {\varepsilon , t - 1}, \varepsilon\right), \sigma_ {\varepsilon , t}\right) \tag {31}
$$
$$
\sigma_ {\varepsilon , t} - \sigma_ {\varepsilon , t - 1} = - \rho_ {t - 1} G \left(\nabla_ {\theta} \mathcal {L} _ {t - 1} \left(\theta_ {\varepsilon , t - 1}, \varepsilon\right), \sigma_ {\varepsilon , t - 1}\right). \tag {32}
$$
In the case of Theorem 2 one would then apply  $\nabla_{\varepsilon|0}$  to the joint system to obtain the update rule. For Theorem 3 one should add additional continuity in  $\varepsilon$  and Lipschitz conditions on  $F$  and  $G$ ; one should then check that these are indeed satisfied for common optimizers like Adam or Adafactor.
F Why do we use the Conjugate Residual method?
Regarding HIF, note that usually the Conjugate Gradient method is used [KL17] for computing inverse Hessian vector products. However, in our experiments the Conjugate Gradient method always yielded a time-series of Pearson Correlations  $R(t)$  oscillating around 0, thus without any predictive power. The reason is that this method assumes the Hessian to be positive-definite, which is not the case for most Neural Networks. A simple fix to the problem is to use the Conjugate Residual method which does not require the Hessian to be positive definite. We recommend to use the Conjugate Residual when applying HIF in Deep Learning; this might look like a minor technical point, but it can avoid reporting that HIF has no predictive power, when instead the issue lies in the numerical method used to compute inverse Hessian vector products.
For more discussion on the Conjugate Residual method see its Wikipedia article.
G Further empirical results on Fading of Influence scores
G.1 Fading of Influence for BERT
In the setting of Figure 2(a) we plot the distance from the confidence interval of the Pearson's r to 0 (Figure 4); once the distance becomes 0, the predictive power of influence scores has become null.
In Figure 5 we zoom in Figure 2 (a: BERT). The fading phenomenon was non affected by checkpoint selection: in Figure 6 we consider a later checkpoint and we see the same qualitative behavior. Moreover, in Figure 6 we also consider TracIn with 3 checkpoints selected using the advice in [PLKS20] – we do not see improvements and the peak is even slightly lower than for TracIn using one checkpoint. Thus, in our further experiments we have used TracIn with a single checkpoint.
In Figure 7 we zoomed in Figure 2 (b: ResNet). In this case the predictive power was never particularly high, e.g. for TracIn it quickly peaked at 0.3 but it takes more steps than in the case of BERT to reach 0. We conjecture that this is in part due to the slower divergence of parameters in
Figure 5: For BERT the predictive power of influence scores on the loss shifts degrades over time very quickly.
Figure 6: For BERT the predictive power of influence scores on the loss shifts degrades over time very quickly. Using multiple checkpoints did not improve performance of TracIn.
Figure 7: For ResNet the predictive power of influence scores faded more slowly and was never particularly high.
Figure 8: For BERT the predictive power of influence scores faded more slowly when using SGD. The "Gradient" method is TracIn.
(b)
Figure 9: The predictive power of influence scores on the loss shifts degrades over time for T5. (a) Correlation with the ground truth, with the shaded area the corresponding  $95\%$  confidence region. (b) Distance from 0 to the confidence interval, which measures when the confidence interval covers 0.
(b)
Figure 10: The predictive power of influence scores on the loss shifts degrades over time for the Vision Transformer (ViT). (a) Full range of re-training steps, (b) zoom-in in the peak and decay phase.
ResNet (compare the x-axis of (a) and (b) in Figure 1) and in part due to the use of SGD. In particular, we verified that a slower fading also takes place when fine-tuning BERT with SGD (Figure 8).
G.2 Fading of Influence for T5
We consider another NLP pre-trained model fine-tuned on SST2, T5. In Figure 9 we illustrate the same effect of fading of influence for the T5 model.
G.3 Fading of Influence for the ViT
In the case of ResNet, in our fading of influence experiments (Fig. 2 (b)) the peak is particularly low. On the same task, we trained a ViT with SGD and observe that the peak is higher, similarly to what happens with BERT, see Figure 10. We conjecture that this difference is due to the different model architecture. Note that in both cases, we observe the fading of influence as predicted by the theory.
H How many steps are needed to correct mis-predictions?
In Figure 11 we plot the average and median number of steps to correct a mis-prediction as a function of  $\varepsilon$ : for any fixed  $\varepsilon$ , the more effective a correction method is, the fewer steps are needed to correct the prediction. For BERT we see that both Proponent-correction and Opponent-tuning result in a large decrease in the number of steps to take, compared with the Baseline. For ResNet, we do not plot Opponent-tuning as it performed poorly on success-rate. For ResNet the gains of Proponent-correction are smaller but consistent as  $\varepsilon$  varies.
(b)
Figure 11: Average and median steps to correct mis-predictions. (a) BERT, (b) ResNet
Table 1: Pairs of misclassified test examples with their mislabeled training proponents
Test example	Top proponent
False positives
1	Agreed. This is dumb.	Yeah i no it sucks.
2	== Iowa-class battleships == Stupid me... I didn't even look... sorry about that... (	By the way, sorry about that stupid edit.
3	It's so silly of me to ask this question but I'm still not getting what you're trying to say. Sorry for being such an idiot! ;)	By the way, sorry about that stupid edit.
4	hahaha this whale is huge and ugly and funky doodle...lol...	A cheeseburger for you! Fatty
5	Cool you can change the content!i love sex!!	Sex in space Sweet. Thanks for the education!
6	=== A lot of poeple got killed that day ===	Yes they were Hitler Stated that Slavs were lesser races and should be salves or killed Ravenkills97 I didn't change the genres on In This Moment, all i did was put commas in between the genres. U gave me a final warning for no reason. All because I put commas in between something, like it fucking erases the genre.
7	== A question regarding Rodriguez's sexual orientation == Would the statement, "I've gone both ways. I do as I please. I am too fucking curious to sit here and not try when I can. Men are intriguing. So are chicks." be enough to label her as bisexual or LGBT?
False negatives
1	your mama is a man	grow up Get a life
2	(...) It's nice to see you finally admit it with your edit summary of, "what a terrible idiot I am". (...)	(...) you're a nerd living with your mom and you have no friends. You see? (...)
3	U r worse than obamas presidency	Are you known as a sock puppet?
4	I don't want to say sorry ,so get a life before i ruin yours buthead	Then grow up and stop spamming on my page. I hear your mommy calling. Run along.
5	(...) it's hard for me to not take the kind of stupidity evident from your edits personally (...)	(...) thats 2 against 1 buddy you lose
6	Your spelling seems to be at about the 3rd grade level, I hope you're not having sex.	Okay then. I'll just have to agree to disagree. I won't read your article and I won't give you fellatio. Now get off my page.
Table 2: Mean influence with variance and metrics for the experiments with the example in Table 1
proponents	mean influence	mean step	median step	retention
True	0.91 (±0.03)	3.0	2.5	98.1%
Permuted	0.50 (±0.21)	5.1	3.5	96.3%
Table 3: Training hyper-parameters for ResNet on CIFAR10
Hyper-parameter	value
Batch-size	128
Epochs	200
Learning rate	10-1
Learning rate scheduler	cosine decay
Optimizer	SGD
l2-regularization	10-4
I Selecting mislabeled proponents for correction
For the experiments in Section 6 we use comparatively clean datasets, so the proponents we retrieve and correct are not mislabeled. In this section we use the notoriously noisy Wikipedia Toxicity Subtypes dataset [WTD17] where for many misclassified test examples we can find mislabeled proponents used during training. Table 1 presents false positives and negatives (that is, test examples incorrectly predicted to be toxic, resp. non-toxic) with a high-scoring proponent which we consider mislabeled. It is worth noting that some examples lack context necessary to confidently verify whether they are toxic or not. In total, we collected 14 such pairs, seven for mispredictions of each kind. $^3$
Table 2 presents the mean and the median number of steps needed to be taken on the corrected proponents to achieve a change in prediction, as well as retention once there is a change. As a point of comparison, we permute the proponents withing the false positive and false negative sets: this is a more challenging baseline than selecting examples for correction randomly. Note that the influence scores for the presented examples range from  $-0.90$  to 0.92, so some of the permuted proponents still have some quite influence on the test examples they get assigned to. Still, the results confirm that correcting unlabeled top-scoring proponents is the fastest way of achieving a change in model predictions, with as few as three steps required on average and with the smallest damage to predictions on other examples (measured with retention).
J Hyper-parameters
J.1 Training
ResNet was trained on a single V100 with the hyper-parameters in Table 3; training data was augmented using torchvision using the transformations transforms.RandomCrop(32, padding=4), and transforms.RandomHorizontalFlip().
Table 4: Training hyper-parameters for BERT on SST2
Hyper-parameter	value
Batch-size	128
Steps	35000
Learning rate	10-5
Learning rate scheduler	None
Optimizer	Adam
l2-regularization	5 × 10-6
<sup>3</sup>We omitted the most upsetting example from the false negatives.
Table 5: Training hyper-parameters for T5 on SST2
Hyper-parameter	value
Batch-size	16
Maximal sequence Length	128
Epochs	4
Learning rate	10-4
Learning rate scheduler	None
Optimizer	Adam
Table 6: Training hyper-parameters for ViT on CIFAR10
Hyper-parameter	value
Batch-size	128
Epochs	300
Peak learning rate	10-3
Learning rate scheduler	1 epoch warm-up and then cosine decay
Optimizer	SGD
The ViT was trained on a single V100 with the hyper-parameters in Table 6; we used the same data augmentation as in the case of ResNet.
BERT was trained on 8 TPUv3 cores using the hyper-parameters in Table 4. The best checkpoint was selected on validation-set accuracy evaluated every 500 steps; it corresponded to 6000 steps of training.
T5 was trained on a GPU V100 using the hyper-parameters in Table 5. We selected a checkpoint based on the validation set accuracy at epoch 4, where the accuracy was  $92.9\%$ . Note that T5 is a generative model, so at prediction time the sequence to classify is encoded with the encoder and then the decoder is used to generate a sequence that represents the desired class label. When computing influence functions, one uses the cross-entropy loss for sequences.
J.2 Correction Experiments
For the correction experiments we used ABIF [SZTS22] because of their computational efficiency as IF scores need to be computed against the whole training set; we used 32 projectors obtained with 64 Arnoldi iterations. The Arnoldi iteration can take up to 2 hours; scoring the data takes a few minutes. We used a learning rate of  $10^{-3}$  and for BERT we loaded the state of the Adam optimizer from the selected checkpoint.
For BERT we selected the checkpoint at 6000 steps which was the best on validation performance. For ResNet we selected the checkpoint after 10 epochs at which about 150 test points are incorrectly classified.
K Illustrating TracIn's modeling error
In Theorem 2 we have derived the correct formula for the Jacobian  $\nabla_{\varepsilon|0}\theta_{\varepsilon,T}$  and we expect TracIn (Eq. 2) to introduce a modeling error in calculating  $\nabla_{\varepsilon|0}\theta_{\varepsilon,T}$ . We can compute the difference between the ground truth  $\theta_{\varepsilon,T}$  and the first order expansion  $\theta_{0,T} + \varepsilon^T\nabla_{\varepsilon|0}\theta_{\varepsilon,T}$  when the Jacobian is computed with TracIn (Eq. 2) or the corrected formula (Eq. 6). We can then compute the relative excess error introduced by TracIn for different values of  $\varepsilon$ . In Figure 12 we observe that except for the extremely small perturbation  $\varepsilon = 10^{-4}$ , TracIn does introduce an additional relative error in modeling the change of parameters when retraining BERT with SGD. The setup of this experiment is the same as those of our parameter divergence experiments.
Figure 12: Illustrating the additional modeling error introduced by TracIn for different values of the perturbation  $\epsilon$ . The error is the ratio between the error of TracIn (Eq. 2) and the error of the exact formula (Eq. 6) when retraining BERT on a perturbed loss with SGD.
L Broader Impact Statement
We believe that our work can benefit model developers who want to debug and correct mis-predictions made by models. Our suggested error-correction procedure is much less compute intensive than previous ones using IF as it does not require model retraining. However, this debugging procedure could introduce new errors in the systems, for example if on a specific problem IF are not effective at predicting loss-shifts or if examples retrieved by IF lead to over-fitting against spurious artifacts present in such examples. Since this could have a negative impact on the users of such systems, mitigation strategies should be put in place regarding the trade-offs between the success rate of fixing specific mis-predictions and the overall retention of the system performance.