SlowGuess/NAACL_2021_OCR huggingface dataset:
Barak A Pearlmutter. 1994. Fast exact multiplication by the hessian. Neural computation, 6(1):147-160.
Pouya Pezeshkpour, Yifan Tian, and Sameer Singh. 2019. Investigating robustness and interpretability of link prediction via adversarial modifications. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3336-3347.
Nazneen Fatema Rajani, Ben Krause, Wengpeng Yin, Tong Niu, Richard Socher, and Caiming Xiong. 2020. Explaining and improving model behavior with k nearest neighbor representations. arXiv preprint arXiv:2010.09030.
Nils Reimers, Iryna Gurevych, Nils Reimers, Iryna Gurevych, Nandan Thakur, Nils Reimers, Johannes Daxenberger, and Iryna Gurevych. 2019. Sentencebert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631-1642.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pages 3319-3328.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112-1122.
Chih-Kuan Yeh, Joon Kim, Ian En-Hsu Yen, and Pradeep K Ravikumar. 2018. Representer point selection for explaining deep neural networks. In Advances in Neural Information Processing Systems, pages 9291-9301.
Datasets To evaluate different attribution methods, we conduct several experiments on sentiment analysis and NLI tasks, following prior work investigating the use of influence functions specifically for NLP (Han et al., 2020). We adopt a binarized version of the Stanford Sentiment Treebank (SST-2) (Socher et al., 2013), consisting of 6920 training samples and 1821 test samples. As our NLI benchmark, we use the Multi-Genre NLI (MNLI) dataset (Williams et al., 2018), which contains  $393\mathrm{k}$  pairs of premise and hypothesis from 10 different genres. For model fine-tuning, we randomly sample  $10\mathrm{k}$  training instances. To evaluate the utility of different instance attribution methods in helping to unearth annotation artifacts in NLI, we use the HANS dataset (McCoy et al., 2019), which comprises examples exhibiting previously identified NLI artifacts such as lexical overlap between hypotheses and premises. We randomly sampled 1000 instances from this benchmark as test data to analyze the behavior of different attribution methods.
Models As discussed in the paper, we define models for both tasks on top of BERT, tuning hyperparameters on validation data via grid search. These hyperparameters include the regularization parameter  $\lambda = [10^{-1}, 10^{-2}, 10^{-3}]$ ; learning rate  $\alpha = [2 \times 10^{-3}, 2 \times 10^{-4}, 2 \times 10^{-5}, 2 \times 10^{-6}]$ ; number of epochs  $\in \{3, 7, 10, 15\}$ ; and the batch size  $\in \{8, 16\}$ . Our final models achieve  $90.6\%$  accuracy on SST and  $71.2\%$  accuracy on MNLI
B Attribution Methods' Correlation
The complete version of Spearman correlation between attribution methods (containing the sentence-BERT) is provided in Figure 3. As expected, similarity-based approaches based on sentence-BERT show a very small correlation with other methods.
We also provide the proportion of shared examples in the top samples retrieved by IF (top-5) and IF (linear) in Figure 4. One can see that there is a very high correlation between these methods in top samples, validating the high quality of simpler version of IF (IF (linear)) in comparison to the more complex method (IF (top-5)).
C Removing 'Important' Samples
In this experiment, we first select 50 random test samples (for both MNLI and SST). Then, for each
one of these instances, we separately remove top-k (we consider  $\mathrm{k} = 50$  and 500) training instances for that test sample, retrain the model, and calculate the change in the model's prediction for that sample. We report the average changed over the prediction of the selected 50 random test samples in Table 1. Moreover, the proportion of common examples in top samples between pairs of attribution methods is depicted in Figures 5 and 6. The very high rate between IF vs GD, RIF vs GC, and NN-EUC vs NN-COS pairs, clarify the reason behind the similar performance of these pairs of methods in leave-some-out experiments.
D Near Training Samples Explanations
To further investigate the quality of the most influential sample based on different attribution methods, we conjecture that a data point very similar to a training sample should recover that sample as the most influential instance. We consider four scenarios to create target points similar to training data: (1) using training samples themselves as the target instances for attribution methods; (2) adding a random token to a random place in each training samples; (3) randomly removing a token from each training samples, and; (4) replacing a random token in each training samples with a random token from the dictionary of tokens. In the MNLI dataset, we apply each modification to both the premise and hypothesis in each training sample.
The result of this analysis is provided in Tables 3 and 4. We observe that similarity-based methods demonstrate a greater ability to recover the original training samples corresponding to the different targets. Moreover, the very low performance of IF, GC, and REP methods is due to the fact that there are training points with high magnitude gradient, which these methods choose as top instances for any target sample.
(a) SST.
(b) MNLI.
Figure 3: Complete version of correlation matrices.
(b) MNLI.
Figure 4: Proportion of common top examples between IF (Top-5) and IF (Linear) Methods. We selected 100 test examples and 500 training examples to compute the attributions over.
(a) Top-10 in SST.
Figure 5: Proportion of common examples in top 10 samples between pairs of attribution methods.
(b) Top-10 in MNLI.
(a) Top-50 in SST.
(b) Top-50 in MNLI.
Figure 6: Proportion of common examples in top 50 samples between pairs of attribution methods.
Method	Train	ADD	Remove	Replace
HIT@1	HIT@10	HIT@1	HIT@10	HIT@1	HIT@10	HIT@1	HIT@10
Sim	NN EUC	100	100	99.9	100	66.5	73.7	99.9	100
NN COS	100	100	99.8	100	67.3	74.6	99.8	100
NN DOT	0.73	2.06	0.73	2.06	0.47	2.19	0.73	2.06
Gradient	IF	0.01	0.34	0.01	0.35	0.04	0.25	0.01	0.35
REP	0.01	0.27	0.01	0.27	0.04	0.22	0.01	0.27
RIF	95.8	96.0	95.9	96.0	65.0	72.2	95.8	96.0
GD	0.01	0.38	0.01	0.38	0.04	0.23	0.01	0.37
GC	95.9	96.0	95.9	96.0	65.3	72.3	95.9	96.0
Table 3: Treating the training samples and their modifications as the target samples for attribution methods over the SST dataset.
Method	Train	ADD	Remove	Replace
HIT@1	HIT@10	HIT@1	HIT@10	HIT@1	HIT@10	HIT@1	HIT@10
Sim	NN EUC	100	100	100	100	36.7	45.8	100	100
NN COS	100	100	100	100	38.1	46.8	100	100
NN DOT	1.30	6.44	1.30	6.44	3.49	10.7	1.30	6.44
Gradient	IF	0.0	0.01	0.0	0.01	0.02	0.10	0.0	0.01
REP	0.0	0.01	0.0	0.01	0.01	0.09	0.0	0.01
RIF	92.5	92.5	92.5	92.5	32.6	41.2	92.5	92.5
GD	0.0	0.01	0.0	0.01	0.10	0.50	0.0	0.01
GC	92.5	92.5	92.5	92.5	32.8	41.2	92.5	92.5
Table 4: Treating the training samples and their modifications as the target samples for attribution methods over the MNLI dataset.