An Empirical Comparison of Instance Attribution Methods for NLP (2021)

Abstract: Widespread adoption of deep models has motivated a pressing need for approaches to interpret network outputs and to facilitate model debugging. Instance attribution methods constitute one means of accomplishing these goals by retrieving training instances that (may have) led to a particular prediction. Influence functions (IF; Koh and Liang 2017) provide machinery for doing this by quantifying the effect that perturbing individual train instances would have on a specific test prediction. However, even approximating the IF is computationally expensive, to the degree that may be prohibitive in many cases. Might simpler approaches (e.g., retrieving train examples most similar to a given test point) perform comparably? In this work, we evaluate the degree to which different potential instance attribution agree with respect to the importance of training samples. We find that simple retrieval methods yield training instances that differ from those identified via gradient-based methods (such as IFs), but that nonetheless exhibit desirable characteristics similar to more complex attribution methods. Code for all methods and experiments in this paper is available at: https://github.com/successar/instance_attributions_NLP .

## introduction interpretability methods are intended to help users understand model predictions @xcite @xcite . in machine learning broadly and nlp specifically, such methods have focused on feature-based explanations that highlight parts of inputs 'responsible for' the specific prediction. feature attribution, however, does not communicate a key basis for model outputs: training data. recent work has therefore considered methods for surfacing training examples that were influential for a specific prediction @xcite @xcite @xcite . while such instance-attribution methods provide an appealing mechanism to identify sources that led to specific predictions (which may reveal potentially problematic training examples), they have not yet been widely adopted, at least in part because even approximating influence functions (koh and liang, 2017)-arguably the most principled attribution method-can be prohibitively expensive in terms of compute. is such complexity necessary to identify 'important' training points? or do simpler methods (e.g., attribution scores based on similarity measures between train and test instances) yield comparable results? in this paper, we set out to evaluate and compare instance attribution methods, including relatively simple and efficient approaches @xcite in the context of nlp (figure 1 ). we design qualitative evaluations intended to probe the following research questions: (1) how correlated are rankings induced by gradient and similarity-based attribution methods (assessing the quality of more efficient approx-imations)? ( 2 ) what is the quality of explanations in similarity methods compared to gradient-based ones (clarifying the necessity of adopting more complex methods)? ## experimental setup datasets to evaluate different attribution methods, we conduct several experiments on sentiment analysis and nli tasks, following prior work investigating the use of if specifically for nlp @xcite . we adopt a binarized version of the stanford sentiment treebank (sst-2; @xcite , and the multi-genre nli (mnli) dataset @xcite . for fine-tuning on mnli, we randomly sample 10k training instances. finally, to evaluate the ability of instance attribution methods to reveal annotation artifacts in nli, we randomly sampled 1000 instances from the hans dataset (more details in the appendix). models we define models for both tasks on top of bert @xcite , tuning hyperparameters on validation data via grid search. our models achieve 90.6% accuracy on sst and 71.2% accuracy on mnli (more details in the appendix). computing the if for bert deriving the if for all parameters θ of a bert-based model requires deriving the corresponding inverse hessian. we compute the inverse hessian vector product (ihvp) h -1 ∇ θ l(x, y, θ) directly because storing the entire matrix of |θ| 2 elements is practically impossible (requiring ∼12 pb of storage). we approximate the ihvp using the lissa algorithm ## experiments in this section, we first investigate the correlation between different methods. then, to study the quality of explanations we conduct leave-some-out experiments, and further analyze attribution methods on hans data. we consider four evaluations (more analyses and experimental details in the appendix). ## conclusions instance attribution methods constitute a promising approach to better understanding how modern nlp models come to make the predictions that they do @xcite . however, approximating if to quantify the importance of train samples is prohibitively expensive. in this work, we investigated whether alternative, simpler and more efficient methods provide similar instance attribution scores. we demonstrated high correlation between (1) gradient-based methods that consider more parameters [if and gd (top-5)] and their simpler counterparts @xcite ], and (2) methods without hessian information, i.e., if vs gd and rif vs gc. we considered even simpler, similarity-based approaches and compared the importance rankings over training instances induced by these to rankings under gradient-based methods. through leave-some-out, randomized-test, and artifact detection experiments, we demonstrated that these simple similarity-based methods are surprisingly competitive. this suggests future directions for work on fast and useful instance attribution methods. all code necessary to reproduce the results reported in this paper is available at: