F ast IF : Scalable Influence Functions for Efficient Model Interpretation and Debugging (2021)

Abstract: Influence functions approximate the “influences” of training data-points for test predictions and have a wide variety of applications. Despite the popularity, their computational cost does not scale well with model and training data size. We present FastIF, a set of simple modifications to influence functions that significantly improves their run-time. We use k-Nearest Neighbors (kNN) to narrow the search space down to a subset of good candidate data points, identify the configurations that best balance the speed-quality trade-off in estimating the inverse Hessian-vector product, and introduce a fast parallel variant. Our proposed method achieves about 80X speedup while being highly correlated with the original influence values. With the availability of the fast influence functions, we demonstrate their usefulness in four applications. First, we examine whether influential data-points can “explain” test time behavior using the framework of simulatability. Second, we visualize the influence interactions between training and test data-points. Third, we show that we can correct model errors by additional fine-tuning on certain influential data-points, improving the accuracy of a trained MultiNLI model by 2.5% on the HANS dataset. Finally, we experiment with a similar setup but fine-tuning on datapoints not seen during training, improving the model accuracy by 2.8% and 1.7% on HANS and ANLI datasets respectively. Overall, our fast influence functions can be efficiently applied to large models and datasets, and our experiments demonstrate the potential of influence functions in model interpretation and correcting model errors.

## introduction language understanding systems are becoming ever more powerful with the recent advances in 1 code is available at https://github.com/ salesforce/fast-influence-functions . large-scale pre-training. as these systems become widely adopted for real-world applications, the ability to interpret the model decisions grows increasingly important. an array of interpretation methods now exist for shedding light on models' decisionmaking processes, with the bulk of work focusing on estimating feature importance @xcite @xcite @xcite ) and interpreting internal model representations @xcite @xcite . these approaches aim to identify features a model uses to make decisions or analyze representations obtained from trained models. in contrast, one might also want to know how particular training data points influence model behavior at test time. this kind of goal is an instance of what @xcite terms "algorithmic transparency," or, transparency "at the level of the learning algorithm itself." the ability to do so would allow researchers to identify and respond to data responsible for problematic test-time behavior. one simple brute-force way to estimate the importance of a training data-point to a test-time decision is the leave-one-out approach @xcite . alternatively, influence functions (koh and liang, 2017) provide a tractable estimate of the effect without the need to repeatedly retrain the model. yet, influence functions are very expensive to compute for moderate-sized model and training data. for instance, finding the most influential examples w.r.t. an evaluation data-point with a model of about 14.7m parameters in a dataset of about 390k examples -a pretty moderate setting -takes more than 2 hours (please see sec. 5.1 for details). consequently, applications might face the dilemma of either accepting the computation cost or resorting to a smaller-scale setting. 2 but why are influence functions computationally expensive? in our early experiments, we found that the main computational bottleneck lies in the following steps: in this work, we present fastif (fast influence functions) to address these challenges through three simple techniques. first, instead of enumerating the full training dataset to find influential data-point, we leverage fast nearest neighbor search @xcite to narrow the search to a small subset of influence-worthy data-points. this operation reduces the computation by about an order of magnitude. second, we identify a set of hyperparameters for the hessian estimation algorithm that reduces computation time by more than half while preserving estimation quality. finally, we describe a simple parallelizable extension, which gives an additional 2x speedup. as a result, we are able to improve the most influential examples search overall by approximately two orders of magnitude in our experiments. so what could we do with faster influence functions? we demonstrate the advantage of fast influence functions via several interesting downstream applications. these require computing in-fluence functions repeatedly and were thus almost intractable without such fast influence functions: the visualizations help us uncover interesting latent structures in the dataset, and the insights could be used to improve model performance. 3. in sec. 6.3 we show a simple application of influence functions in correcting model prediction by fine-tuning on certain influential examples. despite its simplicity, our method can improve a multinli model's accuracy on the hans dataset by more than 5.8% (about 2.5% more than our baseline). 4. influence functions are defined w.r.t. the training data used in the model training. also in sec. 6.3, we extend the above experiment and demonstrate the potential of influence functions to the setting where we want to leverage a new labeled dataset unseen during training. this can improve a multinli model's accuracy on the hans/anli dataset @xcite @xcite ) by more than 5.9%/2.7% (about 2.8%/1.7% more than our baseline). ## experimental setup we use the multinli dataset in analysis experiments (sec. 5), and we include multinli, hans, anli, and amazon-wilds datasets (english) in the applications (sec. 6) @xcite @xcite ). 5 our model is based on pre-trained bertbase model fine-tuned on downstream tasks @xcite . we freeze the first 9 layers based on results in @xcite to reduce computation cost, which leaves us with about 14.7m trainable parameters. we use weight decay of 0.005 during training as recommended by @xcite . please see the appendix for experiment details. ## conclusions we present fastif which, via simple modifications, significantly speeds up the computation to influence functions without significantly impacting their performance. our improvements include using knn to pre-select a subset of influence-worthy data-points, identifying a set of hyperparameters for the inverse hvp estimation algorithm, and a parallel implementation that minimizes communication overhead. we empirically examine the effectiveness of these modifications. then, with the availability of fast influence functions, we demonstrate a few interesting applications that were previously intractable: (1) examining the "explainability" of influential data-points, (2) visualizing data influence-interactions, (3) correcting model predictions using original training data, (4) and correcting model predictions using data from a new dataset. this work presents scalable influence functions for efficient model interpretation and debugging, which would be especially useful for improving model performance for particular categories of model failures after they are identified. recently, @xcite noticed that an adversary could extract training data from largescale language models and recover potentially sensitive information. if properly used, the tool could be helpful for checking whether a model might be vulnerable to such attacks (hence, our tool should be used to encourage the detection of such memorization-based models as opposed to being misused to exploit such models). finally, while the fast influence functions are more compute-efficient than alternatives like retraining and full-scale influence functions, they are nevertheless expensive operations. thus, applying fastif to large-scale datasets and models might be restricted to those who have access to adequate computing power (but in fact, this is why the main purpose of this paper is to make influence functions faster and more compute efficient). section details sec. 5.2 / b.1 100 evaluation data-points, knn with @xmath0, 1×10 2 , 1×10 3 , 1×10 4 , 5×10 4 , 1×10 5 }, r@m with @xmath1, 10 2 , 10 3 }. sec. 5.3 / b.2 8 evaluation data-points, @xmath2, 800, ..., 1300}, @xmath3, 2 1 , ..., 2 7 }, @xmath4, 4}. sec. 5.4 / b.3 20 evaluation data-points for estimating correlations, and 10 for retraining experiments, knn with @xmath5, 10 4 }, @xmath6, 5, 25, 50, 100}. sec. 6.1 / c.1 20 evaluation data-points, 1 fine-tuning data-point, 50 learning rates in log-space from 10 -5 to 10 -2.5 , and repeat for 10 different fine-tuning data-points. sec. 6.2 / c.2 400 evaluation data-points where the model predictions are incorrect, including 100 from the mnli evaluation dataset and 100 for each of the three subsets from the hans evaluation dataset. we use knn with @xmath7. sec. 6.3 / c.3 experiments are repeated 3 times, fine-tuning is applied repeatedly for 10 iterations. for each iteration, we select 10 validation data-points as the "anchor" data-points, update parameters for one gradient step on 10 fine-tuning data-points with learning rate 10 -4 . for amazon-wilds experiments, we use 50 anchor and fine-tuning data-points. results. fig. 9 shows the results of the experiments. for the two figures on the left, we can observe that the computation cost (measured in time) grows with both the batch size and number of recursive iterations j. similarly, the estimation error 12 (the figures on the right) shows that the error decreases with both the batch size and j in general. further, we notice that when the batch size is small (e.g., @xmath10, we can make up the loss in quality by increasing t , which is inherently parallelizable (main paper sec. 3.4). overall, these suggest that we could trade off a small drop in quality with a significant speed-up by the combination of (1) small b, (2) medium j, and (3) large t . 13 11 in the figures, we only include the results of different t for difference norm. this is because practitioners can use parallelism to speed up the computations of each run, which is independent of each other as described in sec. 3.4. thus, the change in time mainly depends on parallelism overhead, which we found to be reasonable in our initial experiments. 12 the estimation error is measured as the difference norm w.r.t. the estimation using the most expensive configuration. 13 we pick @xmath11, 1500, 2000} based on convergence.