On Sample Based Explanation Methods for NLP : Faithfulness, Efficiency and Semantic Evaluation (2021)

Abstract: In the recent advances of natural language processing, the scale of the state-of-the-art models and datasets is usually extensive, which challenges the application of sample-based explanation methods in many aspects, such as explanation interpretability, efficiency, and faithfulness. In this work, for the first time, we can improve the interpretability of explanations by allowing arbitrary text sequences as the explanation unit. On top of this, we implement a hessian-free method with a model faithfulness guarantee. Finally, to compare our method with the others, we propose a semantic-based evaluation metric that can better align with humans’ judgment of explanations than the widely adopted diagnostic or re-training measures. The empirical results on multiple real data sets demonstrate the proposed method’s superior performance to popular explanation techniques such as Influence Function or TracIn on semantic evaluation.

## introduction as complex nlp models such as the transformers family @xcite become an indispensable tool in many applications, there are growing interests to explain the working mechanism of these "black-box" models. among the vast of existing techniques for explaining machine learning models, influence functions @xcite that uses training instances as explanations to a model's behavior have gained popularity in nlp very recently. different from other methods such as using input erasure @xcite , saliency maps or attention matrices @xcite @xcite that only look at how a specific input or input sequence impacts the model decision, explaining with training instances can cast light on the knowledge a model has encoded about a problem, by answering questions like 'what knowledge did the model capture from which training instances so that it makes decision in such a manner during test?'. very recently, the method has been applied to explain bert-based @xcite text classification @xcite and natural language inference @xcite ) models, as well as to aid text generation for data augmentation @xcite using gpt-2 @xcite . although useful, influence function may not be entirely bullet-proof for nlp applications. first, following the original formulation (koh and liang, 2017), the majority of existing works use entire training instances as explanations. however, for long natural language texts that are common in many high-impact application domains (e.g., healthcare, finance, or security), it may be difficult, if not impossible, to comprehend an entire instance as an explanation. for example, a model's decision may depend only on a specific part of a long training instance. second, for modern nlp models and large-scale datasets, the application of influence functions can lead to prohibitive computing costs due to inverse hessian matrix approximation. although hessianfree influence score such as tracin @xcite was introduced very recently, it may not be faithful to the model in question and can result in spurious explanations for the involvement of sub-optimal checkpoints. last, the evaluation of explanation methods, in particular, for the training-instance-based ones, remains an open question. previous evaluation is either under an over-simplified assumption on the agreement of labels between training and test instances @xcite or is based on indirect or manual inspection @xcite @xcite . a method to automatically measure the semantic relations at scale and that highly correlates to human judgment is still missing in the evaluation toolset. to address the above problems, we propose a framework to explain model behavior that includes both a set of new methods and a new metric that can measure the semantic relations between the test instance and its explanations. the new method allows for arbitrary text spans as the explanation unit and is hessian-free while being faithful to the final model. our contributions are: ## preliminaries suppose a model parameterized by θ is trained on classification dataset @xmath0, d test } by empirical risk minimization over d train . let @xmath1, y) ∈ d train and @xmath2, y ) ∈ d test denote a training and a test instance respectively, where x is a token sequence, and y is a scalar. the goal of training instance based explanation is to provide for a given test z an ordered list of training instances as explanation. two notable methods to calculate the influence score are if and tracin: if (koh and liang, 2017) assumes the influence of z can be measured by perturbing the loss function l with a fraction of the loss on z, and obtain tracin @xcite instead assumes the influence of a training instance z is the sum of its contribution to the overall loss all through the entire training history, and conveniently it leads to faithfulness if is faithful to θ since all its calculation is based on a single final model, yet tracin may be less faithful to θ since it obtains gradients from a set of checkpoints 2 . interpretability both methods use the entire training instance as an explanation. explanations with a finer-grained unit, e.g., phrases, may be easier to interpret in many applications where the texts are lengthy. ## proposed method to improve on the above desiderata, a new method should be able to: 1) use any appropriate granularity of span(s) as the explanation unit; 2) avoid the need of hessian while maintaining faithfulness. we discuss the solutions for both in section 3.1 and 3.2, and combine them into one formation in section 3.3 followed by critical implementation details. ## evaluation metrics this section introduces our semantic evaluation method, followed by a description of two other popular metrics for comparison. ## data our criteria for dataset selection are two folds: 1. the dataset should have relatively high classification accuracy so that the trained model can behave rationally; and 2. the dataset should allow for easy identification of critical/useful text spans to compare span-based explanation methods. we chose two aspect-based sentiment analysis (absa) datasets; one is atsa, a subset of mams @xcite for product reviews, where aspects are the terms in the text. the other is sentihood @xcite of location reviews. we can identify the relevant span of an aspect term semiautomatically and train models with high classification accuracy in both datasets. (see section 6.1 for details). data statistics and instances are in ## a case study ## related work popular explanation methods include gradientbased @xcite , attention-based @xcite @xcite , as well as sample-based @xcite @xcite methods. ## future work tracin ++ opens some new questions: 1) how can we generalize tracin ++ to cases where test spans are unknown? 2) can we understand the connection between if and tracin which may spark discoveries on sample-based explanation methods? 3) how can we apply tracin ++ to understand sequence generation models?