HILDIF : I nteractive Debugging of NLI Models Using Influence Functions (2021)

Abstract: Biases and artifacts in training data can cause unwelcome behavior in text classifiers (such as shallow pattern matching), leading to lack of generalizability. One solution to this problem is to include users in the loop and leverage their feedback to improve models. We propose a novel explanatory debugging pipeline called HILDIF, enabling humans to improve deep text classifiers using influence functions as an explanation method. We experiment on the Natural Language Inference (NLI) task, showing that HILDIF can effectively alleviate artifact problems in fine-tuned BERT models and result in increased model generalizability.

## introduction given two sentences, a premise and a hypothesis, natural language inference (nli) is the task of determining whether the premise entails the hypothesis, and it has been considered by many as a sign of language understanding @xcite . although recent deep learning models have shown to achieve good performances on different nli datasets, as in other tasks, they have been shown to learn shallow heuristics. for example, a model is very likely to predict entailment for all hypotheses constructed from words in the premise @xcite . a key challenge is therefore to understand when and why state-of-the-art nli models fail and try to mitigate the problems accordingly. in order to bring to light this kind of pathology, one can use explanation techniques to comprehend how a black box model makes particular predictions. for instance, feature attribution methods explain by identifying parts of inputs that mainly contribute to predictions @xcite @xcite . further, example-based methods, such as influence functions @xcite , identify training data points which are the most important for particular predictions. existing works have proposed ways to improve models by incorporating human feedback, in response to the explanations, by: adding model constraints by fixing certain parameters @xcite , adding training samples @xcite , and adjusting models' weights directly @xcite . in this paper, we propose a novel interactive model debugging pipeline called hildif -human in the loop debugging using influence functions. with the nli task as a target, we use influence functions as an explanation method to help users understand the model reasoning via influential training examples. then, for each influential example shown, the users provide feedback to create augmented training samples for fine tuning the model. using hildif, we effectively mitigate artifact issues of bert models @xcite trained on the mnli dataset @xcite and tested on the hans dataset @xcite , which is a known pathological setting for most deep nli models working on english language. our code can be found at https://github.com/hugozylberajch/hildif . ## related work influence functions. introduced by @xcite , influence functions compute how upweighting individual examples in the training loss changes the model parameters. influential training examples can also be used to study models @xcite . they are particularly useful when feature attribution scores are not sufficient to illustrate how the model reasons. in the nli task, for example, single input words may not suffice to explain a certain prediction, and the overall semantics and structures in the input may be needed. recently, @xcite showed that influence functions can capture key fine-grained interactions among input words and detect the presence of artifacts that lead to incorrect nli predictions. although very appealing, influence functions are computationally expensive. hence, koh and liang (2017) reduced computational complexity by using the linear time stochastic second order algorithm (lissa) for calculating approximations. @xcite proposed fastif, which further speeds up the calculation using the k-nearest neighbors algorithm. they also fine tuned the model with influential training samples of anchor points (i.e., some data points in the validation set) to correct model errors. we will use fastif as a tool to explain bert model's predictions on the nli task in our experiment. explanatory interactive debugging, where we improve a model by leveraging user feedback after presenting explanations for model predictions, was first introduced using simple statistical models such as naïve bayes models or support vector machines with simple explanatory techniques @xcite . recently, explanatory debugging has been applied to more complex models using refined interpretability methods. in find @xcite , a masking matrix is added at the end of a cnn text classifier so as to disable particular cnn filters based on human feedback in response to lrp-based explanations @xcite . in caipi (teso and kersting, 2019), the user investigates and corrects a limebased explanation @xcite for each prediction. then additional training samples, created based on the correction, are used to fine tune the model. for more details on explanatory debugging, we refer interested readers to the survey by @xcite . as in caipi, we will exploit user feedback to control the generation of augmented samples for fine tuning the model. however, our explanations are influential training samples which are more suitable for explaining nli predictions. this is an improvement from @xcite that simply fine tuned the model on influential samples without human feedback involved. ## hildif we propose in algorithm 1 a new pipeline called hildif (human in the loop debugging with influence functions) for debugging deep text clas-algorithm 1: hildif. l is a labeled training set, v is a labeled validation set, t is the number of iteration, and g is a data augmentation method. next, we explain, in detail, each step of hildif, including explanation generation, user feedback collection, and data augmentation. explanation generation. from the validation set v, we can either select anchor points randomly or handpick some that contain particular heuristics we want to debug. after that, the user is presented with a list of top-p most negatively influential training data points for each anchor point. these influential data points contribute to the decrease of the model's loss when upweighted. hence, fine-tuning the model using these data points should improve the model performance as studied by @xcite . however, since hildif relies on fastif which only approximates influence scores, we hypothesize that we can achieve better performance by asking humans to assess relevancy of the influential training samples returned by fastif before fine-tuning. then z ij will obtain a similarity score s ij from 1 to 5 accordingly based on the user's answer. similar in this context means that both samples share the same type of heuristics or lexical artifacts. ## experimental setup datasets and models. we evaluate our pipeline with a pretrained bert-base cased model. we use the mnli dataset @xcite for training and validation, and the hans dataset, which is known to be a dataset where bert performs poorly @xcite , for testing. for the mnli training and validation set, we merge the class neutral and contradiction into a single non-entailment class, following the hans dataset's setting. hans targets three heuristics of nli and includes examples showcasing these heuristics: lexical overlap where the hypothesis is constructed with words from the premise, constituent, where the hypothesis is a subtree of the premise's parse tree, and subsequence, where the hypothesis is a contiguous subsequence of the premise (see implementation details. all our models are implemented using the pytorch library and trained using the adamw optimizer. the hans dataset is held-out during training and fine-tuning and is only used for testing. for computing influence functions, we use the fastif algorithm and faiss library @xcite for k-nearest neighbors search. finally, we ran all our experiment on a single 12gb nvidia tesla k80 gpu. with this setting, the computation of influence scores of 5,000 training points for a corresponding anchor point takes approximately seven minutes. the bert-base model is trained for two epochs on the mnli training dataset. regarding user feedback collection, due to human resources constraints, we did our interactive experiments with one expert user. further experiments could be conducted with more users, and results for the same pair of anchor point and influential point could be aggregated in order to reduce human bias. comparison. we experimented with @xmath0, using five anchor points with 10 and 20 influential samples each. we introduce three binary propositions that will define the debugging pipeline: hs: human scoring, da: data augmentation, and h: handpicked anchor points. without human scoring (¬hs), every influential sample receives a score of 5. without data augmentation (¬da), the fine tuning is done on each influential sample only, and without handpicked anchor points (¬h), anchor points are selected randomly. note that our handpicked anchor points were chosen among the validation samples that contain either the lexical overlap or the subsequence heuristic (see ## results figure 1b shows the accuracies on the hans dataset of the baseline model (i.e., bert trained on mnli), and of four configurations, including (hs, da, h) which is displayed as hildif in the figure. we can see that hildif consistently achieved a higher accuracy in all three categories of heuristics for the non-entailment class and a slightly lower accuracy for the entailment class. actually, we observe the trade-off between the accuracies of both classes in all the four configurations. however, hildif still got higher overall accuracy on the hans dataset than the baseline and the other configurations. moreover, interactive debugging with human scores yielded better accuracies than debugging without human scores for the lexical overlap and the subsequence categories. meanwhile, on the constituent category, handpicking anchor points with targeted heuristic led to a big jump in accuracy that outperformed the configuration with human feedback but random anchor points. therefore, incorporating human knowledge since the selecting anchors step is also helpful when we have prior knowledge about the model bugs. note also, in figure 1a , that the model accuracies on the mnli for hildif and the other configurations stay close to the baseline model's accuracy, as desired. the table in figure 1c shows that fine tuning the model with augmented data samples, instead of the influential samples only, gave better results in most cases. this was likely because data augmentation could help prevent the model from overfitting the influential samples. besides, there was little to no improvement in the model performance when we added user feedback (i.e., human scores) for random anchor points but a substantial improvement for handpicked anchor points. this can be because, during user feedback collection, most of the data samples are difficult to compare as they either satisfy several heuristics or no heuristics relevant to the nli task. when looking at some influential samples for handpicked anchor points, most satisfy the same heuristic and, if not, they can be easily spotted by human eyes. although we are still far from chance performance on the non-entailment class, hildif achieved a substantial increase in accuracy with just five anchor points. ## conclusion we introduced hildif, an interactive explanatory debugging pipeline for deep text classifiers, and ran experiments on the nli task, achieving high accuracies with mnli-trained bert across all categories of the pathological hans dataset. future work includes enhancement of the data augmentation part, including the use of a variational autoencoder or a gpt-2 based generative model for synthetic data generation. also, with more human resources, experiments can be conducted by finetuning more than one iterations (@xmath1. finally, it would be interesting to apply hildif to other text classification tasks, given that, except for the handpicked anchor points that are chosen with knowledge of the task, every step of the pipeline is task-independent. marco tulio ribeiro, sameer singh, and carlos guestrin. 2016. "why should i trust you?" explaining the predictions of any classifier. in proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 1135-1144. daniel smilkov, nikhil thorat, been kim, fernanda viégas, and martin wattenberg. 2017. smoothgrad: removing noise by adding noise. arxiv preprint arxiv:1706.03825. simone stumpf, vidya rajaram, lida li, weng-keen wong, margaret burnett, thomas dietterich, erin sullivan, and jonathan herlocker. 2009. interacting meaningfully with machine learning systems: three experiments. international journal of humancomputer studies, 67(8):639-662. mukund sundararajan, ankur taly, and qiqi yan. 2016. gradients of counterfactuals. arxiv preprint arxiv:1611.02639. stefano teso and kristian kersting. 2019. explanatory interactive machine learning. in proceedings of the 2019 aaai/acm conference on ai, ethics, and society, aies '19, page 239-245, new york, ny, usa. association for computing machinery. adina williams, nikita nangia, and samuel bowman. 2018. a broad-coverage challenge corpus for sentence understanding through inference. in proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long papers), pages 1112-1122, new orleans, louisiana. association for computational linguistics.