KACE : Generating Knowledge Aware Contrastive Explanations for Natural Language Inference (2021)

Abstract: In order to better understand the reason behind model behaviors (i.e., making predictions), most recent works have exploited generative models to provide complementary explanations. However, existing approaches in NLP mainly focus on “WHY A” rather than contrastive “WHY A NOT B”, which is shown to be able to better distinguish confusing candidates and improve data efficiency in other research fields. In this paper, we focus on generating contrastive explanations with counterfactual examples in NLI and propose a novel K nowledge- A ware C ontrastive E xplanation generation framework ( KACE ).Specifically, we first identify rationales (i.e., key phrases) from input sentences, and use them as key perturbations for generating counterfactual examples. After obtaining qualified counterfactual examples, we take them along with original examples and external knowledge as input, and employ a knowledge-aware generative pre-trained language model to generate contrastive explanations. Experimental results show that contrastive explanations are beneficial to fit the scenarios by clarifying the difference between the predicted answer and other possible wrong ones. Moreover, we train an NLI model enhanced with contrastive explanations and achieves an accuracy of 91.9% on SNLI, gaining improvements of 5.7% against ETPA (“Explain-Then-Predict-Attention”) and 0.6% against NILE (“WHY A”).

## introduction in recent years, pre-trained language models @xcite @xcite have been widely adopted in many tasks of natural language processing @xcite @xcite . however, due to the lack of textual explanations, most downstream models become more complicated and difficult to understand. end users, especially those working in critical domains such as healthcare or online education, become more skeptical and reluctant to adopt or trust them, although these models have been proved to improve the decision-making performance. therefore, providing faithful textual explanations has become a promising way to overcome the black-box property of neural networks, which has attracted the attention of academia and industrial communities. recently, the majority of existing methods @xcite @xcite @xcite in natural language processing try to explain the predictions of neural models in a model-intrinsic or model-agnostic (also known as post-hoc) way. while post-hoc models @xcite @xcite provide explanations after making predictions without affecting the overall accuracy, most of them neglect the rationales in inputs and provide textual explanations just in the form of "why a". however, we argue that contrastive explanations in the form of "why a not b" could provide more informative and important clues that are easier to understand and persuade end-users. moreover, we believe that contrastive explanations could benefit downstream tasks (e.g., nli), since such kind of explanations contain more helpful information (e.g. relations between rationales) that can be used to improve model performance. to further enhance the explainability and performance of nli, we propose a novel textual contrastive explanation generation framework in this paper, which is post-hoc and considers rationales, counterfactual examples, and external knowledge. specifically, we first identify rationales (i.e., key phrases) from a premise-hypothesis (p-h) pair with rationale identification counterfactual example generation counterfactual example selection contrastive explanation generation premise: a woman and a young child are making sculptures out of clay. hypothesis: a man and woman painting on canvas. label: contradiction premise: young child, making sculptures, clay hypothesis: man, painting, canvas why contradiction not neutral contrastive explanation: why contradiction: making sculptures out of clay is different from paiting on canvas. why not neutral:making sculptures out of clay is a type of art conceptnet /dictionary concepts knowledge & rationale definition retrieval premise: a woman and a young child are making sculptures out of clay. hypothesis:a child and a woman make something. label:neutral refer to figure 2 label a, and then use them as the key perturbations for transforming and generating candidate counterfactual examples. then we further select one most qualified counterfactual example for any other label b. note that the acquisition of a qualified counterfactual example of class b is essential to generate a meaningful explanation for "why not b", otherwise the resultant contrastive explanation will be groundless or useless. after that, we take the selected examples along with the original p-h pair and related external knowledge as input, and finally employ a knowledge-aware pre-trained language model to generate contrastive explanation, which will specify why the prediction label is a rather than b, and clarify the confusions for end-users. moreover, we train an nli model enhanced with contrastive explanations and achieve the new stateof-art performance on snli. the contributions of this paper are as follows: here, we define the task of contrastive explanation generation for nli. given a trained neural network model f with input x and predicted class a, the problem of generating contrastive explanations (ce) to an input x is to specify why x belongs to category/class a rather than b, defined as: in equation 1, we first identify a set of rationales in given inputs, as described in section 3.1, and in equation 2 we generate counterfactual examples with reversal mechanism as presented in section 3.2. in equation 3, we take the selected counterfactual example along with original example and external knowledge as input, and employ a knowledge-aware generator to produce contrastive explanation as detailed in section 3.3. long, fascinating, soulful. never have i been so sad to see ending credits roll. negative premise hypothesis snli label a woman and a young child are making sculptures out of clay. a man and woman painting on canvas. contradiction a child and woman making something a woman and a young child are making sculptures out of clay. neutral unchanged replaced changed counterfactual example generation for snli and imdb long, boring, blasphemous. never have i been so glad to see ending credits roll. postive changed imdb label replaced token rationale 3 approach ## conclusion in this paper, we focus on knowledge-aware contrastive explanation generation for nli. we generate counterfactual examples by changing identified rationales of given instances. afterwards, we extract concepts knowledge from conceptnet and dictionary to train knowledge-aware explanation generators. we show that contrastive explanations that specify why a model makes prediction a rather than b can provide more faithful information than other "why a" explanations. moreover, contrastive explanations can be used for data augmentation to improve the performance and robustness of existing model. the exploration of contrastive explanation in other nlp tasks (i.e. question answering) and better evaluation metrics for explanation will be performed in the future.