KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations (2023)

Abstract: While recent works have been considerably improving the quality of the natural language explanations (NLEs) generated by a model to justify its predictions, there is very limited research in detecting and alleviating inconsistencies among generated NLEs. In this work, we leverage external knowledge bases to significantly improve on an existing adversarial attack for detecting inconsistent NLEs. We apply our attack to high-performing NLE models and show that models with higher NLE quality do not necessarily generate fewer inconsistencies. Moreover, we propose an off-the-shelf mitigation method to alleviate inconsistencies by grounding the model into external background knowledge. Our method decreases the inconsistencies of previous high-performing NLE models as detected by our attack.

## introduction the accurate yet black-box nature of deep neural networks has accelerated studies on explainable ai. the advent of human-written natural language explanations (nles) datasets @xcite has paved the way for the development of models that provide nles for their predictions. however, by introducing an adversarial attack, which we hereafter refer to as eia (explanation inconsistency attack), @xcite found that an early nle model @xcite was prone to generate inconsistent nles (in-nles). more precisely, two logically contradictory nles generated by a model for two instances that have the same context are considered to form an inconsistency. for example, assume a self-driving car stops in a given traffic environment (the context). if the passenger asks the car q1:"why did you stop?", and it provides nle1: "because the traffic light is red.", and, for the same context, if the passenger instead asks q2: "why did you decide to stop here?" and the car provides nle2: "because the traffic light is green", then nle1 and nle2 form an inconsistency. a model that generates in-nles is undesirable, as it either has a faulty decision-making process (e.g., the traffic light was green, so the car should not have stopped), or it generates nles that are not faithfully describing its decision-making process (e.g., the car stopped for a red traffic light, but states that it was green) @xcite . while recent high-performing nle models have largely improved in terms of the quality (plausibility) of the generated nles, to our knowledge, these models have not been tested against generating inconsistent nles. in this work, we first propose a fast, efficient, and task-generalizable adversarial attack that utilizes external knowledge bases. through experiments on two datasets and four models, we verify the increased efficiency of our approach over the eia attack, the only inconsistency attack for nle models, to our knowledge. we also show that the highperforming nle models are still prone to generating significantly many in-nles and, surprisingly, that a higher nle quality does not necessarily imply fewer inconsistencies. second, we propose a simple yet efficient off-the-shelf method for alleviating inconsistencies that grounds any nle model into background knowledge, leading to fewer inconsistencies. the code for this paper is available at https://github.com/mj-jang/eknowia . ## inconsistency attack we propose eknowia (explanations knowledgegrounded inconsistency attack), which detects more in-nles in a faster and more general manner than eia. ## our know method for alleviating inconsistencies our approach for alleviating inconsistencies in nle models consists of two steps: (1) extraction of knowledge related to the input and (2) knowledge injection. extracting related knowledge. we leverage a knowledge extraction heuristic proposed by @xcite as follows: ## related work a growing number of works focus on building nle models in different areas such as natural language inference @xcite ), question answering @xcite @xcite , medical imaging @xcite , self-driving cars @xcite , and offensiveness classification @xcite . most commonly, the performance of these models is assessed only in terms of how plausible the reasons provided by their nles are. to our knowledge, @xcite is the only work to investigate inconsistencies in nles. we improve their adversarial attack as well as bring an approach to alleviate inconsistencies. works have also been conducted to analyse and make dialogue models generate responses consistent with the dialogue history @xcite @xcite . however, these works are difficult to be applied to nle models, in part because they require specific auxiliary datasets, such as pairs of inconsistent sentences. other works investigated the logical consistency of a model's predictions @xcite @xcite , but would not have straightforward extensions for investigating nles inconsistencies. besides consistency, nles can also be assessed for their faithfulness w.r.t. the decision-making process of the model that they aim to explain @xcite . ## summary and outlook we proposed the eknowia attack, which is more generalizable, successful, and faster than the previous eia attack in detecting in-nles. our experiments show that current nle models generate a significant number of in-nles, and that higher nle quality does not necessarily imply fewer inconsistencies. we also introduced a simple but efficient method that grounds a model into relevant knowledge, decreasing the number of in-nles. our work paves the way for further work on detecting and alleviating inconsistencies in nle models.