Counterfactual Explanations for Natural Language Interfaces (2022)

Abstract: A key challenge facing natural language interfaces is enabling users to understand the capabilities of the underlying system. We propose a novel approach for generating explanations of a natural language interface based on semantic parsing. We focus on counterfactual explanations, which are post-hoc explanations that describe to the user how they could have minimally modified their utterance to achieve their desired goal. In particular, the user provides an utterance along with a demonstration of their desired goal; then, our algorithm synthesizes a paraphrase of their utterance that is guaranteed to achieve their goal. In two user studies, we demonstrate that our approach substantially improves user performance, and that it generates explanations that more closely match the user’s intent compared to two ablations.

## introduction semantic parsing is a promising technique for enabling natural language user interfaces @xcite @xcite . however, a key challenge facing semantic parsing is the richness of human language, which can often encode concepts (e.g., "circle") that do not exist in the underlying system or are encoded using different language (e.g., "ball"). thus, human users can have trouble providing complex compositional commands in the form of natural language to such systems. one approach to addressing this issue is to develop increasingly powerful models for understanding natural language @xcite . while there has been enormous progress in this direction, there remains a wide gap between what these models are capable of compared to human understanding @xcite , manifesting in the fact that these models can 1 code available at: https://github.com/ georgeto20/counterfactual_explanations . thus, an important problem is to devise techniques for explaining these models. generally speaking, a range of techniques have recently been developed for explaining machine learning models. the first technique is to use models that are intrinsically explainable, such as linear regression or decision trees. however, in the case of semantic parsing, such models may achieve suboptimal performance, and furthermore it is not clear that the structure of these models would be useful to end users. a second technique is to train a blackbox model, and then approximate it using an interpretable model. then, the interpretable model can be shown to the human user to explain the high-level decision-making process underlying the blackbox model. however, this approach also suffers from the fact that showing a decision tree or regression model is likely not useful to an end user. instead, we consider an alternative form of explanation called a counterfactual explanation @xcite . these explanations are designed to describe alternative outcomes to the user. in particular, given a prediction for a specific input, they tell the user how they could have minimally modified that input to achieve a different outcome. as an example, suppose a bank is using a machine learning model to help decide whether to provide a loan to an individual; if that individual is denied the loan, then the bank can provide them with a counterfactual explanation describing how they could change their covariates (e.g., increase their income) to qualify for a loan. we propose a novel algorithm for computing counterfactual explanations for semantic parsers. in particular, suppose that a user provides a com-user command 1: "go to the blue circle" user command 2: "go to the top right" our explanation: "go to the blue ball" mand in the form of a natural language utterance. if the natural language interface fails to provide the desired result, then our goal is to explain how the user could have modified their utterance to achieve the desired result. to this end, we have the human additionally provide the desired result. then, we compute an alternative utterance that the semantic parser correctly processes while being as similar as possible to the original utterance. intuitively, this explanation enables the user to modify their language to reliably achieve their goals in future interactions with the system. we evaluate our approach on the babyai environment @xcite , where the human can provide a virtual agent with commands to achieve complex tasks such as "pick up the green ball and place it next to the blue box". we perform two user studies, which demonstrate that our approach both produces correct explanations (i.e., match the user's desired intent), and that it substantially improves the user's ability to provide valid commands. ## experiments we perform two user studies to demonstrate (i) correctness: our explanations preserve the user's original intent, and (ii) usefulness: our explanations improve user performance. ## conclusion we have proposed a technique for explaining how users can adapt their utterances to interact with a natural language interface. our experiments demonstrate how our explanations can be used to significantly improve the usability of semantic parsers when they are limited in terms of their semantic understanding. while any explanations are already very useful, we show that personalizing explanations can further improve performance. a key design choice in our approach is to construct a synthetic grammar from which counterfactual explanations are generated. in a realistic application, the semantic parsing model can be trained on a combination of synthetic data and real-world data, enabling our approach to be used in conjunction with the synthetic grammar. a key direction for future work is extending our approach to settings where such a grammar is not available. in our experience, a key challenge in this setting is that the generated text can be unnatural, possibly due to the constraints imposed on the search space.