Learning to Explain Selectively: A Case Study on Question Answering (2022)

Abstract: Explanations promise to bridge the gap between humans and AI, yet it remains difficult to achieve consistent improvement in AI-augmented human decision making. The usefulness of AI explanations depends on many factors, and always showing the same type of explanation in all cases is suboptimal—so is relying on heuristics to adapt explanations for each scenario. We propose learning to explain”selectively”: for each decision that the user makes, we use a model to choose the best explanation from a set of candidates and update this model with feedback to optimize human performance. We experiment on a question answering task, Quizbowl, and show that selective explanations improve human performance for both experts and crowdworkers.

## introduction recent advances in machine learning (ml) @xcite @xcite sparked new interest in intelligence augmentation-the vision that computers are not mere number-crunching tools but also interactive systems that can augment humans at problem solving and decision making @xcite . the hope is to combine the complementary strengths of human and machine, and to fully harness these models with human intuitions and oversight @xcite . but this agenda is obstructed by the many counterintuitive traits of neural networks (nns) @xcite @xcite and our lack of theoretical understanding @xcite : these models are not interpretable to humans by default, and it is difficult to foresee when they will fail. this lack of interpretability also amplifies the risk of model bias @xcite @xcite , making it difficult to use nn-powered ais in real-world decision making @xcite . to bridge the gap between human and machine, various methods attempt to explain model predictions in human-interpretable terms, e.g., by providing more context to the model's uncertainty estimation @xcite , by highlighting the most important part of the input @xcite @xcite , and by retrieving relevant training examples @xcite . grounded in psychology @xcite , these explanations promise to augment human decision making @xcite . but in applicationgrounded evaluations-with real problems and real humans @xcite , it proves difficult for any single explanation method to achieve consistent improvement in disparate contexts @xcite . a major contributor to this problem is the breadth of context that the explanation methods are applied in. internally, the explanation method can falter when the model reacts badly to novel inputs @xcite ; externally, it faces human users with diverse levels of expertise @xcite , engagement @xcite , and general trust in ai @xcite . our current use of explanations demands a one-size-fits-all solution, but existing methods cannot provide that as they are largely oblivious to the above-mentioned variables. selective explanations. each person is unique, and the right explanation might vary from one example to another, so we propose to show explanations selectively to maximize their utility as decision support. concretely, we assume a given set of explanation methods, but instead of showing all of them for every decision that the human user makes, we use a selector to choose a subset of the explanations to display. we can think of the selector as controlling an on/off switch for each explanation method. the selector is allowed, for example, to show three types of explanations for one example but withhold all of them for the next one. online optimization. for the policy to accurately estimate the utility of explanations in each context, its training data must provide reasonable coverage over the joint distribution of all types of explanations, human users, and examples. this means the dataset will have to include cases where the users receive suboptimal decision support, e.g., with excessive explanations causing information overload @xcite . we focus on the online setting-new information is constantly hurtling toward the user with limited time to carefully select which explanation to pay attention to-which represents real-world scenarios where the opportunity cost of giving suboptimal support cannot be ignored. in this setting, a good policy must balance the trade-off between sticking to tried and true combinations of explanations and exploring new ones; we model this trade-off with a multiarmed bandit formulation @xcite . we evaluate selective explanations on quizbowl-explained in more detail in section 2-using the same platform as @xcite . to mimic real-world decision making, we recruit twenty trivia enthusiasts and ran a multi-player, real-time quizbowl tournament. we compare our method head-to-head against baselines such as showing all explanations for all examples. selective explanations outperform all other strategies, including the best subset of explanations identified by @xcite . we also evaluate our method with mechanical turkers-amateurs whose performance without assistance is far worse than the ai. explanations significantly boost their performance, but only selective explanations can help them reach performance comparable with the ai. ## explaining selectively to support human decisions explanations have many uses in human-ai cooperation; this paper focuses on using explanations as decision support-to improve the quality of human decisions under machine assistance. not all problems benefit from machine assistance @xcite -in this section, we identify three criteria for decision support testbeds. we then introduce our setup based on quizbowl @xcite , a competitive trivia game. ## experiments we run two experiments with real human participants: a single-player experiment with crowdworkers and a multi-player real-time quizbowl tournament with experts. this section first introduces the metric for evaluating quizbowl competency, then provides details about the human players, the ai player, the explanation methods, and the baselines. we show that selective explanation provides personalized decision support and leads to the best augmented human performance. ## conclusion: explanations tailored for users users benefit from collaborating with an ai, and this collaboration can be improved by explaining the ai well. moreover, the benefit is not universal, some users need or thrive with different explanations. however, finding the right combination is not easy; while our bandit approach can find useful explanations, it requires both the user to become acclimatized to human-ai teaming and the bandit to explore the space of explanations. as human-ai collaborations become more common, we must continue to search for better signals and methods to help the teaming minimize stress and acclimation but maximize fun and productivity.