Selective Question Answering under Domain Shift (2020)

Abstract: To avoid giving wrong answers, question answering (QA) models need to know when to abstain from answering. Moreover, users often ask questions that diverge from the model’s training data, making errors more likely and thus abstention more critical. In this work, we propose the setting of selective question answering under domain shift, in which a QA model is tested on a mixture of in-domain and out-of-domain data, and must answer (i.e., not abstain on) as many questions as possible while maintaining high accuracy. Abstention policies based solely on the model’s softmax probabilities fare poorly, since models are overconfident on out-of-domain inputs. Instead, we train a calibrator to identify inputs on which the QA model errs, and abstain when it predicts an error is likely. Crucially, the calibrator benefits from observing the model’s behavior on out-of-domain data, even if from a different domain than the test data. We combine this method with a SQuAD-trained QA model and evaluate on mixtures of SQuAD and five other QA datasets. Our method answers 56% of questions while maintaining 80% accuracy; in contrast, directly using the model’s probabilities only answers 48% at 80% accuracy.

## introduction question answering (qa) models have achieved impressive performance when trained and tested on examples from the same dataset, but tend to perform poorly on examples that are out-of-domain (ood) @xcite @xcite @xcite . deployed qa systems in search engines and personal assistants need to gracefully handle ood inputs, as users often ask questions that fall outside of the system's training distribution. while the ideal system would correctly answer all dataset distributions example question q: what can result from disorders of the immune system? (from squad) q: john wickham legg was recommended by jenner for the post of medical attendant to which eighth child and youngest son of queen victoria and prince albert of saxe-coburg and gotha? (from hotpotqa) q: capote gained fame with this "other" worldly 1948 novel about a teenager in a crumbling southern mansion. (from searchqa) train calibrate test source source known ood source unknown ood figure 1: selective question answering under domain shift with a trained calibrator. first, a qa model is trained only on source data. then, a calibrator is trained to predict whether the qa model was correct on any given example. the calibrator's training data consists of both previously held-out source data and known ood data. finally, the combined selective qa system is tested on a mixture of test data from the source distribution and an unknown ood distribution. ood questions, such perfection is not attainable given limited training data @xcite . instead, we aim for a more achievable yet still challenging goal: models should abstain when they are likely to err, thus avoiding showing wrong answers to users. this general goal motivates the setting of selective prediction, in which a model outputs both a prediction and a scalar confidence, and abstains on inputs where its confidence is low @xcite . in this paper, we propose the setting of selective question answering under domain shift, which captures two important aspects of real-world qa: (i) test data often diverges from the training distribution, and (ii) systems must know when to abstain. we train a qa model on data from a source distribution, then evaluate selective prediction performance on a dataset that includes samples from both the source distribution and an unknown ood distribution. this mixture simulates the likely scenario in which users only sometimes ask questions that are covered by the training distribution. while the sys-tem developer knows nothing about the unknown ood data, we allow access to a small amount of data from a third known ood distribution (e.g., ood examples that they can foresee). we first show that our setting is challenging because model softmax probabilities are unreliable estimates of confidence on out-of-domain data. prior work has shown that a strong baseline for indomain selective prediction is maxprob, a method that abstains based on the probability assigned by the model to its highest probability prediction @xcite . we find that maxprob gives good confidence estimates on in-domain data, but is overconfident on ood data. therefore, maxprob performs poorly in mixed settings: it does not abstain enough on ood examples, relative to in-domain examples. we correct for maxprob's overconfidence by using known ood data to train a calibrator-a classifier trained to predict whether the original qa model is correct or incorrect on a given example @xcite . while prior work in nlp trains a calibrator on in-domain data @xcite , we show this does not generalize to unknown ood data as well as training on a mixture of in-domain and known ood data. figure 1 illustrates the problem setup and how the calibrator uses known ood data. we use a simple random forest calibrator over features derived from the input example and the model's softmax outputs. we conduct extensive experiments using squad @xcite as the source distribution and five other qa datasets as different ood distributions. we average across all 20 choices of using one as the unknown ood dataset and another as the known ood dataset, and test on a uniform mixture of squad and unknown ood data. on average, the trained calibrator achieves 56.1% coverage (i.e., the system answers 56.1% of test questions) while maintaining 80% accuracy on answered questions, outperforming maxprob with the same qa model (48.2% coverage at 80% accuracy), using maxprob and training the qa model on both squad and the known ood data (51.8% coverage), and training the calibrator only on squad data (53.7% coverage). in summary, our contributions are as follows: ## related work our setting combines extrapolation to out-ofdomain data with selective prediction. we also distinguish our setting from the tasks of identifying unanswerable questions and outlier detection. ## problem setup we formally define the setting of selective prediction under domain shift, starting with some notation for selective prediction in general. ## methods recall that our setting differs from the standard selective prediction setting in two ways: unknown ood data drawn from q unk appears at test time, and known ood data drawn from q known is available to the system. intuitively, we expect that systems must use the known ood data to generalize to the unknown ood data. in this section, we present three standard selective prediction methods for indomain data, and show how they can be adapted to use data from q known . ## discussion in this paper, we propose the setting of selective question answering under domain shift, in which systems must know when to abstain on a mixture of in-domain and unknown ood examples. our setting combines two important goals for real-world systems: knowing when to abstain, and handling distribution shift at test time. we show that models are overconfident on ood examples, leading to poor performance in the our setting, but training a calibrator using other ood data can help correct for this problem. while we focus on question answering, our framework is general and extends to any prediction task for which graceful handling of out-of-domain inputs is necessary. across many tasks, nlp models struggle on out-of-domain inputs. models trained on standard natural language inference datasets @xcite generalize poorly to other distributions @xcite . achieving high accuracy on out-of-domain data may not even be possible if the test data requires abilities that are not learnable from the training data @xcite . adversarially chosen ungrammatical text can also cause catastrophic errors @xcite . in all these cases, a more intelligent model would recognize that it should abstain on these inputs. traditional nlu systems typically have a natural ability to abstain. shrdlu recognizes statements that it cannot parse, or that it finds ambiguous @xcite . qualm answers reading comprehension questions by constructing reasoning chains, and abstains if it cannot find one that supports an answer @xcite . nlp systems deployed in real-world settings inevitably encounter a mixture of familiar and unfamiliar inputs. our work provides a framework to study how models can more judiciously abstain in these challenging environments. at 80% accuracy, 62% of the examples maxprob answers are in-domain, compared to 68% for the calibrator. this demonstrates that the calibrator improves over maxprob by answering more indomain questions, which it can do because it is less overconfident on the ood questions.