Towards Improving Selective Prediction Ability of NLP Systems (2022)

Abstract: It’s better to say “I can’t answer” than to answer incorrectly. This selective prediction ability is crucial for NLP systems to be reliably deployed in real-world applications. Prior work has shown that existing selective prediction techniques fail to perform well, especially in the out-of-domain setting. In this work, we propose a method that improves probability estimates of models by calibrating them using prediction confidence and difficulty score of instances. Using these two signals, we first annotate held-out instances and then train a calibrator to predict the likelihood of correctness of the model’s prediction. We instantiate our method with Natural Language Inference (NLI) and Duplicate Detection (DD) tasks and evaluate it in both In-Domain (IID) and Out-of-Domain (OOD) settings. In (IID, OOD) settings, we show that the representations learned by our calibrator result in an improvement of (15.81%, 5.64%) and (6.19%, 13.9%) over ‘MaxProb’ -a selective prediction baseline- on NLI and DD tasks respectively.

## introduction in real-world applications, ai systems often encounter novel inputs that differ from their training data distribution. prior work has shown that even state-of-the-art models tend to make incorrect predictions on such inputs @xcite @xcite . this raises reliability concerns and hinders their adoption in real-world safety-critical domains like biomedical and autonomous robots. selective prediction addresses these concerns by enabling systems to abstain from making predictions when they are likely to be incorrect. avoiding incorrect predictions allows them to maintain high task accuracy and thus makes them more reliable. @xcite proposed 'max-prob' that uses the maximum softmax probability across all answer candidates as the confidence es-timate to selectively make predictions. while performing reasonably well in the in-domain setting, maxprob and other existing selective prediction techniques fail to translate that performance in the out-of-domain setting @xcite . in this work, we propose a selective prediction method that improves probability estimates of models in both in-domain and out-of-domain settings by learning strong representations via calibration. specifically, we calibrate models' outputs using a held-out dataset and use the calibrator as confidence estimator for selective prediction. to this end, we first argue that "all instances are not equally difficult and the model is not equally confident in all its predictions" and then through extensive experiments, we show that prediction confidence is positively correlated with correctness while difficulty score is negatively correlated (5.2). we leverage the above finding to calibrate models' outputs using these two signals. for computing the difficulty scores, we use a model-based technique (3.1) because human perception of difficulty may not always correlate well with machine interpretation. to calibrate a model, we annotate instances of a held-out dataset conditioned on the model's predictive correctness (computed using difficulty score and prediction confidence) and then train a calibrator using these instances. this annotation score represents the likelihood of correctness of the model's prediction. finally, the trained calibrator predicts this likelihood value for test instances and is used as the confidence estimator for selective prediction. to evaluate the efficacy of our method, we conduct comprehensive experiments in in-domain (iid) and out-of-domain (ood) settings for natural language inference (nli) and duplicate detection (dd) tasks. we also compare its performance with existing calibration techniques. on the nli task, our method achieves 15.81% and 5.64% im-provement on auc of risk-coverage curve over maxprob in iid and ood setting respectively. furthermore, on the dd task, it achieves 6.19% and 13.9% improvement in iid and ood setting respectively. finally, we hope that our work will facilitate development of more robust and reliable ai systems making their wide adoption in real-world applications possible. ## selective prediction selective prediction enables a system to abstain on instances where it is likely to be incorrect i.e it consists of a selector (g) that determines if the system should output the prediction. usually, g comprises of a prediction confidence estimator g and a threshold th that controls the abstention level: a selective prediction system makes trade-offs between coverage and risk. for a dataset d, coverage at a threshold th corresponds to the fraction of answered instances (where @xmath0. with the decrease in th, coverage will increase, but the risk will usually also increase. the overall selective prediction performance across all thresholds is measured by the area under risk-coverage curve @xcite . lower the auc, the better the system as it represents lower average risk across all thresholds. ## method we propose to train a confidence estimator that can assign higher scores to correctly predicted instances than incorrectly predicted ones. to this end, we leverage a held-out dataset and annotate it's instances conditioned on the model's predictive correctness. specifically, we infer the model on the held-out dataset and annotate instances with a score such that correctly predicted instances get assigned a higher score than incorrectly predicted instances. this annotation score models the likelihood of the prediction being correct and is computed using the model's prediction confidence and difficulty level of the instance. finally, a calibrator (regression model) is trained using this annotated held-out dataset and used as the confidence estimator for selective prediction. we detail each component of our method and the intuition behind it in the following subsections. ## conclusion and future work we proposed a selective prediction method that calibrates the model outputs using prediction confidence and difficulty level of the instances. through comprehensive experiments, we demonstrated that it achieves considerable improvement over max-prob on nli and duplicate detection tasks in both iid and ood settings. we hope that our work will facilitate development of more robust and reliable ai systems making their wide adoption in real-world applications possible.