Investigating Selective Prediction Approaches Across Several Tasks in IID , OOD , and Adversarial Settings (2022)

Abstract: In order to equip NLP systems with ‘selective prediction’ capability, several task-specific approaches have been proposed. However, which approaches work best across tasks or even if they consistently outperform the simplest baseline MaxProb remains to be explored. To this end, we systematically study selective prediction in a large-scale setup of 17 datasets across several NLP tasks. Through comprehensive experiments under in-domain (IID), out-of-domain (OOD), and adversarial (ADV) settings, we show that despite leveraging additional resources (held-out data/computation), none of the existing approaches consistently and considerably outperforms MaxProb in all three settings. Furthermore, their performance does not translate well across tasks. For instance, Monte-Carlo Dropout outperforms all other approaches on Duplicate Detection datasets but does not fare well on NLI datasets, especially in the OOD setting. Thus, we recommend that future selective prediction approaches should be evaluated across tasks and settings for reliable estimation of their capabilities.

## introduction despite impressive progress made in natural language processing (nlp), it is unreasonable to expect models to be perfect in their predictions. they often make incorrect predictions, especially when inputs tend to diverge from their training data distribution @xcite @xcite . while this is acceptable for tolerant applications like movie recommendations, high risk associated with incorrect predictions hinders the adoption of these systems in realworld safety-critical domains like biomedical and autonomous robots. in such scenarios, selective prediction becomes crucial as it allows maintaining high accuracy by abstaining on instances where error is likely. selective prediction (sp) has been studied in machine learning @xcite and computer vision @xcite , but has only recently gained attention in nlp. @xcite proposed a posthoc calibration-based sp technique for question-answering (qa) datasets. @xcite distill the qa model to filter out error-prone questions. unfortunately, despite the shared goal of making nlp systems robust and reliable for real-world applications, sp has remained underexplored; the community does not know which techniques work best across tasks/settings or even if they consistently outperform the simplest baseline maxprob @xcite (that uses a threshold over the maximum softmax probability for selective prediction). in this work, we address the above point and study selective prediction in a large-scale setup of 17 datasets spanning over natural language inference (nli), duplicate detection, and qa tasks. our comprehensive experiments under in-domain (iid), out-of-domain (ood), and adversarial (adv) settings result in the following findings: 1. none of the existing sp approaches consistently and considerably outperforms maxprob. slight improvement in iid: most of the approaches outperform maxprob in the iid setting; however, the magnitude of improvement is very small (figure 1 ). for instance, mcd achieves an average improvement of just 0.28 on auc value across all nli datasets. negligible improvement in ood: the magnitude of improvement in ood is even lesser (0.08) than that observed in the iid (figure 2a ). in a few cases, we also observe performance degradation (higher auc than maxprob). performance degradation in adv: most of the approaches fail to even match the maxprob's performance in adv setting (figure 2b ). for instance, mcd degrades the auc value by 1.76 on duplicate detection datasets and calibration degrades by 1.27 on nli datasets. 2. approaches do not translate well across tasks: we find that a single approach does not achieve the best performance across all tasks. for instance, mcd outperforms all other approaches on duplicate detection datasets but does not fare well on the nli datasets. 3. existing approaches fail to outperform max-prob despite leveraging additional resources: mcd requires additional computation (for multiple inferences) while calibration-based approaches require a held-out dataset. in contrast, maxprob does not require any such resources and still outperforms them, especially in the adv setting. overall, our results highlight that there is a need to develop stronger selective prediction approaches that perform well across tasks while being computationally efficient. usually, g comprises of a confidence estimator g that indicates f ′ s prediction confidence and a threshold th that controls the abstention level: an sp system makes trade-offs between coverage and risk. for a dataset d, coverage at a threshold th is defined as the fraction of total instances answered by the system (where @xmath0: with decrease in th, coverage will increase, but the risk will usually also increase. the overall sp performance is measured by the area under risk-coverage curve @xcite which plots risk against coverage for all threshold values. lower the auc, the better the sp system as it represents lower average risk across all confidence thresholds. we note that 'confidence calibration' and 'ood detection' are related tasks but are nontrivially different from selective prediction as detailed in appendix a. ## conclusion selective prediction ability is crucial for nlp systems to be reliably deployed in real-world applications and we presented the most systematic study of existing selective prediction approaches. our study involved experiments in iid, ood, and adv settings with 17 datasets across several nlp tasks. we showed that despite leveraging additional resources (held-out data/computation), existing approaches fail to consistently and considerably outperform the simplest baseline (maxprob). furthermore, we demonstrated that these approaches do not translate well across tasks. overall, our results highlight that there is a need to develop stronger selective prediction approaches that perform well across multiple tasks (qa, nli, etc.) and settings (iid, ood, and adv) while being resource-efficient.