Exploring Predictive Uncertainty and Calibration in NLP : A Study on the Impact of Method & Data Scarcity (2022)

Abstract: We investigate the problem of determining the predictive confidence (or, conversely, uncertainty) of a neural classifier through the lens of low-resource languages. By training models on sub-sampled datasets in three different languages, we assess the quality of estimates from a wide array of approaches and their dependence on the amount of available data. We find that while approaches based on pre-trained models and ensembles achieve the best results overall, the quality of uncertainty estimates can surprisingly suffer with more data. We also perform a qualitative analysis of uncertainties on sequences, discovering that a model’s total uncertainty seems to be influenced to a large degree by its data uncertainty, not model uncertainty. All model implementations are open-sourced in a software package.

## introduction in 1877, italian astronomer giovanni schiaparelli described the existence of "canals" on the surface of mars, a finding that was described by a contemporary as a "very important and perplexing @xcite " @xcite . it later turned out that the structures, originally termed canali in italian, were simply mistranslated, since the word can also refer to (natural) channels of water. by that point however, the possibility of irrigation on the red planet had already sept into popular culture, and is still being referenced to this day. in the meantime, translation has become a task that is increasingly performed by neural networks, which -in the face of a word such as canali -might simply fall back on the most likely translation given the training data. and while the error above seems fairly innocuous, there are more safety-critical scenarios in which such ambiguities matter and can potentially have negative real-word consequences. besides translation, there also exist other language-based problems in which the uncertainty surrounding a model prediction can convey critical information, such as medical analyses @xcite , legal case data @xcite or analyzing job applications @xcite . determining model confidence, or, conversely, uncertainty, consequently is an important mean to instill trust in end users and avert harm @xcite . while there exist many works on images @xcite and tabular data @xcite @xcite , the quality of uncertainty estimates provided by neural networks remains underexplored in natural language processing (nlp). in addition, as model underspecification due to insufficient data presents a risk (d'amour et al., 2020), the increasing interest in less-researched languages with limited resources raises the question of how reliably uncertain predictions can be identified. this lets us pose the following research questions: rq1 what are the best approaches in terms of uncertainty quality and calibration? rq2 how are models impacted by the amount of available training data? rq3 what are differences in how the different approaches estimate uncertainty? contributions 1 we address these questions by conducting a comprehensive empirical study of eight different models for uncertainty estimation for classification and evaluate their effectiveness on three languages spanning distinct nlp tasks, involving sequence labeling and classification. @xcite we show that while approaches based on pre-trained models and ensembles achieve the best results overall, the quality of uncertainty estimates on ood data can become worse using more data. @xcite in a qualitative analysis, we also discover that a model's total uncertainty seems to mostly consist of its data uncertainty. @xcite we make our experimental code and model implementations available open-source in separate repositories, aiding future research in this direction. 1 2 related work notions of uncertainty in the absence of additional information, the introductionary example canali has two valid translations -canals and channels. this is an instance of data or aleatoric uncertainty, describing the irreducible ambiguity and noise in the data generating process. the other notion is model or epistemic uncertainty: fitting parameters, there remains a degree of incertitude about the optimal values due to finite data. we can usually reduce this uncertainty by amassing more data, 2 for instance by supplying a translation system with other meanings of canali. these two concepts form the basis for uncertainty estimation in machine learning @xcite . uncertainty in nlp since uncertainty estimation literature is manifold on image data, we dedicate this part to related works in the realm of natural language processing. there are several examples trying to incorporate uncertainty into models to either increase trustworthiness or performance, for instance in machine translation @xcite @xcite , summarization (gidiotis and tsoumakas, 2021), information retrieval @xcite and active learning @xcite . to obtain uncertainties, @xcite use stochastic-gradient langevin dynamics @xcite to obtain posterior weight samples for a lstm. @xcite apply mc dropout with determinantal point processes to transformers for natural language understanding. several authors have also highlighted connections of multihead attention to bayesian inference @xcite . @xcite attempt to transfer the idea of prior networks @xcite onto recurrent neural networks. another line of works investigates uncertainty properties themselves; for instance, @xcite try to explain uncertainty estimates for bert and roberta. another example is given by @xcite , who use predictive uncertainty to explain hallucination in language generation. xu et al. (2020) similarly use uncertainty as a tool to investigate challenges of neural summarization approaches. lastly, due to the way that uncertainty estimates are evaluated, investigating distributional shift in nlp is also of interest, for instance through the work of arora et al. (2021), kamath et al. (2020), who focus on question answering and tan et al. (2019) for text classification. the most similar work to ours is the text classification uncertainty benchmark by van landeghem et al. (2022), however they do not consider the impact of data or language, and test a different selection of models. calibration calibration denotes the property of a model's output to accurately reflect the true chance of a correct prediction -i.e. predicting a class with a confidence of 90% should yield the correct prediction for 90% of similar inputs, when repeated. there have been several studies testing this property in modern neural networks @xcite @xcite and proposing ways to improve it @xcite @xcite @xcite . in nlp, calibration as been explored for pre-trained models @xcite , including on out-of-distribution data @xcite , for neural machine translation @xcite and for question-answering @xcite . likewise, authors have proposed several calibration schemes, for instance by focusing on classes of interest @xcite , generating synthetic examples for regularization @xcite , using richer input representations @xcite and adapting prompts in a zero-shot setting @xcite . ## experiments 4.1 rq1: uncertainty & calibration we present the results from our experiments using the largest training set sizes per dataset in calibration we also see bert models to generally achieve lower calibration errors across all metrics measured, which is in line with previous works @xcite . it is interesting to see that the correct prediction is almost always contained in the 0.95 confidence set across all models, however these number have to be interpreted in the context of the set's width: it becomes apparent that for instance lstms achieve this coverage by spreading probability mass over many classes, while only bertbased models, lstm ensembles as well as the bayesian lstm (on danish) and the variational lstm (on finnish) are confidently correct. uncertainty quality lstm-based model seem to struggle to distinguish in-from out-ofdistribution data based on predictive uncertainty. for danish, only berts perform visibly above chance-level. for finnish, the aupr results suggest that although some ood instances are quickly task id ood calibration id ood uncertainty id ood model acc.↑ f1 ↑ ece↓ ace↓ %cov.↑ ∅width↓ auroc↑ aupr↑ token τ ↑ seq. τ ↑ lstm .79 ±.00 .62 ±.01 77.95 ±.00 .49 ±.01 1.00 ±.00 144.00 ±.00 .88 ±.01 .60 ±.01 .75 ⃝ ±.01 bayesian lstm .59 ±.06 .46 ±.05 77.66 ±.05 .22 ±.01 .88 ±.00 41.99 ±1.94 .86 △ ±.01 .59 ±.01 .66 ⃝ ±.02 lstm ensemble .81 ±.00 .64 ±.01 77.11 ±.00 .09 ±.00 .87 ±.00 4.27 ±.05 .92 ±.00 .71 ±.01 .73 ±.01 variational bert .45 ±.16 .34 ±.13 77.92 ±.02 .22 ±.06 1.00 ±.00 115.11 ±11.38 .80 ±.01 .53 ±.01 .57 ⃝ ±.09 english ddu bert .79 ±.00 .64 ±.01 77.02 ±.00 .00 ±.00 .82 ±.00 1.46 ±.04 .88 ⃝ ±.00 .62 ⃝ ±.01 .87 ⃝ ±.00 lstm .93 ±.00 .92 ±.00 .26 ±.01 .19 ±.01 17.18 ±.00 17.17 ±.00 .16 ±.01 .10 ±.01 1.00 ±.00 1.00 ±.00 19.00 ±.00 19.00 ±.00 .50 ⃝ ±.02 .14 ⃝ ±.01 .50 ⃝ ±.01 .47 ⃝ ±.00 -.26 ±.02 -.28 ⃝ ±.05 variational lstm .90 ±.02 .90 ±.02 .08 ±.02 .09 ±.02 16.74 ±.03 16.72 ±.03 .26 ±.02 .17 ±.01 .99 ±.01 .98 ±.01 6.62 ±.37 6.68 ±.33 .60 ±.04 .21 ±.02 .23 ⃝ ±.06 .23 ⃝ ±.05 -.04 ±.02 -.02 ±.05 st-τ lstm .92 ±.00 .92 ±.00 .12 ±.00 .09 ±.00 16.67 ±.00 16.63 ±.01 .24 ±.01 .15 ±.01 1.00 ±.00 .99 ±.00 7.10 ±.07 7.03 ±.08 .54 ±.01 .15 ±.01 .50 ⃝ ±.00 .48 ⃝ ±.00 -.05 ±.03 -.01 ±.05 bayesian lstm .93 ±.00 .93 ±.00 .07 ±.00 .07 ±.00 16.81 ±.00 16.79 ±.00 .25 ±.01 .18 ±.01 1.00 ±.00 1.00 ±.00 1.68 ±.04 1.70 ±.05 .65 ±.17 .31 ±.30 .53 ⃝ ±.01 .55 ⃝ ±.01 -.01 ±.07 -.02 ±.04 lstm ensemble .95 ±.00 .94 ±.00 .33 ±.01 .25 ±.01 16.37 ±.00 16.35 ±.00 .18 ±.01 .13 ±.01 .98 ±.00 .97 ±.00 1.62 ±.00 1.58 ±.01 .60 ±.02 .18 ±.01 .44 ±.00 .45 ±.00 -.19 ±.01 -.28 ±.01 sngp bert .22 ±.35 .19 ±.34 .03 ±.03 .02 ±.02 17.19 ±.01 17.18 ±.01 .08 ±.01 .06 ±.01 1.00 ±.00 1.00 ±.00 18.84 ±.32 18.83 ±.34 .86 △ ±.06 .49 △ ±.12 .17 ±.09 .26 ±.14 .29 ±.03 .44 ±.11 variational bert .94 ±.00 .89 ±.00 .29 ±.01 .17 ±.00 16.36 ±.00 16.43 ±.00 .20 ±.00 .22 ±.00 .99 ±.00 .98 ±.00 2.25 ±.01 3.86 ±.08 .86 ±.01 .46 ±.02 .42 ⃝ ±.00 .17 ±.00 -.35 ±.01 -.41 ±.01 danish ddu bert .92 ±.00 .89 ±.00 .25 ±.00 .17 ±.00 16.41 ±.00 16.44 ±.00 .19 ±.01 .21 ±.01 .99 ±.00 .99 ±.00 3.48 ±.01 4.04 ±.03 .86 ⃝ ±.01 .39 ⃝ ±.02 .56 ⃝ ±.00 .25 ⃝ ±.01 -.24 ⃝ ±.01 -.38 ⃝ ±.03 lstm .75 ±.00 .69 ±.00 .57 ±.00 .53 ±.00 6.78 ±.00 6.80 ±.00 .40 ±.01 .38 ±.01 1.00 ±.00 1.00 ±.00 16.00 ±.00 16.00 ±.00 .63 △ ±.01 .69 ±.01 .29 ⃝ ±.00 .19 ⃝ ±.01 -.28 ±.02 -.27 ±.02 variational lstm .27 ±.00 .26 ±.00 .03 ±.00 .03 ±.00 6.65 ±.01 6.66 ±.01 .27 ±.01 .28 ±.01 .97 ±.00 .96 ±.00 1.35 ±.23 1.37 ±.21 .51 ±.01 .59 ±.01 .00 △ ±.01 .00 ±.00 .01 △ ±.03 .01 ±.01 st-τ lstm .76 ±.00 .71 ±.00 .58 ±.00 .55 ±.00 6.18 ±.00 6.21 ±.00 .20 ±.01 .22 ±.01 .97 ±.00 .96 ±.00 3.32 ±.01 3.57 ±.01 .62 △ ±.01 .69 ±.01 .31 ⃝ ±.00 .21 ⃝ ±.01 -.14 ±.02 -.12 ±.04 bayesian lstm .27 ±.00 .26 ±.00 .03 ±.00 .03 ±.00 6.84 ±.00 6.85 ±.00 .11 ±.00 .12 ±.00 1.00 ±.00 1.00 ±.00 16.00 ±.00 16.00 ±.00 .51 ±.01 .60 ±.00 .00 ±.00 .00 ±.00 .01 ⃝ ±.01 .04 ±.00 lstm ensemble .81 ±.00 .75 ±.00 .62 ±.00 .57 ±.00 6.18 ±.00 6.22 ±.00 .17 ±.01 .21 ±.00 .99 ±.00 .98 ±.00 3.46 ±.01 3.80 ±.01 .67 ±.01 .74 ±.01 .29 ⃝ ±.00 .19 ⃝ ±.01 -.28 ±.01 -.31 ±.01 variational bert .87 ±.00 .81 ±.00 .74 ±.00 .70 ±.00 6.11 ±.00 6.15 ±.00 .14 ±.00 .18 ±.01 .99 ±.00 .99 ±.00 4.68 ±.03 5.19 ±.02 .64 △ ±.01 .70 ⃝ ±.01 .14 ⃝ ±.00 .08 ±.00 -.19 ±.00 -.16 ±.01 sngp bert .18 ±.10 .17 ±.10 .07 ±.02 .08 ±.02 6.82 ±.00 6.83 ±.00 .16 ±.02 .15 ±.01 1.00 ±.00 .99 ±.01 15.00 ±.00 15.00 ±.00 .54 △ ±.05 .63 △ ±.04 .15 ±.04 .15 ±.03 .12 ±.05 .14 ±.02 finnish ddu bert .87 ±.00 .81 ±.00 .72 ±.03 .68 ±.03 6.01 ±.00 6.03 ±.00 .33 ±.02 .38 ±.02 .94 ±.00 .91 ±.00 2.16 ±.06 2.31 ±.06 .61 ⃝ ±.02 .69 ⃝ ±.02 .39 ⃝ ±.04 .26 ⃝ ±.03 -.07 ⃝ ±.05 -.16 ⃝ ±.04 ## discussion our experiments in section 4 have uncovered interesting nuances about uncertainty estimation in natural language processing. with respect to rq1, we observe that fine-tuning berts and training lstm ensembles on different languages produces high task scores with low calibration errors and high-quality uncertainty estimates, but only so on in-distribution data. on ood data, uncer-tainty estimates from fine-tuned models do actually become less indicative of potential model loss compared to lstm-based models. we also find that among the variety of uncertainty metrics proposed, there does not appear to be a superior metric. differences in kendall's τ on a token and sequence level suggest that loss and uncertainties fluctuate over the course of sequence. answering rq2, it seems that paradoxically more training data seems to decrease the quality of uncertainty estimates on ood data for pre-trained models. we speculate that fine-tuning models increasingly lets them forget relevant features that would produce higher uncertainty. this might explain why for lstm-type models, this effect seems to be smaller. lastly, we conclude about rq3 that all models' total uncertainty behave somewhat similarly, potentially due to the strong influence of aleatoric uncertainty. from these insights, we conclude that the approaches using pre-trained models overall give the best trade-off between task performance, uncertainty quality and calibrations, however their failure on ood samples opens up further directions of research. ensembles can provide an alternative here in data-scarce settings, when the task is sufficiently learnable without the need for pre-training. ## conclusion in this work, we explore the current options for uncertainty estimation in nlp on three different languages and tasks, focusing on the impact of available data on the quality of uncertainty scores in a potential low-resource environment. we conclude the following: fine-tuning pre-trained models produces the best results in terms of task performance, calibration and uncertainty quality, but only on in-distribution data. on out-of-distribution data, lstm-based models produce more reliable figure 4 : uncertainty estimates on single sequences, for predictive entropy of different models on danish (section 4.3) and predictive entropy and mutual information for multi-prediction models on finnish (figure 4b ). future work we see our work as groundwork for future research: while uncertainty estimation is a thriving subject in computer vision, it remains understudied in nlp. our experiments highlight that the model behavior on language data is not well-understood and open several lines for further investigation: one such line is the development of new methods for nlp that a) produce more faithful estimates on ood data while retaining their id performance and b) require less training data to so, in order to be applicable in low-resource settings. additionally, our qualitative analyses along with existing works such as xiao and wang (2021); xu et al. (2020) highlighted the potential to use uncertainty to understand model behavior.