Calibration of Pre-trained Transformers (2020)

Abstract: Pre-trained Transformers are now ubiquitous in natural language processing, but despite their high end-task performance, little is known empirically about whether they are calibrated. Specifically, do these models’ posterior probabilities provide an accurate empirical measure of how likely the model is to be correct on a given example? We focus on BERT and RoBERTa in this work, and analyze their calibration across three tasks: natural language inference, paraphrase detection, and commonsense reasoning. For each task, we consider in-domain as well as challenging out-of-domain settings, where models face more examples they should be uncertain about. We show that: (1) when used out-of-the-box, pre-trained models are calibrated in-domain, and compared to baselines, their calibration error out-of-domain can be as much as 3.5x lower; (2) temperature scaling is effective at further reducing calibration error in-domain, and using label smoothing to deliberately increase empirical uncertainty helps calibrate posteriors out-of-domain.

## introduction neural networks have seen wide adoption but are frequently criticized for being black boxes, offering little insight as to why predictions are made @xcite @xcite and making it difficult to diagnose errors at test-time. these properties are particularly exhibited by pre-trained transformer models @xcite @xcite , which dominate benchmark tasks like superglue @xcite , but use a large number of self-attention heads across many layers in a way that is difficult to unpack @xcite . one step towards understanding whether these models can be trusted is by analyzing whether they are calibrated @xcite @xcite : how aligned their posterior probabilities are with empirical likelihoods @xcite . if a model assigns 70% probability to an event, the event should occur 70% of the time if the model is calibrated. although the model's mechanism itself may be uninterpretable, a calibrated model at least gives us a signal that it "knows what it doesn't know," which can make these models easier to deploy in practice @xcite . in this work, we evaluate the calibration of two pre-trained models, bert @xcite and roberta @xcite , on three tasks: natural language inference @xcite , paraphrase detection @xcite , and commonsense reasoning @xcite . these tasks represent standard evaluation settings for pretrained models, and critically, challenging out-ofdomain test datasets are available for each. such test data allows us to measure calibration in more realistic settings where samples stem from a dissimilar input distribution, which is exactly the scenario where we hope a well-calibrated model would avoid making confident yet incorrect predictions. our experiments yield several key results. first, even when used out-of-the-box, pre-trained models are calibrated in-domain. in out-of-domain settings, where non-pre-trained models like esim @xcite are overconfident, we find that pretrained models are significantly better calibrated. second, we show that temperature scaling @xcite , multiplying non-normalized logits by a single scalar hyperparameter, is widely effective at improving in-domain calibration. finally, we show that regularizing the model to be less certain during training can beneficially smooth probabilities, improving out-of-domain calibration. ## related work calibration has been well-studied in statistical machine learning, including applications in forecasting @xcite @xcite , medicine @xcite , and computer vision @xcite @xcite . past work in natural language processing has studied calibration in the nonneural @xcite and neural @xcite settings across several tasks. however, past work has not analyzed large-scale pre-trained models, and we additionally analyze out-of-domain settings, whereas past work largely focuses on in-domain calibration @xcite . another way of hardening models against outof-domain data is to be able to explicitly detect these examples, which has been studied previously @xcite @xcite . however, this assumes a discrete notion of domain; calibration is a more general paradigm and gracefully handles settings where domains are less quantized. ## posterior calibration a model is calibrated if the confidence estimates of its predictions are aligned with empirical likelihoods. for example, if we take 100 samples where a model's prediction receives posterior probability 0.7, the model should get 70 of the samples correct. formally, calibration is expressed as a joint distribution p (q, y ) over confidences @xmath0, where perfect calibration is achieved when p (@xmath1. this probability can be empirically approximated by binning predictions into k disjoint, equally-sized bins, each consisting of b k predictions. following previous work in measuring calibration @xcite , we use expected calibration error (ece), which is a weighted average of the difference between each bin's accuracy and confidence: for the experiments in this paper, we use @xmath2. ## conclusion posterior calibration is one lens to understand the trustworthiness of model confidence scores. in this work, we examine the calibration of pre-trained transformers in both in-domain and out-of-domain settings. results show bert and roberta coupled with temperature scaling achieve low eces in-domain, and when trained with label smoothing, are also competitive out-of-domain.