Re-Examining Calibration: The Case of Question Answering (2022)

Abstract: For users to trust model predictions, they need to understand model outputs, particularly their confidence — calibration aims to adjust (calibrate) models’ confidence to match expected accuracy. We argue that the traditional calibration evaluation does not promote effective calibrations: for example, it can encourage always assigning a mediocre confidence score to all predictions, which does not help users distinguish correct predictions from wrong ones. Building on those observations, we propose a new calibration metric, MacroCE, that better captures whether the model assigns low confidence to wrong predictions and high confidence to correct predictions. Focusing on the practical application of open-domain question answering, we examine conventional calibration methods applied on the widely-used retriever-reader pipeline, all of which do not bring significant gains under our new MacroCE metric. Toward better calibration, we propose a new calibration method (ConsCal) that uses not just final model predictions but whether multiple model checkpoints make consistent predictions. Altogether, we provide an alternative view of calibration along with a new metric, re-evaluation of existing calibration methods on our metric, and proposal of a more effective calibration method.

## introduction while large pretrained language models have conquered many downstream tasks @xcite , it is sometimes unclear when we should trust them since they often produce false @xcite or hallucinated @xcite predictions. this is important for both model deployment-where low-confidence outputs can be censored-and for end users who need to know whether to trust a model output. the solution is to make sure that models provide reliable figure 1 : distribution of predictions on hotpotqa in an ood setting. we put predictions within the same confidence range into the same bucket (10 fixed-range buckets) and compute the average confidence and accuracy within each bucket. the x-axis represents the confidence range of each bucket, the y-axis represents the average answer accuracy for the dashed line plot and represents the relative bucket sizes for the histogram. before calibration, most predictions have overly high confidence. after temperature scaling, all predictions' confidence values are scaled to become closer to the overall answer accuracy (24.5). moreover, both correct (green bars) and wrong predictions (blue bars) are mixed in the same buckets, making them hard to distinguish. we re-examine calibration and apply it to a complex task with real-world applications: opendomain question answering (odqa; @xcite . the task takes an input question, retrieves evidence passages from a large corpus such as wikipedia, and then returns an answer string. un-like classification, odqa is a pipeline with multiple components: a passage retriever followed by a reader. this complexity is both typical of modern machine learning systems and poses additional challenges. we explore adapting calibration methods for the retriever-reader odqa models on both in-domain and out-of-domain (ood) settings. using the commonly used temperature scaling (ts) method on odqa, models get lower ece, similar to previous findings on multi-class classification tasks @xcite . however, we argue that low ece does not correspond to useful calibration: in fact, it underestimates the true calibration errors due to its bucketing mechanism. ece measures the difference between the confidence and expected accuracy by splitting the confidence values into buckets, taking an average confidence and an average accuracy of each bucket and marginalizing over their differences. however, this allows models with middling confidence to win on the ece metric. for instance, temperature scaling assigns all predictions in the range [0.1, 0.5) on hotpotqa (figure 1 ), which is not useful for users separating correct and wrong predictions because the confidence values are all in a similar range. moreover, the bucketing mechanism causes a cancellation effect where overconfident and under-confident predictions are bucketed together and averaged out, hiding the instancelevel calibration errors. we propose macro-average calibration error (macroce) as an alternative metric that directly focuses on distinguishing correct from wrong predictions (section 4.2). macroce removes the bucketing mechanism and sums calibration error at the instance level. it also takes equal consideration of correct and wrong predictions through macro-averaging to be insensitive to the accuracy level (e.g., when the accuracy is very low, simply lowering confidence on all predictions would lower ece, but not macroce). macroce is insensitive to accuracy shifts, successfully satisfying the desiderata for a stable calibration metric @xcite . we also show that this metric flips the conclusion from the previous section based on ece-four existing calibration methods, including temperature scaling-the ece winner-do not lead to improvements in macroce (section 4). to address this shortcoming, we propose a new method, conscal, which tracks whether the model makes consistent correct predictions over different checkpoints during training. the intuition is that if the same correct prediction is consistent throughout the training trajectory, then it could serve as a strong sign that the model is confident about the prediction. conscal significantly improves macroce on both in-domain and ood evaluation (section 5), including when downstream users must validate model predictions (section 5). in summary, our contributions are: ts achieve very low ece; however, it does not capture if the model effectively distinguishes correct from wrong predictions. 2. we propose a better metric macroce, which captures models' ability to distinguish correct and wrong predictions without being sensitive to the absolute model accuracy. four different calibration methods known to be effective on ece are ineffective on macroce. 3. we introduce a new calibration method (conscal) that uses the consistency of model predictions to estimate confidence, significantly reducing macroce and outperforming all previous baselines. 4. together, we provide an alternative view of calibration, re-evaluation of existing calibration methods, and a proposal of a better calibration method based on our new viewpoint. ## background this section reviews the existing calibration framework, the associated ece metric, and the commonly used temperature scaling method that effectively optimizes the ece metric. ## calibration in open-domain question answering this section adapts the bucketing-based calibration framework for multi-class classification to odqa and evaluates this calibration method on multiple qa benchmarks, both in-and out-of-domain. ## flaws in ece and better alternatives this section takes a closer look at model accuracy and confidence and illustrates how ece is misleading in evaluating model calibration. we provide complementary views of calibration and propose a new calibration metric as an alternative. ## toward better calibration methods this section reviews existing calibration methods other than temperature scaling as baselinesincluding a feature based classifier, a neural reranker and label smoothing (section 5.1)-and introduces our new calibration method based on model consistency throughout training trajectory (conscal; section 5.2). all existing methods do not lower macroce, while conscal significantly improves macroce (section 5.3). ## related work this section reviews prior work on calibration metrics and methods that are relevant to nlp. calibration metrics. brier score @xcite is one of the earliest calibration metrics that sums over squared errors between accuracy and confidence for all instances, but is only applicable to binary classification. naeini et al. (2015) first use bucketing mechanisms to compute calibration errors, and propose metrics like ece and maximum calibration error (mce), nguyen and o'connor (2015) explore other bucketing mechanisms like equal-mass binning. nixon et al. (2019) point out the flaws of the fixed range bucketing mechanism used by ece, and propose an adaptive bucketing mechanism as an alternative. we take inspiration from these prior analyses, identify the problems grounded in a more complex open-domain question answering task, and propose macroce to address the issues. calibration methods. guo et al. (2017) first experiment various post-hoc calibration methods including temperature scaling, and thulasidasan et al. (2019) find that mixup training improves the calibration of image classifiers evaluated by the ece metric. within nlp, previous works explores calibration on multi-class classification (desai and durrett, 2020) and sequence tagging (nguyen and o'connor, 2015). in question answering, kamath et al. (2020) propose the selective question answering setting that aims to abstain on as few questions as possible while maintaining high accuracy. toward this goal, later approaches (zhang et al., 2021; ye and durrett, 2022) extract features to train a binary classifier to decide on which questions to abstain. while selective question answering offers a measurement of calibration, the scale of confidence values is not considered since abstention can be effective as long as correct predictions have higher confidence than wrong ones, regardless of the absolute scales. concurrent work (dhuliawala et al., 2022) explores calibration for retrieverreader odqa, focusing on combining information from the retriever and reader. in addition to spanextraction qa, jiang et al. (2021) and si et al. (2022) also explore calibration for generative qa. however, these works use ece for evaluation. 7 conclusion this paper investigates calibration in the realistic application of odqa where users need to decide whether to trust model predictions based on confidence scores. although confidence scores produced by existing calibration methods improve the popular ece metric, these confidence scores do not help distinguish correct and wrong predictions. we propose to use the macroce metric to remedy the flaws, and existing calibration methods fail on our macroce metric. we further propose a simple and effective calibration method -conscal -that leverages training consistency. our human study confirms both the effectiveness of conscal as well as the alignment between macroce and human preference. our work advocates and paves the path for user-centric calibration, and our con-scal method is a promising direction for better calibration. future work can adapt our calibration metric and method to more diverse tasks (such as generative tasks) and explore other ways to further improve user-centric calibration.