Calibration Meets Explanation: A Simple and Effective Approach for Model Confidence Estimates (2022)

Abstract: Calibration strengthens the trustworthiness of black-box models by producing better accurate confidence estimates on given examples. However, little is known about if model explanations can help confidence calibration. Intuitively, humans look at important features attributions and decide whether the model is trustworthy. Similarly, the explanations may tell us when the model might know and when it does not. Inspired by this, we propose a method named CME that leverages model explanations to make the model less confident with non-inductive attributions. The idea is that when the model is not highly confident, it is difficult to identify strong indications of any class, and the tokens accordingly do not have high attribution scores for any class and vice versa. We conduct extensive experiments on six datasets with two popular pre-trained language models in the in-domain and out-of-domain settings. The results show that CME improves calibration performance in all settings. The expected calibration errors are further reduced when combined with temperature scaling. Our findings highlight that model explanations can help calibrate posterior estimates.

## introduction accurate estimates of posterior probabilities are crucial for neural networks in various natural language processing (nlp) tasks @xcite . for example, it would be helpful for humans if the models deployed in practice abstain or interact when they cannot make a decision with high confidence @xcite . while pre-trained language models (plms) have improved the performance of many nlp tasks @xcite , how to better avoid miscalibration is still an open research problem @xcite . in this paper, we investigate if * corresponding authors positive a fast funny highly enjoyable movie. negative it's about following your dreams no matter what your parents think. and how model explanations can help calibrate the model. explanation methods have attracted considerable research interest in recent years for revealing the internal reasoning processes behind models @xcite @xcite . token attribution scores generated by explanation methods represent the contribution to the prediction @xcite . intuitively, one can draw some insight for analyzing and debugging neural models from these scores if they are correctly attributed, as shown in inspired by this, we propose a simple and effective method named cme that can be applied at training time and improve the performance of the confidence estimates. the estimated confidence measures how confident the model is for a specific example. ideally, reasonable confidence estimates should have higher confidence for correctly classified examples resulting in higher attributions than incorrect ones. hence, given an example pair during training with an inverse classification relationship, we regularize the classifier by comparing the wrong example's attribution magnitude and the correct example's attribution magnitude. our work is related to recent works on incorporating explanations into learning. different from previous studies that leverage explanations to help users predict model decisions @xcite or improve the accuracy @xcite , we focus on answering the following question: are these explanations of black-box models useful for calibration? if so, how should we exploit the predictive power of these explanations? considering the model may be uninterpretable due to the nature of neural networks and limitations of explanation method @xcite , a calibrated model by cme at least can output the unbiased confidence. moreover, we exploit intrinsic explanation during training, which does not require designing heuristics @xcite and additional data augmentation @xcite . we conduct extensive experiments using bert @xcite and roberta @xcite to show the efficacy of our approach on three natural language understanding tasks (i.e., natural language inference, paraphrase detection, and commonsense reasoning) under in-domain (id) and out-of-domain (od) settings. cme achieves the lowest expected calibration error without accuracy drops compared with strong sota methods, e.g., @xcite . when combined with temperature scaling (ts) @xcite , the expected calibration errors are further reduced as better calibrated posterior estimates under these two settings. ## related work as accurate estimates are required for many difficult or sensitive prediction tasks @xcite , probability calibration is an important uncertainty estimation task for nlp. unlike other uncertainty estimation task (e.g., out-of-domain detection, selective inference), calibration focuses on aleatoric uncertainty measured by the probability of the prediction and adjusts the overall model confidence level @xcite @xcite . for example, @xcite propose to adopt multiple predictions with different dropout masks and then combine them to get the confidence estimate. recently, several works focus on the calibration of plms models for nlp tasks @xcite @xcite @xcite . @xcite investigate the calibration properties of different transformer architectures and sizes of bert. in line with recent work @xcite , our work focuses on how explanations can help calibration in three nlp tasks. however, we do not need to learn a calibrator by using model interpretations with heuristics, and also do not compare due to its intensive computation cost when generating attributions. in contrast, we explore whether model explanations are useful for calibrating black-box models during training. ## conclusion we propose a method that leverages model attributions to address calibration estimates of plmsbased models. considering model attributions as facts about model behaviors, we show that cme achieves the lowest eces under most settings for two popular plms.