Rethinking Calibration of Deep Neural Networks: Do Not Be Afraid of Overconfidence

1 Introduction . Modern over-parameterized deep neural networks ( DNNs ) have been shown to be very powerful modeling tools for many prediction tasks involving complex input patterns [ 37 ] . In addition to obtaining accurate predictions , it is also important to capture accurate quantification of prediction uncertainty from deep neural networks in many real-world decision-making applications . A reliable predictive model should be accurate when it is confident about its predictions and indicate high uncertainty when it is likely to be inaccurate . However , modern DNNs trained with cross-entropy ( CE ) loss , despite being highly accurate , have been recently found to predict poorly calibrated probabilities , unlike traditional models trained with the same objective [ 4 ] . The overconfident predictions of DNNs could cause undesired consequences in safety-critical applications such as medical diagnosis and autonomous driving . Bayesian DNNs , which indirectly infer prediction uncertainty through weight uncertainties , have innate abilities to represent the model uncertainty [ 2 , 16 ] . But training and inferring those bayesian models are computationally more expensive and conceptually more complicated than non-bayesian models , and their performance depends on the form of approximation made due to computational constraints . Therefore , the study on uncertainty calibration of deterministic DNNs is important for both development practice and the perspective of understanding DNNs . Post-hoc calibration addresses the miscalibration problem by equipping a given neural network with an additional parameterized calibration component , which can be tuned with a hold-out validation ∗Corresponding author 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . dataset . Guo et al . [ 4 ] experimented with several classical calibration fixes and found that simple post-hoc methods like Temperature Scaling ( TS ) [ 25 ] and Histogram Binning ( HB ) [ 33 ] are significantly effective for DNNs . The authors of [ 10 ] and [ 27 ] proposed to learn linear and non-linear transformation functions to rescale the original output logits respectively . Gupta et al . [ 5 ] proposed to obtain a calibration function by approximating the empirical cumulative distribution of output probabilities via splines . Kumar et al . [ 11 ] proposed to integrate TS with HB to achieve more stable calibration performance . Patel et al . [ 24 ] proposed a mutual information maximization-based binning strategy to solve the severe sample-inefficiency issue in HB . Recently , there is another line of research which presents a possibility of improving the calibration quality of deterministic DNNs via regularization during training . Guo et al . [ 4 ] found that training DNNs with strong weight decay , which used to be the predominant regularization mechanism for training neural networks , has a positive impact on calibration . Müller et al . [ 19 ] showed that training models using the standard CE loss with label smoothing [ 28 ] , instead of one-hot labels , has a very favourable effect on model calibration . Mukhoti et al . [ 18 ] proposed to improve uncertainty calibration by replacing the conventionally used CE loss with the focal loss proposed in [ 14 ] when training DNNs . It is important to note that CE loss with label smoothing and focal loss can be considered as standard CE with an additional maximum-entropy regularizer , which means minimizing these losses is equivalent to minimizing CE loss and maximizing the entropy of the predicted distribution simultaneously [ 18 , 17 ] . Following these studies , a recent work [ 7 ] explored several explicit regularization techniques for improving the predictive uncertainty calibration directly . In this paper , we conduct an empirical study showing that despite the predictions obtained from the regularized models are well calibrated , they suffer from worse calibratable , namely , it is harder to further improve the calibrate performance with post-hoc calibration methods like temperature scaling and histogram binning . We found that the regularization works by simply aligning the average confidence of the whole dataset to the accuracy with some specific regularization strengths , and can not achieve fine-grained calibration . The comparison results show that when post-hoc calibration methods are allowed , the standard CE loss yields better calibration performance than those regularization methods . The extended experiments demonstrate that regularization will make DNNs lose the important information about the hardness of samples , which results in compressing the room of potential improvement by post-hoc calibration . Based on the experimental findings , we raise a natural question : can we design new loss functions in the opposite direction of these regularization methods to further improve the calibration performance ? To this end , we propose inverse focal loss , and empirically found that it can learn more calibratable models in some cases compared with the CE loss , though it causes severer overconfidence problem without post-hoc calibration . Most importantly , our findings show that overconfidence of DNNs is not the nightmare in uncertainty qualification and point out a new direction to improve the calibration of DNNs by considering main training and post-hoc calibration as a unified framework . 2 Preliminaries . Let Y = { 1 , ... , K } denote the label space and X = Rd denote the feature space . Given a sample ( x , y ) ∈ X × Y sampled from an unknown distribution , a learned neural network classifier fθ : X →∆K can produce a probability distribution for x on K classes , where ∆K denotes the K − 1 dimensional unit simplex . Here we assume fθ as a composition of a non-probabilistic K-way classifier gθ and a softmax function σ , i.e . fθ = gθ ◦ σ . For a query instance x , fθ gives its probability of assigning it to label i as exp ( g θ i ( x ) ) ∑K k=1 exp ( g θ k ( x ) ) , where gθi ( x ) denotes the i-th element of the logit vector produced by gθ . Then , ŷ : = arg maxi fθi ( x ) can be returned as the predicted label and p̂ : = maxi f θ i ( x ) can be treated as the associated confidence score . Expected Calibration Error ( ECE ) For a well-calibrated model , p̂ is expected to represent the true probability of correctness . Formally , a perfectly calibrated model satisfies P ( ŷ = y|p̂ = p ) = p for any p ∈ [ 0 , 1 ] . In practice , ECE [ 20 ] is a commonly used calibration metric from finite samples . It works by firstly grouping all samples ( let n denote the number of samples ) into M equally interval bins { Bm } Mm=1 with respect to their confidence scores , then calculating the expected difference between the accuracy and average confidence : ECE = ∑M m=1 |Bm| n |acc ( Bm ) − avgConf ( Bm ) | . Temperature Scaling By scaling the logits produced by gθ with a temperature T , the sharpness of output probabilities can be changed . Formally , after adding TS , the new prediction confidence can be expressed as : p̂ = maxi exp ( gθi ( x ) /T ) ∑K k=1 exp ( g θ k ( x ) /T ) . The temperature softens the output probability with T > 1 and sharpens the probability with T < 1 . As T → 0 , the output probability collapses to one-hot vector . As T → ∞ , the output probability approaches to a uniform distribution . After training of the model , T can be tuned on a hold-out validation set by optimization methods . Histogram Binning is a non-parametric calibration approach . Given an uncalibrated model , all the prediction confidences of validation samples can be divided into mutually exclusive N bins { Bn } Nn=1 according to a set of intervals { In } N+1n=1 which partitions [ 0 , 1 ] . Each bin is assigned a confidence score η , which can be simply set to the corresponding accuracy of samples in each bin . If the uncalibrated confidence p̂ of a query instance falls into bin Bn , then the calibrated confidence is ηn . The bins can be chosen by two simple schemes : equal size binning ( uniformly partitioning the probability interval in [ 0 , 1 ] ) and equal mass binning ( uniformly distributing samples over bins ) . Note that although the HB scheme is simple to implement and was demonstrated to achieve good calibration results in some datasets , it makes the predictor only produce very sparse confidence distribution , and compromises the many legitimately confident predictions . 3 Regularization in Neural Networks for Calibration . In recent years , there is a surge of research on model calibration by leveraging implicit or explicit regularization techniques during training of DNNs , which makes better calibrated predictions by avoiding the overconfident outputs . In this section , we firstly review three representative regularization methods and then empirically show their improvements on ECE compared with the baseline . Label Smoothing is widely used as a means to reduce overfitting of DNNs . The mechanism of LS is simple : when training with CE loss , the one-hot label vector y is replaced with soft label vector ỹ , whose elements can be formally denoted as ỹi = ( 1− ) yi + /K , ∀i ∈ { 1 , ... , K } , where > 0 is a strength coefficient . Müller et al . [ 19 ] demonstrated that label smoothing implicitly calibrates DNNs by preventing the networks from becoming overconfident . Let Lce denote the CE loss , then the following equation holds : Lce ( ỹ , fθ ) = ( 1− ) Lce ( y , fθ ) + Lce ( u , fθ ) ( 1 ) This can be simply proved . Therefore , minimizing CE loss between smoothed labels and the model outputs is equivalent to adding a confidence penalty term , i.e. , a weighted CE loss between the uniform distribution u and the model outputs , to the original CE loss . Lp Norm in the Function Space is one of the explicit regularization methods for calibration investigated by the recent work [ 7 ] . For a real number p ≥ 1 , the Lp Norm of a vector z with dimension n can be expressed as : ‖z‖p = ( ∑n i=1 |zi|p ) 1/p . By adding Lp Norm of logits gθ with a weighting coefficient α into final objective function , i.e . LLp ( y , fθ ) = Lce ( y , fθ ) + α ∥∥gθ∥∥ p , the function complexity of neural networks can be directly penalized during training . Focal Loss is originally proposed to address the class imbalance problem in object detection . By reshaping the standard CE loss through weighting loss components of all samples according to how well the model fits them , focal loss focuses on fitting hard samples and prevents the easy samples from overwhelming the training procedure . Formally , for classification tasks where the target distribution is one-hot encoding , it is defined as : Lf = − ( 1− fθy ) γ log fθy , where γ is a predefined coefficient . Mukhoti et al . [ 18 ] found that the models learned by focal loss produce output probabilities which are already very well calibrated . Interestingly , they also showed that focal loss is an upper bound of the regularized KL-divergence , which can be expressed formally as follows : Lf ≥ KL ( y||fθ ) − γH ( fθ ) ( 2 ) where H ( p ) denotes the entropy of distribution p. This upper bound property shows that replacing the CE loss with focal loss has the effect of adding a maximum-entropy regularizer .

Summary: The work studies methods that improve calibration during training e.g., label smoothing, logits regularization. Work shows that while these methods improve calibration compared with conventional training, they are less liable to improve from post-hoc calibration, such as temperature scaling e.g., TS >> label smoothing/etc. + TS. The work suggests some explanations of this effect and proposes an inverse focal loss that based on the proposed intuition might promote the quality of a post-hoc calibration.