Should Ensemble Members Be Calibrated?

1 INTRODUCTION . Deep learning approaches achieve state-of-the-art performance in a wide range of applications , including image classification . However , these networks tend to be overconfident in their predictions , they often exhibit poor calibration . A system is well calibrated , if when the system makes a prediction with probability of 0.6 then 60 % of the time that prediction is correct . Calibration is very important in deploying system , especially in risk-sensitive tasks , such as medicine ( Jiang et al. , 2012 ) , auto-driving ( Bojarski et al. , 2016 ) , and economics ( Gneiting et al. , 2007 ) . It was shown by Niculescu-Mizil & Caruana ( 2005 ) that shallow neural networks are well calibrated . However , Guo et al . ( 2017 ) found that more complex neural network model with deep structures do not exhibit the same behaviour . This work motivated recent research into calibration for general deep learning systems . Previous research has mainly examined calibration based on samples from the true data distribution { x ( i ) , y ( i ) } Ni=1 ∼ p ( x , ω ) , y ( i ) ∈ { ω1 , ... , ωK } ( Zadrozny & Elkan , 2002 ; Vaicenavicius et al. , 2019 ) . This analysis relies on the limiting behaviour as N → +∞ to define a well calibrated system P ( y = ŷ|P ( ŷ|x ; θ ) = p ) = p ⇐⇒ lim N→+∞ ∑ i∈Spj δ ( y ( i ) , ŷ ( i ) ) |Spj | = p ( 1 ) where Spj = { i|P ( ŷ ( i ) = j|x ( i ) ; θ ) = p , i = 1 , ... , N } and ŷ ( i ) the model prediction for x ( i ) . δ ( s , t ) = 1 if s = t , otherwise 0 . However , Eq . ( 1 ) doesn ’ t explicitly reflect the relation between P ( y = ŷ|P ( ŷ|x ; θ ) = p ) and the underlying data distribution p ( x , y ) . In this work we examine this explicit relationship and use it to define a range of calibration evaluation criteria , including the standard sample-based criteria . One issue with deep-learning approaches is the large number of model parameters associated with the networks . Deep ensembles ( Lakshminarayanan et al. , 2017 ) is a simple , effective , approach for handling this problem . It has been found to improve performance , as well as allowing measures of uncertainty . In recent literature there has been “ contradictory ” empirical observations about the relationship between the calibration of the members of the ensemble and the calibration of the final ensemble prediction ( Rahaman & Thiery , 2020 ; Wen et al. , 2020 ) . In this paper , we examine the underlying theory and empirical results relating to calibration with ensemble methods . We found , both theoretically and empirically , that ensembling multiple calibrated models decreases the confidence of final prediction , resulting in an ill-calibrated ensemble prediction . To address this , strategies to calibrate the final ensemble prediction , rather than individual members , are required . Additionally we empiricaly examine the situation where the ensemble is comprised of models with different topologies , and resulting complexity/performance , requiring non-uniform ensemble averaging . In this study , we focus on post-hoc calibration of ensemble , based on temperature annealing . Guo et al . ( 2017 ) conducted a thorough comparison of various existing post-hoc calibration methods and found that temperature scaling was a simple , fast , and often highly effective approach to calibration . However , standard temperature scaling acts globally for all regions of the input samples , i.e . all logits are scaled towards one single direction , either increasing or decreasing the distribution entropy . To address this constraint , that may hurt some legitimately confident predictions , we investigate the effect of region-specific temperatures . Empirical results demonstrate the effectiveness of this approach , with minimal increase in the number of calibration parameters . 2 RELATED WORK . Calibration is inherently related to uncertainty modeling . Two of the most important scopes of calibration are calibration evaluation and calibration system construction . One method to assessing calibration is the reliability diagram ( Vaicenavicius et al. , 2019 ; Bröcker , 2012 ) . Though informative , It is still desirable to have an overall metric . Widmann et al . ( 2019 ) investigate different distances in the probability simplex for estimating calibration error . Nixon et al . ( 2019 ) point out the problem of fixed spaced binning scheme , bins with few predictions may have low-bias but high-variance measurement . Calibration error measure adaptive to dense populated regions have also been proposed ( Nixon et al. , 2019 ) . Vaicenavicius et al . ( 2019 ) treated the calibration evaluation as hypotheses tests . All these approaches examine calibration criteria from a sample-based perspective , rather than as a function of the underlying data distribution which is used in the thoretical analysis in this work . There are two main approaches to calibrating systems . The first is to recalibrate the uncalibrated systems with post-hoc calibration mapping , e.g . Platt scaling ( Platt et al. , 1999 ) , isotonic regression ( Zadrozny & Elkan , 2002 ) , Dirichlet calibration ( Kull et al. , 2017 ; 2019 ) . The second is to directly build calibrated systems , via : ( i ) improving model structures , e.g . deep convolutional Gaussian processes ( Tran et al. , 2019 ) ; ( ii ) data augmentation , e.g . adversarial samples ( Hendrycks & Dietterich , 2019 ; Stutz et al. , 2020 ) or Mixup ( Zhang et al. , 2018 ) ; ( iii ) minimize calibration error during training ( Kumar et al. , 2018 ) . Calibration based on histogram binning ( Zadrozny & Elkan , 2001 ) , Bayesian binning ( Naeini et al. , 2015 ) and scaling binning ( Kumar et al. , 2019 ) are related to our proposed dynamic temperature scaling , in the sense that the samples are divided into regions and separate calibration mapping are applied . However , our method can preserve the property that all predictions belonging to one sample sum to 1 . The region-based classifier by Kuleshov & Liang ( 2015 ) is also related to our approach . Ensemble diversity has been proposed for improved calibration ( Raftery et al. , 2005 ; Stickland & Murray , 2020 ) . In Zhong & Kwok ( 2013 ) , ensembles of SVM , logistic regressor , boosted decision trees are investigated , where the combination weights of calibrated probabilities is based on AUC of ROC . However , AUC is not comparable between different models as discussed in Ashukha et al . ( 2020 ) . In this work we investigate the combination of different deep neural network structures . The weights assigned to the probabilities is optimised using a likelihood-based metric . 3 CALIBRATION FRAMEWORK . Let X ⊆ Rd be the d-dimensional input space and Y = { ω1 , ... , ωK } be the discrete output space consisting of K classes . The true underlying joint distribution for the data is p ( x , ω ) = P ( ω|x ) p ( x ) , x ∈ X , ω ∈ Y . Given some training data D ∼ p ( x , ω ) , a model θ is trained to predict the distribution P ( ω|x ; θ ) given observation features . For a calibrated system the average predicted posterior probability should equate to the average posterior of the underlying distribution for a specific probability region . Two extreme cases will always yield perfect calibration . First when the predictions that are the same , and equal to the class prior for all inputs , P ( ωj |x ; θ ) = P ( ωj ) . Sec- ond the minimum Bayes ’ risk classifier is obtained , P ( ωj |x ; θ ) = p ( x , ωj ) ∑K k=1 p ( x , ωk ) . Note that perfect calibration doesn ’ t imply high accuracy , as shown by the system predicting the prior distribution . 3.1 DISTRIBUTION CALIBRATION . A system is calibrated if the predictive probability values can accurately indicate the portion of correct predictions . Perfect calibration for a system that yields P ( ω|x ; θ ) when the training and test data are obtained form the joint distribution p ( x , ω ) can be defined as : ∫ x∈Rpj ( θ , ) P ( ωj |x ; θ ) p ( x ) dx = ∫ x∈Rpj ( θ , ) P ( ωj |x ) p ( x ) dx ∀p , ωj , → 0 ( 2 ) Rpj ( θ , ) = { x ∣∣∣|P ( ωj |x ; θ ) − p| ≤ , x ∈ X } ( 3 ) Rpj ( θ , ) denotes the region of input space where the system predictive probability for class ωj is sufficiently close , within error of , to the probability p. A perfectly calibrated system will satisfy this expression for all regions , the expected predictive probability ( left side of Eq . ( 2 ) ) is identical to the expected correctness , i.e. , expected true probability ( right side of Eq . ( 2 ) ) . Rpj ( θ , ) defines the region in which calibration is defined . For top-label calibration , only the most probable class is considered and the region defined in Eq . ( 3 ) is modified to reflect this : R̃pj ( θ , ) = R p j ( θ , ) ∩ { x ∣∣∣ωj = argmax ω P ( ω|x ; θ ) , x ∈ X } ( 4 ) Eq . ( 4 ) is a strict subset of Eq . ( 3 ) . As the two calibration regions are different between calibration and top-label calibration , perfect calibration doesn ’ t imply top-label calibration , and vise versa . A simple illustrative example of this property is given in A.3 . Binary classification , K = 2 , is an exception to this general rule , as the regions for top-label calibration are equivalent to those for perfect calibration , i.e . R̃pj ( θ , ) = R p j ( θ , ) . Hence , perfect calibration is equivalent to top-label calibration for binary classification ( Nguyen & O ’ Connor , 2015 ) . Eq . ( 2 ) defines the requirements for a perfectly calibrated system . It is useful to define metrics that allow how close a system is to perfect calibration to be assessed . Let the region calibration error be : Cpj ( θ , ) = ∫ x∈Rpj ( θ , ) ( P ( ωj |x ; θ ) − P ( ωj |x ) ) p ( x ) dx ( 5 ) This then allows two forms of expected calibration losses to be defined ACE ( θ ) = 1 K ∫ 1 0 ∣∣∣∣∣ K∑ j=1 Cpj ( θ , ) ∣∣∣∣∣dp ; ACCE ( θ ) = 1K K∑ j=1 ∫ 1 0 ∣∣∣Cpj ( θ , ) ∣∣∣dp ( 6 ) All Calibration Error ( ACE ) only considers the expected calibration error for a particular probability , irrespective of the class associated with the data1 ( Hendrycks et al. , 2019 ) . Hence , All Class Calibration Error ( ACCE ) that requires that all classes minimises the calibration error for all probabilities is advocated by Kull et al . ( 2019 ) ; Kumar et al . ( 2019 ) . Nixon et al . ( 2019 ) propose the Thresholded Adaptive Calibration Error ( TACE ) to consider only the prediction larger than a threshold , and it can be described as a special case of ACCE by replacing the integral range . Naeini et al . ( 2015 ) also propose to only consider the region with maximum error . Though measures such as ACE and ACCE require consistency of the expected posteriors with the true distribution , for tasks with multiple classes , particularly large numbers of classes , the same weight is given to the ability of the model to assign low probabilities to highly unlikely classes , and high probabilities to the “ correct ” class . For systems with large numbers of classes this can yield artificially low scores . To address this problem it is more common to replace the regions in Eq . ( 5 ) with the top-label regions in Eq . ( 4 ) , to give a top-label calibration error C̃pj ( θ , ) . This then yields 1In this section the references given refer to the sample-based equivalent versions of the distributional calibration expressions in this paper using the same concepts , rather than identical expressions . the expected top-label equivalents of ACCE and ACE , Expected Class Calibration Error ( ECCE ) and Expected Calibration Error ( ECE ) . Here for example ECE by Guo et al . ( 2017 ) is expressed as ECE ( θ ) = ∫ 1 0 ∣∣∣∣∣ K∑ j=1 ∫ x∈R̃pj ( θ , ) ( P ( ωj |x ; θ ) − P ( ωj |x ) ) p ( x ) dx ∣∣∣∣∣dp ( 7 ) = ∫ 1 0 O ( θ , p ) |Conf ( θ , p ) − Acc ( θ , p ) |dp ( 8 ) whereO ( θ , p ) = ∑K j=1 ∫ x∈R̃pj ( θ , ) p ( x ) dx is the fraction observations that are assigned to that particular probability and Conf ( θ , p ) and Acc ( θ , p ) are the ideal distribution accuracy and confidences from the model for that probability . For more details see the appendix .

Summary: The paper makes an analysis of calibration in ensembles of deep learning models. Through some theoretical developments, the paper supports that a given ensemble cannot be more confident than the average individual members for regions where the ensemble is well calibrated. Empirical results, on CIFAR-100 and three different deep models, report a comparison of ensemble calibration, where calibration is done over all members in order to achieved a calibrated ensemble decision, over individual calibration of members with no feedback from the ensemble decisions. Results show that individual member calibration does not lead to calibrated ensembles, and as such calibrating directly on the ensemble output is required for obtained a proper evaluation of its uncertainty. Different ensemble calibration approaches are also compared.