Calibrated ensembles - a simple way to mitigate ID-OOD accuracy tradeoffs

1 INTRODUCTION . Machine learning models typically suffer large drops in accuracy in the presence of distribution shift where the test distribution is different from the training distribution . As ML systems are widely deployed , it is important to train models that achieve good “ out-of-distribution ” ( OOD ) accuracy . For example , models trained on medical data from a few hospitals should work well when deployed broadly ( Zech et al. , 2018 ; AlBadawy et al. , 2018 ) . Similarly , when predicting poverty from satellite imagery , models trained on data from a few countries should work well on all countries , particularly those where labels are scarce due to resource constraints ( Jean et al. , 2016 ) . There has been a lot of research interest in tackling this robustness problem under various settings such as robustness to spurious correlations ( Heinze-Deml & Meinshausen , 2017 ; Sagawa et al. , 2020a ; Chen et al. , 2020b ; Liu et al. , 2021 ) , robustness to adversarial perturbations ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) , domain generalization ( Arjovsky et al. , 2019 ; Sun & Saenko , 2016 ) , robustness to demographic shifts ( Hashimoto et al. , 2018 ; Duchi et al. , 2019 ) among others . Almost universally across these different settings , an unfortunate tradeoff arises . Robustness interventions typically improve the OOD accuracy but simultaneously cause a drop in the “ in-distribution ” ( ID ) accuracy on new test points from the original distribution . This tradeoff is a major hurdle in using the multitude of proposed robustness interventions . In practice , most inputs are likely to be ID , so it is unsatisfactory to use a “ robust ” model that has high OOD performance but performs less accurately on these majority ID points . On the other hand , “ standard models ” ( trained without robustness interventions ) fail catastrophically in the presence of even small shifts , and it can be highly dangerous to use a standard model even if OOD points are rare . In this work , we ask is there a general strategy by which we can achieve high accuracy both in-distribution and out-of-distribution and mitigate tradeoffs arising in robustness ? We consider four benchmark datasets ( DomainNet , CIFAR → STL , ImageNet → ImageNet-R , and BREEDS-Entity-30 ) and two real world satellite remote sensing datasets ( Landcover and Cropland ) , that have been used in prior work on robustness . Our work spans different types of robustness interventions ( projecting out spurious correlations , zero-shot language prompting , freezing pretrained features ) , data modalities ( image and time series data ) , and model architectures ( vision transformers , deep convolutional networks , time series convolution ) . Averaged across these datasets , robustness interventions increase OOD accuracy from 65 % to 77 % , but decrease ID accuracy from 88 % to 85 % . We first explore the natural strategy of ensembling the standard and robust models to combine their strengths . Concretely , we add the probabilities of each model to obtain a prediction with the hope that when the two models conflict , the more confident model ( with larger probability ) dictates the final prediction . We find that this surprisingly simple baseline already perfoms quite well—on average across all our datasets , this closes 90 % of the gap between the OOD of standard models , while outperforming both models ID . In other words , this simple baseline improves the OOD accuracy over standard models without hurting ID accuracy unlike previous robustness interventions . However , vanilla ensembling still leaves a gap as it underperforms the robust model OOD . We find that simply calibrating both models ID ( adjusting their predicted confidence to match their accuracy , on in-distribution data ) before ensembling them closes this gap . Calibrated ensembles get an average accuracy of 89.3 % ID and 77.9 % OOD , and outperform both the standard and robust model , ID and OOD . The other method in the literature to alleviate robustness induced tradeoffs is self-training that uses large amount of unlabeled data ( Raghunathan et al. , 2020 ; Xie et al. , 2021 ; Khani & Liang , 2021 ) . On the two remote-sensing datasets with additional unlabeled data , we find that calibrated ensembles match self-training on these datasets without requiring any unlabeled data . This shows that calibrated ensembles , though conceptually simple , can be highly effective in mitigating tradeoffs . As a sanity check , we find that the method also works when there is no tradeoff : even when the standard model ( or robust model ) dominates the robust model ( or standard model ) both ID and OOD , the calibrated ensemble accuracy matches up with the better model in both domains . While our method is intuitive in a way , it is also intriguing that it works so well because ensembling seems to rely on good uncertainty estimates while it is common wisdom that uncertainty estimates of deep networks are unreliable out-of-distribution ( Ovadia et al. , 2019 ) . Furthermore , it has been shown that calibration in-domain does not fix the issue of poor uncertainty estimates out-ofdistribution . Indeed , on the six datasets we test on , the models fare poorly on standard uncertainty metrics OOD , even after calibrating ID . The expected calibration error ( which roughly measures the difference between the model ’ s confidence and accuracy ) of the standard model across all datasets is 11 % . A partial answer might be that even if the models have high calibration error on average , what determines the quality of ensembling is the relative confidence between the standard and robust model on a particular data point . For example , on the remote sensing dataset ( Landcover ) , the standard model is on average 6 % more confident in its OOD predictions than the robust model , even though the standard model is less accurate OOD—which seems bad . But at the granularity of individual points , we find the more confident model is more likely to be accurate , enabling calibrated ensembles to achieve higher OOD accuracy than both standard and robust models . While most prior work on confidence estimates has focused on a single model at a population level , our results suggest that examining individual data points and relative confidence of different models is an exciting line of future work . Comparison with regular ensembling . Ensembling is an extremely common trick in ML to boost performance . However , when using ensembles , it is typical to have each ensemble member obtained by running the same training process with a different random seed which controls the stochastic aspects of the training algorithm like initialization , ordering of the training samples in a batch , which features are selected when using drop out and so on . The idea behind ensembling is that with different random seeds , the algorithm might converge to different solutions with decorrelated errors . In this work , we ensemble a standard and robust model which are obtain by minimizing entirely different training losses . This small but subtle change is very crucial . Experimentally , we find that regular ensembling of two standard models does poorly OOD ( 11 % worse than a single robust model ) , and ensembling two robust models does poorly ID ( 3 % worse than a single standard model ) . For these regular ensembles , calibration has no effect—the ID and OOD accuracies stay the same . Why is our proposed ensembling so much better than regular ensembling ? Consider the case of spurious correlations as studied in ( Sagawa et al. , 2020b ) where standard classifiers use spurious features irrespective of their random initialization . In other words , all members of the regular ensemble would have the same failure mode . The intuition of decorrelated error does not pan out with regular ensembles when standard models make systematic errors , as is common in most distribution shift settings . On the other hand , in our ensemble , we have a standard model that uses spurious features to be more accurate but also a robust model trained specifically not to use spurious features . By combining these “ diverse ” models with qualitatively different failure modes , we are able to achieve the best of both ID and OOD performance . In summary , we show that a simple method of calibrating and then ensembling can combine the best of standard and robust models , getting high ID and OOD accuracy—and calibration plays a key role in this method . This is a general practical method to alleviate tradeoffs which can likely be combined with other innovations to completely eliminate robustness induced tradeoffs in practice . This work also raises several interesting conceptual questions regarding ensembling and uncertainty estimation . 2 SETUP . Consider a K-class classification task , where the goal is to predict targets y ∈ [ K ] from inputs x ∈ Rd . Models : A model f : Rd → Rk takes an input x ∈ Rd and outputs f ( x ) ∈ Rk where f ( x ) i denotes the model ’ s confidence that the output is y = i . The model predicts the label ŷ = argmaxk f ( x ) k. The confidences can be converted into probabilities using a softmax function : softmax ( f ( x ) ) i = exp ( f ( x ) i ) ∑k j=1 exp ( f ( x ) j ) softmax : Rk → Rk ( 2.1 ) Training data : Let Pid and Pood denote the underlying distribution of ( x , y ) pairs in-distribution and out-of-distribution , respectively . We have a validation set { ( xvali , yvali ) } nval i=1 ∼ Pid used for early stopping and calibration , a held-out in-distribution test set { ( xtesti , ytesti ) } ntest i=1 ∼ Pid , and a held-out out-of-distribution test set { ( xoodi , yoodi ) } nood i=1 ∼ Pood . Evaluation : All methods can use the ID validation set for tuning hyperparameters , early stopping , and calibration . The ID and OOD test set are only used for evaluation . Given a model f , the ID and OOD accuracies are the average accuracies on these test sets : ID-Acc ( f ) = 1 ntest ntest∑ i=1 I ( ytesti = argmax k f ( xtesti ) k ) ( 2.2 ) OOD-Acc ( f ) = 1 nood nood∑ i=1 I ( yoodi = argmax k f ( xoodi ) k ) ( 2.3 ) 3 METHODS . Calibrated ensembles . Given a standard models fstd and robust model frob , calibrated ensembles first calibrate each model using temperature scaling ( Guo et al. , 2017 ) with the cross-entropy loss l on the in-distribution validation data : Tstd = argmin T 1 nval nval∑ i=1 l ( fstd ( xvali ) T , yvali ) ( 3.1 ) Trob = argmin T 1 nval nval∑ i=1 l ( frob ( xvali ) T , yvali ) ( 3.2 ) The goal of temperature scaling is to adjust each model ’ s confidence on the in-distribution validation data . We then ensemble the two models by adding up the probabilities that they predict ( Lakshminarayanan et al. , 2017 ) . So given an example x , we simply output the average p̂ of the probabilities predicted by the standard and robust models after scaling by Tstd and Trob respectively . p̂ = 1 2 ( softmax ( fstd ( x ) Tstd ) + softmax ( frob ( x ) Trob ) ) ( 3.3 ) Other ensembles . As baselines , we consider a number of additional ensembles . Vanilla Ensembling : Here we simply return the average of the standard and robust model ’ s probabilities , without temperature scaling . p̂ = 1 2 ( softmax ( fstd ( x ) ) + softmax ( frob ( x ) ) ) ( 3.4 ) Tuned Ensembling : As a potentially stronger baseline , we consider outputting a weighted average of the standard and robust model ’ s probabilities , where the weight α ∈ [ 0 , 1 ] is tuned to maximize accuracy on the in-distribution set . p̂ = αsoftmax ( fstd ( x ) ) + ( 1− α ) softmax ( frob ( x ) ) ) ( 3.5 ) Alternative combination methods : We could also combine the logits of the models before taking the softmax , or we could simply output the prediction of the more confident model . We chose to add up the probabilities since that is commonly done by prior work on ensembling ( Lakshminarayanan et al. , 2017 ) and we found it to work slightly better than these alternatives . Other models : Each of these ensembling methods takes two models and combines them . As such , we can apply each of these ensembling methods to two standard models or two robust models as well .

Summary: UPDATE:  I acknowledge that I've read the author responses as well as the other reviews.  I now understand the empirical merit of the paper. Furthermore, the authors clarified the baselines' performance and add further analysis. Therefore, I raise my score to 5 weak reject. However, I'm still not convinced about the novelty of the paper. In ensemble learning, diversity between ensembles is a major factor of success. Therefore, while it is straightforward that the proposed method outperforms robust+robust and standard+standard, temperature scaling is not a new contribution and I'm still skeptical about how 'surprising' the results are. I hope the authors can clarify further issues about the point.  ================================================================================================  The paper studies ensemble methods to achieve improved in-distribution (ID) and out-of-distribution accuracy. The paper proposes a simple fix on concatenating the robust model and the standard model via adopting the idea of classifier-specific temperature parameters, which is optimized on the in-distribution validation dataset. Authors test their methods on various distribution shifts. The proposed method seems to improve the ID-OOD accuracy tradeoff.