Calibrated ensembles - a simple way to mitigate ID-OOD accuracy tradeoffs

1 INTRODUCTION . Machine learning models typically suffer large drops in accuracy in the presence of distribution shift where the test distribution is different from the training distribution . As ML systems are widely deployed , it is important to train models that achieve good “ out-of-distribution ” ( OOD ) accuracy . For example , models trained on medical data from a few hospitals should work well when deployed broadly ( Zech et al. , 2018 ; AlBadawy et al. , 2018 ) . Similarly , when predicting poverty from satellite imagery , models trained on data from a few countries should work well on all countries , particularly those where labels are scarce due to resource constraints ( Jean et al. , 2016 ) . There has been a lot of research interest in tackling this robustness problem under various settings such as robustness to spurious correlations ( Heinze-Deml & Meinshausen , 2017 ; Sagawa et al. , 2020a ; Chen et al. , 2020b ; Liu et al. , 2021 ) , robustness to adversarial perturbations ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) , domain generalization ( Arjovsky et al. , 2019 ; Sun & Saenko , 2016 ) , robustness to demographic shifts ( Hashimoto et al. , 2018 ; Duchi et al. , 2019 ) among others . Almost universally across these different settings , an unfortunate tradeoff arises . Robustness interventions typically improve the OOD accuracy but simultaneously cause a drop in the “ in-distribution ” ( ID ) accuracy on new test points from the original distribution . This tradeoff is a major hurdle in using the multitude of proposed robustness interventions . In practice , most inputs are likely to be ID , so it is unsatisfactory to use a “ robust ” model that has high OOD performance but performs less accurately on these majority ID points . On the other hand , “ standard models ” ( trained without robustness interventions ) fail catastrophically in the presence of even small shifts , and it can be highly dangerous to use a standard model even if OOD points are rare . In this work , we ask is there a general strategy by which we can achieve high accuracy both in-distribution and out-of-distribution and mitigate tradeoffs arising in robustness ? We consider four benchmark datasets ( DomainNet , CIFAR → STL , ImageNet → ImageNet-R , and BREEDS-Entity-30 ) and two real world satellite remote sensing datasets ( Landcover and Cropland ) , that have been used in prior work on robustness . Our work spans different types of robustness interventions ( projecting out spurious correlations , zero-shot language prompting , freezing pretrained features ) , data modalities ( image and time series data ) , and model architectures ( vision transformers , deep convolutional networks , time series convolution ) . Averaged across these datasets , robustness interventions increase OOD accuracy from 65 % to 77 % , but decrease ID accuracy from 88 % to 85 % . We first explore the natural strategy of ensembling the standard and robust models to combine their strengths . Concretely , we add the probabilities of each model to obtain a prediction with the hope that when the two models conflict , the more confident model ( with larger probability ) dictates the final prediction . We find that this surprisingly simple baseline already perfoms quite well—on average across all our datasets , this closes 90 % of the gap between the OOD of standard models , while outperforming both models ID . In other words , this simple baseline improves the OOD accuracy over standard models without hurting ID accuracy unlike previous robustness interventions . However , vanilla ensembling still leaves a gap as it underperforms the robust model OOD . We find that simply calibrating both models ID ( adjusting their predicted confidence to match their accuracy , on in-distribution data ) before ensembling them closes this gap . Calibrated ensembles get an average accuracy of 89.3 % ID and 77.9 % OOD , and outperform both the standard and robust model , ID and OOD . The other method in the literature to alleviate robustness induced tradeoffs is self-training that uses large amount of unlabeled data ( Raghunathan et al. , 2020 ; Xie et al. , 2021 ; Khani & Liang , 2021 ) . On the two remote-sensing datasets with additional unlabeled data , we find that calibrated ensembles match self-training on these datasets without requiring any unlabeled data . This shows that calibrated ensembles , though conceptually simple , can be highly effective in mitigating tradeoffs . As a sanity check , we find that the method also works when there is no tradeoff : even when the standard model ( or robust model ) dominates the robust model ( or standard model ) both ID and OOD , the calibrated ensemble accuracy matches up with the better model in both domains . While our method is intuitive in a way , it is also intriguing that it works so well because ensembling seems to rely on good uncertainty estimates while it is common wisdom that uncertainty estimates of deep networks are unreliable out-of-distribution ( Ovadia et al. , 2019 ) . Furthermore , it has been shown that calibration in-domain does not fix the issue of poor uncertainty estimates out-ofdistribution . Indeed , on the six datasets we test on , the models fare poorly on standard uncertainty metrics OOD , even after calibrating ID . The expected calibration error ( which roughly measures the difference between the model ’ s confidence and accuracy ) of the standard model across all datasets is 11 % . A partial answer might be that even if the models have high calibration error on average , what determines the quality of ensembling is the relative confidence between the standard and robust model on a particular data point . For example , on the remote sensing dataset ( Landcover ) , the standard model is on average 6 % more confident in its OOD predictions than the robust model , even though the standard model is less accurate OOD—which seems bad . But at the granularity of individual points , we find the more confident model is more likely to be accurate , enabling calibrated ensembles to achieve higher OOD accuracy than both standard and robust models . While most prior work on confidence estimates has focused on a single model at a population level , our results suggest that examining individual data points and relative confidence of different models is an exciting line of future work . Comparison with regular ensembling . Ensembling is an extremely common trick in ML to boost performance . However , when using ensembles , it is typical to have each ensemble member obtained by running the same training process with a different random seed which controls the stochastic aspects of the training algorithm like initialization , ordering of the training samples in a batch , which features are selected when using drop out and so on . The idea behind ensembling is that with different random seeds , the algorithm might converge to different solutions with decorrelated errors . In this work , we ensemble a standard and robust model which are obtain by minimizing entirely different training losses . This small but subtle change is very crucial . Experimentally , we find that regular ensembling of two standard models does poorly OOD ( 11 % worse than a single robust model ) , and ensembling two robust models does poorly ID ( 3 % worse than a single standard model ) . For these regular ensembles , calibration has no effect—the ID and OOD accuracies stay the same . Why is our proposed ensembling so much better than regular ensembling ? Consider the case of spurious correlations as studied in ( Sagawa et al. , 2020b ) where standard classifiers use spurious features irrespective of their random initialization . In other words , all members of the regular ensemble would have the same failure mode . The intuition of decorrelated error does not pan out with regular ensembles when standard models make systematic errors , as is common in most distribution shift settings . On the other hand , in our ensemble , we have a standard model that uses spurious features to be more accurate but also a robust model trained specifically not to use spurious features . By combining these “ diverse ” models with qualitatively different failure modes , we are able to achieve the best of both ID and OOD performance . In summary , we show that a simple method of calibrating and then ensembling can combine the best of standard and robust models , getting high ID and OOD accuracy—and calibration plays a key role in this method . This is a general practical method to alleviate tradeoffs which can likely be combined with other innovations to completely eliminate robustness induced tradeoffs in practice . This work also raises several interesting conceptual questions regarding ensembling and uncertainty estimation . 2 SETUP . Consider a K-class classification task , where the goal is to predict targets y ∈ [ K ] from inputs x ∈ Rd . Models : A model f : Rd → Rk takes an input x ∈ Rd and outputs f ( x ) ∈ Rk where f ( x ) i denotes the model ’ s confidence that the output is y = i . The model predicts the label ŷ = argmaxk f ( x ) k. The confidences can be converted into probabilities using a softmax function : softmax ( f ( x ) ) i = exp ( f ( x ) i ) ∑k j=1 exp ( f ( x ) j ) softmax : Rk → Rk ( 2.1 ) Training data : Let Pid and Pood denote the underlying distribution of ( x , y ) pairs in-distribution and out-of-distribution , respectively . We have a validation set { ( xvali , yvali ) } nval i=1 ∼ Pid used for early stopping and calibration , a held-out in-distribution test set { ( xtesti , ytesti ) } ntest i=1 ∼ Pid , and a held-out out-of-distribution test set { ( xoodi , yoodi ) } nood i=1 ∼ Pood . Evaluation : All methods can use the ID validation set for tuning hyperparameters , early stopping , and calibration . The ID and OOD test set are only used for evaluation . Given a model f , the ID and OOD accuracies are the average accuracies on these test sets : ID-Acc ( f ) = 1 ntest ntest∑ i=1 I ( ytesti = argmax k f ( xtesti ) k ) ( 2.2 ) OOD-Acc ( f ) = 1 nood nood∑ i=1 I ( yoodi = argmax k f ( xoodi ) k ) ( 2.3 ) 3 METHODS . Calibrated ensembles . Given a standard models fstd and robust model frob , calibrated ensembles first calibrate each model using temperature scaling ( Guo et al. , 2017 ) with the cross-entropy loss l on the in-distribution validation data : Tstd = argmin T 1 nval nval∑ i=1 l ( fstd ( xvali ) T , yvali ) ( 3.1 ) Trob = argmin T 1 nval nval∑ i=1 l ( frob ( xvali ) T , yvali ) ( 3.2 ) The goal of temperature scaling is to adjust each model ’ s confidence on the in-distribution validation data . We then ensemble the two models by adding up the probabilities that they predict ( Lakshminarayanan et al. , 2017 ) . So given an example x , we simply output the average p̂ of the probabilities predicted by the standard and robust models after scaling by Tstd and Trob respectively . p̂ = 1 2 ( softmax ( fstd ( x ) Tstd ) + softmax ( frob ( x ) Trob ) ) ( 3.3 ) Other ensembles . As baselines , we consider a number of additional ensembles . Vanilla Ensembling : Here we simply return the average of the standard and robust model ’ s probabilities , without temperature scaling . p̂ = 1 2 ( softmax ( fstd ( x ) ) + softmax ( frob ( x ) ) ) ( 3.4 ) Tuned Ensembling : As a potentially stronger baseline , we consider outputting a weighted average of the standard and robust model ’ s probabilities , where the weight α ∈ [ 0 , 1 ] is tuned to maximize accuracy on the in-distribution set . p̂ = αsoftmax ( fstd ( x ) ) + ( 1− α ) softmax ( frob ( x ) ) ) ( 3.5 ) Alternative combination methods : We could also combine the logits of the models before taking the softmax , or we could simply output the prediction of the more confident model . We chose to add up the probabilities since that is commonly done by prior work on ensembling ( Lakshminarayanan et al. , 2017 ) and we found it to work slightly better than these alternatives . Other models : Each of these ensembling methods takes two models and combines them . As such , we can apply each of these ensembling methods to two standard models or two robust models as well .

Summary: This paper discusses the trade-off between in-distribution and out-of-distribution accuracy, where ERM-based learners have good ID but poor OOD performance, and robust learners the opposite. The authors demonstrate that by ensembling both types of learners, and calibrating their ID accuracy, they can outperform the standard and robust models both on ID and OOD data.  The main contribution of the paper is the strong empirical results provided by the simple yet effective approach.