Calibrated ensembles - a simple way to mitigate ID-OOD accuracy tradeoffs
Authors: Ananya Kumar
Aditi Raghunathan
Tengyu Ma
Percy Liang
Venue: ICLR 2022 Submitted (2022)
Keywords: distribution shift
calibration
ensembles

Abstract: We often see undesirable tradeoffs in robust machine learning where out-of-distribution (OOD) accuracy is at odds with in-distribution (ID) accuracy. A ‘robust’ classifier obtained via specialized techniques like removing spurious features has better OOD but worse ID accuracy compared to a ‘standard’ classifier trained via vanilla ERM. On six distribution shift datasets, we find that simply ensembling the standard and robust models is a strong baseline---we match the ID accuracy of a standard model with only a small drop in OOD accuracy compared to the robust model. However, calibrating these models in-domain surprisingly improves the OOD accuracy of the ensemble and completely eliminates the tradeoff and we achieve the best of both ID and OOD accuracy over the original models.

Under review as a conference paper at ICLR 2022
CALIBRATED ENSEMBLES A SIMPLE WAY TO MITI- GATE ID-OOD ACCURACY TRADEOFFS
Anonymous authors Paper under double-blind review
ABSTRACT
We often see undesirable tradeoffs in robust machine learning where out-of- distribution (OOD) accuracy is at odds with in-distribution (ID) accuracy. "ro- bust" classifier obtained via specialized techniques like removing spurious fea- tures has better OOD but worse ID accuracy compared to a "standard" classifier trained via vanilla ERM. On six distribution shift datasets, we find that simply ensembling standard and a robust model is a strong baseline-wy we match the ID accuracy of standard model with only small drop in OOD accuracy compared to the robust model. However, calibrating these models in-domain surprisingly improves the OOD accuracy of the ensemble and eliminates the tradeoff and we achieve the best of both ID and OOD accuracy over the original models.
INTRODUCTION
Machine learning models typically suffer large drops in accuracy in the presence of distribution shift where the test distribution is different from the training distribution. As systems are widely deployed, it is important to train models that achieve good fout-of-distribution" (OOD) accuracy. For example, models trained on medical data from few hospitals should work well when deployed broadly (Zech et al., 2018 AlBadawy et al. 2018). Similarly, when predicting poverty from satellite imagery, models trained on data from a few countries should work well on all countries, particularly those where labels are scarce due to resource constraints (Jean et al., 2016). There has been a lot of research interest in tackling this robustness problem under various settings such as robustness to spurious correlations (Heinze-Deml & Meinshausen [017 Sagawa et al., 2020a Chen et al.) 20206. Liu et al. 202T), robustness to adversarial perturbations (Goodfellow et al. 2015: Madry al.) 2018), domain generalization (Arjovsky et al., 22099; Sun & Saenko (0166), robustness to demographic shifts (Hashimoto et al.) 2018; Duchi et al., 2019) among others. Almost universally across these different settings, an unfortunate tradeoff arises. Robustness interventions typically improve the OOD accuracy but simultaneously cause drop in the "in-distribution" (ID) accuracy on new test points from the original distribution.
This tradeoff is a major hurdle in using the multitude of proposed robustness interventions. In practice, most inputs are likely to be ID, is unsatisfactory to a "robust" model that has high OOD performance but performs less accurately on these majority ID points. On the other hand, "standard models" (trained without robustness interventions) fail catastrophically in the presence of even small shifts, and it can be highly dangerous to use a standard model even if OOD points are rare. In this work, we ask is there general strategy by which we can achieve high accuracy both in in-distribution and out-of- distriibbtttoo and mitigate tradeoffs arising in robustness?
We consider four benchmark datasets (DomainNet, CIFAR ++ STL, ImageNet - ImageNet-R, R, and BREEDS- FEEEDDS-Ettit-300 and two real world satellite remote sensing datasets (Landcover and Cropland), that have been used in prior work on robustness. Our work spans different types of robustness in- terventions (projecting out spurious correlations, zero-shot language prompting. freezing pretrained features), data modalities (image and time series data), and model architectures (vision transformers, deep convolutional networks, time series convolution). Averaged across these datasets, robustness interventions increase OOD accuracy from 65% to 77%, but decrease ID accuracy from 88% to 85%.
We first explore the natural strategy of ensembling the standard and robust models to combine their strengths. Concretely, we add the probabilities of each model to obtain a prediction with the hope

---

Under review as a conference paper at ICLR 2022
accuracy
COD accuracy
(Calibrated] ensemble robust
Calibrated ensemble of standard and robust model
Ensemble (Standard Standard)
96.0%
OOD accuracy
Robust mode!
Ensemble standard and robust model
Robust Model
84.7%
77.2%
(Calibrated] ensembbee standard models
Ensemble (Robust Robust)
84.9%
77.4%
Standard model
Ensemble (Standard + Robust)
89.0%
76.4%
Calibrated ensemble (Standard Robust)
89.3%
77.9%
ID accuracy
Parformance different averaged over 8 real distr bution shifts
(a) Schematic performance of different methods
In many settings, have 'standard' model that performs better in-distribution, and 'robust' model that performs better out-of-distribution. Simply ensembling these two models (c.g., by adding their probabilities), gets better ID accuracy than the standard and robust models, and closes most the OOD gap. Calibrating the models in-distribution (no access to OOD data) before ensembling them leads to further improvements. Note that ensembling two standard two robust models does not close the gap and only leads to small improvements.
Figure 1:
that when the two models conflict, the more confident model (with larger probability) dictates the final prediction. We find that this surprisingly simple baseline already perfoms quite well-on av- erage across all datasets, this closes 90% of the gap between the OOD of standard models, while outperforming both models ID. In other words, this simple baseline improves the OOD ac- curacy over standard models without hurting ID accuracy unlike previous robustness interventions. However, vanilla ensembling still leaves a gap as it underperforms the robust model OOD.
We find that simply calibrating both models ID (adjusting their predicted confidence match their accuracy, on in-distribution data) before ensembling them closes this gap. Calibrated ensembles get an average accuracy of 89.3% ID and 77.9% OOD, and oulperform both the standard and robust model, ID and OOD. The other method in the literature to alleviate robustness induced tradeoffs is self-training that uses large amount of unlabeled data (Raghunathan al., 2020; Xie et al. [ Khani & Liang] (021). On the two remote-sensing datasets with additional unlabeled data, we find that calibrated ensembles match self- self-training on these datasets without requiring any unlabeled data. This shows that calibrated ensembles, though conceptually simple, can be highly effective a sanity check, we find that the method also works when there is no tradeoff: even when the standard model (or robust model) dominates the robust model (or standard model) both ID and OOD, the calibrated ensemble accuracy matches up with better model in both domains.
in mitigating tradeoffs.
While our method intuitive a way, it is also intriguing that works so well because ensem- bling seems to rely good uncertainty estimates while it is common wisdom that uncertainty esti- mates of deep networks are unreliable out-of-distributt been shown that calibration in-domain does not fix the issue of poor uncertainty estimates out-of- distribution. Indeed, on the six datasets we test on, the models fare poorly on standard uncertainty metrics OOD, even after calibrating ID. The expected calibration error (which roughly measures the difference between the model's 's confidence and accuracy) of the standard model across all datasets 11%. A partial answer might be that even if the models have high calibration error on average, what determines the quality of ensembling is the relative confidence between the standard and ro- bust model on particular data point. For example, on the remote sensing dataset (Landcover), the standard model is average 6% more confident in its OOD predictions than the robust model, even though the standard model is less accurate DOD-whtee vidual points, we find the more confident model is more likely to be accurate, enabling calibrated ensembles to achieve higher OOD accuracy than both standard and robust models. While most prior work confidence estimates has focused on single model at population level, our results suggest that examining individual data points and relative confidence different models an exciting line of future work.
(Ovadia et al. 2019). Furthermore, it has
seems bad. But at the granularity of indi-
2

---

Under review as a conference paper at ICLR 2022
Comparison with regular ensembling. Ensembling is extremely common trick in ML boost performance. However, when using ensembles, it is typical to have each ensemble member obtained by rumning the same training process with different random seed which controls the stochastic aspects the training algorithm like initialization, ordering of the training samples in a batch, which features are selected when using drop out and so on. The idea behind ensembling is that with different random seeds, the algorithm might converge to different solutions with decorrelated errors. In this work, we ensemble standard and robust model which are obtain by minimizing entirely different training losses. This small but subtle change is very crucial. Experimentally, we find that regular ensembling of two standard models does poorly OOD (11% worse than a single robust model), and ensembling two robust models does poorly ID (3% worse than single standard model). For these regular ensembles, calibration has no effect- the ID and OOD accuracies stay the same.
Why is our proposed ensembling so much better than regular ensembling? Consider the case of spurious correlations as studied in (Sagawa et al. 2020b) where standard classifiers use spurious features irrespective of their random initialization. In other words, all members of the regular en- semble would have the same failure mode. The intuition of decorrelated error does not pan out with regular ensembles when standard models make systematic errors, as is common most distribution shift settings. On the other hand, in our ensemble, have standard model that uses spurious fea- tures to be more accurate but also a robust model trained specifically not to use spurious features. By combining these "diverse" models with qualitatively different failure modes, we are able achieve the best of both ID and OOD performance.
In summary, we show that simple method of calibrating and then ensembling can combine the best of standard and robust models, getting high ID and OOD accuracy-an calibration plays key role in this method. This a general practical method alleviate tradeoffs which can likely be combined with other innovations to completely eliminate robustness induced tradeoffs in practice. This work also raises several interesting conceptual questions regarding ensembling and uncertainty estimation.
SETUP
Consider a K-class classification task, where the goal is to predict targets y [K] from inputs I E Rd
A model f d - R& takes an input Rd and outputs f(x) E R where f(ø), denotes the model's confidence that the output is = i. The model predicts the label ÿ = arg max; fx)) The confidences be converted into probabilities using a softmax function:
Models:
exp(I(x));
softmax (ottmax((())): Ej= exp(
softmax R*
R
(2.1)
Training data: Let Pia and Pood denote the underlying distribution of (xt, y) pairs in-distribution
and out-of-distribus respectively. We have a validation set {(x)aal y/ Jiml Pia used for early
stopping and calibration, a held-out in-distribution test set {a:jest y Jis1 ~ Pla. and a held-out out-of -fftsrr-bittitiionn test set {( yood) ) Pood-
Evaluation: All methods can use the ID validation for tuning hyperparameters, early stopping, and calibration. The ID and OOD test set are only used for evaluation. Given a model f. the ID and OOD accuracies are the average accuracies on these test sets:
1 Plast
ID-Acc(f)
I(y) test
arg max f( (estt )x)
(2.2)
Thtest
1=1
1
Mood
OOD-Ace(J) Mlood
112
arg max f(x
(2.3)
1=1
3

---

Under review as a conference paper at ICLR 2022
3 METHODS
Calibrated ensembles. Given standard models fstd and robust model frob, calibrated ensembles first calibrate each model using temperature scaling (Guo et al. 2017) with the cross-entropy loss on the in-distribution validation data:
Roal
fstd (a:xal)
Tad arg min
yia
(3.1)
Plyal
froblun
Tob arg min
(3.2)
Thyal
T
The goal of temperature scaling adjust each model's confidence on the in-distribution validation data. We then ensemble the two models by adding up the probabilities that they predict (Lakshmi- marayanan et al., 2017). So given an example æ, we simply output the average the probabilities predicted by the standard and robust models after scaling by Tstd and Tiob respectively.
p 1 softmax
[ Tstd
softmax (rob((2)
(3.3)
Trb
Other ensembles.
As baselines, consider number of additional ensembles.
Vanilla Ensembling: Here we simply return the average of the standard and robust model's proba- bilities, without temperature scaling.
1
p (sottax((fax( (z)) + softmax((feo()))
(3.4)
Tuned Ensembling: As a potentially stronger baseline, consider outputting a weighted average of the standard and robust model's probabilities, where the weight E [0, 1] is tuned to maximize accuracy on the in-distribution set.
p (softmax(fsa(x))) + (1 a00--aa60 )osottmax(fro (x)) (z)))
(3.5)
Alternative combination methods: We could also combine the logits of the models before taking the softmax, or could simply output the prediction of the more confident model. We chose add up the probabilities since that is commonly done by prior work on ensembling (Lakshminarayanan al. 2017) and we found it to work slightly better than these alternatives.
Other models: Each of these ensembling methods takes two models and combines them. As such, can apply each of these ensembling methods two standard models or two robust models as well.
DATASETS
We run experiments spanning three different types of robustness interventions: projecting out spuri- metadata, language prompting, and freezing pretrained features. These experiments span multi- ple data modalities and model architectures.
4.1 SPURIOUS METADATA
We run experiments two remote sensing datasets used in prior work studying ID-OOD trade- offs (Xie et al. 202I). These datasets consist of a core input æ (image data or time series data) and metadata (e.g., location, meteorological climate data). The metadata is spuriously correlated with the target-une using the metadata to predict labels improves accuracy in-distribution (ID), but hurts accuracy out-of-distribution Xie al. (2021) consider standard model that takes in both the inputs and metadata to predict the target, and a robust model that only takes the core inputs and does some additional pretraining. They call these the 'aux-in' and 'aux-out' models respectively.
4

---

Under review as a conference paper at ICLR 2022
Cropland. The goal is to predict whether satellite image is of cropland not. The core input an RGB satellite image, and the metadata & consists of location coordinates and vegetation bands. The original dataset is from Wang et al. (2020), and we use U-net model checkpoints from Xie et al. 2021
Landcover: The goal to predict the land type from satellite data a given location. Here, the core input z is a time series measured by NASA's MODIS satellite (Vermote) 2015). and z is climate data (e.g., lemperature) that location. The dataset from Gislason et al. (006); [Rußwurm et al. (020). We use model checkpoints from Xie et al. (202T) where they use ID) convolutions for time series data.
4.2 ZERO-SHOT LANGUAGE PROMPTING
Radford et al. (2021) (CLIP) pretrain a model on large multi-modal language and vision dataset. The model can then predict the label of an image by comparing the image embedding, with the language embedding for prompts such as 'photo of an apple' or "photo of banana'. They show that this zero-shot language prompting approach can be much more accurate out- nut-of-distribution than the traditional method of fine-tuning the entire model.
ImageNet ImageNet-R. We CLIP vision transformer, specifically ViT-B/16, which the best publicly available model. The robust model uses language prompts to make zero-shot predictions on ImageNet-Renditee
(Hendrycks al., 2020), a dataset containing cartoon, graffiti, video game, etc, renditions of ImageNet classes. The standard model initializes with weights from the CLIP model, and fine-tunes on ImageNet (Russakovsky et al., 2015) training data for 10 epochs with a batch size of 64, initial learning rate of 0.0001 with cosine learning rate decay, before making predictions on ImageNet-R. We note that the robust model gets 10% lower accuracy (on ImageNet validation examples), but gets 30% higher accuracy OOD (on ImageNet-R test examples)
4.3 FREEZING PRETRAINED FEATURES
When adapting a pretrained model to an ID dataset, typically all the model parameters are fine-tuned. Recent work looks 'lighteiihh' fine-tume mne-ttning where only parts of the model are adapted- this can often do better OOD even though the ID performance is worse (Li & Liang] 2021] Houlsby et al., 2019). We consider three distribution shift datasets where the standard model starts from a pretraimed initialization and fine-tunes all parameters an ID dataset, and the robust model only learns the top linear 'head' layer.
DomainNet. A standard domain adaptation dataset (Peng et al. 2019). Here, ID dataset tains 'sketch' images (e.g., drawings of apples, elephants, etc), and the OOD dataset contains 'real' photos of the same categories. We use the version of the dataset from Tan al. (2020). We start from a CLIP pretrained ResNet50 and either fine-tune for 50 epochs with batch size 64 and learning rate 0.001 with cosine learning rate decay (to get a standard model) train the head layer using sklearn logistic regression (to get robust model).
CIFAR-10 + STL. Another standard domain adaptation dataset (French et al. 2018). where the ID is CIFAR-IC 10 (Krizhevsky 2009), and the OOD is STL (Coates et al.] 2011)) We start from a ResNet50 pretrained on unlabeled ImageNet examples using MoCo-v2 (Chen et al. 2020a) and either fine-tune for 20 epochs with batch size of 64 and learning rate of 0.001 with cosine learning rate decay (to get a standard model) or train the head layer using sklearn logistic regression (to get robust model).
Living-17. Part of the BREEDS benchmark (Santurkar et al. 2020). here the goal is to classify an image as one of 17 animal categories such as 'bear"-the ID dataset contains images of black bears and sloth bears and the OOD dataset has images of brown bears and polar bears. We start from a ResNet50 pretrained on unlabeled ImageNet examples using MoCo-v2 (Chen et al. 2020a) and either fine-tur for 20 epochs with batch size 64 and learning rate of 0.001 with cosine learning rate decay (to get a standard model) or train the head layer using sklearn logistic regression (to get robust model).
5

---

Under review as a conference paper at ICLR 2022
Table 1: Out-of-distribution (OOD) accuracies for the standard model, robust model, and calibrated ensembles, across six datasets. Calibrated ensembling matches or outperforms the better model in 5/6 cases, and on average outperforms both the standard and robust models. For the remaining dataset, DomainNet, calibrated ensembles close 96% of the gap between the standard and robust model.
Ent30
DomNet
STL
Land
Crop
ImNet-R
Standard
60.7 (0.1)
55.3 (0.4)
82.4 (0.3)
55.7 (1.1)
85.6 (5.8)
49.6 (-)
Robust
63.2 (1.1)
87.2 (0.1)
85.1 (0.2)
60.4 (1.1)
89.8 (0.4)
77.5 (-)
Cal ensemble
64.7 (0.5)
86.1 (0.2)
87.3 (0.2)
60.8 (0.8)
91.3 (0.8)
77.1 (-)
In-distribution (ID) accuracies for the standard model, robust model, and calibrated ensembling, across six datasets. Calibrated ensembling matches outperforms the hetter model in 5/6 cases, and on average outperforms both the standard and robust models. For the remaining dataset, CIFAR-10, calibrated ensembles close 97% of the gap between the standard and robust model.
Table 2:
Ent30
DomNet
CIFARIO
Land
Crop
ImNet
Standard
93.6 (0.2)
83.9 (1.0)
97.4 (0.1)
76.9 (0.3)
95.3 (0.0)
80.5 (-)
Robust
90.7 (0.2)
89.2 (0.1)
92.0 (0.0)
72.7 (0.2)
95.1 (0.1)
68.4 (-)
Cal ensemble
93.7 (0.1)
91.2 (0.7)
97.2
77.2 (0.2)
95.6 (0.1)
81.1 (-)
5 EXPERIMENTS
5.1 MAIN RESULTS
Strong ID and OOD accuracy: Calibrating and then ensembling a standard and robust model. gets the best of both worlds, typically outperforming the standard and robust model both ID (Table 2 and OOD (Table Averaged across the datasets, calibrated ensembles get 89.3% ID (vs 87.9% the standard model and 84.7% for the robust model) and 77.9% OOD (vs 77.2% for the robust model and 64.9% for the standard model). The method works across the coarb-calibrated ensembles achieve the best performance on 5/6 ID datasets, and on 5/6 OOD datasets. For the remaining two datasets, DomainNet OOD and CIFAR- ID, calibrated ensembles close over 95% of the gap between the standard and robust model (96% for DomainNet, 97% for CIFAR-10).
Competitive with self- teki-rriinng: The remote sensing datasets have lots of unlabeled data so prior work uses self- self-training on these datasets to mitigate the ID-OOD accuracy tradeoff we take check- points from the official implementation in (Xie et al.) (02II) and compare calibrated ensembles with self-training. Table 3 shows that calibrated-ensembles match outperform self-training on both datasets, both ID and OOD. We believe this is an interesting result because calibrated ensembling is simple method and does not need additional unlabeled data.
5.2 ENSEMBLING ABLATIONS
We run ablations to show the importance of calibration, and of including both a robust and a standard model in the ensemble. We find that a strong baseline of tuning the ensemble weights on ID data has lower accuracy than calibrated ensembles OOD.
Ensembling two standard models or two robust models does not mitigate the ID-OOD accuracy tradeoff. As seen in Figure 1, ensembling two standard models gets average OOD accuracy of 66.0%, which is substantially worse than our method (77.9% or even a single robust model (77.2%). Calibrating the standard models does not change its accuracy. Similarly calibrating and ensembling two robust models gets average ID accuracy of 84.9% which is worse than our method (89.3%) 9%)(or or even a single standard model (87.9%) In summary, calibrated ensembles outperform standard and robust models, ID and OOD (Figure 1).
Standard ensembles do not mitigate tradeoffs:

---

Under review as a conference paper at ICLR 2022
Calibrated ensembles competitive with self-training (Xie et al. 2021 which requires unlabeled data.
Table 3:
ID and OOD,
Cropland
Landcover
Acc
OOD Acc
ID) Acc
OOD Acc
Standard model
95.3 (0.0)
85.6 (5.8)
76.9 (0.3)
55.7 (1.1)
Robust model
95.1 (0.1)
89.8 (0.4)
72.7 (0.2)
60.4 (1.1)
Self-training
95.3 (0.2)
90.6 (0.6)
77.0 (0.4)
61.0 (0.7)
Cal ensembling
95.6 (0.1)
91.3 (0.8)
77.2 (0.2)
60.8 (0.8)
Table 4: OOD accuracies: calibrated ensembles outperform vanilla ensembles and even tuned ensembles where the combination weights are tuned to maximize in-distribution accuracy. Averaged across the datasets, calibrated ensembles get an OOD accuracy of 74.6%, while tuned ensembles get accuracy of 71.3%. The in-distribution accuracies of the methods very close (within 0.2% of each other).
Ent30
DomNet
STL
Land
Crop
ImNer-R
Vanilla Logits
64.9 (0.3)
75.7 (1.2)
87.3 (0.2)
60.5 (0.8)
90.9 (0.2)
72.2 (-)
Vanilla Probs
64.6 (0.4)
78.7 (1.3)
87.2 (0.2)
59.5 (1.0)
90.9 (0.2)
77.4 (-)
Tuned Logits
64.6 (0.6)
86.3 (0.6)
85.7 (0.9)
58.7 (1.2)
87.3 (5.7)
(3)1
Tuned Probs
62.8 (0.7)
86.9 (0.2)
81.3)
58.7 (2.2)
86.8 (5.5)
63.8
Calibrated Logits
65.0 (0.4)
84.4 (0.3)
87.5 (0.2)
61.2 (0.8)
91.3 (0.8)
71.7 (-)
Calibrated Probs
64.7 (0.5)
86.1 (0.2)
87.3 (0.2)
60.8 (0.8)
91.3 (0.8)
77.1 (-)
Calibration is important: We compared calibrated ensembles with vanilla ensembles (no calibra- tion) and tuned ensembles where how to weight the predictions of the standard versus robust model learned on ID data. Interestingly, calibrated ensembles substantially outperform tuned ensem- bles OOD (calibrated ensembles: 77.9%, tuned ensembles: 74.0%) with less than 0.1% drop in ID accuracy relative to tuned ensembles. Naturally, we expect the tuned ensemble to the best ID since its weights are tailored for ID-what is surprising is that the calibrated ensembles do so much better OOD without using any OOD data either. Calibrated ensembles outperform vanilla ensembles both ID and OOD well. We show the accuracies for the different types of ensembles in Table4 (OOD) and TID) also ablate between combining the probabilities versus the logits (before softmax) of the standard and robust models.
5.3 CALIBRATION AND RELATIVE CALIBRATION
In this section examine the calibration and relative calibration of models, which provides partial intuitive explanation for the success of calibrated ensembles. Since we calibrated the standard and robust models on in-distribution (ID) data, we expect them be calibrated even on held out ID test data-this follows from standard statistical guarantees for calibration. Table 7 confirms this intuition the ECE of the standard and robust model are very low ID, on average the ID ECE 1.2% for the standard model and 2.0% for the robust model. However, prior work shows that the calibration models degrades substantially with distribution shift shift-indeed, indeed, Table (ooles shows that OOD ECE of the standard (average: 11.0%) and robust (average: 7.3%) models are much higher.
Why do calibrated ensembles mitigate the ID-OOD tradeoff even though neural networks are not calibrated OOD? We show that relative calibration provides partial answer towards this. The final prediction typically made by the model that is more confident. So the robust model more confident than the standard model OOD then the ensemble inherits the high OOD accuracy of the robust model-eve even the absolute ECE the standard and robust models are both bad.
To get handle at relative calibration measure 1. the accuracy gap between the standard and robust model, which the robust model's 's accuracy minus the standard model's I's accuracy, and 2. the confidence gap between the models: the robust model's average confidence minus the standard

---

Under review as a conference paper at ICLR 2022
Table 5: III accuracies: The in-distribution accuracies of calibrated ensembles, tuned ensembles, and vanilla ensembles are very close (within confidence intervals), so any of these methods are acceptable if we are looking at in-distribution aceuracy. However, they perform quite differently when it comes to OOD accuracy (Table
Ent30
DomNet
CIFARIO
Land
Crop
ImNet
Logits
93.7 (0.1)
89.3 (0.6)
97.3 (0.1)
77.4 (0.1)
95.5 (0.1)
80.9 (-)
Probs
93.7 (0.1)
89.1 (0.4)
97.3 (0.1)
(0.2)
95.5 (0.1)
81.0 (-)
Tuned Logits
93.8 (0.0)
91.3 (0.2)
97.4
(7.4
(0.6
81.7
Tuned Probs
93.8 (0.1)
90.6
97.4 (0.1)
77.1
(0.1)
8-)3
Calibrated Logits
93.7 (0.1)
(0.4)
(0.1)
77.2 (0.2)
(0.6
81.0
Calibrated Probs
93.7 (0.1)
91.2 (0.7)
97.2
77.2 (0.2)
95.6 (0.1)
81.1 (-)
Table 6: OOD ECE: The expected calibration error (ECE) of the standard and robust models on OOD test data, after post-calibration in ID validation data. The calibration errors here are high, especially compared to the ID calibration errors in Table
Ent30
DomNet
STL
Land
Crop
ImNet-R
Calibrated Standard
15,4 (0.8)
13.6 (1.5)
5.6 (1.1)
16.4 (0.8)
7.4 (4.8)
7.8 (-)
Calibrated Robust
14.3 (1.5)
5.5 (0.5)
8.2 (0.0)
6.5 (1.1)
5.0 (0.3)
4.0 (-)
model's average confidence. Ideally, the two should align with each other and at least have the same sign: if the robust model is more accurate then it should also be more confident. In Table g we show that this provides reasonable initial explanation for the success calibrated ensembles. The confidence gap and accuracy gap tend to have the same sign, and calibration aligns the two better. However, calibrated ensembles still work on LandCover-even though the relative calibra- tion average across the entire dataset is not correct (the standard model is more confident but less accurate OOD) at the granularity of individual points ensembling is able to get the benefits of both the standard and robust models.
Why might calibrating models ID improve the relative calibration OOD? Standard deep learning models tend to be overparameterized, and can often get near 0 loss the training data. As such they tend to be highly overconfident on ID and OOD data. Robustness interventions typically involve additional constraints (projecting out spurious input features, lightweight fine-tuning, extensive data augmentation) and so robust models generally tend to be less overconfident. Calibrating both mod- els, even just ID, can make their confidences OOD, the hope is that the relative calibration continues to track their accuracy.
comparable-eve though both models degrade
We note that all this is intuition (theory about OOD calibration and ensembling is scarce and highly challenging) hope that future work formalizes these arguments. We think relative calibration might promising future direction even if models are not calibrated OOD, their relative cali- bration could be much better.
RELATED WORKS AND DISCUSSION
Calibration. Calibration has been widely studied in machine learning (Nacini et al. 2014 Guo al. 2017 Kumar et al. 2219)), and applications such as meteorology (Murphy] 1973; DeGrroo & Fienberg 1983 Gneiting & Raftery (205), fairness (Hebert-Johnson et al. 2018), and health- care (Jiang et al. 2012). Many of these works focus on the in-distribution (ID) setting, where models calibrated on the same distribution that they are evaluated on. Ovadia al. (2019) show that if we calibrate (e.g., via temperature scaling) a model ID, it still has poor uncertainties OOD. Jones How-
eet al. (2021) also show that model uncertainties can be quite unreliable ever, we show that despite having poor uncertainties on traditional metrics, calibrated models can be combined effectively mitigate ID-OOD tradeoff'.
8

---

Under review as a conference paper at ICLR 2022
Table 7: ID ECE: The expected calibration enror (ECE) of the standard and robust models ID test data, after post-calibration in ID validation data. The calibration errors are fairly low-note that we only use 500 examples to temperature scale, so for ImageNet have fewer examples than classes for post-calibration, but the models are still fairly well calibrated.
Ent30
DomNet
CIFARIO
Land
Crop
ImNet
Cal Standard
0.7 (0.1)
2.0 (0.3)
0.8 (0.2)
1.1 (0.5)
1,4 (0.3)
1.0 (-)
Cal Robust
1.1 (0.4)
2.2 (0.2)
1.3 (0.2)
1.7 (0.3)
3.5 (0.2)
2.3 (-)
We show the accuracy gap (difference between the accuracy of the standard and robust model) and the confidence gap (difference in confidence between the two) before and after calibra- tion. The accuracy gap and confidence gap typically have the same sign, and this improves after calibrating ID.
Table 8:
Ent30
DomNet
STL
Land
Crop
ImNet-R
Acc Gap
2.5
31.9
2.7
4.7
4.2
27.9
Conf Gap
-2.3
3.6
1.1
-13.5
4.3
18.6
(+ Cal) Conf Gap
1.4
23.7
5.5
-6
1.3
16.6
Ensembling. Ensembling models is common way to get accuracy boost goost-typically the en- semble members are trained on the same data, but with different random seed (Lakssminnrryanan et al. 2017) augmentation (Stickland & Murray 2020). In the setting where the ensemble mem- bers mostly differ by random seeds or augmentations, prior work has shown that calibrating the members of ensemble does not help (Wu & Gales 2021 Ovadia et al., (019). Indeed, find that calibration has minimal effect when we ensemble two standard, or two robust models, that are trained from different seeds. However when we combine very different models (standard and robust), calibration leads to clear improvements.
Mitigating ID-OOD tradeoffs. Tradeoffs between ID and OOD accuracy are widely studied and prior work self-trains large amounts of unlabeled data to mitigate such tradeoff's (Raghunathan ct al. 2020 Xie et 2021 Khani & Liang 2021). In contrast, our approach uses no extra unlabeled data and is simple method where just add up the model probabilities after a quick calibration step. In concurrent and independent work, (Wortsman et al,, 202T) show on ImageNet and variants (e.g. ImageNet-v2. ImageNet-sketch, ImageNet-R) that there exists way to ensemble CLIP zero- shot and fine-tuned model to get good ID and OOD accuracy -houeee learning exactly how to ensemble the models might require OOD data, which is not available. We show that the natural way to learn how to weight ensemble members- members--slecting the weights to optimize in-distribution accuracy- does not mitigate the ID-OOD gap, but calibrated ensembles do.
Discussion. In this paper, we show that a simple method of calibrating and then ensembling mod- els can eliminate the tradeoff between in-distribution (ID) and out-of-distribution (OOD) accuracy. We hope that this leads to more widespread use and deployment of robustness interventions. One mystery is why calibrated ensembles work so well well-especially when tuning ensemble weights ID does not, and the worse standard model can still be more confident OOD.
REFERENCES
EA AlBadawy, Saha, and MA Mazurowski. Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing. Med Phys., 45, 2018.
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. 2019.
arXiv preprint arXiv: /rXiv:/907.0e
Xinlei Chen, Haoqi Fan, Ross B. Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv, 2020a.

---

Under review as a conference paper at ICLR 2022
Yining Chen, Colin Wei, Ananya Kumar, and Tengyu Ma. Self-training avoids using spurious features under domain shift. In Advances in Neural Information Processing Systems (NeurIPS), 2020b.
Adam Coates, Andrew Ng, and Honlak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the Fourteenth International Conference on Artificial Intelli- gence and Statistics, volume 15, pp. 215 213- 2011.
Morris DeGroot and Stephen Fienberg. The comparison and evaluation of forecasters, Journal of the Royal Statistical Society. Series D (The Statistician), 32:12-20 22, 1983.
John Duchi, Tatsunori Hashimoto, and Hongseok Namkoong. Distributionalll robust losses against mixture covariate shifts.
publications: condsss/codcc.cccccmmmm. pdf 2019.
Geoff French, Michal Mackiewicz, and Mark Fisher. Self-ensembling for visual domain adaptation. In International Conference on Learning Representations, 2018.
Pall Oskar Gislason, Jon Atli Benediktsson, and Johannes R. Sveinsson. Random forests for land cover classification. Pattern Recognition Letters, 27(4):294-300 2006.
Tilmann Gneiting and Adrian E. Raftery. Weather forecasting with ensemble methods. Science, 310, 2005.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In International Conference Machine Learning (ICML), pp. 1321-1330, 2017.
Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong, and Perey Liang. Fairness with- out demographics in repeated loss minimization. In International Conference on Machine Learn- ing (ICML.), 2018
Ursula Hebert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibra- tion: Calibration for the (computationally-denniiaa masses. In International Conference Machine Learning (ICML), 2018
Christina Heinze-Deml and Nicolai Meinshausen. Conditional variance penalties and domain shift robustness. arXiv preprint arXiv: 1710. 1469, 2017.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of distribbiitnnn generalization. arXiv preprint arXiv:2006.16244 2020.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. arXiv, 2019.
Neal Jean, Marshall Burke, Michael Xie, W. Matthew Davis, David B. Lobell, and Stefano Ermon. Combining satellite imagery and machine learning to predict poverty. Science, 353, 2016.
Xiaoqian Jiang, Melanie Osl, Jihoon Kim, and Lucila Ohno-Machade Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Asso- ciation, 19(2):263-774, 2012.
Erik Jones, Shiori Sagawa, Pang Wei Koh, Ananya Kumar, and Percy Liang. Selective classification can magnify disparities across groups. International Conference on Learning Representations (ICLR), 2021.
Fereshte Khani and Percy Liang. Removing spurious features can hurt accuracy and affect groups disproportionately. ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2021.
10

---

Under review as a conference paper at ICLR 2022
Alex Krizhevsky, Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Ananya Kumar, Percy Liang, and Tengyu Ma. Verified uncertainty calibration. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
Xiang Lisa Li and Perey Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Association for Computational Linguistics (ACL), 2021.
Evan Zheran Liu, Behzad Haghgoo, Annie S. Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Perey Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning (ICML), 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018.
Allan Murphy. A new vector partition of the probability score. Journal of Applied Meteorology, 72(4):595-600 1973.
Mahdi Pakdaman Naeini, Gregory F. Cooper, and Milos Hauskrecht. Binary classifier calibration: Non-parametric approach. arXiv, 2014.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua V. Dil- lon, Balaji Lakshminarayanan, and Jasper Snoek, Can you trust your model's uncertainty? eval- uating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In International Conference on Computer Vision (ICCV), 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Interna- tional Conference on Machine Learning (ICML), volume 139, pp. 8748-8763, 2021.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C. Duchi, and Percy Liang. Understanding and mitigating the tradeoff between robustness and accuracy. International Conference on Machine Learning (ICML), 2020.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211-252 2015.
Mare RuBwurm, Sherrie Wang, Marco Korner, and David Lobell. Meta-learning for few-shot land cover classification. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition Workshops, 200-201, 2020.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. In International Conference on Learning Representations (ICLR), 2020a.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameterizatioo exacerbates spurious correlations. In International Conference on Machine Learning (ICML), 2020b.
Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. Breeds: Benchmarks for subpopulation shift. arXiv, 2020.
11

---

Under review as a conference paper at ICLR 2022
Asa Cooper Stickland and lain Murray. Diverse ensembles improve calibration. arXiv, 2020.
Baochen Sun and Kate Saenko. Deep coral:
Correlation alignment for deep domain adaptation. In European Conference on Computer Vision (ECCV), 2016.
Shuhan Tan, Xingchao Peng, and Kate Saenko. Class-imbalanced domain adaptation: An empirical odyssey. arXiv preprint arXiv: 10320, 2020.
Vermote, MOD09A1 MODISherra surface reflectance 8-day global 500m SIN grid V006. 006 2015.
Sherrie Wang, William Chen, Sang Michael Xie, George Azzari, and David B. Lobell. Weakly supervised deep learning for segmentation of remote sensing imagery. Remote Sensing, 12, 2020.
Mitchell Wortsman, Gabriel Ilharco, Mike Li, Jong Wook Kim, Hannanch Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. Robust fine-tuning of zero-shot models. arXiv preprint arXiv:2109.01990 2021.
Xixin Wu and M. Gales. Should ensemble members be calibrated? arXiv, 2021.
Sang Michael Xie, Ananya Kumar, Robert Jones, Fereshte Khani. Tengyu Ma. and Percy Liang. In-N-out: Pre-training and self-training using auxiliary information for out-of-distribution robust- ness. In International Conference Learning Representations (ICLR), 2021.
John Zech, Mareus A. Badgeley, Manway Liu, Anthony B. Costa, Joseph Titano, and Eric Karl Oermann. Variable generalization performance of a deep learning model to detect pncumonia in chest radiographs: cross-sectional study. In PLOS Medicine, 2018.
12


Reviews:
[{"review_id": "Q4xidVyxHF2", "reviewer": "Reviewer_7Ez2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper has many strengths:\n- The idea is impactful, easy to implement, and explained clearly.\n\nHowever, there are some areas where the paper could be improved:\n1) More complete baselines. \n\nA number of baselines including vanilla ensembling, tuned ensembling, and alternative combination methods are mentioned in Section 3. However, their relative ID and OOD performances are not sufficiently explored. Table 4 contains the OOD accuracies for some of these methods for all datasets, but in terms of ID performance only the averaged version is available (Figure 1, right), and the Tuned ID baseline remains absent. There is room to include this as the paper is currently just over seven pages, and it would improve the paper. In general, it would be nice to create Figure 1 (left) using real data for each dataset, as currently it is only shown with fake data I believe.\n\nThere are also some baselines that are discussed but absent, e.g., in Section 3 the authors mention that they found combining the softmax probabilities to work better than combining the logits. However, this alternative is absent in all tables / plots. This baseline in particular is interesting for the following reason: I believe that without the softmax, there is a mixing coefficient alpha (e.g., in Eq. 3.5) which exactly matches the proposed method (Eq. 3.3). Going back to the idea from the previous paragraph of creating a scatter plot with real data, it would even be possible to plot the curve from varying alpha and see where the proposed method lies on the curve -- how close is it to the optimal alpha? Calibration could then be interpreted as a way to choose this mixing coefficient, which would be very interesting.\n\nWhen comparing with vanilla ensembling, this paper use a default mixing coefficient of 0.5. However, this paper mentions that [1] only shows there exists a way to ensemble a standard and robust model, and requires OOD data to learn how to do so. As 0.5 is assumed to be the default mixing coefficient for vanilla ensembling, it could be nice to also compare with [1] using, e.g., mixing coefficient 0.5. This seems relatively straightforward for ImageNet-R as the experimental set-up is very similar. Going back to the previous paragraph, it would be a very interesting comparison if you could find the mixing coefficient alpha via calibration then apply it to the method of [1]. As I understand, the concurrent method of [1] is very similar to this work as both propose to ensemble a robust model with a model that performs good ID and find the benefits of both in the ensemble.\n\n2) Clarity in experimental details.\n\nClarity in experimental details could strengthen the paper, for instance when fine-tuning on ImageNet for the ImageNet-R experiments which optimizer and batch size are used. Lack of these details will make reproduction difficult. One drawback of this method is that it is more expensive than a single model, but this experimental detail is absent. Moreover, why is the CLIP zero-shot performance on ImageNet-R extremely low? This is substantially lower than what is reported in [1,2] for the same model type (ViT-B/16).\n\n[1] https://arxiv.org/abs/2109.01903\n[2] https://arxiv.org/abs/2109.01134", "summary_of_the_paper": "A robust model attains high accuracy OOD while a standard model attains high accuracy ID. This paper proposes ensembling a robust and standard model to attain high accuracy ID and OOD. In particular, this paper recommends calibrating both the standard and robust models on ID data before ensembling. ", "summary_of_the_review": "This paper explores an interesting idea of ensembling a robust and standard model to attain high accuracy ID and OOD. However, the paper would substantially benefit from a more complete exploration of baselines and thorough explanation of experimental details. For instance, even some of the baselines mentioned are partially included or absent, and accuracy of ImageNet-R is substantially lower than usually reported. I recommend that the paper could substantially benefit from additional exploration, and may have been rushed in current form.\n\nEdit: many of my concerns were addressed and I have changed my score to 6.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "PiaMg76HIG", "reviewer": "Reviewer_VdiR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The strengths of the submission are that \n\n — the topic is highly relevant and timely\n\n — the range of datasets is broad, and the empirical results are fairly promising\n\nSome points that could strengthen the submission are as follows:\n\n — It would be interesting to investigate qualitatively the OOD cases where the (calibrated) standard model has a higher confidence, as well as a quantitative analysis of how often such events occur. I was hoping to find something along these lines in the Appendix, but it doesn’t look like there’s a supplement. It might also be a good sanity check to look at expected calibration errors for both models.\n\n — While I believe the range of experiments (datasets + models) is sufficiently broad, methods aimed at distributional robustness such as groupDRO [1] or domain adversarial learning [2] (to name some) might be interesting to study under the proposed method (on whatever datasets they’ve been showcased to have been successful upon).\n\n[1] Distributionally robust neural networks for group shifts, ICLR 2020\n\n[2] Domain generalization with adversarial feature learning, CVPR 2018", "summary_of_the_paper": "The submission considers a very topical issue given the active ongoing interest in OOD performance — models developed with OOD robustness in mind can come with a significant loss of in-distribution performance, and since we expect to mostly encounter in-distribution settings in deployment (assuming deployment is being done responsibly), we need to find ways to trade-off performances for in-distribution and unexpected situations.\n\nThe submission illustrates that a fairly simple approach of Platt-scaling done in-distribution for a “standard” and a “robust” model followed by averaging their predictive distributions can sometimes work pretty well: significant performance improvements appear to be achieved both in- as well as out-of-distribution.", "summary_of_the_review": "The submission demonstrates that a fairly straightforward calibration-based approach for an ensemble model can lead to good performance in both in- as well as out-of-distribution settings. While the empirical illustrations could certainly be made more comprehensive, and the mechanism of improvements analyzed a bit more closely, I believe this is an interesting enough illustration to justify drawing attention to it.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3TeE-TuoAkA", "reviewer": "Reviewer_EkrF", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is an interesting paper that discusses an important and timely topic, the trade-off between in-distribution and out-of-distribution performance. The paper empirically analyzes this trade-off and shows that standard ERM-based learners perform better ID at the expense of OOD, while robust learners are the opposite. The authors then approach this problem head-on, and propose a simple yet effective solution to this perceived trade-off. The proposed solution relies on resembling both types of models. Then, to further improve performance, the authors calibrate the models and demonstrate substantial improvements.\n\nWhile the strong empirical results and simple approach are definitely encouraging, I find significant problems with the paper at its current state. \n\nFirst, the lack of any theoretical contribution or even a substantive intuition into why this approach should work, and when it shouldn’t is highly problematic. This is even more problematic considering existing papers that discuss just that [1]. There seems to be substantial agreement between [1] and this paper, along with some disagreement regarding the connection between ID calibration and OOD performance. Discussing this in the paper and showing how the two might agree will surely improve the paper.\n\nSecond, the methods used in the paper can probably be improved. While good performance with a simple method is always a good thing, it would be beneficial to try and further improve performance with a more advanced re-calibration approach or an end-to-end calibrated model. The same critique also stands for the ensemble approaches, which at the moment only consider a combination of two models.\n\nLastly, the paper is quite thin in analysis and discussion. While I don’t have a particular problem with the lack of novel approaches, I do find it redundant adding an equation for softmax and accuracy. If deleted, the authors should use the 1+ pages remaining space to add a detailed analysis of when and why this approach works in reality. Other potential uses of this space include a representative theoretical analysis, more baselines and experiments on widely used OOD benchmarks such as WILDS [2].\n\n\n[1] On Calibration and Out-of-domain Generalization\n[2] WILDS: A Benchmark of in-the-Wild Distribution Shifts\n", "summary_of_the_paper": "This paper discusses the trade-off between in-distribution and out-of-distribution accuracy, where ERM-based learners have good ID but poor OOD performance, and robust learners the opposite. The authors demonstrate that by ensembling both types of learners, and calibrating their ID accuracy, they can outperform the standard and robust models both on ID and OOD data.\n\nThe main contribution of the paper is the strong empirical results provided by the simple yet effective approach.\n", "summary_of_the_review": "While I do think that this paper discusses an important problem and presents methods that perform reasonably well on real data, I think that its current form the paper is a bit too thin and requires some more work before it is published. My main concerns are the lack of theoretical analysis, as well as additional experiments and analysis with more advanced methods. I also point to a previous publication that might help the authors relate to in their analysis. \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-RbDb2h60WS", "reviewer": "Reviewer_u3S8", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n1) A simple algorithm that works in a wide range of datasets.\n\nWeaknesses\n1) I'm not convinced why the proposed method should work. I think the authors should provide more explanations why the newly introduced temperature parameter, which is tuned on the validation dataset, should show improved OOD accuracy.\n2) I'm not sure about the self-training results in Table 3. compared to the original results in the In-N-Out paper [1]. Maybe releasing the author's implementation can demystify the concerns.\n3) The paper lacks any ablation study or analysis to support their claims.\n\n#References \n[1] IN-N-OUT: PRE-TRAINING AND SELF-TRAINING\nUSING AUXILIARY INFORMATION FOR OUT-OF DISTRIBUTION ROBUSTNESS\n", "summary_of_the_paper": "UPDATE:\n\nI acknowledge that I've read the author responses as well as the other reviews.\n\nI now understand the empirical merit of the paper. Furthermore, the authors clarified the baselines' performance and add further analysis. Therefore, I raise my score to 5 weak reject. However, I'm still not convinced about the novelty of the paper. In ensemble learning, diversity between ensembles is a major factor of success. Therefore, while it is straightforward that the proposed method outperforms robust+robust and standard+standard, temperature scaling is not a new contribution and I'm still skeptical about how 'surprising' the results are. I hope the authors can clarify further issues about the point.\n\n================================================================================================\n\nThe paper studies ensemble methods to achieve improved in-distribution (ID) and out-of-distribution accuracy. The paper proposes a simple fix on concatenating the robust model and the standard model via adopting the idea of classifier-specific temperature parameters, which is optimized on the in-distribution validation dataset. Authors test their methods on various distribution shifts. The proposed method seems to improve the ID-OOD accuracy tradeoff.", "summary_of_the_review": "While I think the results in the paper are competitive against the baselines, the authors do not give any explanations for why the proposed method should work. Therefore, I'm leaning towards rejecting the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This is a borderline case and it's quite difficult to decide the recommendation. The paper works on a critically important problem, namely removing or reducing the in-distribution accuracy drop when we need to also take the out-of-distribution accuracy into account. The proposed method is simple and it works, which is great. However, as the reviewers discussed, the demonstrated applications are not very representative, and the authors should consider more popular setups of few-short learning and even other forms of domain generalization. Furthermore, adversarial examples are also OOD (in most cases, since the ID manifolds are thin films and the attacks can easily go out of the ID manifolds), it would be great if adversarial accuracy can be incorporated as a case of OOD accuracy. Since there is still room for improvement, we hope the paper would benefit from a cycle of revisions for a re-submission.
Decision: Reject