Original: Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift
Revised: Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift

Original: Abstract
Revised: Abstract

Original: We often see undesirable tradeoffs in robust machine learning where out-of-distribution (OOD) accuracy is at odds with in-distribution (ID) accuracy. A robust classiﬁer obtained via specialized techniques such as removing spurious features often has better OOD but worse ID accuracy compared to a standard classiﬁer trained viavanilla ERM.
Revised: We often see undesirable tradeoffs in robust machine learning where out-of-distribution (OOD) accuracy is at odds with in-distribution (ID) accuracy: a robust classiﬁer obtained via specialized techniques such as removing spurious features often has better OOD but worse ID accuracy compared to a standard classiﬁer trained via ERM.

Original: In this paper, we ﬁnd that a simple approach of ensembling the standard and robust models, after calibrating on only ID data, outperforms prior state-of-the-art  both ID and OOD.
Revised: In this paper, we ﬁnd that ID-calibrated ensembles—where we simply ensemble the standard and robust models after calibrating on only ID data—outperforms prior state-of-the-art (based on self-training) on both ID and OOD accuracy.

Original: On ten natural distribution shift datasets, ID-calibrated ensembles get the best of both worlds: strong ID accuracyof the standard model and OOD accuracy of the robust model.
Revised: On eleven natural distribution shift datasets, ID-calibrated ensembles obtain the best of both worlds: strong ID accuracy and OOD accuracy.

Original: We analyze this method in stylized settings, and identify two important conditions for ensembles to perform wellon both ID and OOD: (1) standard and robust models should be calibrated (on ID data, because OOD data is unavailable), (2) OOD has no anticorrelated spurious features.
Revised: We analyze this method in stylized settings, and identify two important conditions for ensembles to perform well both ID and OOD: (1) standard and robust models should be calibrated (on ID data, because OOD data is unavailable), (2) OOD has no anticorrelated spurious features.

Original:  INTRODUCTION
Revised:  INTRODUCTION

Original: Machine learning models suffer large drops in accuracy in the presence of distribution shift where the test distribution is different from the training distribution.
Revised: Machine learning models suffer large drops in accuracy in the presence of distribution shift where the test distribution is different from the training distribution.

Original: As ML systems are widely deployed, it is important to train models that achieve good accuracy on unforeseen, out-of-distribution (OOD) examples.
Revised: As ML systems are widely deployed, it is important to train models that achieve good accuracy on unforeseen, out-of-distribution (OOD) examples.

Original: For example, models trained on medical data from a few hospitals should work well when deployed broadly [Zech et al., 2018, AlBadawy et al., 2018].
Revised: For example, models trained on medical data from a few hospitals should work well when deployed broadly [Zech et al., 2018, AlBadawy et al., 2018].

Original: Similarly, when predicting poverty from satellite imagery, models trained on data from a few countries should work well on all countries, particularly those where labels are scarce due to resource constraints [Jean et al., 2016].
Revised: Similarly, when predicting poverty from satellite imagery, models trained on data from a few countries should work well on all countries, particularly those where labels are scarce due to resource constraints [Jean et al., 2016].

Original: There has been a lot of research interest in tackling this robustness problem under various settings such as robustness to spurious correlations [Heinze-Deml and Meinshausen, 2017, Sagawa et al., 2020a], domain generalization [Arjovsky et al., 2019, Sun and Saenko, 2016], demographic shifts [Hashimoto et al., 2018, Duchi et al., 2019] among others.
Revised: There has been a lot of research interest in tackling this robustness problem under various settings such as robustness to spurious correlations [Heinze-Deml and Meinshausen, 2017, Sagawa et al., 2020a], domain generalization [Arjovsky et al., 2019, Sun and Saenko, 2016], demographic shifts [Hashimoto et al., 2018, Duchi et al., 2019] among others.

Original: Almost universally across thesedifferent settings, an unfortunate tradeoff arises: robustness interventions, such as removing spurious features, typically improve the OOD accuracy but cause a drop in the in-distribution (ID) accuracy on new test points from the original distribution.
Revised: Across many of these settings, an unfortunate tradeoff arises: robustness interventions, such as removing spurious features or lightweight ﬁne-tuning, typically improve the OOD accuracy but cause a drop in the in-distribution (ID) accuracy on new test points from the original distribution.

Original: This tradeoff is a major hurdle in using the multitude of proposed robustness interventions.
Revised: This tradeoff is a major hurdle in using the multitude of proposed methods that aim to improve OOD accuracy.

Original: In practice, most inputs are likely to be ID, so it is unsatisfactory to use a “robust” model that has high OOD performance but performs less accurately on these majority ID points.
Revised: In practice, most inputs are likely to be ID, so it is unsatisfactory to use a robust model that has high OOD accuracy but performs less accurately on these majority ID points.

Original: On the other hand, “standard models” (trained without robustness interventions) can fail in the presence of even small shifts, and it can be dangerous to use a standard model even if OOD points are rare.
Revised: On the other hand, standard models (trained without robustness interventions) can fail in the presence of even small shifts, and it can be dangerous to use a standard model even if OOD points are rare.

Original: In this work, we ask: is there a general strategy to harness the strengths of both the standard and robust model to achieve high accuracy both ID and OOD, without using OOD data?
Revised: In this work, we ask: is there a general strategy to harness the strengths of both the standard and robust model to achieve high accuracy both ID and OOD, without using OOD data?

Original: We ﬁnd that  a simple approach of ﬁrst calibrating the standard and robust models on only ID data and then ensembling them, outperforms prior state-of-the-art both ID and OOD.
Revised: We ﬁnd that ID-calibrated ensembles, a simple approach of ﬁrst calibrating the standard and robust models on only ID data and then ensembling them, outperforms prior state-ofthe-art both ID and OOD.

Original: As illustrated in Figure 1, across 10 natural distribution shift datasets (e.g. geographical shift, style shift, subpopulation shift), ID-calibrated ensembles get the best of both worlds : strong ID accuracy of the standard model and robust accuracy of the OOD models.
Revised: As illustrated in Figure 1, across 11 natural distribution shift datasets (e.g. geographical shift, style shift, subpopulation shift), ID-calibrated ensembles get the best of both worlds : strong ID accuracy of the standard model and robust accuracy of the OOD models.

Original: Averaged across these datasets, 
Revised: Averaged across these datasets, ID-calibrated ensembles achieve an

Original: ID-calibrated ensembles achieve an ID accuracy of 91.1% (vs. 89.5% for the standard model and 87.5% for the robust model) and OOD accuracy of 75.3% (vs. 65.6% for the standard model and 73.2% for the robust model).
Revised: ID accuracy of 90.3% (vs. 88.7% for the standard model and 86.8% for the robust model) and OOD accuracy of 74.5% (vs. 65.2% for the standard model and 72.3% for the robust model).

Original: We then analyze when and why ID-calibrated ensembles can get the best of the standard and robust models, under a simplifying assumption that these models provide different and independent signals for the label.
Revised: We then analyze when and why ID-calibrated ensembles can get the best of the standard and robust models, under a simplifying assumption that these models provide different and independent signals for the label.

Original: If the standard and robust models are calibrated ID, the ensembling strategy for the best ID performance is to simply add the predictions of the two models (Proposition 4.1).
Revised: If the standard and robust models are calibrated ID, the ensembling strategy for the best ID performance is to simply add the predictions of the two models (Proposition 4.1).

Original: By the same idea, if the standard and robust models were also calibrated OOD, ensembling would achieve the optimal OOD accuracy.
Revised: By the same idea, if the standard and robust models were also calibrated OOD, ensembling would achieve the optimal OOD accuracy.

Original: However, since we only have ID training data, models can only be calibrated ID and ID calibration is not sufﬁcient for OOD calibration [Ovadia et al., 2019].
Revised: However, since we only have ID training data, models can only be calibrated ID and ID calibration is not sufﬁcient for OOD calibration [Ovadia et al., 2019].

Original: When can calibrated ensembles provide beneﬁts even without OOD calibration?
Revised: When can calibrated ensembles provide beneﬁts even without OOD calibration?

Original: In many natural distribution shifts, standard models pick up on predictive signals in the training data that are absent or suppressed under distribution shift— in these cases, we show that ID-calibrated ensembles obtain the best of both the standard and robust models OOD.
Revised: In many natural distribution shifts, standard models pick up on predictive signals in the training data that are absent or suppressed under distribution shift—in these cases, we show that ID-calibrated ensembles obtain the best of both the standard and robust models OOD.

Original: However, when spurious features become anticorrelated OOD (as is common when the distribution shift is adversarially synthesized), we show that the ensemble performance is in between the standard and robust models.
Revised: However, when spurious features become anticorrelated OOD (as is common when the distribution shift is adversarially synthesized), we show that the ensemble’s OOD accuracy is in between the standard and robust models.

Original: We empirically validate this on three adversarially synthesized shifts [Sagawa et al., 2020a, Jones et al., 2021] where the spurious signals are anticorrelated OOD.
Revised: We empirically validate this on three adversarially synthesized shifts [Sagawa et al., 2020a, Jones et al., 2021] where the spurious signals are anticorrelated OOD.

Original: Finally, we compare  to a number of other alternate ensembling strategies (for example, tuning the weights of the ensemble on ID validation data) and ﬁnd that they do not work as well as ID-calibrated ensembles.
Revised: Finally, we compare ID-calibrated ensembles to a number of other alternate ensembling strategies (for example, tuning the weights of the ensemble on ID validation data) and ﬁnd that they do not work as well as ID-calibrated ensembles.

Original: To summarize, our main contributions are:
Revised: To summarize, our main contributions are:

Original: We revisit the classic idea ofensembling and propose
Revised: We revisit the classic idea of

Revised: ensembling

Original:  a simple, general, and effective method  tomitigate
Revised: and propose a simple, general, and effective method (ID-calibrated ensembles) to

Original:  ID-OOD accuracy tradeoffs.
Revised: mitigate ID-OOD accuracy tradeoffs

Original: This method outperforms prior approaches based on self-training, despite not using any additional unlabeled data.
Revised: This method outperforms prior approaches based on self-training, despite not using any additional unlabeled data

Original: We prove that ensembles of calibrated models are optimal when the models provide independent signals about the label.
Revised: We prove that ensembles of calibrated models are optimal when the models provide independent signals about the label.

Original: However, models can only be calibrated ID from which we have training data, and ID calibration does not imply OOD calibration.
Revised: However, models can only be calibrated ID from which we have training data, and ID calibration does not imply OOD calibration

Revised: In sim-

Revised: ple

Original: In simple and stylized settings, we identify conditions under which ID-calibrated ensembles achieve the best of standard and robust models in terms of OOD performance.
Revised: and stylized settings, we identify conditions under which ID-calibrated ensembles achieve the best of stan- dard and robust models in terms of OOD performance

Original: We validate these insights experimentally and ﬁndthat
Revised: We validate these insights experimentally and ﬁnd

Revised: that ID

Original: ID-calibrated ensembles eliminate tradeoffs under a variety of natural distributionshifts, but can fail when there are adversarially synthesized shifts.
Revised: -calibrated ensembles eliminate tradeoffs under a variety of natural distribution

Revised: shifts,

Revised: but can fail when there are adversarially synthesized

Revised: shifts.

Original:  SETUP
Revised:  SETUP

Original: Consider a K -class classiﬁcation task, where the goal is to predict labels
Revised: Consider a K -class classiﬁcation task, where the goal is to predict labels

Original: y ∈
Revised: y ∈

Original: [ K ] corresponding to inputs x ∈ X .
Revised: [ K ] corresponding to inputs x ∈ X .

Original: Models.
Revised: Models.

Original: A model f : X → R K takes an input x ∈ X and outputs a score f ( x ) ∈ R K where f ( x ) i can be interpreted as the model’s “conﬁdence” that the label y is i .
Revised: A model f : X → R K takes an input x ∈ X and outputs a score f ( x ) ∈ R K where f ( x ) i can be interpreted as the model’s “conﬁdence” that the label y is i .

Original: The model outputs the label pred ( f ( x ))
Revised: The model outputs the label pred ( f ( x ))

Original: =
Revised: =

Original: arg max i f ( x ) i .
Revised: arg max i f ( x ) i .

Original: The conﬁdence scores can be normalized to sum to 1 (and interpreted as probabilities) using the softmax function, softmax ( f ( x ))
Revised: The conﬁdence scores can be normalized to sum to 1 (and interpreted as probabilities) using the softmax function, softmax ( f ( x ))

Original: i =
Revised: i =

Original: [Equation]
Revised: [Equation]

Original: ) for i ∈
Revised: ) for i ∈

Original: [ K ] .
Revised: [ K ] .

Original: Distributions and error.
Revised: Distributions and error.

Original: Let P id and P ood denote the underlying distribution of ( x, y ) pairs in-distribution (ID) and outof-distribution (OOD), respectively.
Revised: Let P id and P ood denote the underlying distribution of ( x, y ) pairs in-distribution (ID) and outof-distribution (OOD), respectively.

Original: We evaluate a model f on the fraction of times it makes a wrong prediction on
Revised: We evaluate a model f on the fraction of times it makes a wrong prediction on

Original: P id and P ood : Err id ( f ) =
Revised: P id and P ood : Err id ( f ) =

Original: E x,y ∼ P i
Revised: E x,y ∼ P i

Original: d
Revised: d

Original: [ pred ( f ( x )) (cid:54) =
Revised: [ pred ( f ( x )) (cid:54) =

Original: y ] and
Revised: y ] and

Original: Err ood
Revised: Err ood

Original: ( f ) =
Revised: ( f ) =

Original: E x,y ∼ P ood
Revised: E x,y ∼ P ood

Original: [ pred ( f ( x )) (cid:54) =
Revised: [ pred ( f ( x )) (cid:54) =

Original: y ] .
Revised: y ] .

Original: Standard and robust models.
Revised: Standard and robust models.

Original: A standard model f std is trained via empirical risk minimization where we minimize some loss on ID training data.
Revised: A standard model f std is trained via empirical risk minimization where we minimize some loss on ID training data.

Original: f std often relies on spurious correlations such as image background or occurence of certain words that are not necessarily predictive OOD.
Revised: f std often relies on spurious correlations such as image background or occurence of certain words that are not necessarily predictive OOD.

Original: In order to improve OOD performance, a robust model f rob is trained via a modiﬁed training procedure (robustness interventions) to discourage models from relying on ID-speciﬁc spurious features.
Revised: In order to improve OOD performance, a robust model f rob is trained via a modiﬁed training procedure (robustness interventions) to discourage models from relying on IDspeciﬁc spurious features.

Original: Formally, we have the followingrelationship between f std and f rob . [Equation]
Revised: Formally, we have the following

Original: Require: in-distribution validation data { ( x val i , y val i ) } n val i =1 ∼ P id , standard and robust models f std , f rob : X → R K 1: f std =
Revised: Require: in-distribution validation data { ( x val i , y val i ) } n val i =1 ∼ P id , standard and robust models f std , f rob : X → R K 1: f std =

Original: Calibrate f std on in-distribution (ID) data 2: f rob =
Revised: Calibrate f std on in-distribution (ID) data 2: f rob =

Original: Calibrate f rob on in-distribution (ID) data 3: Return f ens ( x ) = 12 ( softmax ( f std ( x ))
Revised: Calibrate f rob on in-distribution (ID) data 3: Return f ens ( x ) = 12 ( softmax ( f std ( x ))

Original: + softmax ( f rob ( x )))
Revised: + softmax ( f rob ( x )))

Original:  The precise robustness intervention depends on the task— in Section 4 we model the relationship between f std and f rob in a stylized setting amenable for analysis, and in Sectionwe describe what f std and f rob are in our real datasets.
Revised: relationship between f std and f rob . [Equation] The precise robustness intervention depends on the task— in Section 4 we model the relationship between f std and f rob in a stylized setting amenable for analysis, and in Sectionwe describe what f std and f rob are in our real datasets.

Original: Best of both worlds.
Revised: Best of both worlds.

Original: Our goal is to get the best of both worlds—a classiﬁer f ens that achieves the strong ID accuracy of the standard model, and OOD accuracy of the robust model:
Revised: Our goal is to get the best of both worlds—a classiﬁer f ens that achieves the strong ID accuracy of the standard model, and OOD accuracy of the robust model:

Original: [Equation]
Revised: [Equation]

Original: ID validation data.
Revised: ID validation data.

Original: To get the best of both worlds, we only allow access to ID validation data, { ( x val i , y val i ) } n val i =1 ∼ P id , for tuning hyperparameters.
Revised: To get the best of both worlds, we only allow access to ID validation data, { ( x val i , y val i ) } n val i =1 ∼ P id , for tuning hyperparameters.

Original: Following Xie et al.
Revised: Following Xie et al.

Original: Koh et al.
Revised: Koh et al.

Original: [2021], Gulrajani and Lopez-Paz [2020] we do not use any OOD validation data.
Revised: [2021], Gulrajani and Lopez-Paz [2020] we do not use any OOD validation data.

Original:  METHODS
Revised:  METHODS

Original: Proposed method: ID-calibrated ensembles.
Revised: Proposed method: ID-calibrated ensembles.

Original: Given a standard model f std and robust model f rob , we ﬁrst calibrate each model on the in-distribution validation data, and then add up their predictions (Algorithm 1).
Revised: Given a standard model f std and robust model f rob , we ﬁrst calibrate each model on the in-distribution validation data, and then add up their predictions (Algorithm 1).

Original: In our experiments, we calibrate using temperature scaling [Guo et al., 2017] with the cross-entropy loss (cid:96) :
Revised: In our experiments, we calibrate using temperature scaling [Guo et al., 2017] with the cross-entropy loss (cid:96) :

Original: [Equation]
Revised: [Equation]

Original: We then ensemble the two models by adding up the probabilities that they predict [Lakshminarayanan et al., 2017].
Revised: We then ensemble the two models by adding up the probabilities that they predict [Lakshminarayanan et al., 2017].

Original: [Equation]
Revised: [Equation]

Original: where the predicted label is pred ( f ens ( x ))
Revised: where the predicted label is pred ( f ens ( x ))

Original: [Equation]
Revised: [Equation]

Original: arg max y f ens [Equation]
Revised: arg max y f ens ( x ) y .

Original: Ablations.
Revised: Ablations.

Original: In Section 6 we ablate each component of the method, for example the calibration step, way of combining the models, and we compare to (calibrated) ensembles of two standard models, or of two robust models.
Revised: In Section 6 we ablate each component of the method, for example the calibration step, way of combining the models, and we compare to (calibrated) ensembles of two standard models, or of two robust models.

Original:  INTUITIONS AND ANALYSIS
Revised:  INTUITIONS AND ANALYSIS

Original: In this section, we build basic intuitions for when and why
Revised: In this section, we build basic intuitions for when and why

Original: ID-calibrated ensembles can get the best of both worlds (good ID accuracy of f std and OOD accuracy of f rob ), even without using any OOD data.
Revised: ID-calibrated ensembles can get the best of both worlds (good ID accuracy of f std and OOD accuracy of f rob ), even without using any OOD data.

Original: We ﬁrst deﬁne a stylized setting, and then analyze the ID performance in Section 4.
Revised: We ﬁrst deﬁne a stylized setting, and then analyze the ID performance in Section 4.

Original: and OOD performance in Section 4.2.
Revised: and OOD performance in Section 4.2.

Original: Diverse features.
Revised: Diverse features.

Original: An intuitive and illustrative conceptual setting is the following: we assume inputs have some robust features (that are predictive both ID and OOD) and some spurious features (that are only predictive ID).
Revised: An intuitive and illustrative conceptual setting is the following: we assume inputs have some robust features (that are predictive both ID and OOD) and some spurious features (that are only predictive ID).

Original: f std relies on the spurious features while f rob relies on the robust features, both of which provide independent signals on the label.
Revised: f std relies on the spurious features while f rob relies on the robust features, both of which provide independent signals on the label.

Original: Assumption 4.1.
Revised: Assumption 4.1.

Original: We assume that f rob and f std have diverse features with respect to P id and P ood , that is, f rob ( x ) ⊥ f std ( x ) | y when ( x, y ) ∼ P for P ∈ { P id , P ood } (4.1)
Revised: We assume that f rob and f std have diverse features with respect to P id and P ood , that is, f rob ( x ) ⊥ f std ( x ) | y when ( x, y ) ∼ P for P ∈ { P id , P ood } (4.1)

Original: Connection with prior assumptions.
Revised: Connection with prior assumptions.

Original: The diverse features assumption is weaker than the assumptions in prior conceptual models of distribution shifts Chen et al.
Revised: The diverse features assumption is weaker than the assumptions in prior conceptual models of distribution shifts Chen et al.

Original: [2020], Sagawa et al.
Revised: [2020], Sagawa et al.

Original: [2020b], Nagarajan et al.
Revised: [2020b], Nagarajan et al.

Original: [2020] where robust and spurious features are disjoint parts of the input, each generated independently based on the label.
Revised: [2020] where robust and spurious features are disjoint parts of the input, each generated independently based on the label.

Original: In our setting, the features can be complicated functions of the inputs.
Revised: In our setting, the features can be complicated functions of the inputs.

Original: Ensemble.
Revised: Ensemble.

Original: The ensemble f ens simply adds up the predictions of the standard model f std and robust model f rob .
Revised: The ensemble f ens simply adds up the predictions of the standard model f std and robust model f rob .

Original: This is slightly different from Section 3, but is more amenable to analysis.
Revised: This is slightly different from Section 3, but is more amenable to analysis.

Original: [Equation]
Revised: [Equation]

Original: Class-balanced.
Revised: Class-balanced.

Original: For simplicity of exposition, we assume the class-balanced setting where every label P ( Y = y ) is equally likely.
Revised: For simplicity of exposition, we assume the class-balanced setting where every label P ( Y = y ) is equally likely.

Original: Formally, we say P is class-balanced if
Revised: Formally, we say P is class-balanced if

Original: P ( Y = y )
Revised: P ( Y = y )

Original: = 1 /K for all y ∈
Revised: = 1 /K for all y ∈

Original: [ K ] .
Revised: [ K ] .

Original: We analyze the general setting in Appendix A.
Revised: We analyze the general setting in Appendix A.

Original: 4.1 ID PERFORMANCE OF ENSEMBLES
Revised: 4.1 ID PERFORMANCE OF ENSEMBLES

Original: In this section, we show that if f std and f rob are calibrated with respect to P id , then the ensemble f
Revised: In this section, we show that if f std and f rob are calibrated with respect to P id , then the ensemble f

Original: ens is the best way to combine their predictions.
Revised: ens is the best way to combine their predictions.

Original: Since we have access to validation data from P id , the ﬁrst step of our method (Section 3) is to calibrate f std and f rob ID.
Revised: Since we have access to validation data from P id , the ﬁrst step of our method (Section 3) is to calibrate f std and f rob ID.

Original: We conclude the section by giving intuition for why this calibration step can be particularly important for deep neural networks.
Revised: We conclude the section by giving intuition for why this calibration step can be particularly important for deep neural networks.

Original: Intuitively, calibration means that the probability that a model outputs for an event reﬂects the true frequency of that event: if a model says 1,000 patients have the ﬂu with probability 0.1, approximately 100 of them should indeed have the ﬂu.
Revised: Intuitively, calibration means that the probability that a model outputs for an event reﬂects the true frequency of that event: if a model says 1,000 patients have the ﬂu with probability 0.1, approximately 100 of them should indeed have the ﬂu.

Original: Formally, we look at joint calibration [Murphy, 1973, Brocker, 2009] where a model f is calibrated with respect to a distribution P if for all x ∈ X , y ∈
Revised: Formally, we look at joint calibration [Murphy, 1973, Brocker, 2009] where a model f is calibrated with respect to a distribution P if for all x ∈ X , y ∈

Original: [ K ] :
Revised: [ K ] :

Revised: P

Original: [Equation]
Revised: [Equation]

Original: The following proposition says that if f std and f rob are calibrated on P id , then f ens has lower error on P id than any other way of combining the two models—this also implies that f ens gets higher accuracy than f std and f rob .
Revised: The following proposition says that if f std and f rob are calibrated on P id , then f ens has lower error on P id than any other way of combining the two models—this also implies that f ens gets higher accuracy than f std and f rob .

Original: Proposition 4.1.
Revised: Proposition 4.1.

Original: Suppose that f std and f rob are calibrated with respect to P id , and that P id is class-balanced.
Revised: Suppose that f std and f rob are calibrated with respect to P id , and that P id is class-balanced.

Original: Let h :
Revised: Let h :

Original: R K
Revised: R K

Original: × R K → R K be an arbitrary function that combines the standard and robust model’s predictions, and let f h be the resulting classiﬁer: f h ( x ) =
Revised: × R K → R K be an arbitrary function that combines the standard and robust model’s predictions, and let f h be the resulting classiﬁer: f h ( x ) =

Original: h ( f std ( x ) , f rob ( x )) .
Revised: h ( f std ( x ) , f rob ( x )) .

Original: The ensemble is better than any such combination classiﬁer f h :
Revised: The ensemble is better than any such combination classiﬁer f h :

Original: Err id ( f ens ) ≤ Err id ( f h ) .
Revised: Err id ( f ens ) ≤ Err id ( f h ) .

Original: The proof of Proposition 4.1 is in Appendix A. Intuitively, since f rob ( x ) ⊥ f std ( x ) | y , the Bayes optimal predictor is proportional to multiplying their predicted probabilities, which is equal to adding logits (logits are in log space).
Revised: The proof of Proposition 4.1 is in Appendix A. Intuitively, since f rob ( x ) ⊥ f std ( x ) | y , the Bayes optimal predictor is proportional to multiplying their predicted probabilities, which is equal to adding logits (logits are in log space).

Original: Proposition 4.1 has an important condition: the two models must be calibrated.
Revised: Proposition 4.1 has an important condition: the two models must be calibrated.

Original: In practice, deep learning models are miscalibrated [Guo et al., 2017], so our ﬁrst step (Section 3) is to calibrate the models ID.
Revised: In practice, deep learning models are miscalibrated [Guo et al., 2017], so our ﬁrst step (Section 3) is to calibrate the models ID.

Original: We explain why the ID calibration step is important for deep neural networks.
Revised: We explain why the ID calibration step is important for deep neural networks.

Original: Why neural networks are miscalibrated.
Revised: Why neural networks are miscalibrated.

Original: Deep neural networks are typically large enough to memorize the training dataset, and are encouraged to magnify their weights (and hence their conﬁdence) to decrease the training loss [Mukhoti et al., 2020, Bai et al., 2021].
Revised: Deep neural networks are typically large enough to memorize the training dataset, and are encouraged to magnify their weights (and hence their conﬁdence) to decrease the training loss [Mukhoti et al., 2020, Bai et al., 2021].

Original: The extent of this miscalibration and overconﬁdence depends on the training procedure [Hendrycks et al., 2019, Desai and Durrett, 2020].
Revised: The extent of this miscalibration and overconﬁdence depends on the training procedure [Hendrycks et al., 2019, Desai and Durrett, 2020].

Original: In our case f std and f rob are trained in different ways and have different calibration (Appendix B.3).
Revised: In our case f std and f rob are trained in different ways and have different calibration (Appendix B.3).

Original: Why this miscalibration can hurt ensembling.
Revised: Why this miscalibration can hurt ensembling.

Original: Concretely, consider two models f (cid:48) std and f (cid:48) rob which are calibrated on
Revised: Concretely, consider two models f (cid:48) std and f (cid:48) rob which are calibrated on

Original: P id .
Revised: P id .

Original: Let f std ( x ) =
Revised: Let f std ( x ) =

Original: Mf (cid:48) std ( x ) for large M ∈ R (this magniﬁes its weights as discussed above), and let f rob = f (cid:48) rob .
Revised: Mf (cid:48) std ( x ) for large M ∈ R (this magniﬁes its weights as discussed above), and let f rob = f (cid:48) rob .

Original: f std and f (cid:48) std have the same predictions and therefore accuracy but f std is highly miscalibrated.
Revised: f std and f (cid:48) std have the same predictions and therefore accuracy but f std is highly miscalibrated.

Original: The ensemble is then given by f ens ( x ) =
Revised: The ensemble is then given by f ens ( x ) =

Original: Mf (cid:48) std ( x ) + f (cid:48) rob ( x ) .
Revised: Mf (cid:48) std ( x ) + f (cid:48) rob ( x ) .

Original: For very large M , f ens and f (cid:48)
Revised: For very large M , f ens and f (cid:48)

Original: std have the same predictions—this means that
Revised: std have the same predictions—this means that

Original: Err ood ( f ens ) =
Revised: Err ood ( f ens ) =

Original: Err ood ( f std ) <
Revised: Err ood ( f std ) <

Original: Err ood ( f rob ) , and so ensembling does not get the best of both worlds.
Revised: Err ood ( f rob ) , and so ensembling does not get the best of both worlds.

Original: 4.2 OOD PERFORMANCE OF ENSEMBLES
Revised: 4.2 OOD PERFORMANCE OF ENSEMBLES

Original: We showed that if f std and f rob are calibrated on a distribution P , then f ens is better than both models on P .
Revised: We showed that if f std and f rob are calibrated on a distribution P , then f ens is better than both models on P .

Original: However, our validation data is from P id , so we can only calibrate f std and f rob ID.
Revised: However, our validation data is from P id , so we can only calibrate f std and f rob ID.

Original: Even after this ID calibration step, f std and f rob are very miscalibrated OOD (on P ood —see Appendix B. and Ovadia et al.
Revised: Even after this ID calibration step, f std and f rob are very miscalibrated OOD (on P ood —see Appendix B. and Ovadia et al.

Original: Our goal in this section is to build basic intuitions for when
Revised: Our goal in this section is to build basic intuitions for when

Original: ID-calibrated ensembles can get high OOD accuracy.
Revised: ID-calibrated ensembles can get high OOD accuracy.

Original: We draw inspiration from distribution shift benchmarks but examine simpliﬁed and stylized shifts.
Revised: We draw inspiration from distribution shift benchmarks but examine simpliﬁed and stylized shifts.

Original: A toy version of our analysis is visualized in Figure 2, where the standard model relies on spurious features that change out-of-distribution.
Revised: A toy version of our analysis is visualized in Figure 2, where the standard model relies on spurious features that change out-of-distribution.

Original: If these features are “suppressed” or “missing” OOD, then f ens does better than f std and f rob (Figure 2b).
Revised: If these features are “suppressed” or “missing” OOD, then f ens does better than f std and f rob (Figure 2b).

Original: However, if these features are anticorrelated OOD (correlated with the opposite label) then the accuracy of f ens is between f std and f rob (Figure 2c).
Revised: However, if these features are anticorrelated OOD (correlated with the opposite label) then the accuracy of f ens is between f std and f rob (Figure 2c).

Original: We begin by formalizing these shifts, and then analyze the accuracy under these shifts.
Revised: We begin by formalizing these shifts, and then analyze the accuracy under these shifts.

Original: Missing spurious.
Revised: Missing spurious.

Original: For our ﬁrst setting, we draw inspiration from some distribution shift benchmarks.
Revised: For our ﬁrst setting, we draw inspiration from some distribution shift benchmarks.

Original: Consider Breeds
Revised: Consider Breeds

Original: Living-17 [Santurkar et al., 2020] where the goal is to classify an image as one of 17 animal categories.
Revised: Living-17 [Santurkar et al., 2020] where the goal is to classify an image as one of 17 animal categories.

Original: The category ‘bear’ in the ID training data contains images of black bears and sloth bears while the OOD dataset has images of brown bears and polar bears.
Revised: The category ‘bear’ in the ID training data contains images of black bears and sloth bears while the OOD dataset has images of brown bears and polar bears.

Original: A standard model trained on the ID dataset might latch onto very speciﬁc features about sloth bears (for example the presence of a shaggy mane) which are simply missing in the OOD dataset ( f std ( x ) = 0 ).
Revised: A standard model trained on the ID dataset might latch onto very speciﬁc features about sloth bears (for example the presence of a shaggy mane) which are simply missing in the OOD dataset ( f std ( x ) = 0 ).

Original: A robust model could be trained to project these features out [Xie et al., 2021], so its predictions are still fairly reliable OOD.
Revised: A robust model could be trained to project these features out [Xie et al., 2021], so its predictions are still fairly reliable OOD.

Original: Deﬁnition 4.1 (missing spurious) .
Revised: Deﬁnition 4.1 (missing spurious) .

Original: A distribution P 0 has missing spurious features if for x ∼ P 0 , we have f std ( x ) = 0 almost surely and for some α ∈ R + , P 

Revised: A distribution P 0 has missing spurious features if for x ∼ P 0 , we have f std ( x ) = 0 almost surely and for some α ∈ R + , P

Original: ( Y = y | f rob ( X ) =
Revised: ( Y = y | f rob ( X ) =

Original: f rob ( x ))
Revised: f rob ( x ))

Original: =
Revised: =

Original: softmax ( αf rob ( x ))
Revised: softmax ( αf rob ( x ))

Original: y for all x ∈ X .
Revised: y for all x ∈ X .

Original: Suppressed features.
Revised: Suppressed features.

Original: In some datasets, such as satellite remote sensing datasets [Jean et al., 2016, Xie et al., 2021], a standard model can latch onto country-speciﬁc features that may be less prevalent OOD.
Revised: In some datasets, such as satellite remote sensing datasets [Jean et al., 2016, Xie et al., 2021], a standard model can latch onto country-speciﬁc features that may be less prevalent OOD.

Original: Deﬁnition 4.2 (suppressed features) .
Revised: Deﬁnition 4.2 (suppressed features) .

Original: A distribution P τ is said to have suppressed features if P τ ( Y = y | f ( X ) =
Revised: A distribution P τ is said to have suppressed features if P τ ( Y = y | f ( X ) =

Original: f
Revised: f

Original: ( x ))
Revised: ( x ))

Original: =
Revised: =

Original: softmax ( τf ( x ))
Revised: softmax ( τf ( x ))

Original: y for all x ∈ X and f ∈ { f std , f rob } , where τ ∈ R + .
Revised: y for all x ∈ X and f ∈ { f std , f rob } , where τ ∈ R + .

Original: Anticorrelated spurious.
Revised: Anticorrelated spurious.

Original: In some settings, the spurious feature can be correlated with a label ID but anticorrelated
Revised: In some settings, the spurious feature can be correlated with a label ID but anticorrelated

Original: OOD.
Revised: OOD.

Original: For example, in Waterbirds [Sagawa et al., 2020a], the task is to classify if an image contains a waterbird or a landbird where in the ID dataset, waterbirds are primarily featured with water backgrounds and landbirds with land backgrounds, but in the OOD datasets the backgrounds are  ﬂipped such that landbirds occur with water backgrounds and vice versa.
Revised: For example, in Waterbirds [Sagawa et al., 2020a], the task is to classify if an image contains a waterbird or a landbird where in the ID dataset, waterbirds are primarily featured with water backgrounds and landbirds with land backgrounds, but in the OOD datasets the backgrounds are [Figure] worse than f rob but ﬂipped such that landbirds occur with water backgrounds and vice versa.

Original: This motivates the ﬁnal deﬁnition of spurious shifts where the spurious features (background) are anticorrelated with the label OOD.
Revised: This motivates the ﬁnal deﬁnition of spurious shifts where the spurious features (background) are anticorrelated with the label OOD.

Original: Deﬁnition 4.3 (anticorrelated spurious) .
Revised: Deﬁnition 4.3 (anticorrelated spurious) .

Original: A distribution P adv is said to be anticorrelated spurious if for some α, β > 0 , for all x ∈ X ,
Revised: A distribution P adv is said to be anticorrelated spurious if for some α, β > 0 , for all x ∈ X ,

Original: P adv ( Y = y
Revised: P adv ( Y = y

Original: | f std ( x )) =
Revised: | f std ( x )) =

Original: softmax ( − βf std ( x ))
Revised: softmax ( − βf std ( x ))

Original: y (note the minus sign), while P adv ( Y = y
Revised: y (note the minus sign), while P adv ( Y = y

Original: | f rob ( x ))
Revised: | f rob ( x ))

Original: =
Revised: =

Original: softmax ( αf rob ( x ))
Revised: softmax ( αf rob ( x ))

Original: y .
Revised: y .

Original: [Figure]

Original: If the OOD distribution is a mixture of suppressed features and missing spurious features, then the ensemble f ens gets the best of both worlds.
Revised: If the OOD distribution is a mixture of suppressed features and missing spurious features, then the ensemble f ens gets the best of both worlds.

Original: Proposition 4.2.
Revised: Proposition 4.2.

Original: If the OOD contains a mixture of suppressed features and missing spurious features i.e., P ood = αP τ + (1 − α ) P 0 , and P τ and P 0 are class-balanced, then we have Err ood
Revised: If the OOD contains a mixture of suppressed features and missing spurious features i.e., P ood = αP τ + (1 − α ) P 0 , and P τ and P 0 are class-balanced, then we have Err ood

Original: ( f ens ) ≤
Revised: ( f ens ) ≤

Original: Err ood ( f rob ) and Err ood ( f ens ) ≤
Revised: Err ood ( f rob ) and Err ood ( f ens ) ≤

Original: Err ood ( f std ) .
Revised: Err ood ( f std ) .

Original: On the other hand, if the OOD distribution contains anticorrelated spurious features, then the accuracy of f ens is in between the standard and robust models.
Revised: On the other hand, if the OOD distribution contains anticorrelated spurious features, then the accuracy of f ens is in between the standard and robust models.

Original: Proposition 4.3.
Revised: Proposition 4.3.

Original: If spurious features are anticorrelated
Revised: If spurious features are anticorrelated

Original: OOD so that P ood
Revised: OOD so that P ood

Original: = P adv , then even if P adv is classbalanced, Err ood
Revised: = P adv , then even if P adv is classbalanced, Err ood

Original: ( f rob ) ≤ Err ood ( f ens ) ≤
Revised: ( f rob ) ≤ Err ood ( f ens ) ≤

Original: Err ood ( f std ) .
Revised: Err ood ( f std ) .

Original: The full proofs appear in Appendix A.
Revised: The full proofs appear in Appendix A.

Original:  DATASETS
Revised:  DATASETS

Original: We consider thirteen standard datasets, spanning multiple robustness interventions, types of shifts, and modalities (vision, language, time-series).
Revised: We consider thirteen standard datasets, spanning multiple robustness interventions, types of shifts, and modalities (vision, language, time-series).

Original: We ﬁrst describe the robustness interventions we consider, and then describe the datasets and types of shifts.
Revised: We ﬁrst describe the robustness interventions we consider, and then describe the datasets and types of shifts.

Original: All the datasets have been used by prior works on robustness, so we use their model checkpoints for reliable comparisons.
Revised: All the datasets have been used by prior works on robustness, so we use their model checkpoints for reliable comparisons.

Original: See Appendix B.1 for more details.
Revised: See Appendix B.1 for more details.

Original: Robustness interventions :
Revised: interventions :

Original: In-N-Out: Xie et al.
Revised: In-N-Out: Xie et al.

Original: [2021] use domain knowledge to project out spurious features in the input, and do an additional pretraining step.
Revised: [2021] use domain knowledge to project out spurious features in the input, and do an additional pretraining step.

Original: They call this robust model “aux-out” and show that it improves accuracy OOD, but hurts accuracy ID, compared to ERM. 2.
Revised: They call this robust model “aux-out” and show that it improves accuracy OOD, but hurts accuracy ID, compared to ERM. 2.

Original: Lightweight ﬁne-tuning: We take checkpoints from Kumar et al.
Revised: Lightweight ﬁne-tuning: We take checkpoints from Kumar et al.

Original: [2022] where the standard model ﬁne-tunes all parameters on an ID dataset, and the robust model only learns the top linear ‘head’ layer (which does better OOD but worse ID).
Revised: [2022] where the standard model ﬁne-tunes all parameters on an ID dataset, and the robust model only learns the top linear ‘head’ layer (which does better OOD but worse ID).

Original: Zero-shot language prompting: CLIP [Radford et al., 2021] is a multi-modal model that can predict the label of an image by comparing the image embedding, with prompts such as ‘photo of an apple’.
Revised: Zero-shot language prompting: CLIP [Radford et al., 2021] is a multi-modal model that can predict the label of an image by comparing the image embedding, with prompts such as ‘photo of an apple’.

Original: They show that this zero-shot language prompting approach (robust model) is more accurate OOD than ﬁne-tuning the entire model (standard model), although ID accuracy of the robust model is worse.
Revised: They show that this zero-shot language prompting approach (robust model is more accurate OOD than ﬁne -tuning the entire model (standard model), although ID accuracy of the robust model is worse.

Original: Group distributionally robust optimization (DRO) [Sagawa et al., 2020a]: Standard ERM models often latch on to spurious correlations in a dataset, such as image background color, or the occurrence of certain words in a sentence.
Revised: distributionally robust optimization (DRO) [Sagawa et al., 2020a]: Standard ERM models often latch on to spurious correlations in a dataset, such as image background color, or the occurrence of certain words in a sentence.

Original: Group DRO essentially upweights examples where this spurious correlation is not present.
Revised: Group DRO essentially upweights examples where this spurious correlation is not present.

Original: We consider three types of natural shifts (
Revised: CORAL [Sun and Saenko, 2016] aims to align feature representations across different domains, by penalizing differences in the means and covariances of the feature distributions.

Original: geography
Revised: The hope is that this generalizes better to OOD domains.

Original:  shifts, subpopulation shifts, style shifts), and we also consider adversarially synthesized “anticorrelated” spurious shifts.
Revised: We consider three types of natural shifts (geography shifts, subpopulation shifts, style shifts), and we also consider adversarially synthesized “anticorrelated” spurious shifts.

Original: Geography shifts.
Revised: Geography shifts.

Original: In geography shifts the ID data comes from some locations, and the OOD data comes from a different set of locations.
Revised: In geography shifts the ID data comes from some locations, and the OOD data comes from a different set of locations.

Original: One motivation is that in many developing areas training data may be unavailable because of monetary constraints [Jean et al., 2016].
Revised: One motivation is that in many developing areas training data may be unavailable because of monetary constraints [Jean et al., 2016].

Original: LandCover : The goal is to classify a satellite image into one of 6 land types (e.g., "grassland", "savannas").
Revised: LandCover [Rußwurm et al., 2020]: The goal is to classify a satellite image into one of 6 land types (e g., "grassland", "savannas").

Original: The ID data contains images from outside Africa, and the OOD data consists of images from Africa.
Revised: The ID data contains images from outside Africa, and the OOD data consists of images from Africa.

Original: Xie et al.
Revised: Xie et al.

Original: [2021] use the In-N-Out intervention.
Revised: [2021] use the In-N-Out intervention.

Original: Cropland : The goal is to predict whether a satellite image is of a cropland or not.
Revised: Cropland [Wang et al., 2020]: The goal is to predict whether a satellite image is of a cropland or not.

Original: The ID dataset contains images from Iowa, Missouri, and Illinois, and the OOD dataset contains images from Indiana and Kentucky.
Revised: The ID dataset contains images from Iowa, Missouri, and Illinois and the OOD dataset contains images from Indiana and Kentucky.

Original: Xie et al.
Revised: Xie et al.

Original: [2021] use the In-N-Out intervention.
Revised: [2021] use the In-NOut intervention.

Revised: iWildCam [Beery et al., 2020, Koh et al., 2021]:

Original: Subpopulation shifts.
Revised: The goal is to classify the species of an animal given a photo taken by a camera placed in the wild. The ID dataset consists of photos taken by over 200 cameras, and the OOD dataset consists of photos taken by heldout cameras placed in different locations.

Revised: Koh et al.

Revised: [2021] use the CORAL intervention.

Original:  In subpopulation shifts, the ID data contains a few sub-categories (e.g., black bear and sloth bear), and the OOD data contains different sub-categories (e.g., brown bears and polar bears) of the same parent category (e.g., bears).
Revised: Subpopulation shifts. In subpopulation shifts, the ID data contains a few sub-categories (e.g., black bear and sloth bear), and the OOD data contains different sub-categories (e.g., brown bears and polar bears) of the same parent category (e.g., bears).

Original: For both datasets below, Kumar et al.
Revised: For both datasets below, Kumar et al.

Original: [2022] use the lightweight ﬁne-tuning intervention.
Revised: [2022] use the lightweight ﬁne-tuning intervention.

Original: Living-17 : the goal is to classify an image as one of 17 animal categories such as “bear”, where the ID and OOD datasets have different species of bears.
Revised: Living-17 [Santurkar et al., 2020]: the goal is to classify an image as one of 17 animal categories such as “bear”, where the ID and OOD datasets have different species of bears.

Original: Entiy-30 : similar to Living-17, except the goal is to classify an image as one of 30 entity categories such as “food”, “motor vehicle”, and “insect”.
Revised: Entiy-30 [Santurkar et al., 2020]: similar to Living-17, except the goal is to classify an image as one ofentity categories such as “food”, “motor vehicle”, and “insect”.

Original: Style shifts.
Revised: Style shifts.

Original: In style shifts, the ID data has a certain style (e.g., sketches), and the OOD data has a different style (e.g., real photos, renditions).
Revised: In style shifts, the ID data has a certain style (e.g., sketches), and the OOD data has a different style (e.g., real photos, renditions).

Original: DomainNet [Peng et al., 2019]: a standard domain adaptation dataset.
Revised: DomainNet [Peng et al., 2019]: a standard domain adaptation dataset.

Original: Here, our ID dataset contains “sketch” images (e.g., drawings of apples, elephants, etc), and the OOD dataset contains “real” photos of the same categories.
Revised: Here, our ID dataset contains “sketch” images (e.g., drawings of apples, elephants, etc), and the OOD dataset contains “real” photos of the same categories.

Original: Kumar et al.
Revised: Kumar et al.

Original: [2022] use the lightweight ﬁne-tuning intervention.
Revised: [2022] use the lightweight ﬁne-tuning intervention.

Original: CelebA
Revised: CelebA

Original: [Liu et al., 2015]: the goal is to classify a portrait of a face as “male” or “female” - the ID dataset contains images of people without hats, and the OOD dataset contains images of people wearing hats (some facial features might be “suppressed” or “missing” with hats).
Revised: [Liu et al., 2015]: the goal is to classify a portrait of a face as “male” or “female” - the ID dataset contains images of people without hats, and the OOD dataset contains images of people wearing hats (some facial features might be “suppressed” or “missing” with hats).

Original: Xie et al.
Revised: Xie et al.

Original: [2021] use the In -N-Out intervention.
Revised: [2021] use the In-N-Out intervention.

Original: CIFAR->STL : standard domain adaptation dataset [French et al., 2018], where the ID is CIFAR-10 [Krizhevsky, 2009], and the OOD is 
Revised: CIFAR->STL : standard domain adaptation dataset [French et al., 2018], where the ID is CIFAR-10 [Krizhevsky, 2009], and the OOD is STL

Original: STL [Coates et al., 2011].
Revised: [Coates et al., 2011].

Original: The task is to classify an image into one of 10 categories such as “dog”, “cat”, or “airplane”.
Revised: The task is to classify an image into one of 10 categories such as “dog”, “cat”, or “airplane”.

Original: Kumar et al.
Revised: Kumar et al.

Original: [2022] use the lightweight ﬁne-tuning intervention.
Revised: [2022] use the lightweight ﬁne-tuning intervention.

Original: ImageNet [Russakovsky et al
Revised: ImageNet [Russakovsky et al.,

Original: : a large scale dataset where the goal is to classify an image into one of 1000 categories
Revised: ]: a large scale dataset where the goal is to classify an image into one of 1000 categories.

Original: Radford et al.
Revised: Radford et al.

Original:  use the zero-shot language prompting intervention.
Revised: [2021] use the zero-shot language prompting intervention.

Original: We evaluate on 3 standard OOD datasets: ImageNetV Recht et al
Revised: We evaluate on 3 standard OOD datasets: ImageNetV2 [Recht et al., 2019],

Original: , ImageNet-R
Revised: ImageNet-R

Original: Hendrycks et al., 2020], 
Revised: [Hendrycks et al., 2020], and

Original: and ImageNet-Sketch Wang et al., 
Revised: ImageNet-Sketch [Wang et al., 2019].

Original: Anticorrelated spurious shifts.
Revised: Anticorrelated spurious shifts.

Original: In these adversarially synthesized shifts, the ID dataset contains a feature that is correlated with a label, but this correlation is ﬂipped OOD.
Revised: In these adversarially synthesized shifts, the ID dataset contains a feature that is correlated with a label, but this correlation is ﬂipped OOD.

Original: Jones et al.
Revised: Jones et al.

Original:  use the group DRO intervention
Revised: [2021] use the group DRO intervention.

Original: Waterbirds [Sagawa et al., 2020a]: The goal is to classify an image as a “waterbird” or “landbird”.
Revised: Waterbirds [Sagawa et al., 2020a]: The goal is to classify an image as a “waterbird” or “landbird”.

Original: The dataset is synthetically constructed to have anticorrelated spurious features: “water” backgrounds are correlated with “waterbird” labels in the ID, but anticorrelated OOD.
Revised: The dataset is synthetically constructed to have anticorrelated spurious features: “water” backgrounds are correlated with “waterbird” labels in the ID, but anticorrelated OOD.

Original: MNLI [Williams et al ., 2018]: The goal is to predict whether a hypothesis is entailed, contradicted by, or neutral to an associated premise.
Revised: MNLI [Williams et al., 2018]: The goal is to predict whether a hypothesis is entailed, contradicted by, or neutral to an associated premise.

Original: Sagawa et al.
Revised: Sagawa et al.

Original: [2020a] partition the dataset so that “negation” words are correlated with the contradiction label ID
Revised: [2020a] partition the dataset so that “negation” words are correlated with the contradiction label ID

Original: but these words are anticorrelated with the contradiction label OOD
Revised: but these words are anticorrelated with the contradiction label OOD.

Original: CivilComments [Borkan et al. : The goal is to predict whether a comment is toxic or not.
Revised: CivilComments [Borkan et al., 2019]: The goal is to predict whether a comment is toxic or not.

Original: Jones et al.
Revised: Jones et al.

Original: [2021] partition the dataset so that in the ID split mentions of a Christian identity are correlated with nontoxic comments, but in the OOD split mentions of a Christian identity are correlated with a toxic comment.
Revised: [2021] partition the dataset so that in the ID split mentions of a Christian identity are correlated with nontoxic comments, but in the OOD split mentions of a Christian identity are correlated with a toxic comment.

Original: CivilComments is also used in Koh et al
Revised: CivilComments is also used in Koh et al.

Revised: [Table]

Original:  RESULTS
Revised:  RESULTS

Original: In Section 6.1, we show that ID-calibrated ensembles get the best of both worlds across the 10 natural shifts we consider, but not on the 3 adversarially synthesized anticorrelated spurious shifts, as predicted by our analysis in Section 4.
Revised: In Section 6. [Table] we show that ID-calibrated ensembles get the best of both worlds across the 11 natural shifts we consider, but not on the 3 adversarially synthesized anticorrelated spurious shifts, as predicted by our analysis in Section 4.

Original: ID-calibrated ensembles match or outperform a prior stateof-the art approach based on self-training [Xie et al., 2021], which requires additional unlabeled data.
Revised: ID-calibrated ensembles match or outperform a prior stateof-the art approach based on self-training [Xie et al., 2021], which requires additional unlabeled data.

Original: In Section 6.2, we show ablations of our method.
Revised: In Section 6.2, we show ablations of our method.

Original: Interestingly, we ﬁnd that a common approach of tuning the ensemble weights to optimize ID accuracy can lead to poor OOD performance.
Revised: Interestingly, we ﬁnd that a common approach of tuning the ensemble weights to optimize ID accuracy can lead to poor OOD performance.

Original: 6.1 MAIN RESULTS
Revised: 6.1 MAIN RESULTS

Original: Competitive with self-training.
Revised: Competitive with self-training.

Original: Xie et al.
Revised: Xie et al.

Original: [2021] propose self-training on unlabeled data to mitigate ID-OOD accuracy tradeoffs.
Revised: [2021] propose self-training on unlabeled data to mitigate ID-OOD accuracy tradeoffs.

Original: We run experiments on all 3 datasets they consider (Landcover, Cropland, CelebA), taking checkpoints from the ofﬁcial CodaLab implementation of Xie et al.
Revised: We run experiments on all 3 datasets they consider (Landcover, Cropland, CelebA), taking checkpoints from the ofﬁcial CodaLab implementation of Xie et al.

Original: Table 1 shows that ID-calibrated ensembles match or outperform self-training on all 3 of their datasets, both ID and
Revised: Table 1 shows that ID-calibrated ensembles match or outperform self-training on all 3 of their datasets, both ID and

Original: OOD.
Revised: OOD.

Original: We believe this is interesting because our method is simple and does not need additional unlabeled data (which, for example, the other datasets do not have).
Revised: We believe this is interesting because our method is simple and does not need additional unlabeled data (which, for example, the other datasets do not have).

Original: Strong ID and OOD accuracy.
Revised: Strong ID and OOD accuracy.

Original: Across the 10 natural shifts,
Revised: Across the 11 natural shifts,

Original: ID-calibrated ensembles get the best of both worlds, typically outperforming the standard and robust model both
Revised: ID-calibrated ensembles get the best of both worlds, typically outperforming the standard and robust model both

Original: [Table] [Table] ID (Table 2) and OOD (Table 3).
Revised: ID (Table 2) and OOD (Table 3).

Original: Averaged across the natural shift datasets, ID-calibrated ensembles get 91.1% ID (vs. 89.5% for the standard model and 87.5% for the robust model) and 75.3% OOD (vs. 73.2% for the robust model and 65.6% for the standard model).
Revised: Averaged across the natural shift datasets, ID-calibrated ensembles get 90.3% ID (vs. 88.7% for the standard model and 86.8% for the robust model) and 74.5% OOD (vs. 72.3% for the robust model and 65.2% for the standard model).

Original: The method works across the board—ID-calibrated ensembles achieve the best performance on 7/8 ID natural shifts, and on 9/10 OOD natural shifts.
Revised: The method works across the board—ID-calibrated ensembles achieve the best performance on 8/9 ID natural shifts, and on 10/11 OOD natural shifts.

Original: For the remaining two cases, DomainNet OOD and
Revised: For the remaining two cases, DomainNet OOD and

Original: CIFAR-10 ID, ID-calibrated ensembles close over 95% of the gap between the standard and robust model.
Revised: CIFAR-10 ID, ID-calibrated ensembles close over 95% of the gap between the standard and robust model.

Original: Shift type is important.
Revised: Shift type is important.

Original: Our analysis in Section 4 predicts that ID-calibrated ensembles do not work as well on anticorrelated spurious shifts, where a spurious feature is correlated with the label but anticorrelated OOD.
Revised: Our analysis in Section 4 predicts that ID-calibrated ensembles do not work as well on anticorrelated spurious shifts, where a spurious feature is correlated with the label but anticorrelated OOD.

Original: Indeed, in these cases the OOD accuracy of ID-calibrated ensembles is between the standard and robust model (Table 3).
Revised: Indeed, in these cases the OOD accuracy of ID-calibrated ensembles is between the standard and robust model (Table 3).

Original: Even so, averaged across all 13 datasets ID-calibrated ensembles do well and get 90.5% ID (vs. 89.1% for the standard model, 87.3% for the robust model) and 75.3% OOD (vs. 64.5% for the standard model, 75.5% for the robust model).
Revised: Even so, averaged across all 14 datasets ID-calibrated ensembles do well and get 90.0% ID (vs. 88.6% for the standard model, 86.9% for the robust model) and 74.7% OOD (vs. 64.3% for the standard model, 74.6% for the robust model).

Original: 6.2 ABLATIONS
Revised: 6.2 ABLATIONS

Original: Our proposed method is a simple combination of a calibrated robust and calibrated standard model.
Revised: Our proposed method is a simple combination of a calibrated robust and calibrated standard model.

Original: We vary the components of our method and try: (i) tuned ensembles without calibration, (ii) vanilla ensembles without calibration, and (iii) ensembles of two standard or two robust models.
Revised: We vary the components of our method and try: (i) tuned ensembles without calibration, (ii) vanilla ensembles without calibration, and (iii) ensembles of two standard or two robust models.

Original: Tuned ensembles do not mitigate tradeoffs.
Revised: Tuned ensembles do not mitigate tradeoffs.

Original: A natural way to ensemble the two models is “tuned ensembles”: choosing the ensemble weights to optimize accuracy on the ID validation set.
Revised: A natural way to ensemble the two models is “tuned ensembles”: choosing the ensemble weights to optimize accuracy on the ID validation set.

Original: This approach is also known as stacking, and has performed well on the Netﬂix prize and Kaggle competitions
Revised: This approach is also known as stacking, and has performed well on the Netﬂix prize and Kaggle competitions

Original: [Sill et al., 2009].
Revised: [Sill et al., 2009].

Original: Interestingly, we ﬁnd that tuned ensembles do not do very well OOD, getting an average accuracy of 72.6% across the 13 datasets (vs. 75.3% for
Revised: Interestingly, we ﬁnd that tuned ensembles do not do very well OOD, getting an average accuracy of 72.1% across the 14 datasets (vs. 74.7% for

Original: ID-calibrated ensembles).
Revised: ID-calibrated ensembles).

Original: The ID accuracies are similar— results for all datasets are in Table 4 (ID) and Table 5 (OOD).
Revised: The ID accuracies are similar— results for all datasets are in Table 4 (ID) and Table 5 (OOD).

Original: Calibration helps.
Revised: Calibration helps.

Original: ID-calibrated ensembles (calibration is only done on ID data) outperform vanilla ensembles, getting an average ID accuracy of 90.5% (vs. 89.9% for vanilla ensembles) and an average OOD accuracy of 75.3% (vs. 73.7% for vanilla ensembles).
Revised: ID-calibrated ensembles (calibration is only done on ID data) outperform vanilla ensembles, getting an average ID accuracy of 90.0% (vs. 89.4% for vanilla ensembles) and an average OOD accuracy of 74.7% (vs. 73.1% for vanilla ensembles).

Original: We show results for all datasets in Table 4 (ID) and Table 5 (OOD).[Table]
Revised: We show results for all datasets in Table 4 (ID) and Table 5 (OOD).

Original: Outperforms standard and robust ensembles.
Revised: Outperforms standard and robust ensembles.

Original: As a sanity check, Appendix B.2 shows that our method outperforms 1.
Revised: As a sanity check, Appendix B.2 shows that our method outperforms 1.

Original: ensembling two (calibrated) standard models, and 2.
Revised: ensembling two (calibrated) standard models, and 2.

Original: ensembling two (calibrated) robust models.
Revised: ensembling two (calibrated) robust models.

Original: Models are miscalibrated OOD.
Revised: Models are miscalibrated OOD.

Original: Even after ID calibration, we ﬁnd that the standard and robust models are not calibrated OOD, which matches prior work [Ovadia et al., 2019].
Revised: Even after ID calibration, we ﬁnd that the standard and robust models are not calibrated OOD, which matches prior work [Ovadia et al., [Table] [Table] [Table]

Original: We estimate the expected calibration error (ECE;
Revised: We estimate the expected calibration error (ECE;

Original: Equation 2 in Guo et al.
Revised: Equation 2 in Guo et al.

Original: Since we calibrated on
Revised: Since we calibrated on

Original: ID data, the ECE is low ID (1.4% for the standard model, 2.4% for the robust model; Table 8).
Revised: ID data, the ECE is low ID (1.6% for the standard model, 2.3% for the robust model; Table 8).

Original: However, the ECE is high OOD (11.9% for the standard model, 7.1% for the robust model; Table 9) Appendix B.3 shows that even the relative conﬁdence of the models can be wrong: the standard model can be more conﬁdent but less accurate OOD, after ID-calibration.
Revised: However, the ECE is high OOD (11.3% for the standard model, 6.8% for the robust model; Table 9) Appendix B.3 shows that even the relative conﬁdence of the models can be wrong: the standard model can be more conﬁdent but less accurate OOD, after ID-calibration.

Original: Nonetheless, ID-calibrated ensembles get the best of both worlds, and Section 4 gives some simple intuitions for why this can happen.
Revised: Nonetheless, ID-calibrated ensembles get the best of both worlds—see Section 4 for some simple intuitions for why this can happen.

Original:  RELATED WORKS AND DISCUSSION
Revised:  RELATED WORKS AND DISCUSSION

Original: Calibration.
Revised: Calibration.

Original: Calibration has been widely studied in machine learning [Naeini et al., 2014, Guo et al., 2017, Kumar et al., 2019], and applications such as meteorology [Murphy, 1973, DeGroot and Fienberg, 1983, Gneiting and Raftery, 2005], fairness [Hebert-Johnson et al., 2018], and healthcare [Jiang et al., 2012].
Revised: Calibration has been widely studied in machine learning [Naeini et al., 2014, Guo et al., 2017, Kumar et al., 2019], and applications such as meteorology [Murphy, 1973, DeGroot and Fienberg, 1983, Gneiting and Raftery, 2005], fairness [Hebert-Johnson et al., 2018], and healthcare [Jiang et al., 2012].

Original: Many of these works focus on the in-distribution (ID) setting, where models are calibrated on the same distribution that they are evaluated on.
Revised: Many of these works focus on the in-distribution (ID) setting, where models are calibrated on the same distribution that they are evaluated on.

Original: Ovadia et al.
Revised: Ovadia et al.

Original: [2019], Jones et al.
Revised: [2019], Jones et al.

Original: [2021] show that if we calibrate (e.g., via temperature scaling) a model ID, it still has poor uncertainties OOD.
Revised: [2021] show that if we calibrate (e.g., via temperature scaling) a model ID, it still has poor uncertainties OOD.

Original: However, we show that despite having poor uncertainties on traditional metrics, calibrated models can be combined effectively to mitigate ID-OOD tradeoffs. 
Revised: However, we show that despite having poor uncertainties on traditional metrics, calibrated models can be combined effectively to mitigate ID-OOD tradeoffs. Wald et al.

Revised: [2021] show that if a model is calibrated on many domains (domains > no. of features) in a linear setting, then the model is calibrated (and invariant) on new domains.

Revised: A key difference is that they require a large number of training domains, which may need to be annotated to ensure calibration across them, while we only require access to a single doamin.

Original: Ensembling.
Revised: Ensembling.

Original: Ensembling models is a common way to get an accuracy boost—typically the ensemble members are trained with a different random seed [Lakshminarayanan et al., 2017] or augmentation [Stickland and Murray, 2020].
Revised: Ensembling models is a common way to get an accuracy boost—typically the ensemble members are trained with a different random seed [Lakshminarayanan et al., 2017] or augmentation [Stickland and Murray, 2020].

Original: In the setting where the ensemble members mostly differ by random seeds or augmentations, prior work has shown that calibrating the members of an ensemble does not help [Wu and Gales, 2021, Ovadia et al., 2019].
Revised: In the setting where the ensemble members mostly differ by random seeds or augmentations, prior work has shown that calibrating the members of an ensemble does not help [Wu and Gales, 2021, Ovadia et al., 2019].

Original: However when we combine two very different models (standard and robust), calibration leads to clear improvements.
Revised: However when we combine two very different models (standard and robust), calibration leads to clear improvements.

Original: Mitigating ID-OOD tradeoffs.
Revised: Mitigating ID-OOD tradeoffs.

Original: Tradeoffs between ID and OOD accuracy are widely studied and prior work selftrains on large amounts of unlabeled data to mitigate such tradeoffs [Raghunathan et al., 2020, Xie et al., 2021, Khani and Liang, 2021].
Revised: Tradeoffs between ID and OOD accuracy are widely studied and prior work selftrains on large amounts of unlabeled data to mitigate such tradeoffs [Raghunathan et al., 2020, Xie et al., 2021, Khani and Liang, 2021].

Original: In contrast, our approach uses no extra unlabeled data and is a simple method where we just add up the model probabilities after a quick calibration step.
Revised: In contrast, our approach uses no extra unlabeled data and is a simple method where we just add up the model probabilities after a quick calibration step.

Original: In concurrent and independent work, [Wortsman et al., 2021] show that there exists a way to combine a CLIP zero-shot and ﬁne-tuned model to get good ID and OOD accuracy— however learning how to combine the models may require
Revised: In concurrent and independent work, [Wortsman et al., 2021] show that there exists a way to combine a CLIP zero-shot and ﬁne-tuned model to get good ID and OOD accuracy— however learning how to combine the models may require

Original: OOD data, which is not available.
Revised: OOD data, which is not available.

Original: We show that the natural way to learn how to weight ensemble members—selecting the weights to optimize ID accuracy—does not get the best of both worlds.
Revised: We show that the natural way to learn how to weight ensemble members—selecting the weights to optimize ID accuracy—does not get the best of both worlds.

Original: In addition, their approach does not directly apply to settings where the standard and robust models have different architectures, such as In-N-Out
Revised: In addition, their approach does not directly apply to settings where the standard and robust models have different architectures, such as In-N-Out

Original: [Xie et al., 2021].
Revised: [Xie et al., 2021].

Original: Discussion.
Revised: Conclusion and Future Work.

Original: In this paper, we show that ID-calibrated ensembles, a simple method of calibrating a standard and robust model only on ID data and then ensembling them, can eliminate the tradeoff between in-distribution (ID) and out-of-distribution (OOD) accuracy on a wide range of natural shifts.
Revised: In this paper, we show that ID-calibrated ensembles, a simple method of calibrating a standard and robust model only on ID data and then ensembling them, can eliminate the tradeoff between indistribution (ID) and out-of-distribution (OOD) accuracy on a wide range of natural shifts.

Original: We hope that this leads to more widespread use and deployment of robustness interventions.
Revised: We hope that this leads to more widespread use and deployment of robustness interventions.

Revised: ID-calibrated ensembles were competitive with prior work that used self-training, despite being simpler and not using additional unlabeled data.

Revised: However, self-training may have advantages: we believe self-training may potentially eliminate tradeoffs even in anticorrelated spurious settings—it could be interesting for future work to compare ensembling and self-training theoretically, and see if their beneﬁts are complementary.

Revised: Additionally, ID-calibrated ensembles require twice the compute of a single model (although for fairness, we compared with an ensemble of standard or robust models), while self-training gives us a single model.

Revised: One potential future direction is to see if ID-calibrated ensembles can be distilled into a single model (without additional unlabeled data).

Original:  ACKNOWLEDGEMENTS
Revised:  ACKNOWLEDGEMENTS

Original: This work was in part supported by the Open Philanthropy
Revised: This work was in part supported by the Open Philanthropy

Original: Project and NSF Award Grant No.  and NSF
Revised: Project and NSF Award Grant No. 1805310, and NSF

Original: IIS 2045685.
Revised: IIS 2045685.

Original: AR was supported by an Open Philanthropy
Revised: AR was supported by an Open Philanthropy

Original: Project AI Fellowship.
Revised: Project AI Fellowship.

Original: We would like to thank Robbie Jones and the anonymous reviewers for helpful comments on our draft.
Revised: We would like to thank Robbie Jones and the anonymous reviewers for helpful comments on our draft