Original: Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift
Revised: Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift

Original: Abstract
Revised: Abstract

Original: We often see undesirable tradeoffs in robust machine learning where out-of-distribution (OOD) accuracy is at odds with in-distribution (ID) accuracy.
Revised: We often see undesirable tradeoffs in robust machine learning where out-of-distribution (OOD) accuracy is at odds with in-distribution (ID) accuracy.

Original: A robust classiﬁer obtained via specialized techniques such as removing spurious features often has better OOD but worse ID accuracy compared to a standard classiﬁer trained via vanilla
Revised: A robust classiﬁer obtained via specialized techniques such as removing spurious features often has better OOD but worse ID accuracy compared to a standard classiﬁer trained via vanilla

Original: ERM.
Revised: ERM.

Original: In this paper, we ﬁnd that a simple approach of ensembling the standard and robust models, after calibrating on only ID data, outperforms prior state-of-the-art both ID and OOD.
Revised: In this paper, we ﬁnd that a simple approach of ensembling the standard and robust models, after calibrating on only ID data, outperforms prior state-of-the-art both ID and OOD.

Original: On ten natural distribution shift datasets, ID-calibrated ensembles get the best of both worlds: strong ID accuracy of the standard model and OOD accuracy of the robust model.
Revised: On ten natural distribution shift datasets, ID-calibrated ensembles get the best of both worlds: strong ID accuracy of the standard model and OOD accuracy of the robust model.

Original: We analyze this method in stylized settings, and identify two important conditions for ensembles to perform well on both ID and OOD: (1) standard and robust models should be calibrated (on ID data, because OOD data is unavailable), (2)
Revised: We analyze this method in stylized settings, and identify two important conditions for ensembles to perform well on both ID and OOD: (1) standard and robust models should be calibrated (on ID data, because OOD data is unavailable), (2)

Original: OOD has no anticorrelated spurious features.
Revised: OOD has no anticorrelated spurious features.

Original:  INTRODUCTION
Revised:  INTRODUCTION

Original: Machine learning models suffer large drops in accuracy in the presence of distribution shift where the test distribution is different from the training distribution.
Revised: Machine learning models suffer large drops in accuracy in the presence of distribution shift where the test distribution is different from the training distribution.

Original: As ML systems are widely deployed, it is important to train models that achieve good accuracy on unforeseen, out-of-distribution (OOD) examples.
Revised: As ML systems are widely deployed, it is important to train models that achieve good accuracy on unforeseen, out-of-distribution (OOD) examples.

Original: For example, models trained on medical data from a few hospitals should work well when deployed broadly [Zech et al., 2018, AlBadawy et al., 2018].
Revised: For example, models trained on medical data from a few hospitals should work well when deployed broadly [Zech et al., 2018, AlBadawy et al., 2018].

Original: Similarly, when predicting poverty from satellite imagery, models trained on data from a few countries should work well on all countries, particularly those where labels are scarce due to resource constraints [Jean et al., 2016].
Revised: Similarly, when predicting poverty from satellite imagery, models trained on data from a few countries should work well on all countries, particularly those where labels are scarce due to resource constraints [Jean et al., 2016].

Original: There has been a lot of research interest in tackling this robustness problem under various settings such as robustness to spurious correlations [Heinze-Deml and Meinshausen, 2017, Sagawa et al., 2020a], domain generalization [Arjovsky et al., 2019, Sun and Saenko, 2016], demographic shifts [Hashimoto et al., 2018, Duchi et al., 2019] among others.
Revised: There has been a lot of research interest in tackling this robustness problem under various settings such as robustness to spurious correlations [Heinze-Deml and Meinshausen, 2017, Sagawa et al., 2020a], domain generalization [Arjovsky et al., 2019, Sun and Saenko, 2016], demographic shifts [Hashimoto et al., 2018, Duchi et al., 2019] among others.

Original: Almost universally across these different settings, an unfortunate tradeoff arises: robustness interventions, such as removing spurious features, typically improve the OOD accuracy but cause a drop in the in-distribution (ID) accuracy on new test points from the original distribution.
Revised: Almost universally across these different settings, an unfortunate tradeoff arises: robustness interventions, such as removing spurious features, typically improve the OOD accuracy but cause a drop in the in-distribution (ID) accuracy on new test points from the original distribution.

Original: This tradeoff is a major hurdle in using the multitude of proposed robustness interventions.
Revised: This tradeoff is a major hurdle in using the multitude of proposed robustness interventions.

Original: In practice, most inputs are likely to be ID, so it is unsatisfactory to use a “robust” model that has high OOD performance but performs less accurately on these majority ID points.
Revised: In practice, most inputs are likely to be ID, so it is unsatisfactory to use a “robust” model that has high OOD performance but performs less accurately on these majority ID points.

Original: On the other hand, “standard models” (trained without robustness interventions) can fail in the presence of even small shifts, and it can be dangerous to use a standard model even if OOD points are rare.
Revised: On the other hand, “standard models” (trained without robustness interventions) can fail in the presence of even small shifts, and it can be dangerous to use a standard model even if OOD points are rare.

Original: In this work, we ask: is there a general strategy to harness the strengths of both the standard and robust model to achieve high accuracy both ID and OOD, without using OOD data?
Revised: In this work, we ask: is there a general strategy to harness the strengths of both the standard and robust model to achieve high accuracy both ID and OOD, without using OOD data?

Original: We ﬁnd that a simple approach of ﬁrst calibrating the standard and robust models on only ID data and then ensembling them, outperforms prior state-of-the-art both ID and OOD.
Revised: We ﬁnd that a simple approach of ﬁrst calibrating the standard and robust models on only ID data and then ensembling them, outperforms prior state-of-the-art both ID and OOD.

Original: As illustrated in Figure 1, across 10 natural distribution shift datasets (e.g. geographical shift, style shift, subpopulation shift), ID-calibrated ensembles get the best of both worlds : strong ID accuracy of the standard model and robust accuracy of the OOD models.
Revised: As illustrated in Figure 1, across 10 natural distribution shift datasets (e.g. geographical shift, style shift, subpopulation shift), ID-calibrated ensembles get the best of both worlds : strong ID accuracy of the standard model and robust accuracy of the OOD models.

Original: Averaged across these datasets,
Revised: Averaged across these datasets,

Original: ID-calibrated ensembles achieve an ID accuracy of 91.1% (vs. 89.5% for the standard model and 87.5% for the robust model) and OOD accuracy of 75.3% (vs. 65.6% for the standard model and 73.2% for the robust model).
Revised: ID-calibrated ensembles achieve an ID accuracy of 91.1% (vs. 89.5% for the standard model and 87.5% for the robust model) and OOD accuracy of 75.3% (vs. 65.6% for the standard model and 73.2% for the robust model).

Original: We then analyze when and why ID-calibrated ensembles can get the best of the standard and robust models, under a simplifying assumption that these models provide different and independent signals for the label.
Revised: We then analyze when and why ID-calibrated ensembles can get the best of the standard and robust models, under a simplifying assumption that these models provide different and independent signals for the label.

Original: If the standard and robust models are calibrated ID, the ensembling strategy for the best ID performance is to simply add the predictions of the two models (Proposition 4.1).
Revised: If the standard and robust models are calibrated ID, the ensembling strategy for the best ID performance is to simply add the predictions of the two models (Proposition 4.1).

Original: By the same idea, if the standard and robust models were also calibrated OOD, ensembling would achieve the optimal OOD accuracy.
Revised: By the same idea, if the standard and robust models were also calibrated OOD, ensembling would achieve the optimal OOD accuracy.

Original: However, since we only have ID training data, models can only be calibrated ID and ID calibration is not sufﬁcient for OOD calibration [Ovadia et al., 2019].
Revised: However, since we only have ID training data, models can only be calibrated ID and ID calibration is not sufﬁcient for OOD calibration [Ovadia et al., 2019].

Original: When can calibrated ensembles provide beneﬁts even without OOD calibration?
Revised: When can calibrated ensembles provide beneﬁts even without OOD calibration?

Original: In many natural distribution shifts, standard models pick up on predictive signals in the training data that are absent or suppressed under distribution shift— in these cases, we show that ID-calibrated ensembles obtain the best of both the standard and robust models OOD.
Revised: In many natural distribution shifts, standard models pick up on predictive signals in the training data that are absent or suppressed under distribution shift— in these cases, we show that ID-calibrated ensembles obtain the best of both the standard and robust models OOD.

Original: However, when spurious features become anticorrelated OOD (as is common when the distribution shift is adversarially synthesized), we show that the ensemble performance is in between the standard and robust models.
Revised: However, when spurious features become anticorrelated OOD (as is common when the distribution shift is adversarially synthesized), we show that the ensemble performance is in between the standard and robust models.

Original: We empirically validate this on three adversarially synthesized shifts [Sagawa et al., 2020a, Jones et al., 2021] where the spurious signals are anticorrelated OOD.
Revised: We empirically validate this on three adversarially synthesized shifts [Sagawa et al., 2020a, Jones et al., 2021] where the spurious signals are anticorrelated OOD.

Original: Finally, we compare to a number of other alternate ensembling strategies (for example, tuning the weights of the ensemble on ID validation data) and ﬁnd that they do not work as well as ID-calibrated ensembles.
Revised: Finally, we compare to a number of other alternate ensembling strategies (for example, tuning the weights of the ensemble on ID validation data) and ﬁnd that they do not work as well as ID-calibrated ensembles.

Original: To summarize, our main contributions are:
Revised: To summarize, our main contributions are:

Original: We revisit the classic idea of ensembling and propose
Revised: We revisit the classic idea of ensembling and propose

Original: a simple, general, and effective method to mitigate
Revised: a simple, general, and effective method to mitigate

Original: ID-OOD accuracy tradeoffs.
Revised: ID-OOD accuracy tradeoffs.

Original: This method outperforms prior approaches based on self-training, despite not using any additional unlabeled data.
Revised: This method outperforms prior approaches based on self-training, despite not using any additional unlabeled data.

Original: We prove that ensembles of calibrated models are optimal when the models provide independent signals about the label.
Revised: We prove that ensembles of calibrated models are optimal when the models provide independent signals about the label.

Original: However, models can only be calibrated ID from which we have training data, and ID calibration does not imply OOD calibration.
Revised: However, models can only be calibrated ID from which we have training data, and ID calibration does not imply OOD calibration.

Original: In simple and stylized settings, we identify conditions under which ID-calibrated ensembles achieve the best of standard and robust models in terms of OOD performance.
Revised: In simple and stylized settings, we identify conditions under which ID-calibrated ensembles achieve the best of standard and robust models in terms of OOD performance.

Original: We validate these insights experimentally and ﬁnd that
Revised: We validate these insights experimentally and ﬁnd that

Original: ID-calibrated ensembles eliminate tradeoffs under a variety of natural distribution shifts, but can fail when there are adversarially synthesized shifts.
Revised: ID-calibrated ensembles eliminate tradeoffs under a variety of natural distribution shifts, but can fail when there are adversarially synthesized shifts.

Original:  SETUP
Revised:  SETUP

Original: Consider a K -class classiﬁcation task, where the goal is to predict labels
Revised: Consider a K -class classiﬁcation task, where the goal is to predict labels

Original: y ∈
Revised: y ∈

Original: [ K ] corresponding to inputs x ∈ X .
Revised: [ K ] corresponding to inputs x ∈ X .

Original: Models.
Revised: Models.

Original: A model f : X → R K takes an input x ∈ X and outputs a score f ( x ) ∈ R K where f ( x ) i can be interpreted as the model’s “conﬁdence” that the label y is i .
Revised: A model f : X → R K takes an input x ∈ X and outputs a score f ( x ) ∈ R K where f ( x ) i can be interpreted as the model’s “conﬁdence” that the label y is i .

Original: The model outputs the label pred ( f ( x ))
Revised: The model outputs the label pred ( f ( x ))

Original: =
Revised: =

Original: arg max i f ( x ) i .
Revised: arg max i f ( x ) i .

Original: The conﬁdence scores can be normalized to sum to 1 (and interpreted as probabilities) using the softmax function, softmax ( f ( x ))
Revised: The conﬁdence scores can be normalized to sum to 1 (and interpreted as probabilities) using the softmax function, softmax ( f ( x ))

Original: i =
Revised: i =

Original: [Equation]
Revised: [Equation]

Original: ) for i ∈
Revised: ) for i ∈

Original: [ K ] .
Revised: [ K ] .

Original: Distributions and error.
Revised: Distributions and error.

Original: Let P id and P ood denote the underlying distribution of ( x, y ) pairs in-distribution (ID) and outof-distribution (OOD), respectively.
Revised: Let P id and P ood denote the underlying distribution of ( x, y ) pairs in-distribution (ID) and outof-distribution (OOD), respectively.

Original: We evaluate a model f on the fraction of times it makes a wrong prediction on
Revised: We evaluate a model f on the fraction of times it makes a wrong prediction on

Original: P id and P ood : Err id ( f ) =
Revised: P id and P ood : Err id ( f ) =

Original: E x,y ∼ P i
Revised: E x,y ∼ P i

Original: d
Revised: d

Original: [ pred ( f ( x )) (cid:54) =
Revised: [ pred ( f ( x )) (cid:54) =

Original: y ] and
Revised: y ] and

Original: Err ood
Revised: Err ood

Original: ( f ) =
Revised: ( f ) =

Original: E x,y ∼ P ood
Revised: E x,y ∼ P ood

Original: [ pred ( f ( x )) (cid:54) =
Revised: [ pred ( f ( x )) (cid:54) =

Original: y ] .
Revised: y ] .

Original: Standard and robust models.
Revised: Standard and robust models.

Original: A standard model f std is trained via empirical risk minimization where we minimize some loss on ID training data.
Revised: A standard model f std is trained via empirical risk minimization where we minimize some loss on ID training data.

Original: f std often relies on spurious correlations such as image background or occurence of certain words that are not necessarily predictive OOD.
Revised: f std often relies on spurious correlations such as image background or occurence of certain words that are not necessarily predictive OOD.

Original: In order to improve OOD performance, a robust model f rob is trained via a modiﬁed training procedure (robustness interventions) to discourage models from relying on ID-speciﬁc spurious features.
Revised: In order to improve OOD performance, a robust model f rob is trained via a modiﬁed training procedure (robustness interventions) to discourage models from relying on ID-speciﬁc spurious features.

Original: Formally, we have the following relationship between f std and f rob .
Revised: Formally, we have the following relationship between f std and f rob .

Original: [Equation]
Revised: [Equation]

Original: Require: in-distribution validation data { ( x val i , y val i ) } n val i =1 ∼ P id , standard and robust models f std , f rob : X → R K 1: f std =
Revised: Require: in-distribution validation data { ( x val i , y val i ) } n val i =1 ∼ P id , standard and robust models f std , f rob : X → R K 1: f std =

Original: Calibrate f std on in-distribution (ID) data 2: f rob =
Revised: Calibrate f std on in-distribution (ID) data 2: f rob =

Original: Calibrate f rob on in-distribution (ID) data 3: Return f ens ( x ) = 12 ( softmax ( f std ( x ))
Revised: Calibrate f rob on in-distribution (ID) data 3: Return f ens ( x ) = 12 ( softmax ( f std ( x ))

Original: + softmax ( f rob ( x )))
Revised: + softmax ( f rob ( x )))

Original: The precise robustness intervention depends on the task— in Section 4 we model the relationship between f std and f rob in a stylized setting amenable for analysis, and in Sectionwe describe what f std and f rob are in our real datasets.
Revised: The precise robustness intervention depends on the task— in Section 4 we model the relationship between f std and f rob in a stylized setting amenable for analysis, and in Sectionwe describe what f std and f rob are in our real datasets.

Original: Best of both worlds.
Revised: Best of both worlds.

Original: Our goal is to get the best of both worlds—a classiﬁer f ens that achieves the strong ID accuracy of the standard model, and OOD accuracy of the robust model:
Revised: Our goal is to get the best of both worlds—a classiﬁer f ens that achieves the strong ID accuracy of the standard model, and OOD accuracy of the robust model:

Original: [Equation]
Revised: [Equation]

Original: ID validation data.
Revised: ID validation data.

Original: To get the best of both worlds, we only allow access to ID validation data, { ( x val i , y val i ) } n val i =1 ∼ P id , for tuning hyperparameters.
Revised: To get the best of both worlds, we only allow access to ID validation data, { ( x val i , y val i ) } n val i =1 ∼ P id , for tuning hyperparameters.

Original: Following Xie et al.
Revised: Following Xie et al.

Original: Koh et al.
Revised: Koh et al.

Original: [2021], Gulrajani and Lopez-Paz [2020] we do not use any OOD validation data.
Revised: [2021], Gulrajani and Lopez-Paz [2020] we do not use any OOD validation data.

Original:  METHODS
Revised:  METHODS

Original: Proposed method: ID-calibrated ensembles.
Revised: Proposed method: ID-calibrated ensembles.

Original: Given a standard model f std and robust model f rob , we ﬁrst calibrate each model on the in-distribution validation data, and then add up their predictions (Algorithm 1).
Revised: Given a standard model f std and robust model f rob , we ﬁrst calibrate each model on the in-distribution validation data, and then add up their predictions (Algorithm 1).

Original: In our experiments, we calibrate using temperature scaling [Guo et al., 2017] with the cross-entropy loss (cid:96) :
Revised: In our experiments, we calibrate using temperature scaling [Guo et al., 2017] with the cross-entropy loss (cid:96) :

Original: [Equation]
Revised: [Equation]

Original: We then ensemble the two models by adding up the probabilities that they predict [Lakshminarayanan et al., 2017].
Revised: We then ensemble the two models by adding up the probabilities that they predict [Lakshminarayanan et al., 2017].

Original: [Equation]
Revised: [Equation]

Original: where the predicted label is pred ( f ens ( x ))
Revised: where the predicted label is pred ( f ens ( x ))

Original: [Equation]
Revised: [Equation]

Original: arg max y f ens
Revised: arg max y f ens

Original: [Equation]
Revised: [Equation]

Original: Ablations.
Revised: Ablations.

Original: In Section 6 we ablate each component of the method, for example the calibration step, way of combining the models, and we compare to (calibrated) ensembles of two standard models, or of two robust models.
Revised: In Section 6 we ablate each component of the method, for example the calibration step, way of combining the models, and we compare to (calibrated) ensembles of two standard models, or of two robust models.

Original:  INTUITIONS AND ANALYSIS
Revised:  INTUITIONS AND ANALYSIS

Original: In this section, we build basic intuitions for when and why
Revised: In this section, we build basic intuitions for when and why

Original: ID-calibrated ensembles can get the best of both worlds (good ID accuracy of f std and OOD accuracy of f rob ), even without using any OOD data.
Revised: ID-calibrated ensembles can get the best of both worlds (good ID accuracy of f std and OOD accuracy of f rob ), even without using any OOD data.

Original: We ﬁrst deﬁne a stylized setting, and then analyze the ID performance in Section 4.
Revised: We ﬁrst deﬁne a stylized setting, and then analyze the ID performance in Section 4.

Original: and OOD performance in Section 4.2.
Revised: and OOD performance in Section 4.2.

Original: Diverse features.
Revised: Diverse features.

Original: An intuitive and illustrative conceptual setting is the following: we assume inputs have some robust features (that are predictive both ID and OOD) and some spurious features (that are only predictive ID).
Revised: An intuitive and illustrative conceptual setting is the following: we assume inputs have some robust features (that are predictive both ID and OOD) and some spurious features (that are only predictive ID).

Original: f std relies on the spurious features while f rob relies on the robust features, both of which provide independent signals on the label.
Revised: f std relies on the spurious features while f rob relies on the robust features, both of which provide independent signals on the label.

Original: Assumption 4.1.
Revised: Assumption 4.1.

Original: We assume that f rob and f std have diverse features with respect to P id and P ood , that is, f rob ( x ) ⊥ f std ( x ) | y when ( x, y ) ∼ P for P ∈ { P id , P ood } (4.1)
Revised: We assume that f rob and f std have diverse features with respect to P id and P ood , that is, f rob ( x ) ⊥ f std ( x ) | y when ( x, y ) ∼ P for P ∈ { P id , P ood } (4.1)

Original: Connection with prior assumptions.
Revised: Connection with prior assumptions.

Original: The diverse features assumption is weaker than the assumptions in prior conceptual models of distribution shifts Chen et al.
Revised: The diverse features assumption is weaker than the assumptions in prior conceptual models of distribution shifts Chen et al.

Original: [2020], Sagawa et al.
Revised: [2020], Sagawa et al.

Original: [2020b], Nagarajan et al.
Revised: [2020b], Nagarajan et al.

Original: [2020] where robust and spurious features are disjoint parts of the input, each generated independently based on the label.
Revised: [2020] where robust and spurious features are disjoint parts of the input, each generated independently based on the label.

Original: In our setting, the features can be complicated functions of the inputs.
Revised: In our setting, the features can be complicated functions of the inputs.

Original: Ensemble.
Revised: Ensemble.

Original: The ensemble f ens simply adds up the predictions of the standard model f std and robust model f rob .
Revised: The ensemble f ens simply adds up the predictions of the standard model f std and robust model f rob .

Original: This is slightly different from Section 3, but is more amenable to analysis.
Revised: This is slightly different from Section 3, but is more amenable to analysis.

Original: [Equation]
Revised: [Equation]

Original: Class-balanced.
Revised: Class-balanced.

Original: For simplicity of exposition, we assume the class-balanced setting where every label P ( Y = y ) is equally likely.
Revised: For simplicity of exposition, we assume the class-balanced setting where every label P ( Y = y ) is equally likely.

Original: Formally, we say P is class-balanced if
Revised: Formally, we say P is class-balanced if

Original: P ( Y = y )
Revised: P ( Y = y )

Original: = 1 /K for all y ∈
Revised: = 1 /K for all y ∈

Original: [ K ] .
Revised: [ K ] .

Original: We analyze the general setting in Appendix A.
Revised: We analyze the general setting in Appendix A.

Original: 4.1 ID PERFORMANCE OF ENSEMBLES
Revised: 4.1 ID PERFORMANCE OF ENSEMBLES

Original: In this section, we show that if f std and f rob are calibrated with respect to P id , then the ensemble f
Revised: In this section, we show that if f std and f rob are calibrated with respect to P id , then the ensemble f

Original: ens is the best way to combine their predictions.
Revised: ens is the best way to combine their predictions.

Original: Since we have access to validation data from P id , the ﬁrst step of our method (Section 3) is to calibrate f std and f rob ID.
Revised: Since we have access to validation data from P id , the ﬁrst step of our method (Section 3) is to calibrate f std and f rob ID.

Original: We conclude the section by giving intuition for why this calibration step can be particularly important for deep neural networks.
Revised: We conclude the section by giving intuition for why this calibration step can be particularly important for deep neural networks.

Original: Intuitively, calibration means that the probability that a model outputs for an event reﬂects the true frequency of that event: if a model says 1,000 patients have the ﬂu with probability 0.1, approximately 100 of them should indeed have the ﬂu.
Revised: Intuitively, calibration means that the probability that a model outputs for an event reﬂects the true frequency of that event: if a model says 1,000 patients have the ﬂu with probability 0.1, approximately 100 of them should indeed have the ﬂu.

Original: Formally, we look at joint calibration [Murphy,
Revised: Formally, we look at joint calibration [Murphy,

Original: 1973, Brocker, 2009] where a model f is calibrated with respect to a distribution P if for all x ∈ X , y ∈
Revised: 1973, Brocker, 2009] where a model f is calibrated with respect to a distribution P if for all x ∈ X , y ∈

Original: [ K ] :
Revised: [ K ] :

Original: [Equation]
Revised: [Equation]

Original: The following proposition says that if f std and f rob are calibrated on P id , then f ens has lower error on P id than any other way of combining the two models—this also implies that f ens gets higher accuracy than f std and f rob .
Revised: The following proposition says that if f std and f rob are calibrated on P id , then f ens has lower error on P id than any other way of combining the two models—this also implies that f ens gets higher accuracy than f std and f rob .

Original: Proposition 4.1.
Revised: Proposition 4.1.

Original: Suppose that f std and f rob are calibrated with respect to P id , and that P id is class-balanced.
Revised: Suppose that f std and f rob are calibrated with respect to P id , and that P id is class-balanced.

Original: Let h :
Revised: Let h :

Original: R K
Revised: R K

Original: × R K → R K be an arbitrary function that combines the standard and robust model’s predictions, and let f h be the resulting classiﬁer: f h ( x ) =
Revised: × R K → R K be an arbitrary function that combines the standard and robust model’s predictions, and let f h be the resulting classiﬁer: f h ( x ) =

Original: h ( f std ( x ) , f rob ( x )) .
Revised: h ( f std ( x ) , f rob ( x )) .

Original: The ensemble is better than any such combination classiﬁer f h :
Revised: The ensemble is better than any such combination classiﬁer f h :

Original: Err id ( f ens ) ≤ Err id ( f h ) .
Revised: Err id ( f ens ) ≤ Err id ( f h ) .

Original: The proof of Proposition 4.1 is in Appendix A. Intuitively, since f rob ( x ) ⊥ f std ( x ) | y , the Bayes optimal predictor is proportional to multiplying their predicted probabilities, which is equal to adding logits (logits are in log space).
Revised: The proof of Proposition 4.1 is in Appendix A. Intuitively, since f rob ( x ) ⊥ f std ( x ) | y , the Bayes optimal predictor is proportional to multiplying their predicted probabilities, which is equal to adding logits (logits are in log space).

Original: Proposition 4.1 has an important condition: the two models must be calibrated.
Revised: Proposition 4.1 has an important condition: the two models must be calibrated.

Original: In practice, deep learning models are miscalibrated [Guo et al., 2017], so our ﬁrst step (Section 3) is to calibrate the models ID.
Revised: In practice, deep learning models are miscalibrated [Guo et al., 2017], so our ﬁrst step (Section 3) is to calibrate the models ID.

Original: We explain why the ID calibration step is important for deep neural networks.
Revised: We explain why the ID calibration step is important for deep neural networks.

Original: Why neural networks are miscalibrated.
Revised: Why neural networks are miscalibrated.

Original: Deep neural networks are typically large enough to memorize the training dataset, and are encouraged to magnify their weights (and hence their conﬁdence) to decrease the training loss [Mukhoti et al., 2020, Bai et al., 2021].
Revised: Deep neural networks are typically large enough to memorize the training dataset, and are encouraged to magnify their weights (and hence their conﬁdence) to decrease the training loss [Mukhoti et al., 2020, Bai et al., 2021].

Original: The extent of this miscalibration and overconﬁdence depends on the training procedure [Hendrycks et al., 2019, Desai and Durrett, 2020].
Revised: The extent of this miscalibration and overconﬁdence depends on the training procedure [Hendrycks et al., 2019, Desai and Durrett, 2020].

Original: In our case f std and f rob are trained in different ways and have different calibration (Appendix B.3).
Revised: In our case f std and f rob are trained in different ways and have different calibration (Appendix B.3).

Original: Why this miscalibration can hurt ensembling.
Revised: Why this miscalibration can hurt ensembling.

Original: Concretely, consider two models f (cid:48) std and f (cid:48) rob which are calibrated on
Revised: Concretely, consider two models f (cid:48) std and f (cid:48) rob which are calibrated on

Original: P id .
Revised: P id .

Original: Let f std ( x ) =
Revised: Let f std ( x ) =

Original: Mf (cid:48) std ( x ) for large M ∈ R (this magniﬁes its weights as discussed above), and let f rob = f (cid:48) rob .
Revised: Mf (cid:48) std ( x ) for large M ∈ R (this magniﬁes its weights as discussed above), and let f rob = f (cid:48) rob .

Original: f std and f (cid:48) std have the same predictions and therefore accuracy but f std is highly miscalibrated.
Revised: f std and f (cid:48) std have the same predictions and therefore accuracy but f std is highly miscalibrated.

Original: The ensemble is then given by f ens ( x ) =
Revised: The ensemble is then given by f ens ( x ) =

Original: Mf (cid:48) std ( x ) + f (cid:48) rob ( x ) .
Revised: Mf (cid:48) std ( x ) + f (cid:48) rob ( x ) .

Original: For very large M , f ens and f (cid:48)
Revised: For very large M , f ens and f (cid:48)

Original: std have the same predictions—this means that
Revised: std have the same predictions—this means that

Original: Err  ( f ens ) =
Revised: Err ood ( f ens ) =

Original: Err  ( f std ) <
Revised: Err ood ( f std ) <

Original: Err ood ( f rob ) , and so ensembling does not get the best of both worlds.
Revised: Err ood ( f rob ) , and so ensembling does not get the best of both worlds.

Original: 4.2 OOD PERFORMANCE OF ENSEMBLES
Revised: 4.2 OOD PERFORMANCE OF ENSEMBLES

Original: We showed that if f std and f rob are calibrated on a distribution P , then f ens is better than both models on P .
Revised: We showed that if f std and f rob are calibrated on a distribution P , then f ens is better than both models on P .

Original: However, our validation data is from P id , so we can only calibrate f std and f rob ID.
Revised: However, our validation data is from P id , so we can only calibrate f std and f rob ID.

Original: Even after this ID calibration step, f std and f rob are very miscalibrated OOD (on P ood —see Appendix B. and Ovadia et al.
Revised: Even after this ID calibration step, f std and f rob are very miscalibrated OOD (on P ood —see Appendix B. and Ovadia et al.

Original: Our goal in this section is to build basic intuitions for when
Revised: Our goal in this section is to build basic intuitions for when

Original: ID-calibrated ensembles can get high OOD accuracy.
Revised: ID-calibrated ensembles can get high OOD accuracy.

Original: We draw inspiration from distribution shift benchmarks but examine simpliﬁed and stylized shifts.
Revised: We draw inspiration from distribution shift benchmarks but examine simpliﬁed and stylized shifts.

Original: A toy version of our analysis is visualized in Figure 2, where the standard model relies on spurious features that change out-of-distribution.
Revised: A toy version of our analysis is visualized in Figure 2, where the standard model relies on spurious features that change out-of-distribution.

Original: If these features are “suppressed” or “missing” OOD, then f ens does better than f std and f rob (Figure 2b).
Revised: If these features are “suppressed” or “missing” OOD, then f ens does better than f std and f rob (Figure 2b).

Original: However, if these features are anticorrelated OOD (correlated with the opposite label) then the accuracy of f ens is between f std and f rob (Figure 2c).
Revised: However, if these features are anticorrelated OOD (correlated with the opposite label) then the accuracy of f ens is between f std and f rob (Figure 2c).

Original: We begin by formalizing these shifts, and then analyze the accuracy under these shifts.
Revised: We begin by formalizing these shifts, and then analyze the accuracy under these shifts.

Original: Missing spurious.
Revised: Missing spurious.

Original: For our ﬁrst setting, we draw inspiration from some distribution shift benchmarks.
Revised: For our ﬁrst setting, we draw inspiration from some distribution shift benchmarks.

Original: Consider Breeds
Revised: Consider Breeds

Original: Living-17 [Santurkar et al., 2020] where the goal is to classify an image as one of 17 animal categories.
Revised: Living-17 [Santurkar et al., 2020] where the goal is to classify an image as one of 17 animal categories.

Original: The category ‘bear’ in the ID training data contains images of black bears and sloth bears while the OOD dataset has images of brown bears and polar bears.
Revised: The category ‘bear’ in the ID training data contains images of black bears and sloth bears while the OOD dataset has images of brown bears and polar bears.

Original: A standard model trained on the ID dataset might latch onto very speciﬁc features about sloth bears (for example the presence of a shaggy mane) which are simply missing in the OOD dataset ( f std ( x ) = 0 ).
Revised: A standard model trained on the ID dataset might latch onto very speciﬁc features about sloth bears (for example the presence of a shaggy mane) which are simply missing in the OOD dataset ( f std ( x ) = 0 ).

Original: A robust model could be trained to project these features out [Xie et al., 2021], so its predictions are still fairly reliable OOD.
Revised: A robust model could be trained to project these features out [Xie et al., 2021], so its predictions are still fairly reliable OOD.

Original: Deﬁnition 4.1 (missing spurious) .
Revised: Deﬁnition 4.1 (missing spurious) .

Original: A distribution P 0 has missing spurious features if for x ∼ P 0 , we have f std ( x ) = 0 almost surely and for some α ∈ R + , P 

Revised: A distribution P 0 has missing spurious features if for x ∼ P 0 , we have f std ( x ) = 0 almost surely and for some α ∈ R + , P

Original: ( Y = y | f rob ( X ) =
Revised: ( Y = y | f rob ( X ) =

Original: f rob ( x ))
Revised: f rob ( x ))

Original: =
Revised: =

Original: softmax ( αf rob ( x ))
Revised: softmax ( αf rob ( x ))

Original: y for all x ∈ X .
Revised: y for all x ∈ X .

Original: Suppressed features.
Revised: Suppressed features.

Original: In some datasets, such as satellite remote sensing datasets [Jean et al., 2016, Xie et al., 2021], a standard model can latch onto country-speciﬁc features that may be less prevalent OOD.
Revised: In some datasets, such as satellite remote sensing datasets [Jean et al., 2016, Xie et al., 2021], a standard model can latch onto country-speciﬁc features that may be less prevalent OOD.

Original: Deﬁnition 4.2 (suppressed features) .
Revised: Deﬁnition 4.2 (suppressed features) .

Original: A distribution P τ is said to have suppressed features if P τ ( Y = y | f ( X ) =
Revised: A distribution P τ is said to have suppressed features if P τ ( Y = y | f ( X ) =

Original: f
Revised: f

Original: ( x ))
Revised: ( x ))

Original: =
Revised: =

Original: softmax ( τf ( x ))
Revised: softmax ( τf ( x ))

Original: y for all x ∈ X and f ∈ { f std , f rob } , where τ ∈ R + .
Revised: y for all x ∈ X and f ∈ { f std , f rob } , where τ ∈ R + .

Original: Anticorrelated spurious.
Revised: Anticorrelated spurious.

Original: In some settings, the spurious feature can be correlated with a label ID but anticorrelated
Revised: In some settings, the spurious feature can be correlated with a label ID but anticorrelated

Original: OOD.
Revised: OOD.

Original: For example, in Waterbirds [Sagawa et al., 2020a], the task is to classify if an image contains a waterbird or a landbird where in the ID dataset, waterbirds are primarily featured with water backgrounds and landbirds with land backgrounds, but in the OOD datasets the backgrounds are ﬂipped such that landbirds occur with water backgrounds and vice versa.
Revised: For example, in Waterbirds [Sagawa et al., 2020a], the task is to classify if an image contains a waterbird or a landbird where in the ID dataset, waterbirds are primarily featured with water backgrounds and landbirds with land backgrounds, but in the OOD datasets the backgrounds are ﬂipped such that landbirds occur with water backgrounds and vice versa.

Original: This motivates the ﬁnal deﬁnition of spurious shifts where the spurious features (background) are anticorrelated with the label OOD.
Revised: This motivates the ﬁnal deﬁnition of spurious shifts where the spurious features (background) are anticorrelated with the label OOD.

Original: Deﬁnition 4.3 (anticorrelated spurious) .
Revised: Deﬁnition 4.3 (anticorrelated spurious) .

Original: A distribution P adv is said to be anticorrelated spurious if for some α, β > 0 , for all x ∈ X ,
Revised: A distribution P adv is said to be anticorrelated spurious if for some α, β > 0 , for all x ∈ X ,

Original: P adv ( Y = y
Revised: P adv ( Y = y

Original: | f std ( x )) =
Revised: | f std ( x )) =

Original: softmax ( − βf std ( x ))
Revised: softmax ( − βf std ( x ))

Original: y (note the minus sign), while P adv ( Y = y
Revised: y (note the minus sign), while P adv ( Y = y

Original: | f rob ( x ))
Revised: | f rob ( x ))

Original: =
Revised: =

Original: softmax ( αf rob ( x ))
Revised: softmax ( αf rob ( x ))

Original: y .
Revised: y .

Original: [Figure]
Revised: [Figure]

Original: If the OOD distribution is a mixture of suppressed features and missing spurious features, then the ensemble f ens gets the best of both worlds.
Revised: If the OOD distribution is a mixture of suppressed features and missing spurious features, then the ensemble f ens gets the best of both worlds.

Original: Proposition 4.2.
Revised: Proposition 4.2.

Original: If the OOD contains a mixture of suppressed features and missing spurious features i.e., P ood = αP τ + (1 − α ) P 0 , and P τ and P 0 are class-balanced, then we have Err ood
Revised: If the OOD contains a mixture of suppressed features and missing spurious features i.e., P ood = αP τ + (1 − α ) P 0 , and P τ and P 0 are class-balanced, then we have Err ood

Original: ( f ens ) ≤
Revised: ( f ens ) ≤

Original: Err ood ( f rob ) and Err ood ( f ens ) ≤
Revised: Err ood ( f rob ) and Err ood ( f ens ) ≤

Original: Err ood ( f std ) .
Revised: Err ood ( f std ) .

Original: On the other hand, if the OOD distribution contains anticorrelated spurious features, then the accuracy of f ens is in between the standard and robust models.
Revised: On the other hand, if the OOD distribution contains anticorrelated spurious features, then the accuracy of f ens is in between the standard and robust models.

Original: Proposition 4.3.
Revised: Proposition 4.3.

Original: If spurious features are anticorrelated
Revised: If spurious features are anticorrelated

Original: OOD so that P ood
Revised: OOD so that P ood

Original: = P adv , then even if P adv is classbalanced, Err ood
Revised: = P adv , then even if P adv is classbalanced, Err ood

Original: ( f rob ) ≤ Err ood ( f ens ) ≤
Revised: ( f rob ) ≤ Err ood ( f ens ) ≤

Original: Err ood ( f std ) .
Revised: Err ood ( f std ) .

Original: The full proofs appear in Appendix A.
Revised: The full proofs appear in Appendix A.

Original:  DATASETS
Revised:  DATASETS

Original: We consider thirteen standard datasets, spanning multiple robustness interventions, types of shifts, and modalities (vision, language, time-series).
Revised: We consider thirteen standard datasets, spanning multiple robustness interventions, types of shifts, and modalities (vision, language, time-series).

Original: We ﬁrst describe the robustness interventions we consider, and then describe the datasets and types of shifts.
Revised: We ﬁrst describe the robustness interventions we consider, and then describe the datasets and types of shifts.

Original: All the datasets have been used by prior works on robustness, so we use their model checkpoints for reliable comparisons.
Revised: All the datasets have been used by prior works on robustness, so we use their model checkpoints for reliable comparisons.

Original: See Appendix B.1 for more details.
Revised: See Appendix B.1 for more details.

Original: Robustness interventions :
Revised: Robustness interventions :

Original: In-N-Out: Xie et al.
Revised: In-N-Out: Xie et al.

Original: [2021] use domain knowledge to project out spurious features in the input, and do an additional pretraining step.
Revised: [2021] use domain knowledge to project out spurious features in the input, and do an additional pretraining step.

Original: They call this robust model “aux-out” and show that it improves accuracy OOD, but hurts accuracy ID, compared to ERM. 2.
Revised: They call this robust model “aux-out” and show that it improves accuracy OOD, but hurts accuracy ID, compared to ERM. 2.

Original: Lightweight ﬁne-tuning: We take checkpoints from ?
Revised: Lightweight ﬁne-tuning: We take checkpoints from Kumar et al.

Original:  where the standard model ﬁne-tunes all parameters on an ID dataset, and the robust model only learns the top linear ‘head’ layer (which does better ID but worse OOD).
Revised: [2022] where the standard model ﬁne-tunes all parameters on an ID dataset, and the robust model only learns the top linear ‘head’ layer (which does better OOD but worse ID).

Original: Zero-shot language prompting: CLIP [Radford et al., 2021] is a multi-modal model that can predict the label of an image by comparing the image embedding, with prompts such as ‘photo of an apple’.
Revised: Zero-shot language prompting: CLIP [Radford et al., 2021] is a multi-modal model that can predict the label of an image by comparing the image embedding, with prompts such as ‘photo of an apple’.

Original: They show that this zero-shot language prompting approach (robust model) is more accurate OOD than ﬁne-tuning the entire model (standard model), although ID accuracy of the robust model is worse.4.
Revised: They show that this zero-shot language prompting approach (robust model) is more accurate OOD than ﬁne-tuning the entire model (standard model), although ID accuracy of the robust model is worse.

Original:  (DRO) [Sagawa et al., 2020a]: Standard ERM models often latch on to 
Revised: Group distributionally robust optimization (DRO) [Sagawa et al., 2020a]: Standard ERM models often latch on to spurious correlations in a dataset, such as image background color, or the occurrence of certain words in a sentence.

Original:  spurious correlations
Revised: Group DRO essentially upweights examples where this spurious correlation is not present.

Original: in a dataset, such as image
Revised: We consider three types of natural shifts (

Original: background
Revised: geography

Original: color, or the occurrence of certain words in a sentence Group DRO
Revised: shifts, subpopulation shifts, style shifts), and we

Original: essentially upweights
Revised: also consider adversarially

Original: examples where this spurious correlation is not present. We consider three types of natural shifts (geography shifts, subpopulation shifts, style shifts), and we also consider adversarially synthesized “anticorrelated” spurious shifts.
Revised: synthesized “anticorrelated” spurious shifts.

Original: Geography shifts.
Revised: Geography shifts.

Original: In geography shifts the ID data comes from some locations, and the OOD data comes from a different set of locations.
Revised: In geography shifts the ID data comes from some locations, and the OOD data comes from a different set of locations.

Original: One motivation is that in many developing areas training data may be unavailable because of monetary constraints [Jean et al., 2016].
Revised: One motivation is that in many developing areas training data may be unavailable because of monetary constraints [Jean et al., 2016].

Original: LandCover :
Revised: LandCover :

Original: The goal is to classify a satellite image into one of 6 land types (e.g., "grassland", "savannas").
Revised: The goal is to classify a satellite image into one of 6 land types (e.g., "grassland", "savannas").

Original: The ID data contains images from outside Africa, and the OOD data consists of images from Africa.
Revised: The ID data contains images from outside Africa, and the OOD data consists of images from Africa.

Original: Xie et al.
Revised: Xie et al.

Original: [2021] use the In-N-Out intervention.
Revised: [2021] use the In-N-Out intervention.

Original: Cropland :
Revised: Cropland :

Original: The goal is to predict whether a satellite image is of a cropland or not.
Revised: The goal is to predict whether a satellite image is of a cropland or not.

Original: The ID dataset contains images from Iowa, Missouri, and Illinois, and the OOD dataset contains images from Indiana and Kentucky.
Revised: The ID dataset contains images from Iowa, Missouri, and Illinois, and the OOD dataset contains images from Indiana and Kentucky.

Original: Xie et al.
Revised: Xie et al.

Original: [2021] use the In-N-Out intervention.
Revised: [2021] use the In-N-Out intervention.

Original: Subpopulation shifts.
Revised: Subpopulation shifts.

Original: In subpopulation shifts, the ID data contains a few sub-categories (e.g., black bear and sloth bear), and the OOD data contains different sub-categories (e.g., brown bears and polar bears) of the same parent category (e.g., bears).
Revised: In subpopulation shifts, the ID data contains a few sub-categories (e.g., black bear and sloth bear), and the OOD data contains different sub-categories (e.g., brown bears and polar bears) of the same parent category (e.g., bears).

Original: For both datasets below, ? use the lightweight ﬁne-tuning intervention.
Revised: For both datasets below, Kumar et al. [2022] use the lightweight ﬁne-tuning intervention.

Original: Living-17 :
Revised: Living-17 :

Original: the goal is to classify an image as one of 17 animal categories such as “bear”, where the ID and OOD datasets have different species of bears.
Revised: the goal is to classify an image as one of 17 animal categories such as “bear”, where the ID and OOD datasets have different species of bears.

Original: Entiy-30 : similar to Living-17, except the goal is to classify an image as one of 30 entity categories such as “food”, “motor vehicle”, and “insect”.
Revised: Entiy-30 : similar to Living-17, except the goal is to classify an image as one of 30 entity categories such as “food”, “motor vehicle”, and “insect”.

Original: Style shifts.
Revised: Style shifts.

Original: In style shifts, the ID data has a certain style (e.g., sketches), and the OOD data has a different style (e.g., real photos, renditions).
Revised: In style shifts, the ID data has a certain style (e.g., sketches), and the OOD data has a different style (e.g., real photos, renditions).

Original: DomainNet [Peng et al., 2019]: a standard domain adaptation dataset.
Revised: DomainNet [Peng et al., 2019]: a standard domain adaptation dataset.

Original: Here, our ID dataset contains “sketch” images (e.g., drawings of apples, elephants, etc), and the OOD dataset contains “real” photos of the same categories. ?
Revised: Here, our ID dataset contains “sketch” images (e.g., drawings of apples, elephants, etc), and the OOD dataset contains “real” photos of the same categories. Kumar et al.

Original:  use the lightweight ﬁne-tuning intervention.
Revised: [2022] use the lightweight ﬁne-tuning intervention.

Original: CelebA
Revised: CelebA

Original: [Liu et al., 2015]: the goal is to classify a portrait of a face as “male” or “female” -the ID dataset contains images of people without hats,
Revised: [Liu et al., 2015]: the goal is to classify a portrait of a face as “male” or “female” -

Original:  and the OOD dataset contains images of people wearing hats (some facial features might be “suppressed” or “missing” with hats).
Revised: the ID dataset contains images of people without hats, and the OOD dataset contains images of people wearing hats (some facial features might be “suppressed” or “missing” with hats).

Original: Xie et al.
Revised: Xie et al.

Original: [2021] use the In-N-Out intervention.
Revised: [2021] use the In -N-Out intervention.

Original: CIFAR->STL : standard domain adaptation
Revised: CIFAR->STL : standard domain adaptation

Original: dataset [French et al., 2018], where the ID is
Revised: dataset [French et al., 2018], where the ID is

Original: CIFAR-10 [Krizhevsky, 2009], and the OOD is
Revised: CIFAR-10 [Krizhevsky, 2009], and the OOD is

Original: STL [Coates et al., 2011].
Revised: STL [Coates et al., 2011].

Original: The task is to classify an image into one of 10 categories such as “dog”, “cat”, or “airplane”. ?
Revised: The task is to classify an image into one of 10 categories such as “dog”, “cat”, or “airplane”. Kumar et al.

Original:  use the lightweight ﬁne-tuning intervention.
Revised: [2022] use the lightweight ﬁne-tuning intervention.

Original: ImageNet 
Revised: ImageNet [Russakovsky et al

Original: [Russakovsky et al., 2015]: a large scale dataset where the goal is to classify an image into one of 1000 categories.
Revised: : a large scale dataset where the goal is to classify an image into one of 1000 categories

Original: Radford et al.
Revised: Radford et al.

Original: [2021] use the zero-shot language prompting intervention.
Revised: use the zero-shot language prompting intervention.

Original: We evaluate on 3 standard OOD datasets: ImageNetV2 [Recht et al., 2019],
Revised: We evaluate on 3 standard OOD datasets: ImageNetV Recht et al

Original:  ImageNet-R
Revised: , ImageNet-R

Original: [Hendrycks et al., 2020],and
Revised: Hendrycks et al., 2020],

Original:  ImageNet-Sketch [Wang et al.,2019].
Revised: and ImageNet-Sketch Wang et al.,

Original: Anticorrelated spurious shifts.
Revised: Anticorrelated spurious shifts.

Original: In these

Original: adversarially syn-

Original: thesized shifts , the ID dataset contains a feature that is corre - lated with a label, but this correlation is ﬂipped OOD.
Revised: In these adversarially synthesized shifts, the ID dataset contains a feature that is correlated with a label, but this correlation is ﬂipped OOD.

Original: Jones et al.
Revised: Jones et al.

Original: [2021] use the group DRO intervention.
Revised: use the group DRO intervention

Original: Waterbirds [Sagawa et al., 2020a]: The goal is to classify an image as a “waterbird” or “landbird”.
Revised: Waterbirds [Sagawa et al., 2020a]: The goal is to classify an image as a “waterbird” or “landbird”.

Original: The dataset is synthetically constructed to have anticorrelated spurious features: “water” backgrounds are correlated with “waterbird” labels in the ID, but anticorrelated OOD.
Revised: The dataset is synthetically constructed to have anticorrelated spurious features: “water” backgrounds are correlated with “waterbird” labels in the ID, but anticorrelated OOD.

Original: MNLI Williams
Revised: MNLI [Williams

Original: et al., 2018]
Revised: et al

Original: : The goal is to predict whether a hypothesis is entailed, contradicted by, or neutral to an associated premise.
Revised: ., 2018]: The goal is to predict whether a hypothesis is entailed, contradicted by, or neutral to an associated premise.

Original: Sagawa et al.
Revised: Sagawa et al.

Original: [2020a] partition the dataset so that “negation” words are correlated with the contradiction label ID
Revised: [2020a] partition the dataset so that “negation” words are correlated with the contradiction label ID

Original: but these words are anticorrelated with the contradiction label OOD.
Revised: but these words are anticorrelated with the contradiction label OOD

Original: CivilComments 
Revised: CivilComments [Borkan et al.

Original: [Borkan et al., 2019]: The goal is to predict whether a comment is toxic or not.
Revised: : The goal is to predict whether a comment is toxic or not.

Original: Jones et al.
Revised: Jones et al.

Original: [2021] partition the dataset so that in the ID split mentions of a Christian identity are correlated with nontoxic comments, but in the OOD split mentions of a Christian identity are correlated with a toxic comment.
Revised: [2021] partition the dataset so that in the ID split mentions of a Christian identity are correlated with nontoxic comments, but in the OOD split mentions of a Christian identity are correlated with a toxic comment.

Original: CivilComments is also used in Koh et al.
Revised: CivilComments is also used in Koh et al

Original:  RESULTS
Revised:  RESULTS

Original: In Section 6.1, we show that ID-calibrated ensembles get the best of both worlds across the 10 natural shifts we consider, but not on the 3 adversarially synthesized anticorrelated spurious shifts, as predicted by our analysis in Section 4.
Revised: In Section 6.1, we show that ID-calibrated ensembles get the best of both worlds across the 10 natural shifts we consider, but not on the 3 adversarially synthesized anticorrelated spurious shifts, as predicted by our analysis in Section 4.

Original: ID-calibrated ensembles match or outperform a prior stateof-the art approach based on self-training [Xie et al., 2021], which requires additional unlabeled data.
Revised: ID-calibrated ensembles match or outperform a prior stateof-the art approach based on self-training [Xie et al., 2021], which requires additional unlabeled data.

Original: In Section 6.2, we show ablations of our method.
Revised: In Section 6.2, we show ablations of our method.

Original: Interestingly, we ﬁnd that a common approach of tuning the ensemble weights to optimize ID accuracy can lead to poor OOD performance.
Revised: Interestingly, we ﬁnd that a common approach of tuning the ensemble weights to optimize ID accuracy can lead to poor OOD performance.

Original: 6.1 MAIN RESULTS
Revised: 6.1 MAIN RESULTS

Original: Competitive with self-training.
Revised: Competitive with self-training.

Original: Xie et al.
Revised: Xie et al.

Original: [2021] propose self-training on unlabeled data to mitigate ID-OOD accuracy tradeoffs.
Revised: [2021] propose self-training on unlabeled data to mitigate ID-OOD accuracy tradeoffs.

Original: We run experiments on all 3 datasets they consider (Landcover, Cropland, CelebA), taking checkpoints from the ofﬁcial CodaLab implementation of Xie et al.
Revised: We run experiments on all 3 datasets they consider (Landcover, Cropland, CelebA), taking checkpoints from the ofﬁcial CodaLab implementation of Xie et al.

Original: Table 1 shows that ID-calibrated ensembles match or outperform self-training on all 3 of their datasets, both ID and
Revised: Table 1 shows that ID-calibrated ensembles match or outperform self-training on all 3 of their datasets, both ID and

Original: OOD.
Revised: OOD.

Original: We believe this is interesting because our method is simple and does not need additional unlabeled data (which, for example, the other datasets do not have).
Revised: We believe this is interesting because our method is simple and does not need additional unlabeled data (which, for example, the other datasets do not have).

Original: Strong ID and OOD accuracy.
Revised: Strong ID and OOD accuracy.

Original: Across the 10 natural shifts,
Revised: Across the 10 natural shifts,

Original: ID-calibrated ensembles get the best of both worlds, typically outperforming the standard and robust model both
Revised: ID-calibrated ensembles get the best of both worlds, typically outperforming the standard and robust model both

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ID (Table 2) and OOD (Table 3).
Revised: ID (Table 2) and OOD (Table 3).

Original: Averaged across the natural shift datasets, ID-calibrated ensembles get 91.1% ID (vs. 89.5% for the standard model and 87.5% for the robust model) and 75.3% OOD (vs. 73.2% for the robust model and 65.6% for the standard model).
Revised: Averaged across the natural shift datasets, ID-calibrated ensembles get 91.1% ID (vs. 89.5% for the standard model and 87.5% for the robust model) and 75.3% OOD (vs. 73.2% for the robust model and 65.6% for the standard model).

Original: The method works across the board—ID-calibrated ensembles achieve the best performance on 7/8 ID natural shifts, and on 9/10 OOD natural shifts.
Revised: The method works across the board—ID-calibrated ensembles achieve the best performance on 7/8 ID natural shifts, and on 9/10 OOD natural shifts.

Original: For the remaining two cases, DomainNet OOD and
Revised: For the remaining two cases, DomainNet OOD and

Original: CIFAR-10 ID, ID-calibrated ensembles close over 95% of the gap between the standard and robust model.
Revised: CIFAR-10 ID, ID-calibrated ensembles close over 95% of the gap between the standard and robust model.

Original: Shift type is important.
Revised: Shift type is important.

Original: Our analysis in Section 4 predicts that ID-calibrated ensembles do not work as well on anticorrelated spurious shifts, where a spurious feature is correlated with the label but anticorrelated OOD.
Revised: Our analysis in Section 4 predicts that ID-calibrated ensembles do not work as well on anticorrelated spurious shifts, where a spurious feature is correlated with the label but anticorrelated OOD.

Original: Indeed, in these cases the OOD accuracy of ID-calibrated ensembles is between the standard and robust model (Table 3).
Revised: Indeed, in these cases the OOD accuracy of ID-calibrated ensembles is between the standard and robust model (Table 3).

Original: Even so, averaged across all 13 datasets ID-calibrated ensembles do well and get 90.5% ID (vs. 89.1% for the standard model, 87.3% for the robust model) and 75.3% OOD (vs. 64.5% for the standard model, 75.5% for the robust model).
Revised: Even so, averaged across all 13 datasets ID-calibrated ensembles do well and get 90.5% ID (vs. 89.1% for the standard model, 87.3% for the robust model) and 75.3% OOD (vs. 64.5% for the standard model, 75.5% for the robust model).

Original: 6.2 ABLATIONS
Revised: 6.2 ABLATIONS

Original: Our proposed method is a simple combination of a calibrated robust and calibrated standard model.
Revised: Our proposed method is a simple combination of a calibrated robust and calibrated standard model.

Original: We vary the components of our method and try: (i) tuned ensembles without calibration, (ii) vanilla ensembles without calibration, and (iii) ensembles of two standard or two robust models.
Revised: We vary the components of our method and try: (i) tuned ensembles without calibration, (ii) vanilla ensembles without calibration, and (iii) ensembles of two standard or two robust models.

Original: Tuned ensembles do not mitigate tradeoffs.
Revised: Tuned ensembles do not mitigate tradeoffs.

Original: A natural way to ensemble the two models is “tuned ensembles”: choosing the ensemble weights to optimize accuracy on the ID validation set.
Revised: A natural way to ensemble the two models is “tuned ensembles”: choosing the ensemble weights to optimize accuracy on the ID validation set.

Original: This approach is also known as stacking, and has performed well on the Netﬂix prize and Kaggle competitions[ ? ].
Revised: This approach is also known as stacking, and has performed well on the Netﬂix prize and Kaggle competitions

Original:  Interestingly, we ﬁnd that tuned ensembles do not do very well OOD, getting an average accuracy of 72.6% across the 13 datasets (vs. 75.3% for ID-calibrated ensembles).
Revised: [Sill et al., 2009]. Interestingly, we ﬁnd that tuned ensembles do not do very well OOD, getting an average accuracy of 72.6% across the 13 datasets (vs. 75.3% for ID-calibrated ensembles).

Original: The ID accuracies are similar—results for all datasets are in Table 4 (ID) and Table 5 (OOD).
Revised: The ID accuracies are similar— results for all datasets are in Table 4 (ID) and Table 5 (OOD).

Original: Calibration helps.
Revised: Calibration helps.

Original: ID-calibrated ensembles (calibration is only done on ID data) outperform vanilla ensembles, getting an average ID accuracy of 90.5% (vs. 89.9% for vanilla ensembles) and an average OOD accuracy of 75.3% (vs. 73.7% for vanilla ensembles).
Revised: ID-calibrated ensembles (calibration is only done on ID data) outperform vanilla ensembles, getting an average ID accuracy of 90.5% (vs. 89.9% for vanilla ensembles) and an average OOD accuracy of 75.3% (vs. 73.7% for vanilla ensembles).

Original: We show results for all datasets in Table 4 (ID) and Table 5 (OOD).
Revised: We show results for all datasets in Table 4 (ID) and Table 5 (OOD).

Original: [Table]
Revised: [Table]

Original: Outperforms standard and robust ensembles.
Revised: Outperforms standard and robust ensembles.

Original: As a sanity check, Appendix B.2 shows that our method outperforms 1.
Revised: As a sanity check, Appendix B.2 shows that our method outperforms 1.

Original: ensembling two (calibrated) standard models, and 2.
Revised: ensembling two (calibrated) standard models, and 2.

Original: ensembling two (calibrated) robust models.
Revised: ensembling two (calibrated) robust models.

Original: Models are miscalibrated OOD.
Revised: Models are miscalibrated OOD.

Original: Even after ID calibration, we ﬁnd that the standard and robust models are not calibrated OOD, which matches prior work [Ovadia et al., 2019].
Revised: Even after ID calibration, we ﬁnd that the standard and robust models are not calibrated OOD, which matches prior work [Ovadia et al., 2019].

Original: We estimate the expected calibration error (ECE;
Revised: We estimate the expected calibration error (ECE;

Original: Equation 2 in Guo et al.
Revised: Equation 2 in Guo et al.

Original: Since we calibrated on
Revised: Since we calibrated on

Original: ID data, the ECE is low ID (1.4% for the standard model, 2.4% for the robust model; Table 8).
Revised: ID data, the ECE is low ID (1.4% for the standard model, 2.4% for the robust model; Table 8).

Original: However, the ECE is high OOD (11.9% for the standard model, 7.1% for the robust model; Table 9) Appendix B.3 shows that even the relative conﬁdence of the models can be wrong: the standard model can be more conﬁdent but less accurate OOD, after
Revised: However, the ECE is high OOD (11.9% for the standard model, 7.1% for the robust model; Table 9) Appendix B.3 shows that even the relative conﬁdence of the models can be wrong: the standard model can be more conﬁdent but less accurate OOD, after

Original: ID-calibration.
Revised: ID-calibration.

Original: Nonetheless, ID-calibrated ensembles get the best of both worlds, and Section 4 gives some simple intuitions for why this can happen.
Revised: Nonetheless, ID-calibrated ensembles get the best of both worlds, and Section 4 gives some simple intuitions for why this can happen.

Original:  RELATED WORKS AND DISCUSSION
Revised:  RELATED WORKS AND DISCUSSION

Original: Calibration.
Revised: Calibration.

Original: Calibration has been widely studied in machine learning [Naeini et al., 2014, Guo et al., 2017, Kumar et al., 2019], and applications such as meteorology [Murphy, 1973, DeGroot and Fienberg, 1983, Gneiting and Raftery, 2005], fairness [Hebert-Johnson et al., 2018], and healthcare [Jiang et al., 2012].
Revised: Calibration has been widely studied in machine learning [Naeini et al., 2014, Guo et al., 2017, Kumar et al., 2019], and applications such as meteorology [Murphy, 1973, DeGroot and Fienberg, 1983, Gneiting and Raftery, 2005], fairness [Hebert-Johnson et al., 2018], and healthcare [Jiang et al., 2012].

Original: Many of these works focus on the in-distribution (ID) setting, where models are calibrated on the same distribution that they are evaluated on.
Revised: Many of these works focus on the in-distribution (ID) setting, where models are calibrated on the same distribution that they are evaluated on.

Original: Ovadia et al.
Revised: Ovadia et al.

Original: [2019], Jones et al.
Revised: [2019], Jones et al.

Original: [2021] show that if we calibrate (e.g., via temperature scaling) a model ID, it still has poor uncertainties OOD.
Revised: [2021] show that if we calibrate (e.g., via temperature scaling) a model ID, it still has poor uncertainties OOD.

Original: However, we show that despite having poor uncertainties on traditional metrics, calibrated models can be combined effectively to mitigate ID-OOD tradeoffs.
Revised: However, we show that despite having poor uncertainties on traditional metrics, calibrated models can be combined effectively to mitigate ID-OOD tradeoffs.

Original: Ensembling.
Revised: Ensembling.

Original: Ensembling models is a common way to get an accuracy boost—typically the ensemble members are trained with a different random seed [Lakshminarayanan et al., 2017] or augmentation [Stickland and Murray, 2020].
Revised: Ensembling models is a common way to get an accuracy boost—typically the ensemble members are trained with a different random seed [Lakshminarayanan et al., 2017] or augmentation [Stickland and Murray, 2020].

Original: In the setting where the ensemble members mostly differ by random seeds or augmentations, prior work has shown that calibrating the members of an ensemble does not help [Wu and Gales, 2021, Ovadia et al., 2019].
Revised: In the setting where the ensemble members mostly differ by random seeds or augmentations, prior work has shown that calibrating the members of an ensemble does not help [Wu and Gales, 2021, Ovadia et al., 2019].

Original: However when we combine two very different models (standard and robust), calibration leads to clear improvements.
Revised: However when we combine two very different models (standard and robust), calibration leads to clear improvements.

Original: Mitigating ID-OOD tradeoffs.
Revised: Mitigating ID-OOD tradeoffs.

Original: Tradeoffs between ID and OOD accuracy are widely studied and prior work selftrains on large amounts of unlabeled data to mitigate such tradeoffs [Raghunathan et al., 2020, Xie et al., 2021, Khani and Liang, 2021].
Revised: Tradeoffs between ID and OOD accuracy are widely studied and prior work selftrains on large amounts of unlabeled data to mitigate such tradeoffs [Raghunathan et al., 2020, Xie et al., 2021, Khani and Liang, 2021].

Original: In contrast, our approach uses no extra unlabeled data and is a simple method where we just add up the model probabilities after a quick calibration step.
Revised: In contrast, our approach uses no extra unlabeled data and is a simple method where we just add up the model probabilities after a quick calibration step.

Original: In concurrent and independent work, [Wortsman et al., 2021] show that there exists a way to combine a CLIP zero-shot and ﬁne-tuned model to get good ID and OOD accuracy— however learning how to combine the models may require
Revised: In concurrent and independent work, [Wortsman et al., 2021] show that there exists a way to combine a CLIP zero-shot and ﬁne-tuned model to get good ID and OOD accuracy— however learning how to combine the models may require

Original: OOD data, which is not available.
Revised: OOD data, which is not available.

Original: We show that the natural way to learn how to weight ensemble members—selecting the weights to optimize ID accuracy—does not get the best of both worlds.
Revised: We show that the natural way to learn how to weight ensemble members—selecting the weights to optimize ID accuracy—does not get the best of both worlds.

Original: In addition, their approach does not directly apply to settings where the standard and robust models have different architectures, such as In-N-Out
Revised: In addition, their approach does not directly apply to settings where the standard and robust models have different architectures, such as In-N-Out

Original: [Xie et al., 2021].
Revised: [Xie et al., 2021].

Original: Discussion.
Revised: Discussion.

Original: In this paper, we show that ID-calibrated ensembles, a simple method of calibrating a standard and robust model only on ID data and then ensembling them, can eliminate the tradeoff between in-distribution (ID) and out-of-distribution (OOD) accuracy on a wide range of natural shifts.We hope that this leads to more widespread use and deployment of robustness interventions. A PROOFS FOR SECTION We begin with some standard background on Bayes optimal classiﬁers.
Revised: In this paper, we show that ID-calibrated ensembles, a simple method of calibrating a standard and robust model only on ID data and then ensembling them, can eliminate the tradeoff between in-distribution (ID) and out-of-distribution (OOD) accuracy on a wide range of natural shifts.

Original: When then prove the results in Section 4.

Original: By default, expectations are taken over all random variables.

Original: A.1 BACKGROUND ON BAYES-OPTIMAL CLASSIFIERS

Original: These results are all standard, but we include it as background information since different texts use different notations.

Original: Let

Original: Z ∈ Z denotes some features (that can be complicated functions of the input x , for example the output of a neural network), and let Y ∈ Y denote the label.

Original: Let P be a distribution over ( Z, Y ) .

Original: The Bayes-optimal classiﬁer predicts the most likely label y given features z .

Original: Deﬁnition A.1.

Original: The Bayes-optimal classiﬁer for P given features z is given by:

Original: [Equation]

Original: The Bayes-optimal classiﬁer has the minimum misclassiﬁcation error of all possible classiﬁers that use z ∈ Z to predict y ∈ Y .

Original: Formally, the error of a classiﬁer (cid:98) y is the probability that it gets the label incorrect.

Original: Deﬁnition A.2.

Original: The error of a predictor (cid:98)

Original: y :

Original: Z → Y on distribution P is given by:

Original: [Equation]

Original: Alternatively, we can look at the error for each Z , and then take the average over Z , which gives us:

Original: Lemma A.1.

Original: The error of a predictor (cid:98) y :

Original: Z → Y on distribution P can be written as:

Original: [Equation] Proof. We can write the misclassiﬁcation probability as an expectation over an indicator and then apply the law of total expectation. [Equation]
Revised: We hope that this leads to more widespread use and deployment of robustness interventions.

Original: And then just write the inner expectation as a probability.

Original: [Equation]

Original: The Bayes-optimal classiﬁer selects the y with the highest probability given z , so we have:

Original: Lemma A.2.

Original: The error of the Bayes-optimal classiﬁer y ∗ on a distributon P can be written as (where Z ∼ P ):

Original: [Equation]

Original: Proof.

Original: The proof is immediate by substituting the deﬁnition of the Bayes-optimal classiﬁer (Deﬁnition A.1) into the alternative formula for the error in Lemma A.1.

Original: From the above, it is clear that the Bayes-optimal classiﬁer has lower error than any other classiﬁer that uses only z , formalized below.

Original: Lemma A.3.

Original: The bayes-optimal classiﬁer (for P ) has lower error than all classiﬁers (cid:98)

Original: y :

Original: Z → Y :

Original: [Equation]

Original: [Equation]

Original: Proof.

Original: Beginning from Lemma A.1, we have:

Original: [Equation]

Original: As a simple corollary, we note that the accuracy of the Bayes-optimal classiﬁer is at least the frequency of the most common label.

Original: Corollary A.1.

Original: If y ∗ is bayes-optimal for P then,

Original: [Equation]

Original: So for example if P is balanced, then the Bayes-opt classiﬁer will have accuracy at least 1 /K , where K is the number of classes.

Original: Note that calibrated classiﬁers are Bayes-optimal given their outputs.

Original: Formally, let P be a distribution over ( x, y ) , and suppose f is calibrated with respect to P .

Original: Let z

Original: = f ( x ) and let P (cid:48) be the induced distribution over ( z, y ) .

Original: Then f is

Original: Bayes-optimal for P (cid:48) given features z .

Original: The label distributions P (cid:48) ( y ) and P ( y ) are the same, so Lemma A.3 applies to any calibrated classiﬁer.

Original: A.2 PROOF OF PROPOSITION 4.

Original: Restatement of Proposition 4.1.

Original: Suppose that f std and f rob are calibrated with respect to P id , and that P id is class-balanced.

Original: Let h : R K × R K

Original: → R K be an arbitrary function that combines the standard and robust model’s predictions, and let f h be the resulting classiﬁer: f h ( x ) =

Original: h ( f std ( x ) , f rob ( x )) .

Original: The ensemble is better than any such combination classiﬁer f h :

Original: Err id ( f ens ) ≤ Err id ( f h ) .

Original: We ﬁrst show that in the setting of the Proposition, we can write P ( y | f rob ( x ) , f rob ( x )) in terms of f rob ( x ) and f std ( x ) .

Original: Lemma A.4.

Original: In the setting of Proposition 4.1, let m ∈

Original: R K be the log of the marginal probabilities P ( y ) :

Original: [Equation]

Original: Then we have:

Original: [Equation]

Original: In the balanced setting, where P ( y ) = 1 /K for all y , this simpliﬁes to:

Original: [Equation]

Original: Proof.

Original: Fix r

Original: = f rob ( x ) and s = f std ( x ) , where r, s ∈ R K .

Original: We ﬁrst rewrite the probability of y given the robust and standard model outputs P ( y | r, s ) in terms of the probability of y given each of the individual model outputs: P ( y | r ) and P ( y | s ) .

Original: We do this for discrete random variables for simplicity, but the same result follows by using Bayes rule for general random variables.

Original: [Equation]

Original: [Bayes

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: [Bayes

Original: [Equation]

Original: A

Original: [Equation]

Original: [Equation]

Original: [Algebra] (A.20) (A.21)

Original: Since r, s are ﬁxed, we can denote the terms that do not depend on y by a constant c 1 ,

Original: [Equation]

Original: So then we can write:

Original: [Equation]

Original: Now, we assumed P ( Y = y | r ) =

Original: softmax ( r ) y and P ( Y = y | s )

Original: = softmax ( s ) y for all y ∈

Original: [ K ] .

Original: For some constants c 2 , c 3 ∈ R , we can write this as: P ( Y = y | r ) =

Original: exp( r y ) /c

Original: 2 and P ( Y = y | s ) =

Original: exp( s y ) /c

Original: 3 for all y ∈

Original: [ K ] .

Original: Substituting this into Equation A.23, we get:

Original: [Equation]

Original: Writing 1 /P

Original: [-( y ) as exp( − log P ( y )) , and setting c 
-]
Revised: [-( y ) as exp( − log P ( y )) , and setting c 
-]

Original: =

Original: c 1 c 2 c 3 , this gives us:

Original: [Equation]

Original: Since the LHS is a probability, these must sum to 1 and so c 4 must be a normalizing constant, that is, c

Original: 4 = 1 / ( (cid:80) y ∈

Original: [ K ] exp( r y + s y − log P ( y ))) .

Original: This gives us:

Original: [Equation]

Original: which is precisely Equation A.15.

Original: In the balanced setting, we have P ( Y ) =

Original: 1 /K

Original: so we simply fold P ( Y ) into the constant c 4 , and get:

Original:  [Equation] s ) y [Equation]
Revised:  ACKNOWLEDGEMENTS

Original: for all y ∈

Original: [ K ]

Original: [Equation]

Original: which is precisely Equation A.16.

Original: Now we are ready to prove Proposition 4.1.

Original: Proof of Proposition 4.1.

Original: We assumed the “balanced” setting where P ( y )

Original: =

Original: 1 /K for all y .

Original: From Lemma A.4, letting f ens ( x ) = f std ( x ) + f rob ( x ) , we have:

Original: [Equation]

Original: So this means that the ensemble prediction is the Bayes optimal given ( f std ( x ) , f rob ( x )) :

Original: [Equation]

Original: But then from Lemma A.3, any other predictor which uses only ( f rob ( x ) , f std ( x )) must have higher error.

Original: This completes the proof.

Original: Note that the inequality in the above proof is a strict inequality except in degenerate cases: as long as f std and f rob sometimes disagree in their predictions, and in some of these cases f std assigns a higher probability to its predictions, and in some cases f rob assigns a higher probability to its prediction, the inequalities will be strict inequalities.

Original: A.3 PROOF OF PROPOSITION 4.

Original: Restatement of Proposition 4.2.

Original: If the OOD contains a mixture of suppressed features and missing spurious features i.e., P ood = αP τ + (1 − α ) P 0 , and P τ and P 0 are class-balanced, then we have Err ood

Original: ( f ens ) ≤

Original: Err ood ( f rob ) and

Original: Err ood ( f ens ) ≤

Original: Err ood ( f std ) .

Original: Proof.

Original: We ﬁrst note that errors are additive.

Original: That is, letting:

Original: [Equation]

Original: y ] , where x, y ∼ P,

Original: [Equation]

Original: we have:

Original: [Equation]

Original: So it sufﬁces to prove that the ensemble is better than the standard and robust models for P τ and P 0 separately.

Original: Suppressed features.

Original: Let f rob ( x ) = τf rob ( x ) and f std ( x ) = τf std ( x ) be scaled versions of the standard and robust models.

Original: Deﬁnition 4.2 implies that f rob and f std are calibrated.

Original: Since we assumed P τ is balanced, by Proposition 4.1, f ens given by f ens ( x ) = τf rob ( x ) + τf std ( x ) has optimal error on P τ .

Original: But for all x , the predictions of f ens and f ens are the same (multiplying the outputs of a model by a constant does not change the predicted output, which is the arg max ).

Original: So f

Original: ens also has optimal error on P τ :

Original: [Equation]

Original: Note that these inequalities are strict inequalities except in degenerate cases: as long as f std and f rob sometimes disagree in their predictions, and in some of these cases f std assigns a higher probability to its predictions, and in some cases f rob assigns a higher probability to its prediction, the inequalities will be strict inequalities.

Original: Missing spurious.

Original: If f std ( x ) = 0 almost surely, then f ens ( x ) =

Original: f rob ( x ) + f std ( x ) =

Original: f rob ( x ) almost surely.

Original: Furthermore, if f std ( x ) = 0 then its error is lower bounded by 1 − max y P 0 ( y ) .

Original: On the other hand, f rob ( x ) is calibrated and therefore

Original: Bayes-optimal given z = f rob ( x ) so from Lemma A.1 (e.g., see the the discussion below the Lemma for more details) has error at most 1 − max y P 0 ( y ) .

Original: So we have:

Original: [Equation]

Original: [-Note that the inequality is a strict inequality except in a degenerate case (where the probability that f rob predicts for the most common class arg max y P 
-]
Revised: [-Note that the inequality is a strict inequality except in a degenerate case (where the probability that f rob predicts for the most common class arg max y P 
-]

Original: ( y ) is the same for all inputs).

Original: A.4 PROOF OF PROPOSITION 4.

Original: Restatement of Proposition 4.3.

Original: If spurious features are anticorrelated OOD so that P ood

Original: =

Original: P

Original: adv , then even if P adv is class-balanced, Err ood ( f rob )

Original: ≤ Err ood ( f ens ) ≤

Original: Err ood ( f std ) .

Original: Proof.

Original: Let X, Y ∼ P ood , and let Z =

Original: ( f std ( X ) , f rob ( X )) be the predictions of the standard and robust models.

Original: Fix z

Original: = ( f std ( x ) , f rob ( x )) , and let s = f std ( x ) and r = f rob ( x ) .

Original: We will analyze the errors for ﬁxed Z =

Original: z (showing that the robust model is better than the ensemble, which is better than the standard model).

Original: Since this is true for all z , we then use

Original: Lemma A.1 (which is basically the law of total expectation), to get the desired result.

Original: Bayes-opt classiﬁer.

Original: Recall that for some α, β > 0 , we have P

Original: adv ( Y = y

Original: | f std ( x )) =

Original: softmax ( − βf std ( x ))

Original: y for all x (note the minus sign), while P adv ( Y = y

Original: | f rob ( x ))

Original: =

Original: softmax ( αf rob ( x ))

Original: y .

Original: Then, applying Lemma A.4, we have:

Original: [Equation]

Original: Rewriting this in terms of z, r, s , we have:

Original: [Equation]

Original: Ensemble vs. robust classiﬁer.

Original: Let j rob =

Original: arg max y r

Original: y be the robust model’s prediction, and j ens = arg max y ( r + s )

Original: y be the ensemble model’s prediction.

Original: Because j rob is the arg max of r , we have:

Original: [Equation]

Original: Because j ens is the arg max of r + s , we have:

Original: [Equation]

Original: Taking the negation of this, we get:

Original: [Equation]

Original: Adding β times Inequality A.38 to ( α + β ) times Inequality A.36, we get:

Original: [Equation]

Original: Since softmax is monotonic, we have:

Original: [Equation]

Original: But from Equation A.35 the LHS is the same as the robust model’s probability of getting the label correct, and the RHS is the same as the ensemble’s probability of getting the label correct:

Original: [Equation]

Original: Taking negations (to get the error), and then the expectation over Z

Original: = z , we get (note that below we write the error, which is why the sign is now ﬂipped):

Original: [Equation]

Original: Which is what we wanted to show.

Original: Ensemble vs. standard classiﬁer.

Original: The argument is fairly analogous to the previous case, with some minor differences in the algebra in the ﬁrst part.

Original: Let j std = arg max y s

Original: y be the standard model’s prediction.

Original: Because j std is the arg max of s , we have:

Original: [Equation]

Original: Taking the negation of this, we get:

Original: [Equation]

Original: Because j ens is the arg max of r + s , we have:

Original: [Equation]

Original: Adding α times Inequality A.45 with ( α + β ) times Inequality A.44, we get:

Original: [Equation]

Original: [Equation]

Original: The rest of this step is the same as in the comparison between the ensemble and the robust model.

Original: Since softmax is monotonic, we have:

Original: [Equation]

Original: But from Equation A.35 the LHS is the same as the robust model’s probability of getting the label correct, and the RHS is the same as the ensemble’s probability of getting the label correct:

Original: [Equation]

Original: Taking negations (to get the error), and then the expectation over Z

Original: = z , we get (note that below we write the error, which is why the sign is now ﬂipped):

Original: [Equation]

Original: Which is what we wanted to show.

Original: B MORE INFORMATION ON EXPERIMENTS B.1 ADDITIONAL DETAILS ON DATASETS

Original: Here we describe the robustness interventions and datasets in more detail.

Original: Robustness interventions :

Original: In-N-Out [Xie et al., 2021].

Original: Many datasets contain a core input x (image or time series data), and metadata z (e.g., location or climate data).

Original: Xie et al.

Original: [2021] show that using the metadata (in addition to x ) improves accuracy in-distribution (ID), but hurts accuracy out-of-distribution.

Original: Xie et al.

Original: [2021] consider a standard model that takes in both the core inputs and metadata to predict the target, and a robust model that only takes in the core inputs and does some additional pretraining.

Original: We use ofﬁcial checkpoints

Original: from

Original: their CodaLab worksheet https://worksheets.codalab.org/ worksheets/0x2613c72d4f3f4fbb94e0a32c17ce5fb

Original: , and compare to the results tagged as “In-N-Out” on each dataset.

Original: They also show results after doing additional self-training on (unlabeled) OOD data, but we do not compare to this because 1.

Original: OOD data is assumed to be unavailable in our setting, and 2.

Original: if OOD unlabeled data is available, we can also start from ID-calibrated ensembles and do additional self-training.

Original: Lightweight ﬁne-tuning [ ? ]: When adapting a pretrained model to an ID dataset, typically all the model parameters are ﬁne-tuned.

Original: Recent works show that tuning only parts of the model can often do better OOD even though the ID performance is worse [Li and Liang, 2021, Houlsby et al., 2019].

Original: On four distribution shift datasets, we take checkpoints from ?

Original: where the standard model starts from a pretrained initialization and ﬁne-tunes all parameters on an ID dataset, and the robust model only learns the top linear ‘head’ layer.

Original: Zero-shot language prompting: Radford et al.

Original: [2021] pretrain a model on a large multi-modal language and vision dataset.

Original: The model can then predict the label of an image by comparing the image embedding, with the language embedding for prompts such as ‘photo of an apple’ or ‘photo of a banana’.

Original: They show that this zero-shot language prompting approach (robust model) can be much more accurate OOD than the traditional method of ﬁne-tuning the entire model (standard model), although ID accuracy of the robust model is worse.

Original: We use model checkpoints and datasets from Radford et al.

Original: Group distributionally robust optimization (DRO) [Sagawa et al., 2020a]: Standard ERM models often latch on to spurious correlations in a dataset, such as image background color, or the occurrence of certain words in a sentence.

Original: Group DRO essentially upweights examples where this spurious correlation is not present.

Original: The original formulation in Sagawa et al.

Original: [2020a] assumes the spurious correlations are annotated, but newer variants [Liu et al., 2021] can work even without these annotations.

Original: We consider three types of natural shifts (geography shifts, subpopulation shifts, style shifts) and we also consider adversarial spurious shifts.

Original: Geography shifts.

Original: In geography shifts the ID data comes from some locations, and the OOD data comes from a different set of locations.

Original: One motivation is that in many developing areas training data may be unavailable because of monetary constraints [Jean et al., LandCover :
Revised: This work was in part supported by the Open Philanthropy

Original: The goal is to classify a satellite images into one of 6 land types (e.g., "grassland", "savannas").

Original: The ID data contains images from outside Africa, and the OOD data consists of images from Africa.

Original: We take model checkpoints from Xie et al.

Original: [2021] where they use the In-N-Out intervention—the core feature x is time series data measured by Nasa’s MODIS satellite, and the spurious metadata z consists of climate data (e.g., temperature) at that location.

Original: We use the dataset from Rußwurm et al.

Original: [2020] which is the same as what Xie et al.

Original: [2021] use.

Original: Cropland :

Original: The goal is to predict whether a satellite image is of a cropland or not.

Original: The ID dataset contains images from Iowa

Original: , Missouri, and Illinois, and the OOD dataset contains images from Indiana and Kentucky.

Original: We take model checkpoints from Xie et al.

Original: [2021] where they use the In-N-Out intervention—the core feature x is an RGB satellite image, and the spurious metadata z consists of location coordinates and vegetation bands.

Original: We use the dataset from Wang et al.

Original: [2020] which is the same as what Xie et al.

Original: [2021] use.

Original: Subpopulation shifts.

Original: In subpopulation shifts, the ID data contains a few sub-categories (e.g., black bear and sloth bear), and the OOD data contains different sub-categories (e.g., brown bears and polar bears) or the same parent category (e.g., bears).

Original: For both datasets below, we take model checkpoints from ? where they use the lightweight ﬁne-tuning intervention, starting from a MoCo-v2 ResNet-50 model pretrained on unlabeled ImageNet images.

Original: The datasets are from Santurkar et al.

Original: Living-17 :

Original: the goal is to classify an image as one of 17 animal categories such as “bear” - the ID dataset contains images of black bears and sloth bears and the OOD dataset has images of brown bears and polar bears.

Original: Entiy-30 : similar to Living-17, except the goal is to classify an image as one of 30 entity categories such as “food”, “motor vehicle”, and “index”.

Original: Style shifts.

Original: In style shifts, the

Original: ID data contains data

Original: in

Original: a

Original: certain

Original: style

Original: (e.g.,

Original: sketches

Original: ), and the OOD data contains data in a different style (

Original: e g real photos renditions

Original: DomainNet [Peng et al., 2019]: a standard domain adaptation dataset.

Original: Here, our ID dataset contains “sketch” images (e.g., drawings of apples, elephants, etc), and the OOD dataset contains “real” photos of the same categories.

Original: We take model checkpoints from ? where they use the lightweight ﬁne-tuning intervention, starting from a CLIP ResNetmodel.

Original: CelebA

Original: [Liu et al., 2015]: the goal is to classify a portrait of a face as “male” or “female” - the ID dataset contains images of people without hats, and the OOD dataset contains images of people wearing hats (some facial features might be “suppressed” or “missing” with hats).

Original: We take model checkpoints from Xie et al.

Original: [2021] where they use the In-N-Out intervention—the core feature x is the RGB image, and the spurious metadata z consists of 7 attribute tags annotated in the dataset (e.g., presence of makeup, beard).

Original: CIFAR->STL : standard domain adaptation dataset [French et al., 2018], where the ID is CIFAR-10 [Krizhevsky, 2009], and the OOD is STL

Original: [Coates et al., 2011].

Original: The task is to classify an image into one of 10 categories such as “dog”, “cat”, or “airplane”.

Original: We take model checkpoints from ? where they use the lightweight ﬁne-tuning intervention, starting from a MoCo-v2 ResNet-50 model pretrained

Original: on unlabeled ImageNet images.

Original: ImageNet [Russakovsky et al., 2015]: a large scale dataset where the goal is to classify an image into one ofcategories.

Original: We use the zero-shot language prompting intervention using a CLIP ViT-B/16 vision transformer model.

Original: We evaluate on 3 standard OOD datasets: ImageNetV2 [Recht et al., 2019],

Original: ImageNet-R

Original: [Hendrycks et al., 2020], and

Original: ImageNet-Sketch [Wang et al., 2019].

Original: Adversarial spurious shifts.

Original: In adversarial spurious shifts, the ID dataset contains a feature that is correlated with a label, but this correlation is ﬂipped OOD.

Original: Waterbirds [Sagawa et al., 2020a]: The goal is to classify an image as a “waterbird” or “landbird”.

Original: The dataset is

Original: synthetically constructed to have

Original: adversarially

Original: spurious

Original: features

Original: : “water” backgrounds are

Original: correlated

Original: with “waterbird”

Original: labels in the ID

Original: but anticorrelated OOD.

Original: We use checkpoints from Jones et al [2021] where they use the group DRO
Revised: Project and NSF Award Grant No. and NSF

Original: intervention

Original: MNLI [Williams et al., 2018]: The goal is to predict whether a hypothesis is entailed

Original: contradicted by

Original: , or

Original: neutral to an associated premise.

Original: We use the splits in Sagawa et al.

Original: [2020a

Original: —they partition

Original: the dataset so that in
Revised: IIS 2045685.

Original: distribution “negation” words “nobody”

Original: “no”

Original: “never”

Original: and “nothing” are correlated

Original: with

Original: the contradiction label

Original: however in the OOD dataset these words are anticorrelated with the
Revised: AR was supported by an Open Philanthropy

Original: contradiction label.

Original: We use checkpoints from Jones

Original: et

Original: al

Original: where

Original: they use the

Original: group DRO intervention

Original: CivilComments [Borkan et al.

Original: , 2019]: The goal is to predict whether a

Original: comment

Original: is toxic or not

Original: We use the splits in

Original: Sagawa et al. 2020a ]—they partition the dataset
Revised: Project AI Fellowship.

Original: where in the ID split mentions of a Christian identity are correlated with non-toxic comments, but in the OOD split mentions of a Christian identity are correlated with a toxic comment. We use checkpoints from Jones et al. [2021] where they use the group DRO intervention.
Revised: We would like to thank Robbie Jones and the anonymous reviewers for helpful comments on our draft.

Original: CivilComments is also used in Koh et al.

Original: B.2 PER-DATASET RESULTS ON ENSEMBLING ABLATIONS

Original: In Section 6.2 we ablated calibrated ensembles with “tuned” ensembles where the ensemble weights are tuned on indistribution validation data, and with vanilla ensembles.

Original: Here, we show per-dataset results both ID (Table 4) and OOD (Table 5).

Original: In Section 6.2, We also compared calibrated ensembles (of one standard and one robust model) with ensembles of two standard models, and ensembles of two robust models, where for a fair comparison all models are calibrated.

Original: We ran this ablation on 7 of the 13 datasets (Entity-30, DomainNet, CIFAR → STL, Living-17, Landcover, Cropland, and CelebA) because it requires multiple standard and multiple robust models, which were not available or very expensive to run on large datasets like ImageNet.

Original: Calibrated ensembles get an average ID accuracy of 92.0% (vs. 89.6% for a robust-robust ensemble

Original: [Table]

Original: and 91.4% for a standard-standard ensemble), and an average OOD accuracy of 78.7% (vs. 78.2% for a robust-robust ensemble and 70.8% for a standard-standard ensemble).

Original: We show per-dataset results in Table 6 (ID) and Table 7 (OOD).

Original: We show per-dataset results both ID (Table 6) and OOD (Table 7).

Original: B.3 PER-DATASET RESULTS ON CALIBRATION AND CONFIDENCE

Original: Relative conﬁdence can be incorrect.

Original: We measure the conﬁdence of a model f on a distribution P as conf ( f, P ) =

Original: E x ∼ P [max i f ( x ) i ] .

Original: Even if the models are not calibrated OOD, one intuitive intuition for why calibrated ensembles work is that that robust model has higher conﬁdence OOD, so that the ensemble primarily uses the (more accurate) robust model’s predictions OOD.

Original: However, on the remote sensing dataset Landcover we ﬁnd that the robust model is 6% less conﬁdent on OOD data than the standard model even though the robust model is 5% more accurate OOD than the standard model.

Original: Interestingly, calibrated ensembles are able to combine the models in a more ﬁne-grained way to get the best of both worlds, which is captured in our stylized setting in Section 4.

Original: We show the average conﬁdence of the standard and robust models for each dataset ID (Table 10) and OOD (Table 11).

Original: Per-dataset results for ECE.

Original: In Section ??

Original: , we talked about the ECE of the standard and robust models after calibrating on

Original: ID data .

Original: Here we show the results for each dataset ID (Table 8) and OOD (Table 9).

Original: We also show the ECE of the standard and robust models before calibrating on ID data , on ID (Table 12) and on OOD (Table 13).

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: The standard model is typically more conﬁdence than the robust model, which is reasonable since the standard model is also typically more accurate.

Original: There are a few exceptions such as DomainNet, CelebA

Original: [Table]

Original: and WaterBirds where the standard model is less conﬁdent than the robust model

Original: [Table]

Original: but the standard model is also less accurate in these cases, so this is also reasonable.

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]