Original: ID AND OOD P ERFORMANCE A RE S OMETIMES I NVERSELY C ORRELATED ON R EAL - WORLD D ATASETS
Revised: ID AND OOD P ERFORMANCE A RE S OMETIMES I NVERSELY C ORRELATED ON R EAL - WORLD D ATASETS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Context.
Revised: Context.

Original: Several studies have empirically compared in-distribution (ID) and outof-distribution (OOD) performance of various models.
Revised: Several studies have empirically compared in-distribution (ID) and outof-distribution (OOD) performance of various models.

Original: They report frequent positive correlations on benchmarks in computer vision and NLP.
Revised: They report frequent positive correlations on benchmarks in computer vision and NLP.

Original: Surprisingly, they never observe inverse correlations suggesting necessary trade-offs.
Revised: Surprisingly, they never observe inverse correlations suggesting necessary trade-offs.

Original: This matters to determine whether ID performance can serve as a proxy for OOD generalization.
Revised: This matters to determine whether ID performance can serve as a proxy for OOD generalization.

Original: Findings.
Revised: Findings.

Original: This paper shows that inverse correlations between ID and OOD performance do happen in real-world benchmarks.
Revised: This paper shows that inverse correlations between ID and OOD performance do happen in real-world benchmarks.

Original: They could be missed in past studies because of a biased selection of models.
Revised: They could be missed in past studies because of a biased selection of models.

Original: We show an example on the WILDSCamelyon17 dataset, using models from multiple training epochs and random seeds.
Revised: We show an example on the WILDSCamelyon17 dataset, using models from multiple training epochs and random seeds.

Original: Our observations are particularly striking with models trained with a regularizer that diversifies the solutions to the ERM objective (Teney et al., 2022a).
Revised: Our observations are particularly striking with models trained with a regularizer that diversifies the solutions to the ERM objective (Teney et al., 2022a).

Original: Implications.
Revised: Implications.

Original: We nuance recommendations and conclusions made in past studies.
Revised: We nuance recommendations and conclusions made in past studies.

Original: • High OOD performance may sometimes require trading off ID performance.
Revised: • High OOD performance may sometimes require trading off ID performance.

Original: • Focusing on ID performance alone may not lead to optimal OOD performance: it can lead to diminishing and eventually negative returns in OOD performance.
Revised: • Focusing on ID performance alone may not lead to optimal OOD performance: it can lead to diminishing and eventually negative returns in OOD performance.

Original: • Our example reminds that empirical studies only chart regimes achievable with existing methods: care is warranted in deriving prescriptive recommendations.
Revised: • Our example reminds that empirical studies only chart regimes achievable with existing methods: care is warranted in deriving prescriptive recommendations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Past observations.
Revised: Past observations.

Original: This paper complements existing studies that empirically compare indistribution (ID) and out-of-distribution 1 (OOD) performance of deep learning models (Andreassen et al., 2021;
Revised: This paper complements existing studies that empirically compare indistribution (ID) and out-of-distribution 1 (OOD) performance of deep learning models (Andreassen et al., 2021;

Original: Djolonga et al., 2021; Miller et al., 2021; Mania & Sra, 2020; Miller et al., 2020; Taori et al., 2020; Wenzel et al., 2022).
Revised: Djolonga et al., 2021; Miller et al., 2021; Mania & Sra, 2020; Miller et al., 2020; Taori et al., 2020; Wenzel et al., 2022).

Original: It has long been known that models applied to OOD data suffer a drop in performance, e.g. in classification accuracy.
Revised: It has long been known that models applied to OOD data suffer a drop in performance, e.g. in classification accuracy.

Original: The above studies show that, despite this gap, ID and OOD performance are often positively correlated 2 across models on benchmarks in computer vision (Miller et al., 2021) and NLP (Miller et al., 2020).
Revised: The above studies show that, despite this gap, ID and OOD performance are often positively correlated 2 across models on benchmarks in computer vision (Miller et al., 2021) and NLP (Miller et al., 2020).

Original: Past explanations.
Revised: Past explanations.

Original: Frequent positive correlations are surprising because nothing forbids opposite, inverse ones.
Revised: Frequent positive correlations are surprising because nothing forbids opposite, inverse ones.

Original: Indeed, ID and OOD data contain different associations between labels and features.
Revised: Indeed, ID and OOD data contain different associations between labels and features.

Original: One could imagine e.g. that an image background is associated with class C 1 ID and class C
Revised: One could imagine e.g. that an image background is associated with class C 1 ID and class C

Original: OOD.
Revised: OOD.

Original: The more a model relies on the presence of this background, the better its ID performance but the worse its OOD performance, resulting in an inverse correlation.
Revised: The more a model relies on the presence of this background, the better its ID performance but the worse its OOD performance, resulting in an inverse correlation.

Original: Never observing inverse correlations has been explained with the possibility that real-world benchmarks might contain
Revised: Never observing inverse correlations has been explained with the possibility that real-world benchmarks might contain

Original: [Figure]
Revised: [Figure]

Original: OO D
Revised: OO D

Original: A cc u r a c y
Revised: A cc u r a c y

Original: ID Accuracy
Revised: ID Accuracy

Original: OO D
Revised: OO D

Original: A cc u r a c y
Revised: A cc u r a c y

Original: This paper shows, with a counterexample, that inverse correlations are possible and can be accidentally overlooked.
Revised: This paper shows, with a counterexample, that inverse correlations are possible and can be accidentally overlooked.

Original: The possible need for an ID / OOD trade-off is thus not merely theoretical and should be envisioned, e.g. preventing blind reliance on ID performance for model selection.
Revised: The possible need for an ID / OOD trade-off is thus not merely theoretical and should be envisioned, e.g. preventing blind reliance on ID performance for model selection.

Original: only mild distribution shifts (Mania & Sra, 2020).
Revised: only mild distribution shifts (Mania & Sra, 2020).

Original: We will show that such observations can also be an artefact of study design.
Revised: We will show that such observations can also be an artefact of study design.

Original: A recent large-scale study.
Revised: A recent large-scale study.

Original: Wenzel et al.
Revised: Wenzel et al.

Original: (2022) show that not all datasets display a clear positive correlation.
Revised: (2022) show that not all datasets display a clear positive correlation.

Original: The authors observe other patterns that sometimes reveal underspecification (D’Amour et al., 2020; Teney et al., 2022b; Lee et al., 2022), or severe shifts that prevent any training / test transfer.
Revised: The authors observe other patterns that sometimes reveal underspecification (D’Amour et al., 2020; Teney et al., 2022b; Lee et al., 2022), or severe shifts that prevent any training / test transfer.

Original: Surprisingly, they never observe inverse correlations: “ We did not observe any trade-off between accuracy and robustness, where more accurate models would overfit to spurious features that do not generalize. ”
Revised: Surprisingly, they never observe inverse correlations: “ We did not observe any trade-off between accuracy and robustness, where more accurate models would overfit to spurious features that do not generalize. ”

Original: (Wenzel et al., 2022)
Revised: (Wenzel et al., 2022)

Original: On the contrary, we do observe such cases and showcase it on a dataset from the above study.
Revised: On the contrary, we do observe such cases and showcase it on a dataset from the above study.

Original: Explaining inverse correlations.
Revised: Explaining inverse correlations.

Original: We name the underlying cause a misspecification , by extension of underspecification which was previously used to explain why models with similar ID performance can vary in OOD performance (D’Amour et al., 2020; Teney et al., 2022b; Lee et al., 2022).
Revised: We name the underlying cause a misspecification , by extension of underspecification which was previously used to explain why models with similar ID performance can vary in OOD performance (D’Amour et al., 2020; Teney et al., 2022b; Lee et al., 2022).

Original: In cases of misspecification, the standard ERM objective (empirical risk minimization), which drives
Revised: In cases of misspecification, the standard ERM objective (empirical risk minimization), which drives

Original: ID performance, conflicts with the goal of OOD performance.
Revised: ID performance, conflicts with the goal of OOD performance.

Original: ID and OOD metrics can then vary independently and inversely to one another.
Revised: ID and OOD metrics can then vary independently and inversely to one another.

Original: In Section 5, we present a minimal theoretical example that illustrate how an inverse correlation pattern originates from the presence of both robust and spurious features in the data.
Revised: In Section 5, we present a minimal theoretical example that illustrate how an inverse correlation pattern originates from the presence of both robust and spurious features in the data.

Original: In Section 6, we show that different patterns of ID /
Revised: In Section 6, we show that different patterns of ID /

Original: OOD performance occur with different magnitudes of distribution shifts.
Revised: OOD performance occur with different magnitudes of distribution shifts.

Original: Summary of contributions.
Revised: Summary of contributions.

Original: • An empirical examination of ID vs. OOD performance on the WILD-Camelyon17 dataset (Koh et al., 2021) that shows an inverse correlation pattern conflicting with past evidence (Section 3).
Revised: • An empirical examination of ID vs. OOD performance on the WILD-Camelyon17 dataset (Koh et al., 2021) that shows an inverse correlation pattern conflicting with past evidence (Section 3).

Original: • An explanation and empirical verification that past studies could miss such patterns because of a biased sampling of models (Section 4).
Revised: • An explanation and empirical verification that past studies could miss such patterns because of a biased sampling of models (Section 4).

Original: • A theoretical analysis showing when inverse correlations patterns can occur (Sections 5–6).
Revised: • A theoretical analysis showing when inverse correlations patterns can occur (Sections 5–6).

Original: • A revision of conclusions and recommendations made in past studies (Section 7).
Revised: • A revision of conclusions and recommendations made in past studies (Section 7).

Original:  P REVIOUSLY - OBSERVED PATTERNS OF ID VS . OOD PERFORMANCE
Revised:  P REVIOUSLY - OBSERVED PATTERNS OF ID VS . OOD PERFORMANCE

Original: Past studies conclude that ID and OOD performance tend to vary jointly across models on many real-world datasets (Djolonga et al., 2021; Miller et al., 2021; Taori et al., 2020).
Revised: Past studies conclude that ID and OOD performance tend to vary jointly across models on many real-world datasets (Djolonga et al., 2021; Miller et al., 2021; Taori et al., 2020).

Original: Millet al. report an almost-systematic linear correlation 3 between probit-scaled ID and OOD accuracies.
Revised: Millet al. report an almost-systematic linear correlation 3 between probit-scaled ID and OOD accuracies.

Original: Mania &
Revised: Mania &

Original: Sra (2020) explain this trend with the fact that real-world benchmarks contain only mild distribution shifts.
Revised: Sra (2020) explain this trend with the fact that real-world benchmarks contain only mild distribution shifts.

Original: 4 Andreassen et al.
Revised: 4 Andreassen et al.

Original: (2021) find that pretrained models perform “above the linear trend” in the early stages of fine-tuning.
Revised: (2021) find that pretrained models perform “above the linear trend” in the early stages of fine-tuning.

Original: Their OOD accuracy rises more quickly than their ID accuracy early on, even though the final accuracies agree with a linear trend.
Revised: Their OOD accuracy rises more quickly than their ID accuracy early on, even though the final accuracies agree with a linear trend.

Original: Most recently, the large-scale study of Wenzel et al.
Revised: Most recently, the large-scale study of Wenzel et al.

Original: (2022) is more nuanced: they observe a linear trend only on some datasets.
Revised: (2022) is more nuanced: they observe a linear trend only on some datasets.

Original: Their setup consists in fine-tuning an ImageNet-pretrained model on a chosen dataset and evaluating it on matching ID and OOD test sets.
Revised: Their setup consists in fine-tuning an ImageNet-pretrained model on a chosen dataset and evaluating it on matching ID and OOD test sets.

Original: They repeat the procedure with a variety of datasets, architectures, and implementation options such as data augmentations.
Revised: They repeat the procedure with a variety of datasets, architectures, and implementation options such as data augmentations.

Original: The scatter plots of ID / OOD accuracy in Wenzel et al.
Revised: The scatter plots of ID / OOD accuracy in Wenzel et al.

Original: (2022) show four typical patterns (Figure 2).
Revised: (2022) show four typical patterns (Figure 2).

Original: [Figure]
Revised: [Figure]

Original: Increasing line (positive correlation): mild distribution shift.
Revised: Increasing line (positive correlation): mild distribution shift.

Original: ID and OOD accuracies are positively correlated.
Revised: ID and OOD accuracies are positively correlated.

Original: Focusing on classical generalization and ID performance brings concurrent OOD improvements.
Revised: Focusing on classical generalization and ID performance brings concurrent OOD improvements.

Original: Vertical line: underspecification (D’Amour et al., 2020; Lee et al., 2022; Teney et al., 2022b).
Revised: Vertical line: underspecification (D’Amour et al., 2020; Lee et al., 2022; Teney et al., 2022b).

Original: Different models obtain a similar high ID performance but different OOD performance.
Revised: Different models obtain a similar high ID performance but different OOD performance.

Original: The objective of high ID performance does not sufficiently constrains the learning.
Revised: The objective of high ID performance does not sufficiently constrains the learning.

Original: Typically, multiple features in the data (a.k.a.
Revised: Typically, multiple features in the data (a.k.a.

Original: biased or spurious features) can be used to obtain high ID performance, but not all of them are equally reliable on OOD data.
Revised: biased or spurious features) can be used to obtain high ID performance, but not all of them are equally reliable on OOD data.

Original: To improve OOD performance, additional task-specific information is necessary, e.g. additional supervision or inductive biases (custom architectures, regularizers, etc.).
Revised: To improve OOD performance, additional task-specific information is necessary, e.g. additional supervision or inductive biases (custom architectures, regularizers, etc.).

Original: Horizontal line, low OOD accuracy: severe distribution shift.
Revised: Horizontal line, low OOD accuracy: severe distribution shift.

Original: No model performs well OOD.
Revised: No model performs well OOD.

Original: A severe shift prevents any transfer between training and OOD test data.
Revised: A severe shift prevents any transfer between training and OOD test data.

Original: The task needs to be significantly more constrained e.g. with task-specific inductive biases
Revised: The task needs to be significantly more constrained e.g. with task-specific inductive biases

Original: No clear trend: underspecification.
Revised: No clear trend: underspecification.

Original: Models show a variety of
Revised: Models show a variety of

Original: ID and OOD
Revised: ID and OOD

Original: accuracies.
Revised: accuracies.

Original: The difference
Revised: The difference

Original: with
Revised: with

Original: (2) is
Revised: (2) is

Original: the wider variety along the ID axis, e
Revised: the wider variety along the ID axis, e

Original: g. because a difficult learning task yields solutions of lower ID accuracy from local minima of the ERM objective.
Revised: g. because a difficult learning task yields solutions of lower ID accuracy from local minima of the ERM objective.

Original: The authors note the absence of decreasing patterns,
Revised: The authors note the absence of decreasing patterns,

Original: which
Revised: which

Original: are
Revised: are

Original: however possible in
Revised: however possible in

Original: theory
Revised: theory

Original: Decreasing line (inverse correlation): misspecification.
Revised: Decreasing line (inverse correlation): misspecification.

Original: The highest accuracy ID and OOD are achieved by different models.
Revised: The highest accuracy ID and OOD are achieved by different models.

Original: Optima of the ERM objective, which are expected to be optima in ID performance, do not correspond to optima in OOD performance.
Revised: Optima of the ERM objective, which are expected to be optima in ID performance, do not correspond to optima in OOD performance.

Original: This implies a trade-off: higher OOD performance is possible at the cost of lower ID performance.
Revised: This implies a trade-off: higher OOD performance is possible at the cost of lower ID performance.

Original: When does an inverse correlation occur between ID and OOD performance?
Revised: When does an inverse correlation occur between ID and OOD performance?

Original: Intuitively, it can occur when there is a pattern in the data that is predictive in one distribution and misleading in the other.
Revised: Intuitively, it can occur when there is a pattern in the data that is predictive in one distribution and misleading in the other.

Original: For example, object classes C 1 and C 2 are respectively associated with image backgrounds B 1 and B 2 in ID data, and respectively B 2 in B 1 (swapped) in OOD data.
Revised: For example, object classes C 1 and C 2 are respectively associated with image backgrounds B 1 and B 2 in ID data, and respectively B 2 in B 1 (swapped) in OOD data.

Original: Relying on the background can improve performance on either distribution but not both simultaneously.
Revised: Relying on the background can improve performance on either distribution but not both simultaneously.

Original: While such severe shifts might be rare, the next section presents an actual example.
Revised: While such severe shifts might be rare, the next section presents an actual example.

Original:  N EW OBSERVATIONS : INVERSELY CORRELATED ID / OOD PERFORMANCE
Revised:  N EW OBSERVATIONS : INVERSELY CORRELATED ID / OOD PERFORMANCE

Original: We use the WILDS-Camelyon17 dataset described by Koh et al.
Revised: We use the WILDS-Camelyon17 dataset described by Koh et al.

Original: (2021) in a manner similar to Wenzel et al.
Revised: (2021) in a manner similar to Wenzel et al.

Original: These authors evaluate different architectures, assuming different inductive biases will produce models covering a range of ID / OOD accuracies.
Revised: These authors evaluate different architectures, assuming different inductive biases will produce models covering a range of ID / OOD accuracies.

Original: For simplicity, we rely instead on different random seeds since D’Amour et al.
Revised: For simplicity, we rely instead on different random seeds since D’Amour et al.

Original: (2020) showed this to be sufficient to cover a variety of
Revised: (2020) showed this to be sufficient to cover a variety of

Original: ID / OOD accuracies on this dataset.
Revised: ID / OOD accuracies on this dataset.

Original: To increase this variety even further without manually picking architectures, we also train models with the diversity-inducing method of Teney et al.
Revised: To increase this variety even further without manually picking architectures, we also train models with the diversity-inducing method of Teney et al.

Original: (2022a).
Revised: (2022a).

Original: Experimental details.
Revised: Experimental details.

Original: We use DenseNet-121 models pretrained by the authors of the dataset using 10 different seeds.
Revised: We use DenseNet-121 models pretrained by the authors of the dataset using 10 different seeds.

Original: For each of these 10 models, we re-train the last linear layer from a random initialization for 1 to 10 epochs, while keeping the other layers frozen.
Revised: For each of these 10 models, we re-train the last linear layer from a random initialization for 1 to 10 epochs, while keeping the other layers frozen.

Original: These are referred to as “ ERM models ”.
Revised: These are referred to as “ ERM models ”.

Original: We perform this re-training with 10 different seeds which gives 10 3 ERM models (10 pretraining seeds × 10 re-training seeds × 10 numbers of epochs).
Revised: We perform this re-training with 10 different seeds which gives 10 3 ERM models (10 pretraining seeds × 10 re-training seeds × 10 numbers of epochs).

Original: In addition, we repeat this re-training of the last layer with the diversity-inducing method of Teney et al.
Revised: In addition, we repeat this re-training of the last layer with the diversity-inducing method of Teney et al.

Original: (2022a) (details in the box below).
Revised: (2022a) (details in the box below).

Original: These are referred to as “ diverse models ”.
Revised: These are referred to as “ diverse models ”.

Original: Each run of the methods producesdifferent models, giving a total of 10 3 .
Revised: Each run of the methods producesdifferent models, giving a total of 10 3 .

Original: 24 such models ( 10 3 as above × 24).
Revised: 24 such models ( 10 3 as above × 24).

Original: Background: learning diverse solutions to a learning task.
Revised: Background: learning diverse solutions to a learning task.

Original: A range of methods exist to identify multiple neural networks of similar high ID performance but that differ in other desirable properties such as OOD performance, interpretability, adversarial robustness, etc.
Revised: A range of methods exist to identify multiple neural networks of similar high ID performance but that differ in other desirable properties such as OOD performance, interpretability, adversarial robustness, etc.

Original: These methods are relevant in cases of underspecification (D’Amour et al., 2020) i.e. when the standard ERM objective does not constrain the solution space to a unique one.
Revised: These methods are relevant in cases of underspecification (D’Amour et al., 2020) i.e. when the standard ERM objective does not constrain the solution space to a unique one.

Original: Recent methods consist of optimizing multiple models while encouraging diversity in their feature space (Heljakka et al., 2022; Yashima et al., 2022), prediction space (Pagliardini et al., 2022; Lee et al., 2022), or gradient space (Ross et al., 2018; 2020; Teney et al., 2022a;b).
Revised: Recent methods consist of optimizing multiple models while encouraging diversity in their feature space (Heljakka et al., 2022; Yashima et al., 2022), prediction space (Pagliardini et al., 2022; Lee et al., 2022), or gradient space (Ross et al., 2018; 2020; Teney et al., 2022a;b).

Original: This study uses the method of Teney et al.
Revised: This study uses the method of Teney et al.

Original: (2022a) which encourages gradient diversity .
Revised: (2022a) which encourages gradient diversity .

Original: The method trains many copies of the same model in parallel – in our case, a linear classifer on top of a frozen DenseNet backbone (see Figure 3).
Revised: The method trains many copies of the same model in parallel – in our case, a linear classifer on top of a frozen DenseNet backbone (see Figure 3).

Original: [Figure]
Revised: [Figure]

Original: Each training image x goes through a frozen pretrained DenseNet to produce features h
Revised: Each training image x goes through a frozen pretrained DenseNet to produce features h

Original: = f ( x ) .
Revised: = f ( x ) .

Original: We train a set of linear classifiers { g i } ni = 1 on these features.
Revised: We train a set of linear classifiers { g i } ni = 1 on these features.

Original: A diversity loss minimizes the pairwise similarity between their input gradients.
Revised: A diversity loss minimizes the pairwise similarity between their input gradients.

Original: The models are optimized by standard SGD to minimize the sum of a standard classification loss (cross-entropy) with a diversity loss that encourages diversity across models.
Revised: The models are optimized by standard SGD to minimize the sum of a standard classification loss (cross-entropy) with a diversity loss that encourages diversity across models.

Original: Using λ a weight hyperparameter, the complete loss is L =
Revised: Using λ a weight hyperparameter, the complete loss is L =

Original: L classification + λ L diversity .
Revised: L classification + λ L diversity .

Original: The second term encourages each copy to rely on different features by minimizing the mutual alignment of input gradients:
Revised: The second term encourages each copy to rely on different features by minimizing the mutual alignment of input gradients:

Original: [Equation]
Revised: [Equation]

Original: g
Revised: g

Original: [Equation]
Revised: [Equation]

Original: ( h ) .
Revised: ( h ) .

Original: ∇
Revised: ∇

Original: [Equation]
Revised: [Equation]

Original: h ) with h
Revised: h ) with h

Original: = f ( x ) .
Revised: = f ( x ) .

Original: These pairwise dot products quantify the mutual alignment of the gradients.
Revised: These pairwise dot products quantify the mutual alignment of the gradients.

Original: Intuitively, minimizing this loss makes each model locally sensitive along different directions in its input space.
Revised: Intuitively, minimizing this loss makes each model locally sensitive along different directions in its input space.

Original: Assuming that g produces a vector of logits (as many as there are classes), ∇ h g ( · ) refers to the gradient of the largest logit w.r.t.
Revised: Assuming that g produces a vector of logits (as many as there are classes), ∇ h g ( · ) refers to the gradient of the largest logit w.r.t.

Original: the classifier’s input h .
Revised: the classifier’s input h .

Original: We use n =24 copies and a weight λ
Revised: We use n =24 copies and a weight λ

Original: = that were selected for giving a wide range of ID accuracies.
Revised: = that were selected for giving a wide range of ID accuracies.

Original: See Teney et al.
Revised: See Teney et al.

Original: (2022a) for details about the method.
Revised: (2022a) for details about the method.

Original: Variations were also described by Ross et al.
Revised: Variations were also described by Ross et al.

Original: (2020) and Teney et al.
Revised: (2020) and Teney et al.

Original: (2022b).
Revised: (2022b).

Original: [Figure]
Revised: [Figure]

Original: Results with ERM models.
Revised: Results with ERM models.

Original: In Figure 4 we plot the ID vs. OOD accuracy of ERM models as red dots ( ).
Revised: In Figure 4 we plot the ID vs. OOD accuracy of ERM models as red dots ( ).

Original: Each corresponds to a different pretraining seed.
Revised: Each corresponds to a different pretraining seed.

Original: The variation across panels (note the different Y-axis limits) shows that OOD performance varies across pre-training seeds even though the ID accuracy is similar, as noted by Koh et al.
Revised: The variation across panels (note the different Y-axis limits) shows that OOD performance varies across pre-training seeds even though the ID accuracy is similar, as noted by Koh et al.

Original: Our new observations are visible within each panel.
Revised: Our new observations are visible within each panel.

Original: The dots (models) in any panel differ in their re-training seed and/or number of epochs.
Revised: The dots (models) in any panel differ in their re-training seed and/or number of epochs.

Original: The seeds induce little variation, but the number of epochs produce patterns of decreasing trend (negative correlation).
Revised: The seeds induce little variation, but the number of epochs produce patterns of decreasing trend (negative correlation).

Original: Despite the narrow ID variation (X axis), careful inspection confirms that the pattern appears in nearly all cases (see Appendix A for zoomed-in plots).
Revised: Despite the narrow ID variation (X axis), careful inspection confirms that the pattern appears in nearly all cases (see Appendix A for zoomed-in plots).

Original: Results with diverse models.
Revised: Results with diverse models.

Original: We plot models trained with the diversity-inducing method (Teney et al., 2022a) as gray dots ( ).
Revised: We plot models trained with the diversity-inducing method (Teney et al., 2022a) as gray dots ( ).

Original: These models cover a wider range of accuracies and form patterns that extend those of ERM models.
Revised: These models cover a wider range of accuracies and form patterns that extend those of ERM models.

Original: The decreasing trend is now obvious.
Revised: The decreasing trend is now obvious.

Original: This trend is also clearly juxtaposed with a rising trend where ID / OOD performance are positively correlated.
Revised: This trend is also clearly juxtaposed with a rising trend where ID / OOD performance are positively correlated.

Original: This suggests a point of highest OOD performance after which the model overfits to ID data.
Revised: This suggests a point of highest OOD performance after which the model overfits to ID data.

Original: Appendix A shows similar results with other pretraining seeds.
Revised: Appendix A shows similar results with other pretraining seeds.

Original: The patterns are not always clearly discernible because large regions of the performance landscape are not covered, despite the diversity-inducing method.
Revised: The patterns are not always clearly discernible because large regions of the performance landscape are not covered, despite the diversity-inducing method.

Original: We further discuss this issue next.
Revised: We further discuss this issue next.

Original:  W HY PAST STUDIES MISSED NEGATIVE CORRELATIONS : A BIASED SAMPLING OF MODELS
Revised:  W HY PAST STUDIES MISSED NEGATIVE CORRELATIONS : A BIASED SAMPLING OF MODELS

Original: We identified several factors explaining the discrepancy between our observations and past studies.
Revised: We identified several factors explaining the discrepancy between our observations and past studies.

Original: • ERM models alone do not always form clear patterns (red dots in Figure 4).
Revised: • ERM models alone do not always form clear patterns (red dots in Figure 4).

Original: In our observations, the models trained with a diversity-inducing method (gray dots ) were key in making the suspected patterns more obvious, because they cover a wider range of accuracies.
Revised: In our observations, the models trained with a diversity-inducing method (gray dots ) were key in making the suspected patterns more obvious, because they cover a wider range of accuracies.

Original: • The ID / OOD trade-off varies during training, as noted by Andreassen et al.
Revised: • The ID / OOD trade-off varies during training, as noted by Andreassen et al.

Original: This variation across training epochs is responsible for much of the newly observed patterns.
Revised: This variation across training epochs is responsible for much of the newly observed patterns.

Original: However, models of different architectures or pretraining seeds are not always comparable with one another because of shifts in their overall performance (see e
Revised: However, models of different architectures or pretraining seeds are not always comparable with one another because of shifts in their overall performance (see e

Original: g. different Y-axis limits across panels in Figure 4).
Revised: g. different Y-axis limits across panels in Figure 4).

Original: Therefore the performance across epochs should be analyzed individually per model.
Revised: Therefore the performance across epochs should be analyzed individually per model.

Original: • The “inverse correlation” patterns are not equally visible with all pretraining seeds .
Revised: • The “inverse correlation” patterns are not equally visible with all pretraining seeds .

Original: In some cases, a careful examination of zoomed-in plots is necessary, see Appendix A.
Revised: In some cases, a careful examination of zoomed-in plots is necessary, see Appendix A.

Original: This is a reminder that stochastic factors in deep learning can have large effects and that empirical studies should randomize them as much as possible.
Revised: This is a reminder that stochastic factors in deep learning can have large effects and that empirical studies should randomize them as much as possible.

Original: To demonstrate these points, we plot our data (same as in Figure 4) while keeping only the ERM models trained for 10 epochs and including all pretraining seeds on the same panel.
Revised: To demonstrate these points, we plot our data (same as in Figure 4) while keeping only the ERM models trained for 10 epochs and including all pretraining seeds on the same panel.

Original: Figure 5 shows that these small changes reproduce the vertical line observed by Wenzel et al.
Revised: Figure 5 shows that these small changes reproduce the vertical line observed by Wenzel et al.

Original: (2022), which completely misses the inverse correlations patterns visible in Figure 4.
Revised: (2022), which completely misses the inverse correlations patterns visible in Figure 4.

Original: [Figure]
Revised: [Figure]

Original: a c
Revised: a c

Original: [Figure]
Revised: [Figure]

Original: t e s t
Revised: t e s t

Original: [Figure]
Revised: [Figure]

Original: sampledregime
Revised: sampledregime

Original: A general explanation is that past studies undersample regions of the ID / OOD performance space .
Revised: A general explanation is that past studies undersample regions of the ID / OOD performance space .

Original: They usually consider a variety of architectures in an attempt to sample this space.
Revised: They usually consider a variety of architectures in an attempt to sample this space.

Original: However, different architectures do not necessarily behave very differently from one another (see the box below).
Revised: However, different architectures do not necessarily behave very differently from one another (see the box below).

Original: We lack methods to reliably identify models of high OOD performance, but the diversityinducing method that we use yields models spanning a wide range of the performance spectrum.
Revised: We lack methods to reliably identify models of high OOD performance, but the diversityinducing method that we use yields models spanning a wide range of the performance spectrum.

Original: Why isn’t it sufficient to evaluate a variety of architectures?
Revised: Why isn’t it sufficient to evaluate a variety of architectures?

Original: Different architectures do not necessarily induce radically different behaviour.
Revised: Different architectures do not necessarily induce radically different behaviour.

Original: Even CNNs and vision transformers were shown to have similar failure modes (Pinto et al., 2022).
Revised: Even CNNs and vision transformers were shown to have similar failure modes (Pinto et al., 2022).

Original: Distinct architectures can share similar inductive biases that are e.g. due to SGD, such as the simplicity bias (Scimeca et al., 2022; Shah et al., 2020) or neural anisotropies (Ortiz-Jimenez et al., 2021).
Revised: Distinct architectures can share similar inductive biases that are e.g. due to SGD, such as the simplicity bias (Scimeca et al., 2022; Shah et al., 2020) or neural anisotropies (Ortiz-Jimenez et al., 2021).

Original: Therefore, independently trained models are not necessarily diverse despite the variety of architectures.
Revised: Therefore, independently trained models are not necessarily diverse despite the variety of architectures.

Original: ID / OOD performance may only vary along similar directions across models.
Revised: ID / OOD performance may only vary along similar directions across models.

Original:  T HEORETICAL ANALYSIS OF A LINEAR CASE
Revised:  T HEORETICAL ANALYSIS OF A LINEAR CASE

Original: In this section, we present a minimal case that shows a trade-off between ID and OOD performance and aids understanding the cause of such a pattern.
Revised: In this section, we present a minimal case that shows a trade-off between ID and OOD performance and aids understanding the cause of such a pattern.

Original: Let y ∈ R be a target variable to be predicted by a model, and x the features used as input to the model.
Revised: Let y ∈ R be a target variable to be predicted by a model, and x the features used as input to the model.

Original: These features are a concatenation of invariant and spurious features (defined implicitly below): x = [ x inv ; x spu ] with x inv ∈ R d inv and x spu ∈ R d spu .
Revised: These features are a concatenation of invariant and spurious features (defined implicitly below): x = [ x inv ; x spu ] with x inv ∈ R d inv and x spu ∈ R d spu .

Original: Following Arjovsky et al.
Revised: Following Arjovsky et al.

Original: ; Rosenfeld et al.
Revised: ; Rosenfeld et al.

Original: ; Zhou et al.
Revised: ; Zhou et al.

Original: (2022), we consider the simple data-generating process defined by the following structural equations:
Revised: (2022), we consider the simple data-generating process defined by the following structural equations:

Original: [Equation]
Revised: [Equation]

Original: where e ∈ { e ID , e OOD } is an environment index referring to ID or OOD data.
Revised: where e ∈ { e ID , e OOD } is an environment index referring to ID or OOD data.

Original: The random variables ϵ inv and ϵ spu represent symmetric independent random noise with zero-mean, sub-Gaussian tail probabilities, and Var ( ϵ inv ) > 0 , Var( ϵ spu ,i ) > 0 , ∀ i ∈ [1 , d spu ] .
Revised: The random variables ϵ inv and ϵ spu represent symmetric independent random noise with zero-mean, sub-Gaussian tail probabilities, and Var ( ϵ inv ) > 0 , Var( ϵ spu ,i ) > 0 , ∀ i ∈ [1 , d spu ] .

Original: The vector γ ∈ R d inv determines the relation between the target variable and the invariant features and is identical across environments.
Revised: The vector γ ∈ R d inv determines the relation between the target variable and the invariant features and is identical across environments.

Original: In contrast, the vector α e affects the spurious features and varies among environments.
Revised: In contrast, the vector α e affects the spurious features and varies among environments.

Original: Therefore the invariant features are similarly predictive in ID and OOD data while the spurious ones are not.
Revised: Therefore the invariant features are similarly predictive in ID and OOD data while the spurious ones are not.

Original: To study the relationship between ID and OOD performance of a hypothetical predictive model, we assume that this model relies on a subset of the features x .
Revised: To study the relationship between ID and OOD performance of a hypothetical predictive model, we assume that this model relies on a subset of the features x .

Original: This subset is identified by a binary mask Φ ∈ { 0 , 1 } d inv + d spu .
Revised: This subset is identified by a binary mask Φ ∈ { 0 , 1 } d inv + d spu .

Original: Suppose we have already selected ˆ d inv invariant features and ˆ d spu spurious features, such that ( ˆ d inv + ˆ d spu ) =
Revised: Suppose we have already selected ˆ d inv invariant features and ˆ d spu spurious features, such that ( ˆ d inv + ˆ d spu ) =

Original: ˆ d = ||
Revised: ˆ d = ||

Original: Φ ˆ d
Revised: Φ ˆ d

Original: || 1 where the subscript ˆ d denotes the number of selected features.
Revised: || 1 where the subscript ˆ d denotes the number of selected features.

Original: The features selected by Φ ˆ d are [ x inv , 1 , ..., x inv , ˆ d inv , x spu , 1 , ..., x spu , ˆ d spu ] .
Revised: The features selected by Φ ˆ d are [ x inv , 1 , ..., x inv , ˆ d inv , x spu , 1 , ..., x spu , ˆ d spu ] .

Original: Let E denote either the in- and out-of-domain expectation as E ID and E OOD .
Revised: Let E denote either the in- and out-of-domain expectation as E ID and E OOD .

Original: We use β to denote the optimal parameter of the linear regression for a certain domain, i.e., β ˆ d = E
Revised: We use β to denote the optimal parameter of the linear regression for a certain domain, i.e., β ˆ d = E

Original: [Φ ˆ d ( x ) ⊤ Φ ˆ d ( x )]
Revised: [Φ ˆ d ( x ) ⊤ Φ ˆ d ( x )]

Original: E [Φ ˆ d ( x ) ⊤ y ] .
Revised: E [Φ ˆ d ( x ) ⊤ y ] .

Original: Then the MSE loss of the fitted linear regressor is E [ y − Φ ˆ d ( x ) ⊤ β ˆ d ] 2 .
Revised: Then the MSE loss of the fitted linear regressor is E [ y − Φ ˆ d ( x ) ⊤ β ˆ d ] 2 .

Original: Further, let [ λ ˆ d 1 , λ ˆ d 2 , ..., λ ˆ d ˆ d ] denote the eigenvalues of E [Φ ˆ d ( x ) ⊤ Φ ˆ d ( x )] and [ v ˆ d 1 , v ˆ d 2 , ..., v ˆ d ˆ d ] the corresponding eigenvectors.
Revised: Further, let [ λ ˆ d 1 , λ ˆ d 2 , ..., λ ˆ d ˆ d ] denote the eigenvalues of E [Φ ˆ d ( x ) ⊤ Φ ˆ d ( x )] and [ v ˆ d 1 , v ˆ d 2 , ..., v ˆ d ˆ d ] the corresponding eigenvectors.

Original: Given a feature mask Φ ˆ d , we now examine how the ID and OOD losses of the model vary when including an additional spurious feature feature into Φ ˆ d (see Appendix B for a proof).
Revised: Given a feature mask Φ ˆ d , we now examine how the ID and OOD losses of the model vary when including an additional spurious feature feature into Φ ˆ d (see Appendix B for a proof).

Original: Theorem 1.
Revised: Theorem 1.

Original: Including an additional spurious feature leads to the following change in loss L :
Revised: Including an additional spurious feature leads to the following change in loss L :

Original: [Equation]
Revised: [Equation]

Original: with Q 1 , Q 2 , Q 3 defined as follows:
Revised: with Q 1 , Q 2 , Q 3 defined as follows:

Original: [Equation]
Revised: [Equation]

Original: Further, if the new feature is sufficiently unstable in the test domain, i.e. if (( α ID ˆ d +1 ) 2 − ( α OOD ˆ d +1 ) 2 ) 2 is sufficiently large such that:
Revised: Further, if the new feature is sufficiently unstable in the test domain, i.e. if (( α ID ˆ d +1 ) 2 − ( α OOD ˆ d +1 ) 2 ) 2 is sufficiently large such that:

Original: [Equation]
Revised: [Equation]

Original: then we have Q 3 >
Revised: then we have Q 3 >

Original: | Q 1 + Q 2 | and therefore L OOD (Φ ˆ d +1 )
Revised: | Q 1 + Q 2 | and therefore L OOD (Φ ˆ d +1 )

Original: − L OOD (Φ ˆ d ) > 0 .
Revised: − L OOD (Φ ˆ d ) > 0 .

Original: Because ϵ spu is a zero-mean symmetric random noise, the sign of α ˆ d +1 does not the matter in the results.
Revised: Because ϵ spu is a zero-mean symmetric random noise, the sign of α ˆ d +1 does not the matter in the results.

Original: Theorem 1 shows that adding a spurious feature to those used by the model can affect its
Revised: Theorem 1 shows that adding a spurious feature to those used by the model can affect its

Original: ID and OOD losses in opposite directions, implying a trade-off between ID and OOD accuracy.
Revised: ID and OOD losses in opposite directions, implying a trade-off between ID and OOD accuracy.

Original: In other words, this minimal case shows that a simple model without/with an extra (spurious) feature can exhibit an inverse correlation between its ID and OOD performance.
Revised: In other words, this minimal case shows that a simple model without/with an extra (spurious) feature can exhibit an inverse correlation between its ID and OOD performance.

Original:  O RDERING ID / OOD PATTERNS ACCORDING TO SHIFT MAGNITUDE
Revised:  O RDERING ID / OOD PATTERNS ACCORDING TO SHIFT MAGNITUDE

Original: The above analysis shows that inverse correlation patterns are essentially due to the presence of spurious features, i.e. features whose predictive relation with the target in ID data becomes misleading
Revised: The above analysis shows that inverse correlation patterns are essentially due to the presence of spurious features, i.e. features whose predictive relation with the target in ID data becomes misleading

Original: OOD.
Revised: OOD.

Original: Occurrences of spurious features increase with the magnitude of the distribution shift.
Revised: Occurrences of spurious features increase with the magnitude of the distribution shift.

Original: Therefore, the possible patterns in ID / OOD performance presented in Section 2 can be ordered according to the magnitude of the distribution shift they are likely to occur with (see Figure 6).
Revised: Therefore, the possible patterns in ID / OOD performance presented in Section 2 can be ordered according to the magnitude of the distribution shift they are likely to occur with (see Figure 6).

Original: [Table]
Revised: [Table]

Original: (Too?)
Revised: (Too?)

Original: Severe
Revised: Severe

Original: Typical pattern
Revised: Typical pattern

Original: Positive correlation
Revised: Positive correlation

Original: Vertical line/no clear trend
Revised: Vertical line/no clear trend

Original: Negative correlation
Revised: Negative correlation

Original: Low horizontal line (toy representation)
Revised: Low horizontal line (toy representation)

Original: ID Accuracy
Revised: ID Accuracy

Original: OO D
Revised: OO D

Original: A cc u r a c y
Revised: A cc u r a c y

Original: ID Accuracy
Revised: ID Accuracy

Original: OO D
Revised: OO D

Original: A cc u r a c y
Revised: A cc u r a c y

Original: ID Accuracy
Revised: ID Accuracy

Original: OO D
Revised: OO D

Original: A cc u r a c y
Revised: A cc u r a c y

Original: ID Accuracy
Revised: ID Accuracy

Original: OO D
Revised: OO D

Original: A cc u r a c y
Revised: A cc u r a c y

Original: Valid approaches
Revised: Valid approaches

Original: Simply focus
Revised: Simply focus

Original: Task-relevant inductive biases e.g. arch.
Revised: Task-relevant inductive biases e.g. arch.

Original: , regularizers.
Revised: , regularizers.

Original: Open for improving OOD on improving
Revised: Open for improving OOD on improving

Original: Data augmentation with task-relevant transformations.
Revised: Data augmentation with task-relevant transformations.

Original: question performance
Revised: question performance

Original: ID performance
Revised: ID performance

Original: Non-i.i.d.
Revised: Non-i.i.d.

Original: training data e.g. multiple training domains.
Revised: training data e.g. multiple training domains.

Original: Example
Revised: Example

Original: ImageNet →
Revised: ImageNet →

Original: PACS sketch →
Revised: PACS sketch →

Original: WILDS
Revised: WILDS

Original: DomainNet datasets
Revised: DomainNet datasets

Original: ImageNet v photograph
Revised: ImageNet v photograph

Original: Camelyon infograph → quickdraw (Recht et al., 2019) (Li et al., 2017)
Revised: Camelyon infograph → quickdraw (Recht et al., 2019) (Li et al., 2017)

Original: (Koh et al., 2021) (Koh et al., 2021)
Revised: (Koh et al., 2021) (Koh et al., 2021)

Original: With the smallest distribution shifts (leftmost case in Figure 6), for example training on ImageNet and testing on its replication ImageNet v2 (Recht et al., 2019), ID validation performance closely correlates with OOD test performance.
Revised: With the smallest distribution shifts (leftmost case in Figure 6), for example training on ImageNet and testing on its replication ImageNet v2 (Recht et al., 2019), ID validation performance closely correlates with OOD test performance.

Original: This OOD setting is the easiest because one can focus on improving classical generalization and reap concurrent improvements OOD.
Revised: This OOD setting is the easiest because one can focus on improving classical generalization and reap concurrent improvements OOD.

Original: With a larger distribution shift, more features are likely to be spurious, which is likely to break the ID / OOD correlation.
Revised: With a larger distribution shift, more features are likely to be spurious, which is likely to break the ID / OOD correlation.

Original: The task of improving OOD performance is likely to be under- or misspecified, i.e. there is not enough information to determine which features a model should rely on to perform well OOD.
Revised: The task of improving OOD performance is likely to be under- or misspecified, i.e. there is not enough information to determine which features a model should rely on to perform well OOD.

Original: Valid approaches include modifying the objective function, injecting taskspecific information with custom architectures (e.g. building-in invariance to rotations as in Teney & Hebert (2016)), well-chosen data augmentations, or inhomogeneous training data such as multiple training environments (Li et al., 2017) and counterfactual examples (Teney et al., 2020a).
Revised: Valid approaches include modifying the objective function, injecting taskspecific information with custom architectures (e.g. building-in invariance to rotations as in Teney & Hebert (2016)), well-chosen data augmentations, or inhomogeneous training data such as multiple training environments (Li et al., 2017) and counterfactual examples (Teney et al., 2020a).

Original: With extreme distribution shifts, most predictive features are overwhelmingly spurious and it is very difficult to learn any one relevant in OOD data (rightmost case in Figure 6).
Revised: With extreme distribution shifts, most predictive features are overwhelmingly spurious and it is very difficult to learn any one relevant in OOD data (rightmost case in Figure 6).

Original: The proposed ordering of patterns is rather informal and could be further developed following the two axes of diversity shifts and correlation shifts proposed by Ye et al.
Revised: The proposed ordering of patterns is rather informal and could be further developed following the two axes of diversity shifts and correlation shifts proposed by Ye et al.

Original: (2022) (see also Wiles et al.
Revised: (2022) (see also Wiles et al.

Original: More recently, Wang & Veitch (2022) showed that the suitability of various methods for
Revised: More recently, Wang & Veitch (2022) showed that the suitability of various methods for

Original: OOD generalization depends on particularities of the underlying causal structure of the task – which must therefore be known to select a suitable method.
Revised: OOD generalization depends on particularities of the underlying causal structure of the task – which must therefore be known to select a suitable method.

Original: Identifying which ID / OOD patterns occur with particular causal structures might serve as a tool to understand the type of OOD situation one is facing and identify a suitable method.
Revised: Identifying which ID / OOD patterns occur with particular causal structures might serve as a tool to understand the type of OOD situation one is facing and identify a suitable method.

Original:  R EVISITING RECOMMENDATIONS MADE IN PAST STUDIES
Revised:  R EVISITING RECOMMENDATIONS MADE IN PAST STUDIES

Original: We have established that observations in past studies were incomplete.
Revised: We have established that observations in past studies were incomplete.

Original: We now bring nuance to some recommendations and conclusions made in these studies.
Revised: We now bring nuance to some recommendations and conclusions made in these studies.

Original: • Focusing on a single metric.
Revised: • Focusing on a single metric.

Original: “ We see the following potential prescriptive outcomes (...) correlation between OOD and ID performance can simplify model development since we can focus on a single metric . ”
Revised: “ We see the following potential prescriptive outcomes (...) correlation between OOD and ID performance can simplify model development since we can focus on a single metric . ”

Original: (Miller et al., 2021)
Revised: (Miller et al., 2021)

Original: We demonstrated that inverse correlations are a possibility, hence there exist scenarios where an
Revised: We demonstrated that inverse correlations are a possibility, hence there exist scenarios where an

Original: ID metric would be misleading.
Revised: ID metric would be misleading.

Original: In general, relying on a single metric during model development is ill-advised (Teney et al., 2020b).
Revised: In general, relying on a single metric during model development is ill-advised (Teney et al., 2020b).

Original: Even more so here since it cannot capture trade-offs along multiple axes.
Revised: Even more so here since it cannot capture trade-offs along multiple axes.

Original: A model with a suboptimal ID performance may have learned features that enable better OOD generalization.
Revised: A model with a suboptimal ID performance may have learned features that enable better OOD generalization.

Original: Our recommendation is to track multiple metrics e.g. performance on multiple distributions or qualitative interpretable predictions on representative test points.
Revised: Our recommendation is to track multiple metrics e.g. performance on multiple distributions or qualitative interpretable predictions on representative test points.

Original: Improving ID performance for OOD robustness.
Revised: Improving ID performance for OOD robustness.

Original: “ If practitioners want to make the model more robust on OOD data, the main focus should be to improve the ID classification error .
Revised: “ If practitioners want to make the model more robust on OOD data, the main focus should be to improve the ID classification error .

Original: We speculate that the risk of overfitting large pretrained models to the downstream test set is minimal, and it seems to be not a good strategy to, e.g., reduce the capacity of the model in the hope of better OOD generalization. ”
Revised: We speculate that the risk of overfitting large pretrained models to the downstream test set is minimal, and it seems to be not a good strategy to, e.g., reduce the capacity of the model in the hope of better OOD generalization. ”

Original: (Wenzel et al., 2022)
Revised: (Wenzel et al., 2022)

Original: This recommendation assumes the persistence of a positive correlation.
Revised: This recommendation assumes the persistence of a positive correlation.

Original: On the opposite, we saw that a positive correlation can precede a regime of inverse correlation (Figure 4, left panels).
Revised: On the opposite, we saw that a positive correlation can precede a regime of inverse correlation (Figure 4, left panels).

Original: If the goal is to improve OOD, focusing on ID performance is a blind alley since this goal requires to increase ID performance at times, and reduce it at other times.
Revised: If the goal is to improve OOD, focusing on ID performance is a blind alley since this goal requires to increase ID performance at times, and reduce it at other times.

Original: • Future achievable OOD performance.
Revised: • Future achievable OOD performance.

Original: As obvious as it is, it feels necessary to point out that empirical studies only chart regimes achievable with existing methods.
Revised: As obvious as it is, it feels necessary to point out that empirical studies only chart regimes achievable with existing methods.

Original: Observations have limited predictive power, hence more care seems warranted when deriving prescriptive recommendations from empirical evidence.
Revised: Observations have limited predictive power, hence more care seems warranted when deriving prescriptive recommendations from empirical evidence.

Original: Evidently, the best possible performance on Camelyon17 is not limited to the Pareto front observed in our experiments.
Revised: Evidently, the best possible performance on Camelyon17 is not limited to the Pareto front observed in our experiments.

Original: For example,
Revised: For example,

Original: the current state of the art on this dataset (Robey et al., 2021; WILDS Leaderboard) injects additional task-relevant knowledge to bypass the under/misspecification of the ERM objective, and exceeds both our highest ID and OOD performance.
Revised: the current state of the art on this dataset (Robey et al., 2021; WILDS Leaderboard) injects additional task-relevant knowledge to bypass the under/misspecification of the ERM objective, and exceeds both our highest ID and OOD performance.

Original: The important message remains that a given hypothesis class (DenseNet architecture in our case) admits parametrizations whose ID and OOD performance do not necessarily correlate.
Revised: The important message remains that a given hypothesis class (DenseNet architecture in our case) admits parametrizations whose ID and OOD performance do not necessarily correlate.

Original: • Possible invalidation of existing studies.
Revised: • Possible invalidation of existing studies.

Original: The possibility of inverse correlations may invalidate studies that implicitly assume a positive one.
Revised: The possibility of inverse correlations may invalidate studies that implicitly assume a positive one.

Original: For example, Angarano et al.
Revised: For example, Angarano et al.

Original: (2022) evaluate the OOD robustness of backbone computer vision architectures.
Revised: (2022) evaluate the OOD robustness of backbone computer vision architectures.

Original: They find that modern architectures surpass domain generalization (DG) methods.
Revised: They find that modern architectures surpass domain generalization (DG) methods.

Original: However, they discard any model but those with the highest ID performance, a.k.a.
Revised: However, they discard any model but those with the highest ID performance, a.k.a.

Original: “ training domain validation ” in Gulrajani & Lopez-Paz (2021).
Revised: “ training domain validation ” in Gulrajani & Lopez-Paz (2021).

Original: This means that any model with a high
Revised: This means that any model with a high

Original: OOD performance but non-optimal ID is ignored.
Revised: OOD performance but non-optimal ID is ignored.

Original: They also train every model for a fixed, large number of epochs (30).
Revised: They also train every model for a fixed, large number of epochs (30).

Original: This may additionally prevent from finding models with high OOD performance since robustness is progressively lost during fine-tuning (Andreassen et al., 2021).
Revised: This may additionally prevent from finding models with high OOD performance since robustness is progressively lost during fine-tuning (Andreassen et al., 2021).

Original: By design, this study is therefore incapable of finding OOD benefits of any architecture / hyperparameters / DG method that requires trading off some ID performance.
Revised: By design, this study is therefore incapable of finding OOD benefits of any architecture / hyperparameters / DG method that requires trading off some ID performance.

Original: Most importantly, once the implicit assumption of a positive correlation is enacted by throwing away models with non-maximal ID performance, there is no more opportunity to demonstrate its validity.  
Revised: Most importantly, once the implicit assumption of a positive correlation is enacted by throwing away models with non-maximal ID performance, there is no more opportunity to demonstrate its validity.  R ELATED WORK

Revised: Inverse correlations can be found in the literature, although not always highlighted by their authors

Original: D ISCUSSION
Revised: • An close examination of Gokhale et al. (2022, Table 2) reveals a clear inverse pattern with three benchmarks for natural language inference (NLI). The task is known for biases and shortcuts in the training data, and the OOD test sets in these benchmarks correspond to severe distribution shifts. Our proposed explanation (right end of the spectum in Figure 6) therefore aligns with these observations.

Revised: Experiments on question answering from the same authors use data with milder distribution shifts.

Revised: Correspondingly, they show instead a positive correlation.

Revised: • Kaplun et al.

Revised: (2022, Figure 7) find that the CIFAR dataset contains a subset (CIFAR-10-Neg) on which the performance of visual classifiers is inversely correlated with their ID performance.

Revised: • Xie et al.

Revised: (2020, Section 5.3) discusses cases of inverse correlation on the CelebA dataset with their In-N-Out method – albeit within an overall positive trend.

Revised: • McCoy et al.

Revised: (2019) show that BERT models trained with different random seeds vary widely on their performance on the HANS benchmark for NLI, while their ID performance on the MNLI dataset is similar.

Revised: • Naganuma et al.

Revised: (2022) published an extensive evaluation of OOD benchmarks after the initial release of this paper.

Revised: They consider a much wider range of hyperparameters than previous studies.

Revised: As expected from our claims, they do observe new patterns of inverse correlation.

Revised: This brings strong empirical evidence that supports the message of this paper.

Revised: • Work on adversarial examples has examined the trade-off between standard and adversariallyrobust accuracy Raghunathan et al.

Revised: ; Yang et al.

Revised: ; Zhang et al.

Revised: This agrees with our explanation since adversarial inputs correspond to extreme distribution shifts.

Revised: • Work on transfer learning has occasionally shown cases of negative transfer, a related phenomenon where improving performance in one domain hurts in others.

Revised:  D ISCUSSION

Original: This paper showed that inverse correlations between ID / OOD performance are possible not only theoretically, but also happen in real-world data.
Revised: This paper showed that inverse correlations between ID / OOD performance are possible not only theoretically, but also happen in real-world data.

Original: We do not know how frequent this situation is.
Revised: We do not know how frequent this situation is.

Original: Although we examined a single counterexample, 5 we also showed that past studies may have systematically overlooked such cases.
Revised: Although we examined a single counterexample, 5 we also showed that past studies may have systematically overlooked such cases.

Original: This suffices to show that one cannot know a priori where a given task falls on the spectrum of Figure 6.
Revised: This suffices to show that one cannot know a priori where a given task falls on the spectrum of Figure 6.

Original: It is thus ill-advised to blindly make the assumption of a positive correlation , which was suggested in the past.
Revised: It is thus ill-advised to blindly make the assumption of a positive correlation , which was suggested in the past.

Original: Can we avoid inverse correlations with a larger training set?
Revised: Can we avoid inverse correlations with a larger training set?

Original: Scaling alone without data curation seems unlikely to prevent inverse correlations.
Revised: Scaling alone without data curation seems unlikely to prevent inverse correlations.

Original: Fang et al.
Revised: Fang et al.

Original: (2022) examined a more general question and determined that the impressive robustness of the large vision-and-language model CLIP is determined by the distribution of its training data rather than its quantity.
Revised: (2022) examined a more general question and determined that the impressive robustness of the large vision-and-language model CLIP is determined by the distribution of its training data rather than its quantity.

Original: Similarly, inverse correlations stem from biases in the training distribution (e.g. a class C 1 appearing more frequently with image background B 1 than any other).
Revised: Similarly, inverse correlations stem from biases in the training distribution (e.g. a class C 1 appearing more frequently with image background B 1 than any other).

Original: And biases in a distribution do not vanish with more i.i.d.
Revised: And biases in a distribution do not vanish with more i.i.d.

Original: samples.
Revised: samples.

Original: Indeed, more data can cover more of the support of the distribution, but this coverage will remain uneven, i.e. biased.
Revised: Indeed, more data can cover more of the support of the distribution, but this coverage will remain uneven, i.e. biased.

Original: The problem can become one of “subpopulation shift” (Santurkar et al., 2020) rather than distribution shift, but it remains similarly challenging.
Revised: The problem can become one of “subpopulation shift” (Santurkar et al., 2020) rather than distribution shift, but it remains similarly challenging.

Original: Training full networks with a diversity-inducing method.
Revised: Training full networks with a diversity-inducing method.

Original: We showed inverse correlations with standard ERM models and – even more strikingly – with linear classifiers trained with a diversityinducing method (Teney et al., 2022a).
Revised: We showed inverse correlations with standard ERM models and – even more strikingly – with linear classifiers trained with a diversityinducing method (Teney et al., 2022a).

Original: To the best of our knowledge, this method has not been applied to deep models because of its computational expense.
Revised: To the best of our knowledge, this method has not been applied to deep models because of its computational expense.

Original: It would be interesting to confirm our observations on networks trained entirely with this or other diversity-inducing methods (Section 3).
Revised: It would be interesting to confirm our observations on networks trained entirely with this or other diversity-inducing methods (Section 3).

Original: Qualitative differences along the Pareto frontier.
Revised: Qualitative differences along the Pareto frontier.

Original: We focused on quantitative performance.
Revised: We focused on quantitative performance.

Original: Interpretability methods could also be used to examine whether models of various ID / OOD trade-offs rely on different features and generalization strategies, as done in NLP by Juneja et al.
Revised: Interpretability methods could also be used to examine whether models of various ID / OOD trade-offs rely on different features and generalization strategies, as done in NLP by Juneja et al.

Original: Recent work by Eastwood et al.
Revised: Recent work by Eastwood et al.

Original: (2022) also recognizes issues of model selection for OOD generalization.
Revised: (2022) also recognizes issues of model selection for OOD generalization.

Original: They get around selection based on either ID or OOD validation performance with a new  domain generalization method (Quantile Risk Minimization) with a tunable trade-off.
Revised: They get around selection based on either ID or OOD validation performance with a new The dataset was not picked post hoc because of unusual results. domain generalization method (Quantile Risk Minimization) with a tunable trade-off.

Original: Other exciting advances by Wang & Veitch (2022) examine existing approaches to improve OOD performance with their suitability to different distribution shifts and underlying causal structures.
Revised: Other exciting advances by Wang & Veitch (2022) examine existing approaches to improve OOD performance with their suitability to different distribution shifts and underlying causal structures.

Original: A A DDITIONAL RESULTS
Revised: A A DDITIONAL RESULTS

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: cc u r a c
Revised: cc u r a c

Original: y
Revised: y

Original: ( % , v a l - oo d )
Revised: ( % , v a l - oo d )

Original: [Figure]
Revised: [Figure]

Original: c y
Revised: c y

Original: [Figure]
Revised: [Figure]

Original: cc u r a c y (
Revised: cc u r a c y (

Original: [Figure]
Revised: [Figure]

Original: v a l - oo d )
Revised: v a l - oo d )

Original: [Figure]
Revised: [Figure]

Original: a l
Revised: a l

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: B P ROOF OF T HEOREM
Revised: B P ROOF OF T HEOREM

Original: Let x ˆ d :=
Revised: Let x ˆ d :=

Original: Φ ˆ d ( m x )[ x inv , 1 , ..., x inv , ˆ d inv , x spu , 1 , ..., x spu , ˆ d spu ] be the ˆ d features already selected and x ˆ d +1 :
Revised: Φ ˆ d ( m x )[ x inv , 1 , ..., x inv , ˆ d inv , x spu , 1 , ..., x spu , ˆ d spu ] be the ˆ d features already selected and x ˆ d +1 :

Original: =
Revised: =

Original: Φ ˆ d +1 ( m x ) the features after adding a new spurious feature x spu , ˆ d spu +1 to x ˆ d .
Revised: Φ ˆ d +1 ( m x ) the features after adding a new spurious feature x spu , ˆ d spu +1 to x ˆ d .

Original: Let [ λ ˆ d 1 , λ ˆ d 2 , ..., λ ˆ d ˆ d ] denote eigenvalues of E [ x ⊤ ˆ d x ˆ d ] and [ v ˆ d 1 , v ˆ d 2 , ..., v ˆ d ˆ d ] corresponding eigenvectors.
Revised: Let [ λ ˆ d 1 , λ ˆ d 2 , ..., λ ˆ d ˆ d ] denote eigenvalues of E [ x ⊤ ˆ d x ˆ d ] and [ v ˆ d 1 , v ˆ d 2 , ..., v ˆ d ˆ d ] corresponding eigenvectors.

Original: Assumption 1.
Revised: Assumption 1.

Original: The projection of E [ x ⊺ ˆ d v ˆ di ] on each basis corresponding to feature is non zero, i.e.
Revised: The projection of E [ x ⊺ ˆ d v ˆ di ] on each basis corresponding to feature is non zero, i.e.

Original: [Equation]
Revised: [Equation]

Original: This ensures that coefficients of a feature can not be always 0 , otherwise we can simply remove it.
Revised: This ensures that coefficients of a feature can not be always 0 , otherwise we can simply remove it.

Original: Proof.
Revised: Proof.

Original: Let β ID and β OOD denote the solution of linear regression in the ID and OOD domains, i.e.,
Revised: Let β ID and β OOD denote the solution of linear regression in the ID and OOD domains, i.e.,

Original: [Equation]
Revised: [Equation]

Original: Now let us compare the OOD loss after we include x spu , ˆ d spu +1 .
Revised: Now let us compare the OOD loss after we include x spu , ˆ d spu +1 .

Original: In practice, we can only obtain β ID and then apply it on both the ID and OOD domains, which elicits the following errors:
Revised: In practice, we can only obtain β ID and then apply it on both the ID and OOD domains, which elicits the following errors:

Original: [Equation]
Revised: [Equation]

Original: It is well known that the residual of the linear fitting y by x ˆ d on the ID domain is
Revised: It is well known that the residual of the linear fitting y by x ˆ d on the ID domain is

Original: [Equation]
Revised: [Equation]

Original: Similarly, we have
Revised: Similarly, we have

Original: [Equation]
Revised: [Equation]

Original: Since x spu , ˆ d spu+1 does not lies in the space spaned by x ˆ d , so the space spanned by x ˆ d +1 is strictly larger than x ˆ d .
Revised: Since x spu , ˆ d spu+1 does not lies in the space spaned by x ˆ d , so the space spanned by x ˆ d +1 is strictly larger than x ˆ d .

Original: Togther with Assumption 1, we have
Revised: Togther with Assumption 1, we have

Original: [Equation]
Revised: [Equation]

Original: 9) and also
Revised: 9) and also

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: By the proof in Appendix B.6.3 (above Eq. 29) in Zhou et al.
Revised: By the proof in Appendix B.6.3 (above Eq. 29) in Zhou et al.

Original: (2022), we have
Revised: (2022), we have

Original: [Equation]
Revised: [Equation]

Original: By Eq.
Revised: By Eq.

Original: (20) in Zhou et al.
Revised: (20) in Zhou et al.

Original: (2022), we have
Revised: (2022), we have

Original: [Equation]
Revised: [Equation]

Original: So we have:
Revised: So we have:

Original: [Equation]
Revised: [Equation]

Original: From Eq.
Revised: From Eq.

Original: 11 and 13, we have the desired result.
Revised: 11 and 13, we have the desired result

Revised: [Equation]