Original: U NDERSTANDING THE FAILURE MODES OF OUT - OFDISTRIBUTION GENERALIZATION
Revised: U NDERSTANDING THE FAILURE MODES OF OUT - OFDISTRIBUTION GENERALIZATION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Empirical studies suggest that machine learning models often rely on features, such as the background, that may be spuriously correlated with the label only during training time, resulting in poor accuracy during test-time.
Revised: Empirical studies suggest that machine learning models often rely on features, such as the background, that may be spuriously correlated with the label only during training time resulting in poor accuracy during test- time

Original: In this work, we identify the fundamental factors that give rise to this behavior, by explaining why models fail this way even in easy-to-learn tasks where one would expect these models to succeed.
Revised: In this work, we identify the fundamental factors that give rise to this behavior , by explaining why models fail this way even in easy-to-learn tasks where one would expect these models to succeed.

Original: In particular, through a theoretical study of gradient-descenttrained linear classiﬁers on some easy-to-learn tasks, we uncover two complementary failure modes.
Revised: In particular , through a theoretical study of gradient-descenttrained linear classiﬁers on some easy -to- learn tasks, we uncover two complementary failure modes.

Original: These modes arise from how spurious correlations induce two kinds of skews in the data: one geometric in nature, and another, statistical in nature.
Revised: These modes arise from how spurious correlations induce two kinds of skews in the data : one geometric in nature, and another statistical in na - ture.

Original: Finally, we construct natural modiﬁcations of image classiﬁcation datasets to understand when these failure modes can arise in practice.
Revised: Finally, we construct natural modiﬁcations of image classiﬁcation datasets to understand when these failure modes can arise in practice.

Original: We also design experiments to isolate the two failure modes when training modern neural networks on these datasets.
Revised: We also design experiments to isolate the two failure modes when training modern neural networks on these datasets

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: A machine learning model in the wild (e.g., a self-driving car) must be prepared to make sense of its surroundings in rare conditions that may not have been well-represented in its training set.
Revised: A machine learning model in the wild (e.g., a self-driving car) must be prepared to make sense of its surroundings in rare conditions that may not have been well-represented in its training set.

Original: This could range from conditions such as mild glitches in the camera to strange weather conditions.
Revised: This could range from conditions such as mild glitches in the camera to strange weather conditions.

Original: This out-of-distribution (OoD) generalization problem has been extensively studied within the framework of the domain generalization setting (Blanchard et al., 2011; Muandet et al., 2013).
Revised: This out-of-distribution (OoD) generalization problem has been extensively studied within the framework of the domain generalization setting (Blanchard et al., 2011; Muandet et al., 2013).

Original: Here, the classiﬁer has access to training data sourced from multiple “domains” or distributions, but no data from test domains.
Revised: Here, the classiﬁer has access to training data sourced from multiple “domains” or distributions, but no data from test domains.

Original: By observing the various kinds of shifts exhibited by the training domains, we want the classiﬁer can learn to be robust to such shifts.
Revised: By observing the various kinds of shifts exhibited by the training domains, we want the classiﬁer can learn to be robust to such shifts.

Original: The simplest approach to domain generalization is based on the Empirical Risk Minimization (ERM) principle (Vapnik, 1998): pool the data from all the training domains (ignoring the “domain label” on each point) and train a classiﬁer by gradient descent to minimize the average loss on this pooled dataset.
Revised: The simplest approach to domain generalization is based on the Empirical Risk Minimization (ERM) principle (Vapnik, 1998): pool the data from all the training domains (ignoring the “domain label” on each point) and train a classiﬁer by gradient descent to minimize the average loss on this pooled dataset.

Original: Alternatively, many recent studies (Ganin et al., 2016; Arjovsky et al., 2019; Sagawa et al., 2020a) have focused on designing more sophisticated algorithms that do utilize the domain label on the datapoints e.g., by enforcing certain representational invariances across domains.
Revised: Alternatively, many recent studies (Ganin et al., 2016; Arjovsky et al., 2019; Sagawa et al., 2020a) have focused on designing more sophisticated algorithms that do utilize the domain label on the datapoints e.g., by enforcing certain representational invariances across domains.

Original: A basic premise behind pursuing such sophisticated techniques, as emphasized by Arjovsky et al.
Revised: A basic premise behind pursuing such sophisticated techniques, as emphasized by Arjovsky et al.

Original: (2019), is the empirical observation that ERM-based gradient-descent-training (or for convenience, just ERM) fails in a characteristic way.
Revised: (2019), is the empirical observation that ERM-based gradient-descent-training (or for convenience, just ERM) fails in a characteristic way.

Original: As a standard illustration, consider a cow-camel classiﬁcation task (Beery et al., 2018) where the background happens to be spuriously correlated with the label in a particular manner only during training — say, most cows are found against a grassy background and most camels against a sandy one.
Revised: As a standard illustration, consider a cow-camel classiﬁcation task (Beery et al., 2018) where the background happens to be spuriously correlated with the label in a particular manner only during training — say, most cows are found against a grassy background and most camels against a sandy one.

Original: Then, during test-time, if the correlation is completely ﬂipped (i.e., all cows in deserts, and all camels in meadows), one would observe that the accuracy of ERM drops drastically.
Revised: Then, during test-time, if the correlation is completely ﬂipped (i.e., all cows in deserts, and all camels in meadows), one would observe that the accuracy of ERM drops drastically.

Original: Evidently, ERM, in its unrestrained attempt at ﬁtting the data, indiscriminately relies onall kinds of informative features, including unreliable spurious features like the background.
Revised: Evidently, ERM, in its unrestrained attempt at ﬁtting the data, [Figure] classiﬁers ( margin) rely on the spurious feature in many real Fig

Original:  However, an algorithm that carefully uses domain label information can hope to identify and rely purely on invariant features (such as the shape of the animal) that are consistently informative across domains.
Revised: We identify two fundamental factors behind this behavior. In doing so, we also identify and explain other kinds of vulnerabilities such as the one in Fig 1d (see Sec 4) indiscriminately relies on all kinds of informative features, including unreliable spurious features like the background. However, an algorithm that carefully uses domain label information can hope to identify and rely purely on invariant features (or “core” features (Sagawa et al., 2020b)).

Original: While the above narrative is an oft-stated motivation behind developing sophisticated OoD generalization algorithms, there is little formal explanation as to why ERM fails in this characteristic way.
Revised: While the above narrative is an oft-stated motivation behind developing sophisticated OoD generalization algorithms, there is little formal explanation as to why ERM fails in this characteristic way.

Original: Existing works (Sagawa et al., 2020b; Tsipras et al., 2019; Arjovsky et al., 2019; Shah et al., 2020)[Figure] provide valuable answers to this question through concrete theoretical examples; however, their examples critically rely on certain factors to make the task difﬁcult enough for ERM to rely on the spurious features.
Revised: Existing works (Sagawa et al., 2020b; Tsipras et al., 2019; Arjovsky et al., 2019; Shah et al., 2020) provide valuable answers to this question through concrete theoretical examples; however, their examples critically rely on certain factors to make the task difﬁcult enough for ERM to rely on the spurious features.

Original: For example, many of these examples have invariant features that are only partially predictive of the label (see Fig 1a).
Revised: For instance, many of these examples have invariant features that are only partially predictive of the label (see Fig 1a).

Original: Surprisingly though, ERM relies on spurious features even in much easier-to-learn tasks where these complicating factors are absent — such as in tasks with fully predictive invariant features e.g., Fig 1c or the Waterbirds/CelebA examples in Sagawa et al.
Revised: Surprisingly though, ERM relies on spurious features even in much easier-to-learn tasks where these complicating factors are absent — such as in tasks with fully predictive invariant features e.g., Fig 1c or the Waterbirds/CelebA examples in Sagawa et al.

Original: (2020a) or for that matter, in any real-world situation where the object shape perfectly determines the label.
Revised: (2020a) or for that matter, in any real-world situation where the object shape perfectly determines the label.

Original: This failure in easy-to-learn tasks, as we argue later, is not straightforward to explain (see Fig 1b for brief idea).
Revised: This failure in easy-to-learn tasks, as we argue later, is not straightforward to explain (see Fig 1b for brief idea).

Original: This evidently implies that there must exist factors more general and fundamental than those known so far, that cause ERM to fail.
Revised: This evidently implies that there must exist factors more general and fundamental than those known so far, that cause ERM to fail.

Original: Our goal in this work is to uncover these fundamental factors behind the failure of ERM.
Revised: Our goal in this work is to uncover these fundamental factors behind the failure of ERM.

Original: The hope is that this will provide a vital foundation for future work to reason about OoD generalization.
Revised: The hope is that this will provide a vital foundation for future work to reason about OoD generalization.

Original: Indeed, recent empirical work (Gulrajani & Lopez-Paz, 2020) has questioned whether existing alternatives necessarily outperform ERM on OoD tasks; however, due to a lack of theory, it is not clear how to hypothesize about when/why one algorithm would outperform another here.
Revised: Indeed, recent empirical work (Gulrajani & Lopez-Paz, 2020) has questioned whether existing alternatives necessarily outperform ERM on OoD tasks; however, due to a lack of theory, it is not clear how to hypothesize about when/why one algorithm would outperform another here.

Original: Through our theoretical study, future work can hope to be better positioned to precisely identify the key missing components in these algorithms, and bridge these gaps to better solve the OoD generalization problem.
Revised: Through our theoretical study, future work can hope to be better positioned to precisely identify the key missing components in these algorithms, and bridge these gaps to better solve the OoD generalization problem.

Original: Our contributions.
Revised: Our contributions.

Original: To identify the most fundamental factors causing OoD failure, our strategy is to (a) study tasks that are “easy” to succeed at, and (b) to demonstrate that ERM relies on spurious features despite how easy the tasks are.
Revised: To identify the most fundamental factors causing OoD failure, our strategy is to (a) study tasks that are “easy” to succeed at, and (b) to demonstrate that ERM relies on spurious features despite how easy the tasks are.

Original: More concretely:
Revised: More concretely:

Original: We formulate a set of constraints on how our tasks must be designed so that they are easy to succeed at (e.g., the invariant feature must be fully predictive of the label).
Revised: We formulate a set of constraints on how our tasks must be designed so that they are easy to succeed at (e.g., the invariant feature must be fully predictive of the label).

Original: Notably, this class of easy-to-learn tasks provides both a theoretical test-bed for reasoning about OoD generalization and also a simpliﬁed empirical test-bed.
Revised: Notably, this class of easy-to-learn tasks provides both a theoretical test-bed for reasoning about OoD generalization and also a simpliﬁed empirical test-bed.

Original: In particular, this class encompasses simpliﬁed MNIST and CIFAR10-based classiﬁcation tasks where we establish empirical failure of ERM. 2.
Revised: In particular, this class encompasses simpliﬁed MNIST and CIFAR10-based classiﬁcation tasks where we establish empirical failure of ERM. 2.

Original: We identify two complementary mechanisms of failure of ERM that arise from how spurious correlations induce two kinds of skews in the data: one that is geometric and the other statistical .
Revised: We identify two complementary mechanisms of failure of ERM that arise from how spurious correlations induce two kinds of skews in the data: one that is geometric and the other statistical .

Original: In particular, we theoretically isolate these failure modes by studying linear classiﬁers trained by gradient descent (on logistic/exponential loss) and its inﬁnite-time-trained equivalent, the maxmargin classiﬁer (Soudry et al., 2018; Ji & Telgarsky, 2018) on the easy-to-learn-tasks.
Revised: In particular, we theoretically isolate these failure modes by studying linear classiﬁers trained by gradient descent (on logistic/exponential loss) and its inﬁnite-time-trained equivalent, the maxmargin classiﬁer (Soudry et al., 2018; Ji & Telgarsky, 2018) on the easy-to-learn-tasks.

Original: We also show that in any easy-to-learn task that does not have these geometric or statistical skews, these models do not rely on the spurious features.
Revised: We also show that in any easy-to-learn task that does not have these geometric or statistical skews, these models do not rely on the spurious features.

Original: This suggests that these skews are not only a sufﬁcient but also a necessary factor for failure of these models in easy-to-learn tasks.
Revised: This suggests that these skews are not only a sufﬁcient but also a necessary factor for failure of these models in easy-to-learn tasks.

Original: To empirically demonstrate the generality of our theoretical insights, we (a) experimentally validate these skews in a range of MNIST and CIFAR10-based tasks and (b) demonstrate their effects on fully-connected networks (FNNs) and ResNets.
Revised: To empirically demonstrate the generality of our theoretical insights, we (a) experimentally validate these skews in a range of MNIST and CIFAR10-based tasks and (b) demonstrate their effects on fully-connected networks (FNNs) and ResNets.

Original: We also identify and explain failure in scenar- ios where standard notions of spurious correlations do not apply (see Fig 1d).
Revised: We also identify and explain failure in scenarios where standard notions of spurious correlations do not apply (see Fig 1d).

Original: We perform similar experiments on a non-image classiﬁcation task in App E.
Revised: We perform similar experiments on a non-image classiﬁcation task in App E.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Revised: Spurious correlations.

Revised: Empirical work has shown that deep networks ﬁnd superﬁcial ways to predict the label, such as by relying on the background (Beery et al., 2018;

Revised: Ribeiro et al., 2016) or other kinds of shortcuts (McCoy et al., 2019; Geirhos et al., 2020).

Original:  Understanding failure of ERM.
Revised: Such behavior is of practical concern because accuracy can deteriorate under shifts in those features (Rosenfeld et al., 2018; Hendrycks & Dietterich, 2019). It can also lead to unfair biases and poor performance on minority groups (Dixon et al., 2018; Zhao et al., 2017; Sagawa et al., 2020b). Understanding failure of ERM.

Original: While the fact that ERM relies on spurious correlations has become empirical folk wisdom, only a few studies have made efforts to carefully model this.
Revised: While the fact that ERM relies on spurious correlations has become empirical folk wisdom, only a few studies have made efforts to carefully model this.

Original: Sagawa et al. (2020b) explain how themax-margin classiﬁer relies on spurious correlations in a concrete, linear setting with a single invariantfeature and  a single spurious feature (and many “non-sense” features).
Revised: Broadly, there are two kinds of existing models that explain this phenomenon. One existing model is to imagine that both the invariant and the spurious features are only partially predictive of the label (Tsipras et al., 2019; Sagawa et al., 2020b; Arjovsky et al., 2019; Ilyas et al., 2019; Khani & Liang, 2020), as a result of which the classiﬁer that maximizes accuracy cannot ignore the spurious feature (see Fig 1a). The other existing model is based on the “simplicity bias” of gradient-descent based deep network training (Rahaman et al., 2018; Neyshabur et al., 2015;

Original: Tsipras et al. (2019) similarly explain failure of ERM in a setting with high-dimensional spurious features.
Revised: Kalimeris et al., 2019; Arpit et al., 2017; Xu et al., 2019; des Combes et al., 2018).

Original: Both these works provide concrete upper bounds on OoD accuracy, demonstrating failure under the crucial condition that  the invariant feature is sufﬁciently noisy (i.e., it is only partially informative of the label). Shah et al. (2020) take an important step forward by studying a fully predictive  invariant feature.
Revised: In particular, this model typically assumes that both the invariant and spurious features are fully predictive of the label, but crucially posits that the spurious features are more simpler to learn (e.g., more linear) than the invariant features, and therefore gradient descent prefers to use them (Shah et al., 2020;

Original: In particular, their invariant feature can separate the data with a concretely-deﬁned non-linear boundary, while the spurious feature helps separate (most of) the data with a simple linear boundary.
Revised: Nar et al., 2019; Hermann & Lampinen, 2020).

Original: Their insight is that neural networks in particular are biased towards simpler boundaries, and hence rely on the spurious feature.
Revised: While both these models offer simple-to-understand and useful explanations for why classiﬁers use spurious correlations, we argue that a more fundamental explanation exists.

Original: In contrast to these works, we demonstrate and explain how ERM can rely on the spurious feature even when the invariant feature is fully predictive and  corresponds to a simple linear boundary. Furthermore, we study a much more abstract class of tasks (without a concretely-deﬁned distribution) in order derive more general insights. Importantly, our class of tasks is able to faithfully model the key easy-to-learn aspects of the cow/camel task that is not captured by prior studies. Note that because we study an abstract setting, we do not provide concrete OoD accuracy bounds.
Revised: In particular, we empirically and theoretically demonstrate how ERM can rely on the spurious feature even in much easier tasks where these explanations would fall apart: these are tasks where unlike in the ﬁrst model (a) the invariant feature is fully predictive and unlike in the second model, (b) the invariant feature corresponds to a simple linear boundary and (c) the spurious feature is not fully predictive of the label.

Original: We also go beyond the max-margin setting and analyze the dynamics ofa ﬁnite-time gradient-descent trained classiﬁer on logistic loss.
Revised: Further, we go beyond the max-margin settings analyzed in these works to analyze the dynamics of ﬁnite-time gradient-descent trained classiﬁer on logistic loss.

Original: Finally, we also refer the reader to an orthogonal line of theory work (Fawzi et al., 2018; Gilmer et al., 2018) that has investigated the failure of standard classiﬁers in the adversarial perturbation setting.
Revised: We would also like to point the reader to concurrent work of Khani & Liang (2021) that has proposed a different model addressing the above points. While their model sheds insight into the role of overparameterization in the context of spurious features (and our results are agnostic to that), their model also requires the spurious feature to be “dependent” on the invariant feature, an assumption we don’t require (see Sec 3).

Original: Algorithms for OoD generalization.
Revised: Algorithms for OoD generalization.

Original: Due to the empirical shortcomings of ERM, a wide range of sophisticated algorithms have been developed for domain generalization.
Revised: Due to the empirical shortcomings of ERM, a wide range of sophisticated algorithms have been developed for domain generalization.

Original: The most popular strategy is to learn useful features while constraining them to have similar distributions across domains (Ganin et al., 2016; Li et al., 2018b; Albuquerque et al., 2020).
Revised: The most popular strategy is to learn useful features while constraining them to have similar distributions across domains (Ganin et al., 2016; Li et al., 2018b; Albuquerque et al., 2020).

Original: Other works constrain these features in a way that one can learn a classiﬁer that is simultaneously optimal across all domains (Peters et al., 2016; Arjovsky et al., 2019; Krueger et al., 2020).
Revised: Other works constrain these features in a way that one can learn a classiﬁer that is simultaneously optimal across all domains (Peters et al., 2016; Arjovsky et al., 2019; Krueger et al., 2020).

Original: As discussed in Gulrajani & Lopez-Paz (2020), there are also many other existing non-ERM based methods, including that of meta-learning (Li et al., 2018a), parameter-sharing (Sagawa et al., 2020a) and data augmentation (Zhang et al., 2018).
Revised: As discussed in Gulrajani & Lopez-Paz (2020), there are also many other existing non-ERM based methods, including that of meta-learning (Li et al., 2018a), parameter-sharing (Sagawa et al., 2020a) and data augmentation (Zhang et al., 2018).

Original: Through their extensive empirical survey of many of the above algorithms, Gulrajani & Lopez-Paz (2020) suggest that ERM may be just as competitive as the state-of-the-art.
Revised: Through their extensive empirical survey of many of the above algorithms, Gulrajani & Lopez-Paz (2020) suggest that ERM may be just as competitive as the state-of-the-art.

Original: But we must emphasize that this doesn’t vindicate ERM of its failures but rather indicates that we are yet to develop a substantial improvement over ERM.
Revised: But we must emphasize that this doesn’t vindicate ERM of its failures but rather indicates that we may be yet to develop a substantial improvement over ERM.

Original:  E ASY - TO - LEARN DOMAIN GENERALIZATION TASKS
Revised:  E ASY - TO - LEARN DOMAIN GENERALIZATION TASKS

Original: Below, we ﬁrst set up the basic domain generalization setting and the idea of ERM.
Revised: Below, we ﬁrst set up the basic domain generalization setting and the idea of ERM.

Original: Then, in Section 3.1, we formulate a class of domain-generalization tasks that are in many aspects “easy” for the learnerto succeed at (such as fully informative invariant features — what  makes a task “easy” will bemore precisely discussed in Section 3.1).
Revised: Then, in Section 3.1, we formulate a class of domain-generalization tasks that are in many aspects “easy” for the learner (such as fully informative invariant features) – what exactly makes a task “easy” will be discussed in Section 3.1).

Original: This discussion sets the ground for the later sections to show how ERM can fail even in these easy tasks, which will help uncover the fundamental factors behind its failure.
Revised: This discussion sets the ground for the later sections to show how ERM can fail even in these easy tasks, which will help uncover the fundamental factors behind its failure.

Original: Notations.
Revised: Notations.

Original: Consider an input (vector) space X and label space Y ∈ {− 1 , 1 } .
Revised: Consider an input (vector) space X and label space Y ∈ {− 1 , 1 } .

Original: For any distribution
Revised: For any distribution

Original: D over X × Y , let p D ( · ) denote its probability density function (PDF).
Revised: D over X × Y , let p D ( · ) denote its probability density function (PDF).

Original: Let H denote a class of classiﬁers h : X → R .
Revised: Let H denote a class of classiﬁers h : X → R .

Original: Let the error of h on D be denoted as L D ( h ) :
Revised: Let the error of h on D be denoted as L D ( h ) :

Original: =
Revised: =

Original: E ( x ,y ) ∼D
Revised: E ( x ,y ) ∼D

Original: [ h ( x ) · y < 0] .
Revised: [ h ( x ) · y < 0] .

Original: The domain generalization setting and ERM.
Revised: The domain generalization setting and ERM.

Original: In the domain generalization setting, one considers an underlying class D of data distributions over X × Y corresponding to different possible domains.
Revised: In the domain generalization setting, one considers an underlying class D of data distributions over X × Y corresponding to different possible domains.

Original: The learner is given training data collected from multiple distributions from D .
Revised: The learner is given training data collected from multiple distributions from D .

Original: For an ERM-based learner in particular, the training data will be pooled together, so we can model the data as com- ing from a single (pooled) distribution D train , which for simplicity, can be assumed to belong to D .
Revised: For an ERM-based learner in particular, the training data will be pooled together, so we can model the data as coming from a single (pooled) distribution D train , which for simplicity, can be assumed to belong to D .

Original: Given this data, the learner outputs a hypothesis ˆ h ∈ H that is tested on a new distribution D test picked from D .
Revised: Given this data, the learner outputs a hypothesis ˆ h ∈ H that is tested on a new distribution D test picked from D .

Original: This can be potentially modeled by assuming that all test and training distributions are drawn from a common hyper-distribution over D .
Revised: This can be potentially modeled by assuming that all test and training distributions are drawn from a common hyper-distribution over D .

Original: However, this assumption becomes pointless in most practical settings(e.g., PACS (Asadi et al., 2019), VLCS (Fang et al., 2013)) where the training domains are not more than three to four in number, and therefore hardly representative of any hyper-distribution.
Revised: However, this assumption becomes pointless in most practical settings where the training domains are not more than three to four in number (e.g., PACS (Asadi et al., 2019), VLCS (Fang et al., 2013)), and therefore hardly representative of any hyper-distribution.

Original: Here, the problem becomes as hard as ensuring good performance on a worst-case test-distribution without any hyper-distribution assumption; this boils down to minimizing max D∈ D L D (ˆ h ) .
Revised: Here, the problem becomes as hard as ensuring good performance on a worst-case test-distribution without any hyper-distribution assumption; this boils down to minimizing max D∈ D L D (ˆ h ) .

Original: Indeed, most works have studied the worst-case setting, both theoretically (Sagawa et al., 2020b) and empirically (Arjovsky et al., 2019; Sagawa et al., 2020a).
Revised: Indeed, most works have studied the worst-case setting, both theoretically (Sagawa et al., 2020b) and empirically (Arjovsky et al., 2019; Sagawa et al., 2020a).

Original: Similarly, for this work, we focus on the worst-case setting and deﬁne the optimal target function h (cid:63) to be h (cid:63) =
Revised: Similarly, for this work, we focus on the worst-case setting and deﬁne the optimal target function h (cid:63) to be h (cid:63) =

Original: arg min h ∈ H max D∈ D L D ( h ) .
Revised: arg min h ∈ H max D∈ D L D ( h ) .

Original: Then, we deﬁne the features that this “robust” classiﬁer uses as invariant features X inv
Revised: Then, we deﬁne the features that this “robust” classiﬁer uses as invariant features X inv

Original: (e.g., the shape of the object), and the rest as spurious features X sp (e.g., the background).
Revised: (e.g., the shape of the object), and the rest as spurious features X sp (e.g., the background).

Original: To formalize this, we assume that there exists a mapping Φ : X inv ×X sp → X such that each D ∈ D is induced by a distribution over X inv × X sp (so we can denote any x as Φ( x inv , x sp ) ).
Revised: To formalize this, we assume that there exists a mapping Φ : X inv ×X sp → X such that each D ∈ D is induced by a distribution over X inv × X sp (so we can denote any x as Φ( x inv , x sp ) ).

Original: With an abuse of notation we will use p D ( · ) to also denote the PDF of the distribution over X inv × X sp .
Revised: With an abuse of notation we will use p D ( · ) to also denote the PDF of the distribution over X inv ×X sp .

Original: Then, the fact that X sp are features that h (cid:63) does not rely on, is mathematically stated as: ∀ x inv , ∀ x sp (cid:54) =
Revised: Then, the fact that X sp are features that h (cid:63) does not rely on, is mathematically stated as: ∀ x inv and ∀ x sp (cid:54) =

Original: x (cid:48) sp , h (cid:63) (Φ( x inv , x sp ))
Revised: x (cid:48) sp , h (cid:63) (Φ( x inv , x sp ))

Original: =
Revised: =

Original: h (cid:63) (Φ( x inv , x (cid:48) sp )) .
Revised: h (cid:63) (Φ( x inv , x (cid:48) sp )) .

Original: Finally, we note that, to make this learning problem tractable, one has to impose further restrictions; we’ll provide more details on those when we discuss the class of easy-to-learn domain generalization tasks in Sec 3.1.
Revised: Finally, we note that, to make this learning problem tractable, one has to impose further restrictions; we’ll provide more details on those when we discuss the class of easy-to-learn domain generalization tasks in Sec 3.1.

Original: Empirical failure of ERM.
Revised: Empirical failure of ERM.

Original: To guide us in constructing the easy-to-learn tasks, let us ground our study in a concrete empirical setup where an ERM-based linear classiﬁer shows OoD failure.
Revised: To guide us in constructing the easy-to-learn tasks, let us ground our study in a concrete empirical setup where an ERM-based linear classiﬁer shows OoD failure.

Original: Specifically, consider the following Binary-MNIST based task (the ﬁrst ﬁve digits and the remaining ﬁve digits form the two classes).
Revised: Specifically, consider the following Binary-MNIST based task, where the ﬁrst ﬁve digits and the remaining ﬁve digits form the two classes.

Original: First, we letthe Φmapping be the identity, and so x = ( x inv , x sp ) .
Revised: First, we let Φ be the identity mapping, and so x = ( x inv , x sp ) .

Original: Then, we let x inv be a random ReLU features representation of the MNIST digit i.e., if x raw represents theMNIST image,
Revised: Then, we let x inv be a random ReLU features representation of the MNIST digit i.e., if x raw represents the

Original:  then x inv = ReLU ( W x raw ) where W is a matrix with Gaussian entries.
Revised: MNIST image, then x inv = ReLU ( W x raw ) where W is a matrix with Gaussian entries.

Original: We make this representation sufﬁciently high-dimensional so that the data becomes linearly separable.
Revised: We make this representation sufﬁciently high-dimensional so that the data becomes linearly separable.

Original: Next, we let the spurious feature take values in { + B , −B} (for some B > 0 ) imitating the two possible background colors in the camel-cow dataset.
Revised: Next, we let the spurious feature take values in { + B , −B} for some B > 0 , imitating the two possible background colors in the camel-cow dataset.

Original: Finally, on D train , for any y , we pick the image x inv from the corresponding class and independently set the “background color” x sp so that there is some spurious correlation i.e., Pr D train [ x sp · y > 0] > 0 .
Revised: Finally, on D train , for any y , we pick the image x inv from the corresponding class and independently set the “background color” x sp so that there is some spurious correlation i.e., Pr D train [ x sp · y > 0] > 0 .

Original: During test time however, we ﬂip this correlation around so that Pr D test [ x sp · y > 0] = 0 . 0 .
Revised: During test time however, we ﬂip this correlation around so that Pr D test [ x sp · y > 0] = 0 . 0 .

Original: In this task, we observe in Fig 2a (shown later under
Revised: In this task, we observe in Fig 2a (shown later under

Original: Sec 4)
Revised: Sec 4)

Original: that as we vary the train-time spurious correlation from none ( Pr D train [ x sp · y > 0] = 0 . 5 ) to its maximum ( Pr D train [ x sp · y > 0] = 1 . 0 ), the OoD accuracy of a max-margin classiﬁer progressively deteriorates.
Revised: that as we vary the train-time spurious correlation from none ( Pr D train [ x sp · y > 0] = 0 . 5 ) to its maximum ( Pr D train [ x sp · y > 0] = 1 . 0 ), the OoD accuracy of a max-margin classiﬁer progressively deteriorates.

Original: (We present similar results for a CIFAR10 setting, and all experiment details in
Revised: (We present similar results for a CIFAR10 setting, and all experiment details in

Original: App C.1.)
Revised: App C.1.)

Original: Our goal is now to theoretically demonstrate why ERM fails this way (or equivalently, why it relies on the spurious feature) even in tasks as “easy-to-learn” as these.
Revised: Our goal is now to theoretically demonstrate why ERM fails this way (or equivalently, why it relies on the spurious feature) even in tasks as “easy-to-learn” as these.

Original: 3. C ONSTRAINTS DEFINING EASY - TO - LEARN DOMAIN - GENERALIZATION TASKS .
Revised: 3. C ONSTRAINTS DEFINING EASY - TO - LEARN DOMAIN - GENERALIZATION TASKS .

Original: To formulate a class of easy-to-learn tasks , we enumerate a set of constraints that the tasks must satisfy; notably, this class of tasks will encompass the empirical example described above.
Revised: To formulate a class of easy-to-learn tasks , we enumerate a set of constraints that the tasks must satisfy; notably, this class of tasks will encompass the empirical example described above.

Original: The motivation behind this exercise is that restricting ourselves to the constrained set of tasks yields stronger insights — it prevents us from designing complex examples where ERM is forced to rely on spurious features due to a not-so-fundamental factor.
Revised: The motivation behind this exercise is that restricting ourselves to the constrained set of tasks yields stronger insights — it prevents us from designing complex examples where ERM is forced to rely on spurious features due to a not-so-fundamental factor.

Original: Indeed, each constraint here forbids a unique, less fundamental failure mode of ERM from occuring in the easy-to-learn tasks.
Revised: Indeed, each constraint here forbids a unique, less fundamental failure mode of ERM from occuring in the easy-to-learn tasks.

Original: Constraint 1.
Revised: Constraint 1.

Original: (Fully predictive invariant features.)
Revised: (Fully predictive invariant features.)

Original: For all D ∈ D , L D ( h (cid:63) ) = 0 .
Revised: For all D ∈ D , L D ( h (cid:63) ) = 0 .

Original: Arguably, our most important constraint is that the invariant features (which is what h (cid:63) purely relies on) are perfectly informative of the label.
Revised: Arguably, our most important constraint is that the invariant features (which is what h (cid:63) purely relies on) are perfectly informative of the label.

Original: The motivation is that, when this is not the case (e.g., as in noisy invariant features like Fig 1a or in Sagawa et al.
Revised: The motivation is that, when this is not the case (e.g., as in noisy invariant features like Fig 1a or in Sagawa et al.

Original: (2020b)
Revised: (2020b)

Original: ; Tsipras et al.
Revised: ; Tsipras et al.

Original: , failure can arise from the fact that the spurious features provide vital extra information that the invariant features cannot provide (see App A for more formal argument).
Revised: , failure can arise from the fact that the spurious features provide vital extra information that the invariant features cannot provide (see App A for more formal argument).

Original: However this explanation quickly falls apart when the invariant feature in itself is fully predictive of the label.
Revised: However this explanation quickly falls apart when the invariant feature in itself is fully predictive of the label.

Original: Constraint 2.
Revised: Constraint 2.

Original: (Identical invariant distribution.)
Revised: (Identical invariant distribution.)

Original: Across all D ∈ D , p D ( x inv ) is identical.
Revised: Across all D ∈ D , p D ( x inv ) is identical.

Original: This constraint demands that the (marginal) invariant feature distribution must remain stable across domains (like in our binary-MNIST example).
Revised: This constraint demands that the (marginal) invariant feature distribution must remain stable across domains (like in our binary-MNIST example).

Original: While this may appear to be unrealistic, (the exact distribution of the different types of cows and camels could vary across domains), we must emphasize that it is easier to make ERM fail when the invariant features are not stable (see example in
Revised: While this may appear to be unrealistic, (the exact distribution of the different types of cows and camels could vary across domains), we must emphasize that it is easier to make ERM fail when the invariant features are not stable (see example in

Original: App A Fig 4a).
Revised: App A Fig 4a).

Original: Constraint 3.
Revised: Constraint 3.

Original: (Conditional independence.)
Revised: (Conditional independence.)

Original: For all D ∈ D , x sp ⊥⊥ x inv | y .
Revised: For all D ∈ D , x sp ⊥⊥ x inv | y .

Original: This constraint reﬂects the fact that in the MNIST example, we chose the class label and then picked the color feature independent of the actual hand-written digit picked from that class.
Revised: This constraint reﬂects the fact that in the MNIST example, we chose the class label and then picked the color feature independent of the actual hand-written digit picked from that class.

Original: This prevents us from designing complex relationships between the background and the object shape to show failure (see example in App A Fig 4b).
Revised: This prevents us from designing complex relationships between the background and the object shape to show failure (see example in App A Fig 4b or the failure mode in Khani & Liang (2021)).

Original: Constraint 4.
Revised: Constraint 4.

Original: (Two-valued spurious features.)
Revised: (Two-valued spurious features.)

Original: We set X sp =
Revised: We set X sp =

Original: R and the support of x sp in D train is {−B , + B} .
Revised: R and the support of x sp in D train is {−B , + B} .

Original: This constraint 1 captures the simplicity of the cow-camel example where the background color is limited to yellow/green.
Revised: This constraint 1 captures the simplicity of the cow-camel example where the background color is limited to yellow/green.

Original: Notably, this excludes failure borne out of high-dimensional (Tsipras et al., 2019) or carefully-constructed continuous-valued (Sagawa et al., 2020b; Shah et al., 2020) spurious features.
Revised: Notably, this excludes failure borne out of high-dimensional (Tsipras et al., 2019) or carefully-constructed continuous-valued (Sagawa et al., 2020b; Shah et al., 2020) spurious features.

Original: Constraint 5.
Revised: Constraint 5.

Original: (Identity mapping.)
Revised: (Identity mapping.)

Original: Φ is the identity mapping i.e., x = ( x inv , x sp ) .
Revised: Φ is the identity mapping i.e., x = ( x inv , x sp ) .

Original: This ﬁnal constraint 2 , also implicitly made in Sagawa et al.
Revised: This ﬁnal constraint 2 , also implicitly made in Sagawa et al.

Original: (2020b)
Revised: (2020b)

Original: ; Tsipras et al.
Revised: ; Tsipras et al.

Original: (2019), prevents
Revised: (2019), prevents

Original: ERM from failing because of a hard-to-disentangle representation (see example in App A Fig 4c).
Revised: ERM from failing because of a hard-to-disentangle representation (see example in App A Fig 4c).

Original: Before we connect these constraints to our main goal, it is worth mentioning their value beyond that goal.
Revised: Before we connect these constraints to our main goal, it is worth mentioning their value beyond that goal.

Original: First, as brieﬂy discussed above and elaborated in App A, each of these constraints in itself corresponds to a unique failure mode of ERM, one that is worth exploring in future work.
Revised: First, as brieﬂy discussed above and elaborated in App A, each of these constraints in itself corresponds to a unique failure mode of ERM, one that is worth exploring in future work.

Original: Second, the resulting class of easy-to-learn tasks provides a theoretical (and a simpliﬁed empirical) test-bed that would help in broadly reasoning about OoD generalization.
Revised: Second, the resulting class of easy-to-learn tasks provides a theoretical (and a simpliﬁed empirical) test-bed that would help in broadly reasoning about OoD generalization.

Original: For example, any algorithm for solving OoD generalization should at the least hope to solve these easy-to-learn tasks well.
Revised: For example, any algorithm for solving OoD generalization should at the least hope to solve these easy-to-learn tasks well.

Original: Why is it hard to show that ERM relies on the spurious feature in easy-to-learn tasks?
Revised: Why is it hard to show that ERM relies on the spurious feature in easy-to-learn tasks?

Original: Consider the simplest easy-to-learn 2D task.
Revised: Consider the simplest easy-to-learn 2D task.

Original: Speciﬁcally, during training we set x inv = y (and so Constraintis satisﬁed) and x sp to be y B with probability p ∈
Revised: Speciﬁcally, during training we set x inv = y (and so Constraintis satisﬁed) and x sp to be y B with probability p ∈

Original: [0 . 5 , 1) and − y B with probability 1 − p (hence satisfying both Constraint 3 and 4).
Revised: [0 . 5 , 1) and − y B with probability 1 − p (hence satisfying both Constraint 3 and 4).

Original: During test-time, the only shifts allowed are on the distribution of x sp (to respect Constraint 2).
Revised: During test-time, the only shifts allowed are on the distribution of x sp (to respect Constraint 2).

Original: Observe from Fig 1b that this distribution has a support of the four points in {− 1 , +1 } × {−B , + B} and is hence an abstract form of the cow-camel dataset which also has four groups of points: (a majority of) cows/camels against grass/sand and (a minority of) cows/camels against sand/grass.
Revised: Observe from Fig 1b that this distribution has a support of the four points in {− 1 , +1 } × {−B , + B} and is hence an abstract form of the cow-camel dataset which also has four groups of points: (a majority of) cows/camels against grass/sand and (a minority of) cows/camels against sand/grass.

Original: Fitting a max-margin classiﬁer on D train leads us to a simple yet key observation: owing to the geometry of the four groups of points, the max-margin classiﬁer has no reliance on the spurious feature despite the spurious correlation .
Revised: Fitting a max-margin classiﬁer on D train leads us to a simple yet key observation: owing to the geometry of the four groups of points, the max-margin classiﬁer has no reliance on the spurious feature despite the spurious correlation .

Original: In other words, even though this dataset distills what seem to be the core aspects of the cow/camel dataset, we are unable to reproduce the corresponding behavior of ERM.
Revised: In other words, even though this dataset distills what seem to be the core aspects of the cow/camel dataset, we are unable to reproduce the corresponding behavior of ERM.

Original: In the next two sections, we will try to resolve this apparent paradox.
Revised: In the next two sections, we will try to resolve this apparent paradox.

Revised: would hold the we study are rotation-

Revised: But for we’ll with the

Revised: [Figure]

Original:  F AILURE DUE TO GEOMETRIC SKEWS
Revised:  F AILURE DUE TO GEOMETRIC SKEWS

Original: A pivotal piece in solving this puzzle is a particular geometry underlying how the invariant features in real-world distributions are separated.
Revised: A pivotal piece in solving this puzzle is a particular geometry underlying how the invariant features in real-world distributions are separated.

Original: To describe this, consider the random features representation of MNIST (i.e., x inv ) and ﬁt a max-margin classiﬁer on it, i.e., the least-norm w inv that achieves a margin of y · ( w inv · x inv + b ) ≥ 1 on all data (for some b ).
Revised: To describe this, consider the random features representation of MNIST (i e., x inv ) and ﬁt a max-margin classiﬁer on it, i.e., the least-norm w inv that achieves a margin of y · ( w inv · x inv + b ) ≥ 1 on all data (for some b ).

Original: Then, we’d observe that as the number of training points increase, the (cid:96) 2 norm of this max-margin classiﬁer grows (see Figure 2b); similar observations also hold for CIFAR10 (see App C.2).
Revised: Then, we’d observe that as the number of training points increase, the (cid:96) 2 norm of this max-margin classiﬁer grows (see Figure 2b); similar observations also hold for CIFAR10 (see App C.2).

Original: This observation (which stems from the geometry of the dataset) builds on ones that were originally made in Neyshabur et al.
Revised: This observation (which stems from the geometry of the dataset) builds on ones that were originally made in Neyshabur et al.

Original: (2017); Nagarajan & Kolter (2017; 2019) for norms of overparameterized neural networks.
Revised: (2017); Nagarajan & Kolter (2017; 2019) for norms of overparameterized neural networks.

Original: Our contribution here is to empirically establish this for a linear overparameterized model and then to theoretically relate this to OoD failure.
Revised: Our contribution here is to empirically establish this for a linear overparameterized model and then to theoretically relate this to OoD failure.

Original: In the following discussion, we will take this “increasing norms” observation as a given, and use it to explain why the (original) max-margin classiﬁer (i.e., the one trained on all the The rest of our discussion hold work
Revised: In the following discussion, we will take this “increasing norms” observation as a given 3 ), and use it to explain why the (original) max-margin classiﬁer (i.e., the one trained on all the features including x sp ) relies on the spurious feature.

Original: [Figure] Consider the dataset where every inputis x inv  concatenated with a feature x sp ∈ {−B , B} that is spuriously correlated with the label i [Figure]
Revised: Imagine every input x inv to be concatenated with a feature x sp ∈ {−B , B} that is spuriously correlated with the label i.e., Pr[ x sp · y > 0] > 0 .

Original: e., Pr[ x sp · y > 0] > 0 . The underlying spurious correlation implicitly induces two disjoint groups in thewhole dataset S : a majority group S maj where x sp · y >e.g., cows/camels with green/yellow backgrounds [Figure] and a minority group S min where x sp · y < 0 e.g., cows/camels with yellow/green backgrounds.
Revised: The underlying spurious correlation implicitly induces two disjoint groups in the dataset S : a majority group S maj where x sp · y > 0 e.g., cows/camels with green/yellow backgrounds, and a minority group S min where x sp · y < 0 e.g., cows/camels with yellow/green backgrounds.

Original: Next, let w all ∈ X inv denote the least-norm vector that (a) lies in the invariant space and (b) classiﬁes all of S by a margin of at least 1 .
Revised: Next, let w all ∈ X inv denote the least-norm vector that (a) lies in the invariant space and (b) classiﬁes all of S by a margin of at least 1 .

Original: Similarly, let w min ∈ X invsimilarly denote a least-norm, purely-invariant vector that classiﬁes all of S min by a margin of at least 1 .
Revised: Similarly, let w min ∈ X inv denote a least-norm, purely-invariant vector that classiﬁes all of S min by a margin of at least 1 .

Original: Crucially, since S min has much fewer points than S , by the “increasing-norm” property, we can say that (cid:107) w min (cid:107) (cid:28) (cid:107) w all (cid:107) .
Revised: Crucially, since S min has much fewer points than S , by the “increasing-norm” property, we can say that (cid:107) w min (cid:107) (cid:28) (cid:107) w all (cid:107) .

Original: We informally refer to the gap in these (cid:96) 2 norms as a geometric skew.
Revised: We informally refer to the gap in these (cid:96) 2 norms as a geometric skew.

Original: Given this skew, we can intuitively see why the max-margin classiﬁer must use the spurious feature.
Revised: Given this skew, we can intuitively see why the max-margin classiﬁer must use the spurious feature.

Original: One way to classify the data is to use only the invariant feature, which would cost an (cid:96) 2 norm of (cid:107) w all (cid:107) .
Revised: One way to classify the data is to use only the invariant feature, which would cost an (cid:96) 2 norm of (cid:107) w all (cid:107) .

Original: But there is another alternative: use the spurious feature as a short-cut to classify the majority of the dataset S maj (by setting w sp > 0 ) and combine it with w min to classify the remaining minority S min .
Revised: But there is another alternative: use the spurious feature as a short-cut to classify the majority of the dataset S maj (by setting w sp > 0 ) and combine it with w min to classify the remaining minority S min .

Original: Since (cid:107) w min (cid:107) (cid:28) (cid:107) w all (cid:107) , the latter strategy requires lesser (cid:96) 2 norm, and is therefore the strategy opted by the max-margin classiﬁer.
Revised: Since (cid:107) w min (cid:107) (cid:28) (cid:107) w all (cid:107) , the latter strategy requires lesser (cid:96) 2 norm, and is therefore the strategy opted by the max-margin classiﬁer.

Original: We concretely illustrate this failure mechanism in a 2D dataset in Fig 2c.
Revised: We concretely illustrate this failure mechanism in a 2D dataset in Fig 2c.

Original: Here, we have explicitly designed the data to capture the “increasing norms” property: the distance between purely-invariant classiﬁer boundary (i.e., a vertical separator through the origin) and the closest point in S maj is smaller than that of the closest point in S min .
Revised: Here, we have explicitly designed the data to capture the “increasing norms” property: the distance between purely-invariant classiﬁer boundary (i.e., a vertical separator through the origin) and the closest point in S maj is smaller than that of the closest point in S min .

Original: In other words, a purely-invariant classiﬁer would require much greater norm to classify the majority group by a margin of 1 than to classify the minority group.
Revised: In other words, a purely-invariant classiﬁer would require much greater norm to classify the majority group by a margin of 1 than to classify the minority group.

Original: We can then visually see that the max-margin classiﬁer would take the orientation of a diagonal separator that uses the spurious feature rather than a vertical, purely-invariant one.
Revised: We can then visually see that the max-margin classiﬁer would take the orientation of a diagonal separator that uses the spurious feature rather than a vertical, purely-invariant one.

Original: The following result formalizes the above failure mechanism.
Revised: The following result formalizes the above failure.

Original: In particular, for any arbitrary easyto-learn task, we provide lower and upper bounds on| w sp| that are larger for smaller values of (cid:107) w min (cid:107) / (cid:107) w all (cid:107) .
Revised: In particular, for any arbitrary easy-to-learn task, we provide lower and upper bounds on w sp that are larger for smaller values of (cid:107) w min (cid:107) / (cid:107) w all (cid:107) .

Original: For readability, we state only an informal,distilled version of our theorem below, and in particular for B = 1 .
Revised: For readability, we state only an informal, version of our theorem below.

Original: In App B.1, we present the full, precise resultfor a generic value B along with the proof.
Revised: In App B.1, we present the full, precise result along with the proof.

Original: Theorem 1.
Revised: Theorem 1.

Original: (informal)
Revised: (informal)

Original: Let H be the set of linear classiﬁers, h ( x ) =
Revised: Let H be the set of linear classiﬁers, h ( x ) =

Original: w inv x inv + w sp x sp + b .
Revised: w inv x inv + w sp x sp + b .

Original: Then for any task satisfying all the constraints in Sec 3.1 with B = 1 , the max-margin classiﬁer satisﬁes:
Revised: Then for any task satisfying all the constraints in Sec 3.1 with B = 1 , the max-margin classiﬁer satisﬁes:

Original: [Equation]
Revised: [Equation]

Original: A salient aspect of this result is that it explains the varying dynamics between the underlying spurious correlation and spurious-feature-reliance in the classiﬁer.
Revised: A salient aspect of this result is that it explains the varying dynamics between the underlying spurious correlation and spurious-feature-reliance in the classiﬁer.

Original: First, as the correlation increases ( Pr D train [ x sp · y > 0] → 1 . 0 ), the size of the minority group decreases to zero.
Revised: First, as the correlation increases ( Pr D train [ x sp · y > 0] → 1 . 0 ), the size of the minority group decreases to zero.

Original: Then, we empirically know that (cid:107) w min (cid:107) / (cid:107) w all (cid:107) progressively shrinks all the way down to 0 .
Revised: Then, we empirically know that (cid:107) w min (cid:107) / (cid:107) w all (cid:107) progressively shrinks all the way down to 0 .

Original: Then, we can invoke the lower bound which implies that w sp grows to ≈ 1 .
Revised: Then, we can invoke the lower bound which implies that w sp grows to ≈ 1 .

Original: This implies serious vulnerability to the test-time shifts: any ﬂip in the sign of the spurious feature can reduce the original margin of ≈ 1 by a value of 2 | w sp x sp | ≈ 2 (since B = 1 here) making the margin negative (implying misclassiﬁcation).
Revised: This implies serious vulnerability to the test-time shifts: any ﬂip in the sign of the spurious feature can reduce the original margin of ≈ 1 by a value of 2 | w sp x sp | ≈ 2 (since B = 1 here) making the margin negative (implying misclassiﬁcation).

Original: On the other hand, when spurious correlations diminish ( Pr D train [ x sp · y > 0] → 0 . 5 ) , the value of (cid:107) w min (cid:107) grows comparable to (cid:107) w all (cid:107) (although not exactly as large), and our upper bound suggests that the spurious component must shrink towards ≈ 0 , thereby implying robustness to these shifts.
Revised: On the other hand, when spurious correlations diminish ( Pr D train [ x sp · y > 0] → 0 . 5 ) , the value of (cid:107) w min (cid:107) grows comparable to (cid:107) w all (cid:107) (although not exactly as large), and our upper bound suggests that the spurious component must shrink towards ≈ 0 , thereby implying robustness to these shifts.

Original: Broader empirical implications.
Revised: Broader empirical implications.

Original: While our theorem explains failure in linear, easy-to-learn settings, the underlying geometric argument can be used to intuitively understand failure in more general settings i.e., setting where the classiﬁer is non-linear and/or the task is not easy-to-learn.
Revised: While our theorem explains failure in linear, easy-to-learn settings, the underlying geometric argument can be used to intuitively understand failure in more general settings i.e., setting where the classiﬁer is non-linear and/or the task is not easy-to-learn.

Original: For illustration, we identify a few such unique non-linear tasks involving the failure of a neural network.
Revised: For illustration, we identify a few such unique non-linear tasks involving the failure of a neural network.

Original: The ﬁrst two tasks below can be informally thought of as easy-to-learn tasks 3 :
Revised: The ﬁrst two tasks below can be informally thought of as easy-to-learn tasks 4 :

Original: • In Fig 1c, we consider a CIFAR10 task where we add a line to with its color spuriously correlated with the class only during training.
Revised: • In Fig 1c, we consider a CIFAR10 task where we add a line to with its color spuriously correlated with the class only during training.

Original: The (cid:38) 10% OoD accuracy drop of a ResNet here, we argue (in App C.3.1), arises from the fact that it takes greater norms for the ResNet to ﬁt larger proportions CIFAR10.
Revised: The (cid:38) 10% OoD accuracy drop of a ResNet here, we argue (in App C.3.1), arises from the fact that it takes greater norms for the ResNet to ﬁt larger proportions CIFAR10.

Original: • In Fig 1d, we consider a colored Cats vs. Dogs task (Elson et al., 2007), where a majority of the datapoints are blue-ish and a minority are green-ish.
Revised: • In Fig 1d, we consider a colored Cats vs. Dogs task (Elson et al., 2007), where a majority of the datapoints are blue-ish and a minority are green-ish.

Original: During testing, we color all datapoints to be green-ish.
Revised: During testing, we color all datapoints to be green-ish.

Original: Crucially, even though there is no correlation between the label and the color of the images, the OoD accuracy of an ResNet drops by (cid:38) 20% .
Revised: Crucially, even though there is no correlation between the label and the color of the images, the OoD accuracy of an ResNet drops by (cid:38) 20% .

Original: To explain this, in App.
Revised: To explain this, in App.

Original: C.3.3, we identify an “implicit”, non-visual kind of spurious correlation in this dataset, one between the label and a particular component of the difference between the two channels.
Revised: C.3.3, we identify an “implicit”, non-visual kind of spurious correlation in this dataset, one between the label and a particular component of the difference between the two channels.

Original: Next, we enumerate two not-easy-to-learn tasks.
Revised: Next, we enumerate two not-easy-to-learn tasks.

Original: Here, one of the easy-to-learn constraints is disobeyed signiﬁcantly enough to make the task hard, and this is essential in causing failure.
Revised: Here, one of the easy-to-learn constraints is disobeyed signiﬁcantly enough to make the task hard, and this is essential in causing failure.

Original: In other words, the failure modes here correspond to ones that were outlined in Sec 3.1.
Revised: In other words, the failure modes here correspond to ones that were outlined in Sec 3.1.

Original: Nevertheless, we argue in App C.3 that even these modes can be reasoned geometrically as a special case of Theorem 1.
Revised: Nevertheless, we argue in App C.3 that even these modes can be reasoned geometrically as a special case of Theorem 1.

Original: The two not-easy-to-learn tasks are as follows:
Revised: The two not-easy-to-learn tasks are as follows:

Original: • In Fig 2d, we add a line to the last channel of CIFAR10 images regardless of the label, and make the line brighter during testing resembling a camera glitch, which results in a (cid:38) 27% drop in a ResNet’s accuracy.
Revised: • In Fig 2d, we add a line to the last channel of CIFAR10 images regardless of the label, and make the line brighter during testing resembling a camera glitch, which results in a (cid:38) 27% drop in a ResNet’s accuracy.

Original: We geometrically argue how this failure arises from breaking Constraint 5.
Revised: We geometrically argue how this failure arises from breaking Constraint 5.

Original: In App C.3.5, we consider an MNIST setting inspired by Tsipras et al.
Revised: In App C.3.5, we consider an MNIST setting inspired by Tsipras et al.

Original: (2019), where failure arises (geometrically) due to high-dimensional spurious features (breaking Constraint 4).
Revised: (2019), where failure arises (geometrically) due to high-dimensional spurious features (breaking Constraint 4).

Original: We hope that these examples, described in greater detail in App C.3, provide (a) a broader way to think about how spurious correlations manifest
Revised: We hope that these examples, described in greater detail in App C.3, provide (a) a broader way to think about how spurious correlations manifest

Original:  and (b) how a variety of resulting failure modes can be reasoned geometrically
Revised: , and (b) how a variety of resulting failure modes can be reasoned geometrically.

Original:  F AILURE DUE TO STATISTICAL SKEWS
Revised:  F AILURE DUE TO STATISTICAL SKEWS

Original: Having theoretically studied max-margin classiﬁers, let us now turn our attention to studying linear classiﬁers trained by gradient descent on logistic/exponential loss.
Revised: Having theoretically studied max-margin classiﬁers, let us now turn our attention to studying linear classiﬁers trained by gradient descent on logistic/exponential loss.

Original: Under some conditions, on linearly separable datasets, these classiﬁers would converge to the max-margin classiﬁer given inﬁnite time (Soudry et al., 2018; Ji & Telgarsky, 2018).
Revised: Under some conditions, on linearly separable datasets, these classiﬁers would converge to the max-margin classiﬁer given inﬁnite time (Soudry et al., 2018; Ji & Telgarsky, 2018).

Original: So it is reasonable to say that even these classiﬁers would suffer from the geometric skews, even if stopped in some ﬁnite time.
Revised: So it is reasonable to say that even these classiﬁers would suffer from the geometric skews, even if stopped in some ﬁnite time.

Original: However, are there any other failure modes that would arise here?
Revised: However, are there any other failure modes that would arise here?

Original: To answer this, let us dial back to the easiest-to-learn task: the setting with four points {− 1 , +1 } × {−B , + B} , where in the training distribution (say D 2-dim ), we have x inv =
Revised: To answer this, let us dial back to the easiest-to-learn task: the setting with four points {− 1 , +1 } × {−B , + B} , where in the training distribution (say D 2-dim ), we have x inv =

Original: y and x sp to be y B with probability p ∈
Revised: y and x sp to be y B with probability p ∈

Original: [0 . 5 , 1) and − y B with probability 1 − p .
Revised: [0 . 5 , 1) and − y B with probability 1 − p .

Original: Even though the max-marginclassiﬁer here does not rely on x sp for any level of spurious correlation p ∈ [0 . 5 , 1) — there are no geometric skews here after all — the story is more complicated when we empirically evaluate via gradient descent stopped in ﬁnite time t .
Revised: Here, even though the max-margin does not rely on x sp for any level of spurious correlation p ∈ [0 . 5 , 1) — there are no geometric skews here after all — the story is more complicated when we empirically evaluate via gradient descent stopped in ﬁnite time t .

Original: Speciﬁcally, for various values of p we plot w sp / √ w 2 inv + w 2 sp vs. t (here, looking at w sp alone does not make sense since the weight norm grows unbounded).
Revised: Speciﬁcally, for various values of p we plot w sp / √ w 2 inv + w 2 sp vs. t (here, looking at w sp alone does not make sense since the weight norm grows unbounded).

Original: We observe in Fig 3a that the spurious component appears to stagnate around a value proportional to p , even after sufﬁciently long training, and even though it is supposed to converge to 0 .
Revised: We observe in Fig 3a that the spurious component appears to stagnate around a value proportional to p , even after sufﬁciently long training, and even though it is supposed to converge to 0 .

Original: Thus, even though max-margin doesn’t fail in this dataset, ﬁnite-time-stopped gradient descent fails.
Revised: Thus, even though max-margin doesn’t fail in this dataset, ﬁnite-time-stopped gradient descent fails.

Original: Why does this happen?
Revised: Why does this happen?

Original: To explain this behavior, a partial clue already exists in Soudry et al.
Revised: To explain this behavior, a partial clue already exists in Soudry et al.

Original: (2018); Ji & Telgarsky (2018): gradient descent can have a frustratingly slow logarithmic rate of convergence to the max-margin i.e,.
Revised: (2018); Ji & Telgarsky (2018): gradient descent can have a frustratingly slow logarithmic rate of convergence to the max-margin i.e,.

Original: the ratio | w sp /w
Revised: the ratio | w sp /w

Original: inv
Revised: inv

Original: | could decay to zero as slow as 1 / ln t .
Revised: | could decay to zero as slow as 1 / ln t .

Original: However, this bound is a distributionindependent one that does not explain why the convergence varies with the spurious correlation.
Revised: However, this bound is a distributionindependent one that does not explain why the convergence varies with the spurious correlation.

Original: To this end, we build on this result to derive a distribution-speciﬁc convergence bound in terms of p , that applies to any easy-to-learn task (where x inv may be higher dimensional unlike in D 2-dim ).
Revised: To this end, we build on this result to derive a distribution-speciﬁc convergence bound in terms of p , that applies to any easy-to-learn task (where x inv may be higher dimensional unlike in D 2-dim ).

Original: For convenience, we focus on continuous-time gradient descent under the exponential loss exp( − yh ( x ))
Revised: For convenience, we focus on continuous-time gradient descent under the exponential loss exp( − yh ( x ))

Original: (the dynamics of which is similar to that of logistic loss as noted in Soudry et al.
Revised: (the dynamics of which is similar to that of logistic loss as noted in Soudry et al.

Original: Then we consider any easy-to-learn task and informally speaking, any corresponding dataset without geometric skews, so the max-margin wouldn’t rely on the spurious feature.
Revised: Then we consider any easy-to-learn task and informally speaking, any corresponding dataset without geometric skews, so the max-margin wouldn’t rely on the spurious feature.

Original: We then study the convergence rate of w sp ( t ) ·B / w inv ( t ) · x inv to 0 i.e., the rate at which the ratio between the output of the spurious component to that of the invariant component converges to its corresponding max-margin value.
Revised: We then study the convergence rate of w sp ( t ) ·B / w inv ( t ) · x inv to 0 i.e., the rate at which the ratio between the output of the spurious component to that of the invariant component converges to its corresponding max-margin value.

Original: We show that the convergence rate is Θ(1 / ln t ) , crucially scaled by an extra factor that monotonically increases in [0 , ∞ ) as a function of the spurious correlation, p ∈ [0 . 5 , 1) , thus capturing slower convergence for larger spurious correlation.
Revised: We show that the convergence rate is Θ(1 / ln t ) , crucially scaled by an extra factor that monotonically increases in [0 , ∞ ) as a function of the spurious correlation, p ∈ [0 . 5 , 1) , thus capturing slower convergence for larger spurious correlation.

Original: Another notable aspect of our result is that when there is no spurious correlation ( p = 0 . 5 ), both the upper and lower bound reduce to 0 , indicating quick convergence.
Revised: Another notable aspect of our result is that when there is no spurious correlation ( p = 0 . 5 ), both the upper and lower bound reduce to 0 , indicating quick convergence.

Original: We provide the full statement and proof of this bound in App B.2.
Revised: We provide the full statement and proof of this bound in App B.2.

Original: For completeness, we also provide a more precise analysis of the dynamics for a 2D setting under both exponential and logistic loss in Theorem 5 and Theorem 6 in App B.2.
Revised: For completeness, we also provide a more precise analysis of the dynamics for a 2D setting under both exponential and logistic loss in Theorem 5 and Theorem 6 in App B.2.

Original: Theorem 2.
Revised: Theorem 2.

Original: (informal)
Revised: (informal)

Original: Let H be the set of linear classiﬁers h ( x ) =
Revised: Let H be the set of linear classiﬁers h ( x ) =

Original: w inv · x inv + w sp x sp .
Revised: w inv · x inv + w sp x sp .

Original: Then, for any easy-to-learn task, and for any dataset without geometric skews, continuous-time gradient descent training of w inv ( t ) · x inv + w sp ( t ) x sp to minimize the exponential loss, satisﬁes:
Revised: Then, for any easy-to-learn task, and for any dataset without geometric skews, continuous-time gradient descent training of w inv ( t ) · x inv + w sp ( t ) x sp to minimize the exponential loss, satisﬁes:

Original: [Equation]
Revised: [Equation]

Original: where p :=
Revised: where p :=

Original: Pr D [Equation] [ x sp · y > 0] ∈ [0 . 5 , 1) .
Revised: Pr D train [ x sp · y > 0] ∈ [0 . 5 , 1) .

Original: The intuition behind this failure mode is that in the initial epochs, when the loss exp ( − y w · x ) on all points are more or less the same, the updates 1 / | S | · (cid:80) ( x
Revised: The intuition behind this failure mode is that in the initial epochs, when the loss exp ( − y w · x ) on all points are more or less the same, the updates 1 / | S | · (cid:80) ( x

Original: ,y ) ∈ S y x exp( − y w · x ) roughly push along the direction, 1 / | S | · (cid:80) ( x
Revised: ,y ) ∈ S y x exp( − y w · x ) roughly push along the direction, 1 / | S | · (cid:80) ( x

Original: ,y ) ∈
Revised: ,y ) ∈

Original: S y x .
Revised: S y x .

Original: This is the precise (mis)step where gradient descent “absorbs” the spurious correlation, as this step pushes w sp along p B −
Revised: This is the precise (mis)step where gradient descent “absorbs” the spurious correlation, as this step pushes w sp along p B −

Original: (1 − p ) B
Revised: (1 − p ) B

Original: = (2 p − 1) B .
Revised: = (2 p − 1) B .

Original: While this update would be near-zero when there is only little spurious correlation ( p ≈ 0 . 5 ), it takes larger values for larger levels of spurious correlation ( p ≈ 1 ).
Revised: While this update would be near-zero when there is only little spurious correlation ( p ≈ 0 . 5 ), it takes larger values for larger levels of spurious correlation ( p ≈ 1 ).

Original: Unfortunately, under exponential-type losses, the gradients decay with time, and so the future gradients, even if they eventually get rid of this absorbed spurious component, take an exponentially long time to do so.
Revised: Unfortunately, under exponential-type losses, the gradients decay with time, and so the future gradients, even if they eventually get rid of this absorbed spurious component, take an exponentially long time to do so.

Original: Broader empirical implications.
Revised: Broader empirical implications.

Original: We now demonstrate the effect of statistical skews in more general empirical settings consisting of a non-linear easy-to-learn task learned using a neural network.
Revised: We now demonstrate the effect of statistical skews in more general empirical settings consisting of a non-linear easy-to-learn task learned using a neural network.

Revised: [Figure]

Original: Isolating the statistical skew effect is however challenging in practice: any gradient-descent-trained model is likely to be hurt by both statistical skews and geometric skews, and we’d have to somehow disentangle the two effects.
Revised: Isolating the statistical skew effect is however challenging in practice: any gradient-descent-trained model is likely to be hurt by both statistical skews and geometric skews, and we’d have to somehow disentangle the two effects.

Original: We handle this by designing the following experiment.
Revised: We handle this by designing the following experiment.

Original: We ﬁrst create a control dataset S con where there are no geometric or statistical skews.
Revised: We ﬁrst create a control dataset S con where there are no geometric or statistical skews.

Original: For this, we take a set of images S inv (with no spurious features), and create two copies of it, S maj and S min where we add spurious features, positively and negatively aligned with the label, respectively, and deﬁne
Revised: For this, we take a set of images S inv (with no spurious features), and create two copies of it, S maj and S min where we add spurious features, positively and negatively aligned with the label, respectively, and deﬁne

Original: S con =
Revised: S con =

Original: S maj ∪ S min .
Revised: S maj ∪ S min .

Original: Next, we create an experimental dataset S exp with a statistical skew in it.
Revised: Next, we create an experimental dataset S exp with a statistical skew in it.

Original: We do this by taking S con and duplicating S maj in it so that the ratio | S maj | : | S min | becomes 10 : 1 .
Revised: We do this by taking S con and duplicating S maj in it so that the ratio | S maj | : | S min | becomes 10 : 1 .

Original: Importantly, this dataset has no geometric skews, since merely replicating points does not affect the [Figure]
Revised: Importantly, this dataset has no geometric skews, since merely replicating points does not affect the geometry.

Original: geometry/max-margin norms.

Original: Then, if we were to observe thattraining (stochastic) gradient descent on S exp results in greater spurious-feature-reliance than S con , we would have isolated the effect of statistical skews.
Revised: Then, if we were to observe that (stochastic) gradient descent on S exp results in greater spurious-feature-reliance than S con , we would have isolated the effect of statistical skews.

Original: Indeed, we demonstrate this in two easy-to-learn tasks.
Revised: Indeed, we demonstrate this in two easy-to-learn tasks.

Original: First, we consider a Binary-MNIST task learned by an FNNs.
Revised: First, we consider a Binary-MNIST task learned by an fully-connected network.

Original: Here, we concatenate a spurious channel where either all the pixels are “on” or “off”.
Revised: Here, we concatenate a spurious channel where either all the pixels are “on” or “off”.

Original: Second, we consider a (multiclass) CIFAR10 task (where we add a spuriously colored line) learned using a ResNet (He et al., 2016).
Revised: Second, we consider a (multiclass) CIFAR10 task (where we add a spuriously colored line) learned using a ResNet (He et al., 2016).

Original: In Fig 3, we demonstrate that training on S exp leads to less robust models than on S con in both these tasks.
Revised: In Fig 3, we demonstrate that training on S exp leads to less robust models than on S con in both these tasks.

Original: In other words, gradient descent on S exp leads to greater spurious-feature-reliance, thus validating the effect of statistical skews in practice.
Revised: In other words, gradient descent on S exp leads to greater spurious-feature-reliance, thus validating the effect of statistical skews in practice.

Original: More details are provided in App C.4.
Revised: More details are provided in App C.4.

Original:  C ONCLUSIONS AND FUTURE WORK
Revised:  C ONCLUSIONS AND FUTURE WORK

Original: We identify that spurious correlations during training can induce two distinct skews in the training set, one geometric and another statistical.
Revised: We identify that spurious correlations during training can induce two distinct skews in the training set, one geometric and another statistical.

Original: These skews result in two complementary ways by which empirical risk minimization (ERM) via gradient descent is guaranteed to rely on those spurious correlations.
Revised: These skews result in two complementary ways by which empirical risk minimization (ERM) via gradient descent is guaranteed to rely on those spurious correlations.

Original: At the same time, our theoretical results (in particular, the upper bounds on the spurious component of the classiﬁer) show that when these skews do disappear, there is no failure within the considered tasks.
Revised: At the same time, our theoretical results (in particular, the upper bounds on the spurious component of the classiﬁer) show that when these skews do disappear, there is no failure within the considered tasks.

Original: This suggests that within the class of easy-to-learn tasks and for gradient-descenttrained linear models, the above discussion likely captures all possible failure modes.
Revised: This suggests that within the class of easy-to-learn tasks and for gradient-descenttrained linear models, the above discussion likely captures all possible failure modes.

Original: However, when we do venture into the real-world to face more complicated tasks and use non-linear, deep models, many other kinds of failure modes would crop up (such as the ones we enumerate in
Revised: However, when we do venture into the real-world to face more complicated tasks and use non-linear, deep models, many other kinds of failure modes would crop up (such as the ones we enumerate in

Original: Sec 3.1, in addition to the fundamental ones mentioned above).
Revised: Sec 3.1, in addition to the fundamental ones mentioned above).

Original: Indeed, the central message of our work is that there is no one unique mechanism by which classiﬁers fail under spurious correlations, even in the simplest of tasks.
Revised: Indeed, the central message of our work is that there is no one unique mechanism by which classiﬁers fail under spurious correlations, even in the simplest of tasks.

Original: This in turn has a key practical implication: in order to improve our solutions to OoD generalization, it would be valuable to ﬁgure out whether or not a uniﬁed solution approach is sufﬁcient to tackle all these failure mechanisms.
Revised: This in turn has a key practical implication: in order to improve our solutions to OoD generalization, it would be valuable to ﬁgure out whether or not a uniﬁed solution approach is sufﬁcient to tackle all these failure mechanisms.

Original: While we outline some solutions in
Revised: While we outline some solutions in

Original: App D, we hope that the foundation we have laid in this study helps future work in better tackling out-of-distribution challenges.
Revised: App D, we hope that the foundation we have laid in this study helps future work in better tackling out-of-distribution challenges.

Revised: Acknowledgements.

Revised: We thank Hanie Sedghi for providing useful feedback on the draft.

Original: A C ONSTRAINTS AND OTHER FAILURE MODES
Revised: A C ONSTRAINTS AND OTHER FAILURE MODES

Original: Here, we elaborate on how each of the constraints we impose on the easy-to-learn tasks corresponds to eliminating a particular complicated kind of failure of ERM from happening.
Revised: Here we elaborate on how each of the constraints we impose on the easy-to-learn tasks corresponds to eliminating a particular complicated kind of failure of ERM from happening.

Original: In particular, for each of these constraints, we’ll construct a task that betrays that constraint (but obeys all the others) and that causes a unique kind of failure of ERM.
Revised: In particular, for each of these constraints, we’ll construct a task that betrays that constraint (but obeys all the others) and that causes a unique kind of failure of ERM.

Original: We hope that laying these out concretely can provide a useful starting point for future work to investigate these various failure modes.
Revised: We hope that laying these out concretely can provide a useful starting point for future work to investigate these various failure modes.

Original: Finally, it is worth noting that all the failure modes here can be explained via a geometric argument, and furthermore the failure modes for Constraint 5 and Constraint 4 can be explained as a special case of the argument in Theorem1.
Revised: Finally, it is worth noting that all the failure modes here can be explained via a geometric argument and furthermore the failure modes for Constraint 5 and Constraint 4 can be explained as a special case of the argument in Theorem

Original: Failure due to weakly predictive invariant feature (breaking Constraint 1).
Revised: Failure due to weakly predictive invariant feature (breaking Constraint 1).

Original: We enforced in
Revised: We enforced in

Original: Constraint 1 that the invariant feature be fully informative of the label.
Revised: Constraint 1 that the invariant feature be fully informative of the label.

Original: For a setting that breaks this constraint, consider a 2D task with noisy invariant features, where across all domains, we have a
Revised: For a setting that breaks this constraint, consider a 2D task with noisy invariant features, where across all domains, we have a

Original: Gaussian invariant feature of the form x inv ∼ N ( y, σ 2 inv ) —- this sort of a noisy invariant feature was critically used in Tsipras et al.
Revised: Gaussian invariant feature of the form x inv ∼ N ( y, σ 2 inv ) —- this sort of a noisy invariant feature was critically used in Tsipras et al.

Original: ; Sagawa et al.
Revised: ; Sagawa et al.

Original: (2020b);
Revised: (2020b);

Original: Arjovsky et al.
Revised: Arjovsky et al.

Original: to explain failure of ERM.
Revised: to explain failure of ERM.

Original: Now, assume that during training we have a spurious feature x sp ∼ N ( y, σ 2 sp ) (say with relatively larger variance, while positively correlated with y ).
Revised: Now, assume that during training we have a spurious feature x sp ∼ N ( y, σ 2 sp ) (say with relatively larger variance, while positively correlated with y ).

Original: Then, observe that the Bayes optimal classiﬁer on D train is sgn ( x inv /σ inv + x sp /σ sp ) i.e., it must rely on the spurious feature.
Revised: Then, observe that the Bayes optimal classiﬁer on D train is sgn ( x inv /σ inv + x sp /σ sp ) i.e., it must rely on the spurious feature.

Original: However, also observe that if one were to eliminate noise in the invariant feature by setting σ inv →(thus making the invariant feature perfectly informative of the label like in our MNIST example), the Bayes optimal classiﬁer approaches sgn ( x inv ) , thus succeeding after all.
Revised: However, also observe that if one were to eliminate noise in the invariant feature by setting σ inv →(thus making the invariant feature perfectly informative of the label like in our MNIST example), the Bayes optimal classiﬁer approaches sgn ( x inv ) , thus succeeding after all.

Original: Failure due to “unstable” invariant feature (breaking Constraint 2).
Revised: Failure due to “unstable” invariant feature (breaking Constraint 2).

Original: If during test-time, we push the invariant features closer to the decision boundary (e.g., partially occlude the shape of every camel and cow), test accuracy will naturally deteriorate, and this embodies a unique failure mode.
Revised: If during test-time, we push the invariant features closer to the decision boundary (e.g., partially occlude the shape of every camel and cow), test accuracy will naturally deteriorate, and this embodies a unique failure mode.

Original: Concretely, consider a domain generalization task where across all domains x inv ≥ − 0 .
Revised: Concretely, consider a domain generalization task where across all domains x inv ≥ − 0 .

Original: 5 determines the true boundary (see Fig 4a).
Revised: 5 determines the true boundary (see Fig 4a).

Original: Now, assume that in the training domains, we see x inv = 2 y or x inv = 3 y .
Revised: Now, assume that in the training domains, we see x inv = 2 y or x inv = 3 y .

Original: This would result in learning a max-margin classiﬁer of the form x inv ≥ 0 .
Revised: This would result in learning a max-margin classiﬁer of the form x inv ≥ 0 .

Original: Now,[Figure] during test-time, if one were to provide “harder” examples that are closer to the true boundary in that x inv = − 0 .
Revised: Now, during test-time, if one were to provide “harder” examples that are closer to the true boundary in that x inv = − 0 .

Original: 5 + 0 .
Revised: 5 + 0 .

Original: 1 y , then all the positive examples would end up being misclassiﬁed.
Revised: 1 y , then all the positive examples would end up being misclassiﬁed.

Original: Failure due to complex conditional dependencies (breaking Constraint 3).
Revised: Failure due to complex conditional dependencies (breaking Constraint 3).

Original: This constraint imposed x inv ⊥⊥ x sp | y .
Revised: This constraint imposed x inv ⊥⊥ x sp | y .

Original: Stated more intuitively, given that a particular image is that of a camel, this constraint captures the fact that knowing the background color does not tell us too much about the precise shape of the camel.
Revised: Stated more intuitively, given that a particular image is that of a camel, this constraint captures the fact that knowing the background color does not tell us too much about the precise shape of the camel.

Original: An example where this constraint is broken is one where x inv and x sp share a neat geometric relationship during training, but not during testing, which then results in failure as illustrated below.
Revised: An example where this constraint is broken is one where x inv and x sp share a neat geometric relationship during training, but not during testing, which then results in failure as illustrated below.

Original: Consider the example in Fig 4b where for the positive class we set x inv + x sp = 1 and for the negative class we set x inv + x sp = − 1 , thereby breaking this constraint.
Revised: Consider the example in Fig 4b where for the positive class we set x inv + x sp = 1 and for the negative class we set x inv + x sp = − 1 , thereby breaking this constraint.

Original: Now even though the invariant feature in itself is fully informative of the label, while the spurious feature is not, the max-margin here is parallel to the line x inv + x sp
Revised: Now even though the invariant feature in itself is fully informative of the label, while the spurious feature is not, the max-margin here is parallel to the line x inv + x sp

Original: = c and is therefore reliant on the spurious feature.
Revised: = c and is therefore reliant on the spurious feature.

Original: Failure mode due to high-dimensional spurious features (lack of Constraint 4).
Revised: Failure mode due to high-dimensional spurious features (lack of Constraint 4).

Original: Akin to the setting in Tsipras et al.
Revised: Akin to the setting in Tsipras et al.

Original: (2019), albeit without noise, consider a task where the spurious feature has
Revised: (2019), albeit without noise, consider a task where the spurious feature has

Original: D different co-ordinates, X sp
Revised: D different co-ordinates, X sp

Original: = {− 1 , 1 } D and the invariant feature just one X inv = {− 1 , +1 } .
Revised: = {− 1 , 1 } D and the invariant feature just one X inv = {− 1 , +1 } .

Original: Then, assume that the i th spurious feature x sp ,i independently the value y with probability p i and − y with probability 1 − p i , where without loss of generality p i > 1 / 2 .
Revised: Then, assume that the i th spurious feature x sp ,i independently the value y with probability p i and − y with probability 1 − p i , where without loss of generality p i > 1 / 2 .

Original: Here, with high probability, all datapoints in S can be separated simply by summing up the spurious features (given D is large enough).
Revised: Here, with high probability, all datapoints in S can be separated simply by summing up the spurious features (given D is large enough).

Original: Then, we argue that the max-margin classiﬁer would provide some non-zero weight to this direction because it helps maximize its margin (see visualization in Fig 4d).
Revised: Then, we argue that the max-margin classiﬁer would provide some non-zero weight to this direction because it helps maximize its margin (see visualization in Fig 4d).

Original: One way to see why this is true is by invoking a special case of Theorem 1.
Revised: One way to see why this is true is by invoking a special case of Theorem 1.

Original: In particular, if we deﬁne (cid:80) x sp ,i to be a single dimensional spurious feature x sp , this feature satisﬁes x sp · y > 0 on all training points.
Revised: In particular, if we deﬁne (cid:80) x sp ,i to be a single dimensional spurious feature x sp , this feature satisﬁes x sp · y > 0 on all training points.

Original: In other words, this is an extreme scenario with no minority group.
Revised: In other words, this is an extreme scenario with no minority group.

Original: Then, Theoremwould yield a positive lower bound on the weight given to x sp , explaining why the classiﬁer relies on the spurious pixels.
Revised: Then, Theoremwould yield a positive lower bound on the weight given to x sp , explaining why the classiﬁer relies on the spurious pixels.

Original: For the sake of completeness, we formalize all this discussion below:
Revised: For the sake of completeness, we formalize all this discussion below:

Original: Proposition 1.
Revised: Proposition 1.

Original: Let c be a constant such that for all i , p j > 12 + c 2 .
Revised: Let c be a constant such that for all i , p j > 12 + c 2 .

Original: Let D be sufﬁciently large so that D ≥ 12 c (cid:112) 2 ln mδ where m is the number of training datapoints in S .
Revised: Let D be sufﬁciently large so that D ≥ 12 c (cid:112) 2 ln mδ where m is the number of training datapoints in S .

Original: Then, w.h.p.
Revised: Then, w.h.p.

Original: of 1 − δ over the draws of S , the max-margin classiﬁer corresponds is of the form w inv x inv + w sp x sp where:
Revised: of 1 − δ over the draws of S , the max-margin classiﬁer corresponds is of the form w inv x inv + w sp x sp where:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: First, we’ll show that on S there exists a classiﬁer that relies purely on the spurious features to separate the data.
Revised: First, we’ll show that on S there exists a classiﬁer that relies purely on the spurious features to separate the data.

Original: In particular, consider w (cid:48) sp where the i th dimension is 1 / √ D if p i > 1 / 2 , and − 1 / √ D otherwise.
Revised: In particular, consider w (cid:48) sp where the i th dimension is 1 / √ D if p i > 1 / 2 , and − 1 / √ D otherwise.

Original: By the Hoeffding’s inequality, we have that with high probability 1 − δ , on all the m training datapoints, y w (cid:48) 
Revised: By the Hoeffding’s inequality, we have that with high probability 1 − δ , on all the m training datapoints, y w (cid:48) sp · x sp ≥ 1 √ D (cid:80) (2 p i − 1) − (cid:113) 2 D ln mδ ≥ c 2 √ D .

Original: [Table]
Revised: [Figure]

Original: Now, for the max-margin classiﬁer, assume that w 2 inv = α .
Revised: Now, for the max-margin classiﬁer, assume that w 2 inv = α .

Original: Further, assume that the margin contributed by w sp x sp + b equals √ 1 − α 2 m for some m .
Revised: Further, assume that the margin contributed by w sp x sp + b equals √ 1 − α 2 m for some m .

Original: Observe that m must satisfy m ≥ c 2 √ D (as otherwise, we can replace w sp with √ 1 − α 2 w (cid:48) sp to achieve a better margin).
Revised: Observe that m must satisfy m ≥ c 2 √ D (as otherwise, we can replace w sp with √ 1 − α 2 w (cid:48) sp to achieve a better margin).

Original: Now, for the resulting margin to be maximized, α must satisfy
Revised: Now, for the resulting margin to be maximized, α must satisfy

Original: [Equation]
Revised: [Figure]

Original: Failure mode due to hard-to-separate features (lack of Constraint 5)
Revised: Failure mode due to hard-to-separate features (lack of Constraint 5)

Original: Here we assumed that the feature space can be orthogonally decomposed into invariant and spurious features.
Revised: Here we assumed that the feature space can be orthogonally decomposed into invariant and spurious features.

Original: Now let us imagine a 2D task, visualized in Fig 4c where this is not respected in that each datapoint is written as ( x inv , x inv + x sp ) , assuming that x inv =
Revised: Now let us imagine a 2D task, visualized in Fig 4c where this is not respected in that each datapoint is written as ( x inv , x inv + x sp ) , assuming that x inv =

Original: y and x sp ∈ {− 0 .
Revised: y and x sp ∈ {− 0 .

Original: A practical example of this sort of structure is the example in Fig 2d where we add a line to the last channel of CIFAR10, and then vary the brightness of the line during test-time.
Revised: A practical example of this sort of structure is the example in Fig 2d where we add a line to the last channel of CIFAR10, and then vary the brightness of the line during test-time.

Original: To understand why failure occurs in this 2D example, observe that, regardless of the correlation between x sp and y , we’d have that ( x inv + x sp ) · y > 0 .
Revised: To understand why failure occurs in this 2D example, observe that, regardless of the correlation between x sp and y , we’d have that ( x inv + x sp ) · y > 0 .

Original: In other words, the second co-ordinate is fully informative of the label.
Revised: In other words, the second co-ordinate is fully informative of the label.

Original: The max-margin classiﬁer, due to its bias, relies on both the ﬁrst and the second co-ordinate to maximize its margin i.e., the classiﬁer would be of the form w 1 x inv + w 2 ( x inv + x sp ) where w 2 > 0 .
Revised: The max-margin classiﬁer, due to its bias, relies on both the ﬁrst and the second co-ordinate to maximize its margin i.e., the classiﬁer would be of the form w 1 x inv + w 2 ( x inv + x sp ) where w 2 > 0 .

Original: (Again, like in our discussion of the failure mode of Constraint 4, we can argue this via Thm 1 by considering x inv + x sp itself as a spurious feature, and by observing that there’s no minority group here.)
Revised: (Again, like in our discussion of the failure mode of Constraint 4, we can argue this via Thm 1 by considering x inv + x sp itself as a spurious feature, and by observing that there’s no minority group here.)

Original: Hence, by assigning a positive weight to the second co-ordinate it inadvertently becomes susceptible to the spurious feature that may shift during testing.
Revised: Hence, by assigning a positive weight to the second co-ordinate it inadvertently becomes susceptible to the spurious feature that may shift during testing.

Original: B P ROOFS B. P ROOF OF T HEOREM 1 ON FAILURE DUE GEOMETRIC SKEWS
Revised: B P ROOFS B. P ROOF OF T HEOREM 1 ON FAILURE DUE GEOMETRIC SKEWS

Original: Below we provide a proof for our result analyzing the failure mode arising from geometric skews in the data.
Revised: Below we provide a proof for our result analyzing the failure mode arising from geometric skews in the data.

Original: Recall that given a dataset S , where the spurious feature can take only values in {−B , + B} , we partitioned S into two subsets S maj and S min where in S maj the points satisfy x sp · y > 0 and in S min the points satisfy x sp · y < 0 .
Revised: Recall that given a dataset S , where the spurious feature can take only values in {−B , + B} , we partitioned S into two subsets S maj and S min where in S maj the points satisfy x sp · y > 0 and in S min the points satisfy x sp · y < 0 .

Original: Next, we deﬁne two key notations.
Revised: Next, we deﬁne two key notations.

Original: First, for any dataset T ⊆ S , let v ( T ) ∈
Revised: First, for any dataset T ⊆ S , let v ( T ) ∈

Original: X inv denote a least-norm vector (purely in the invariant space) that achieves a margin of at least 1 on all datapoints in T (see Appendix B.1 for clearer deﬁnition).
Revised: X inv denote a least-norm vector (purely in the invariant space) that achieves a margin of at least 1 on all datapoints in T (we’ll deﬁne this formally a few paragraphs below).

Original: Similarly, let ˜ v
Revised: Similarly, let ˜ v

Original: ( T ) ∈
Revised: ( T ) ∈

Original: X inv denote a least-norm vector that achieves a margin of at least 1 on T , and a margin of at least 0 on S \ T (see full deﬁnition below).
Revised: X inv denote a least-norm vector that achieves a margin of at least 1 on T , and a margin of at least 0 on S \ T (again, full deﬁnition will shortly follow).

Original: While by deﬁnition, (cid:107) v ( T ) (cid:107) ≤ (cid:107) ˜ v ( T ) (cid:107) , we can informally treat these quantities as the same since empirically (cid:107) v ( T ) (cid:107) ≈ (cid:107) ˜ v ( T ) (cid:107) .
Revised: While by deﬁnition, (cid:107) v ( T ) (cid:107) ≤ (cid:107) ˜ v ( T ) (cid:107) , we can informally treat these quantities as the same since empirically (cid:107) v ( T ) (cid:107) ≈ (cid:107) ˜ v ( T ) (cid:107) .

Original: We show these plots in Sec C.1 for both MNIST and CIFAR10.
Revised: We show these plots in Sec C.1 for both MNIST and CIFAR10.

Original: But importantly, both these quantities grow with | T | as noted earlier.
Revised: But importantly, both these quantities grow with the size of T .

Original: Then, by virtue of the small size of the minority group S min , we can say, for example that (cid:107) v ( S min ) (cid:107) is smaller than both (cid:107) v ( S maj ) (cid:107) and (cid:107) v ( S ) (cid:107) .
Revised: Then, by virtue of the small size of the minority group S min , we can say that (cid:107) v ( S min ) (cid:107) is smaller than both (cid:107) v ( S maj ) (cid:107) and (cid:107) v ( S ) (cid:107) .

Original: We refer to this gap as a geometric skew.
Revised: We refer to this gap as a geometric skew.

Original: When this skew is prominent enough (e.g., (cid:107) v ( S min ) (cid:107) / (cid:107) v ( S ) (cid:107) ≈ 0 ), our result below argues that the spurious component in the overall maxmargin classiﬁer must be sufﬁciently large (and positive).
Revised: When this skew is prominent enough (e.g., (cid:107) v ( S min ) (cid:107) / (cid:107) v ( S ) (cid:107) ≈ 0 ), our result below argues that the spurious component in the overall maxmargin classiﬁer must be sufﬁciently large (and positive).

Original: On the ﬂip side, we also show that when the skew is negligible enough (e.g., (cid:107) v ( S min ) (cid:107) / (cid:107) v ( S ) (cid:107) ≈ 1 ), then the spurious component has to be sufﬁciently small.
Revised: On the ﬂip side, we also show that when the skew is negligible enough (e.g., (cid:107) v ( S min ) (cid:107) / (cid:107) v ( S ) (cid:107) ≈ 1 ), then the spurious component has to be sufﬁciently small.

Original: To be able to better visualize these bounds, we write these as bounds on |B w sp | (i.e., | w sp x sp | rather than | w sp | ).
Revised: To be able to better visualize these bounds, we write these as bounds on |B w sp | (i.e., | w sp x sp | rather than | w sp | ).

Original: Then we can think of a lower bound of the form |B w sp | (cid:38) 1 as demonstrating serious failure as a shift in the correlation can adversely reduce the original margin of ≈ 1 .
Revised: Then we can think of a lower bound of the form |B w sp | (cid:38) 1 as demonstrating serious failure as a shift in the correlation can adversely reduce the original margin of ≈ 1 .

Original: Before we state the result, for clarity, we state the full mathematical deﬁnition of v and ˜ v as follows.
Revised: Before we state the result, for clarity, we state the full mathematical deﬁnition of v and ˜ v as follows.

Original: For any T ⊆ S :
Revised: For any T ⊆ S :

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Using these notations, we state our full theorem and provide its proof below.
Revised: Using these notations, we state our full theorem and provide its proof below.

Original: Theorem 3.
Revised: Theorem 3.

Original: Let H be the set of linear classiﬁers, h ( x ) =
Revised: Let H be the set of linear classiﬁers, h ( x ) =

Original: w inv x inv + w sp x sp + b .
Revised: w inv x inv + w sp x sp + b .

Original: Let the geometric skews in a dataset S be quantiﬁed through the terms κ 1 = (cid:107) v ( S min ) (cid:107) / (cid:107) v ( S ) (cid:107) , κ 2 = (cid:107) v ( S min ) (cid:107) / (cid:107) v ( S maj ) (cid:107) and ˜ κ 1 := (cid:107) ˜ v ( S min ) (cid:107) / (cid:107) ˜ v ( S ) (cid:107) , ˜ κ 2 :
Revised: Let the geometric skews in a dataset S be quantiﬁed through the terms κ 1 = (cid:107) v ( S min ) (cid:107) / (cid:107) v ( S ) (cid:107) , κ 2 = (cid:107) v ( S min ) (cid:107) / (cid:107) v ( S maj ) (cid:107) and ˜ κ 1 := (cid:107) ˜ v ( S min ) (cid:107) / (cid:107) ˜ v ( S ) (cid:107) , ˜ κ 2 :

Original: = (cid:107) ˜ v ( S min ) (cid:107) / (cid:107) ˜ v ( S maj ) (cid:107) .
Revised: = (cid:107) ˜ v ( S min ) (cid:107) / (cid:107) ˜ v ( S maj ) (cid:107) .

Original: Then for any task satisfying all the constraints in Sec 3.1 the max-margin classiﬁer satisﬁes the inequalities (where for readability, we will use c 1 := 1 / (2 (cid:107) ˜ v ( S ) (cid:107)B ) , c 2 := 1 / (2 (cid:107) ˜ v ( S maj ) (cid:107)B ) ):
Revised: Then for any task satisfying all the constraints in Sec 3.1 the max-margin classiﬁer satisﬁes the inequalities (where for readability, we will use c 1 := 1 / (2 (cid:107) ˜ v ( S ) (cid:107)B ) , c 2 := 1 / (2 (cid:107) ˜ v ( S maj ) (cid:107)B ) ):

Original: [Equation]
Revised: [Equation]

Original: if 
Revised: if ˜

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: ifκ
Revised: if

Original: [Equation]
Revised: [Equation]

Original: ≤ 1 . For readability, it helps to think of c 1 and c 2 as small constants here (also see remark below).
Revised: For readability, it helps to think of c 1 and c 2 as small constants here (also see remark below).

Original: Furthermore, for readability, one can also imagine that all the κ terms are  similar to each other.
Revised: Furthermore, for readability, one can also imagine that all the κ terms are numerically similar to each other.

Original: We make a few remarks below before providing the proof.
Revised: We make a few remarks below before providing the proof.

Original: Remark 1.
Revised: Remark 1.

Original: For the lower bound on w sp to be positive, we need c 1 and c 2 to be small.
Revised: For the lower bound on w sp to be positive, we need c 1 and c 2 to be small.

Original: This would be true when either B or  v ( S )  (or  ˜ v ( S maj )  ) is sufﬁciently large.
Revised: This would be true when either B or (cid:107) v ( S ) (cid:107) (or (cid:107) ˜ v ( S maj ) (cid:107) ) is sufﬁciently large.

Original: This is intuitive: after all, if B is too small (say 0 ) there is no spurious feature effectively and therefore the max-margin has no incentive to use it; similarly, if v ( S ) is too small (say 0 ), then the max-margin has no incentive to use any feature besides the invariant feature, which is already quite cheap.
Revised: This is intuitive: after all, if B is too small (say 0 ) there is no spurious feature effectively and therefore the max-margin has no incentive to use it; similarly, if v ( S ) is too small (say 0 ), then the max-margin has no incentive to use any feature besides the invariant feature, which is already quite cheap.

Original: Remark 2.
Revised: Remark 2.

Original: The above result is not intended to be a numerically tight/upper lower bound.
Revised: The above result is not intended to be a numerically tight/upper lower bound.

Original: In fact, the proof can be tightened in numerous places which we however avoid to keep the result and the proof simple.
Revised: In fact, the proof can be tightened in numerous places which we however avoid, to keep the result and the proof simple.

Original: The bound is rather meant to be instructive of the effect of the geometric skew (i.e., the gap between the max-margin norms on the minority and whole/majority dataset) on the spurious component.
Revised: The bound is rather meant to be instructive of the effect of the geometric skew (i.e., the gap between the max-margin norms on the minority and whole/majority dataset) on the spurious component.

Original: Proof.
Revised: Proof.

Original: We present the proof of lower bound ﬁrst, followed by the upper bound.
Revised: We present the proof of lower bound ﬁrst, followed by the upper bound.

Original: Proof of lower bound.
Revised: Proof of lower bound.

Original: First, we will show that there exists a classiﬁer of norm 1 that relies on the spurious feature to create a sufﬁciently large margin.
Revised: First, we will show that there exists a classiﬁer of norm 1 that relies on the spurious feature to create a sufﬁciently large margin.

Original: We’ll let this classiﬁer be of the form α ˜ v ( S min ) (cid:107) ˜ v ( S min ) (cid:107) · x inv + √ 1 − α 2 x sp + αb min .
Revised: We’ll let this classiﬁer be of the form α ˜ v ( S min ) (cid:107) ˜ v ( S min ) (cid:107) · x inv + √ 1 − α 2 x sp + αb min .

Original: By the deﬁnition of ˜ v ( S min ) , the margin of this classiﬁer on any datapoint in S min is at least [Equation]
Revised: By the deﬁnition of ˜ v ( S min ) , the margin of this classiﬁer on any datapoint in S min is at least α (cid:107) ˜ v ( S min ) (cid:107) − √ 1 − α 2 B .

Original: √ 1 − α 2 B . Again, by the deﬁnition of ˜ v ( S min ) , the margin on S maj is at least √ 1 − α 2 B .
Revised: Again, by the deﬁnition of ˜ v ( S min ) , the margin on S maj is at least √ 1 − α 2 B .

Original: Let us pick an α such that these two quantities are equal.
Revised: Let us pick an α such that these two quantities are equal.

Original: Such an α would satisfy
Revised: Such an α would satisfy

Original: [Equation]
Revised: [Equation]

Original: α 2 = 2 (cid:107) ˜ v ( S min ) (cid:107)B .
Revised: [Equation]

Original: By plugging this back, we get that the resulting margin of this classiﬁer on the whole dataset S is at least
Revised: By plugging this back, we get that the resulting margin of this classiﬁer on the whole dataset S is at least

Original: B √

Original: [Equation]

Original: [Equation]

Original: v

Original: [Equation]
Revised: [Equation]

Original: S min ) [Equation] 2 B 2 . In other words, this also means the least norm classiﬁer  with a margin of at least 1 on S has a norm of at most √
Revised: In other words, this also means the least norm classiﬁer w with a margin of at least 1 on S has its norm upper bounded as:

Original: [Equation]
Revised: [Equation]

Original: Now, assume that  the least norm classiﬁer with a margin of 1 on S  is of the form w inv x inv + w sp x sp + b .
Revised: Now, assume that w (i.e., the least norm classiﬁer with a margin of 1 on S ) is of the form w inv x inv + w sp x sp + b .

Original: First, we derive a lower bound on | w sp | , and then show that w sp > 0 .
Revised: First, we derive a lower bound on | w sp | , and after that we’ll show that w sp > 0 .

Original: For this we’ll consider two cases, one where | w sp | ≥ 1 B (and so we already have a lower bound) and another case where | w sp | <  B .
Revised: For this we’ll consider two cases, one where | w sp | ≥ 1 B (and so we already have a lower bound) and another case where | w sp | < 1 B .

Original: In the latter case, we will need the invariant part of the max-margin classiﬁer, namely w inv x inv + b , to have to have a margin of at least 1 − | w sp |B on S ; if it were any lesser, the contribution from  spurious component which is at most | w sp x sp | will be unable to bump this margin up to 1 .
Revised: In the latter case, we will need the invariant part of the max-margin classiﬁer, namely w inv x inv + b , to have to have a margin of at least 1 − | w sp |B on S ; if it were any lesser, the contribution from the spurious component which is at most | w sp x sp | will be unable to bump this margin up to 1 .

Original: Now, for the invariant part of the max-margin classiﬁer to have a margin of at least 1 −| w sp |B (a non-negative quantity since | w sp | ≤ 1 / B ) on S , the norm of the  classiﬁer must be at least (1 − | w sp |B )
Revised: Now, for the invariant part of the max-margin classiﬁer to have a margin of at least 1 − | w sp |B (a non-negative quantity since | w sp | ≤ 1 / B ) on S , the norm of the (invariant part of the) classiﬁer must be at least (1 − | w sp |B )

Original: (cid:107)˜ v ( S ) (cid:107) (which follows from the deﬁnition of˜ v ( S ) ).
Revised: (cid:107) v ( S ) (cid:107) (which follows from the deﬁnition of v ( S ) ).

Original: Therefore, for the overall classiﬁer to be the maxmargin classiﬁer, we need (1 − | w sp |B ) (cid:107) v ( S ) (cid:107) ≤ √
Revised: This implies,

Original: [Equation]
Revised: [Equation]

Original:  Rearranging this gives us the bound that B| {+w sp | ≥ 1 − 2 (cid:113) (cid:107) ˜ v ( S min ) (cid:107) 2 (cid:107) v ( S ) (cid:107) 
+}
Revised: (cid:107) v ( S ) (cid:107) . Combining this with Eq 1, we get (1 − | w sp |B ) (cid:107) v ( S ) (cid:107) ≤ √ 1+4 (cid:107) ˜ v ( S min ) (cid:107) 2 B 2 B . Rearranging this gives us the bound that B| {+w sp | ≥ 1 − 2 (cid:113) (cid:107) ˜ v ( S min ) (cid:107) 2 (cid:107) v ( S ) (cid:107) 
+}

Original: [Table]
Revised: + 1 4 B 2 (cid:107) v ( S ) (cid:107) 2 .

Original: [Table]
Revised: Finally note that v ( S ) is the same as ˜ v ( S ) . Hence we can interchange these terms in the ﬁnal result (which we’ve done for readability) to arrive at |B w sp | ≥ 1 − 2 (cid:112) ˜ κ

Original: [Table]
Revised: 1 + c 21 ).

Original: What remains now is to show that w sp > 0 .
Revised: What remains now is to show that w sp > 0 .

Original: For this we do the same argument as above but with a slight modiﬁcation.
Revised: For this we do the same argument as above but with a slight modiﬁcation.

Original: First, if w sp > 1 / B , we are done.
Revised: First, if w sp > 1 / B , we are done.

Original: So, assume that w sp ≤ 1 / B .
Revised: So, assume that w sp ≤ 1 / B .

Original: Then, we can say that the invariant part of the max-margin classiﬁer i.e., w inv x inv + b , must achieve a margin of 1 − w sp B (a non-negative quantity since w sp ≤ 1 / B ) speciﬁcally on S maj .
Revised: Then, we can say that the invariant part of the max-margin classiﬁer i.e., w inv x inv + b , must achieve a margin of 1 − w sp B (a non-negative quantity since w sp ≤ 1 / B ) speciﬁcally on S maj .

Original: Then, by the deﬁnition of v ( S maj ) , it follows that the norm of our overall max-margin classiﬁer must be at least (1 − w sp B ) (cid:107) v ( S maj ) (cid:107) .
Revised: Then, by the deﬁnition of v ( S maj ) , it follows that the norm of our overall max-margin classiﬁer must be at least (1 − w sp B ) (cid:107) v ( S maj ) (cid:107) .

Original: Again, for the overall classiﬁer to be the max-margin classiﬁer, we need (1 − | w sp |B ) (cid:107) v ( S maj ) (cid:107) ≤ √ 1+4 (cid:107) ˜ v ( S min ) (cid:107) 2 B 2 B , which when rearranged gives us B w sp ≥ 1 − √ 
Revised: Again, for the overall classiﬁer to be the max-margin classiﬁer, we need (1 − | w sp |B ) (cid:107) v ( S maj ) (cid:107) ≤ √ 1+4 (cid:107) ˜ v ( S min ) (cid:107) 2 B 2 B , which when rearranged gives us B w sp ≥ 1 − √ 4 B 2 (cid:107) ˜ v ( S min ) (cid:107)

Original: [Table]
Revised: [Equation]

Original: The R.H.S is at least [-
-]
Revised: The R.H.S is at least [-
-]0 when

Original: when 14 (cid:16) 1 −

Original: [Table] [Equation] [Table] [Equation]
Revised: [Equation]

Original: [Table]

Original: [Equation]
Revised: [Equation]

Original: (cid:17) ≥

Original: [Table]

Original: S min [Table] [Equation] (cid:107) 2 (i.e., (cid:113) 14 − c 22 ≥ ˜ κ 2 ).
Revised: i.e., (cid:113) 14 − c 22 ≥ ˜ κ 2 ).

Original: In other words when ˜ κ 2 ≤ (cid:113) 14 − c 22 , we have w sp ≥ 0 .
Revised: In other words when ˜ κ 2 ≤ (cid:113) 14 − c 22 , we have w sp ≥ 0 .

Original: Proof of upper bound.
Revised: Proof of upper bound.

Original: The spurious component of the classiﬁer w inv x inv + w sp x sp
Revised: The spurious component of the classiﬁer w inv x inv + w sp x sp

Original: +
Revised: +

Original: b positively contributes to the margin of one of the groups (i.e.,  S min and S maj ), and negatively contributes to the other group, dependingon the sign of w sp .
Revised: b positively contributes to the margin of one of the groups (i.e., one of S min and S maj ), and negatively contributes to the other group, depending on the sign of w sp .

Original: On whichever group the spurious component negatively contributes to the margin, the invariant part of the classiﬁer, w inv x inv + b , must counter this and achieve a margin of 1 + | w sp |B .
Revised: On whichever group the spurious component negatively contributes to the margin, the invariant part of the classiﬁer, w inv x inv + b , must counter this and achieve a margin of 1 + | w sp |B .

Original: To manage this, we’d requirethat (cid:107) w inv (cid:107) ≥ (1 + | w sp |B )
Revised: To manage this, we’d require (cid:107) w inv (cid:107) ≥ (1 + | w sp |B )

Original: min( (cid:107) v ( S min ) (cid:107) , (cid:107) v ( S maj ) (cid:107) ) .
Revised: min( (cid:107) v ( S min ) (cid:107) , (cid:107) v ( S maj ) (cid:107) ) .

Original: In other words, the (cid:96) 2 norm of the overall  classiﬁer winv x inv + w sp x sp + b must be at least (1 + | w sp |B ) min( (cid:107) v ( S min ) (cid:107) , (cid:107) v ( Smaj ) (cid:107) ) .
Revised: In other words, for the overall max-margin classiﬁer w , we have: [Equation] At the same time, we also know from the deﬁnition of v ( S ) that

Original: If this (cid:96) 2 norm lower bound i.e., (cid:107) w inv (cid:107) , was larger than (cid:107) v ( S ) (cid:107) we’d run into a contradiction as that would result in v ( S ) being the overall min-norm classiﬁer.
Revised: [Equation]

Original: Therefore, (1 + | w sp |B ) min( (cid:107) v ( S min ) (cid:107) , (cid:107) v ( S maj ) (cid:107) ) ≤ (cid:107) v ( S ) (cid:107) .
Revised: Combining the above two equations, we can say (1+ | w sp |B ) min( (cid:107) v ( S min ) (cid:107) , (cid:107) v ( S maj ) (cid:107) ) ≤ (cid:107) v ( S ) (cid:107) .

Original: Since we are given κ 2 ≤ 1 , it means that min( (cid:107) v ( S min ) (cid:107) , (cid:107) v ( S maj ) (cid:107) )
Revised: Since we are given κ 2 ≤ 1 , it means that min( (cid:107) v ( S min ) (cid:107) , (cid:107) v ( S maj ) (cid:107) )

Original: =
Revised: =

Original: (cid:107) v ( S min ) (cid:107) , this simpliﬁes to (1 + | w sp |B ) (cid:107) v ( S min ) (cid:107) ≤ (cid:107) v ( S ) (cid:107) , which when rearranged reaches the result B w sp ≤ 1 κ 1 − 1 .
Revised: (cid:107) v ( S min ) (cid:107) , this simpliﬁes to (1 + | w sp |B ) (cid:107) v ( S min ) (cid:107) ≤ (cid:107) v ( S ) (cid:107) , which when rearranged reaches the result B w sp ≤ 1 κ 1 − 1 .

Original: To get the other upper bound here, observe that for w inv x inv + w sp x sp + b to be the overall min-norm classiﬁer, its (cid:96) 2 norm, which is lower bounded by | w sp | must not be larger than the (cid:96) 2 norm of v ( S ) .
Revised: To get the other upper bound here, observe that for w inv x inv + w sp x sp + b to be the overall min-norm classiﬁer, its (cid:96) 2 norm, which is lower bounded by | w sp | must not be larger than the (cid:96) 2 norm of v ( S ) .

Original: Hence | w sp | ≤ (cid:107) v ( S ) (cid:107) .
Revised: Hence | w sp | ≤ (cid:107) v ( S ) (cid:107) .

Original: B. P ROOF OF T HEOREM 2 ON FAILURE DUE TO STATISTICAL SKEWS
Revised: B. P ROOF OF T HEOREM 2 ON FAILURE DUE TO STATISTICAL SKEWS

Original: Below we state the full form of Theorem 2 and its proof demonstrating the effect of statistical skews in easy-to-learn tasks.
Revised: Below we state the full form of Theorem 2 and its proof demonstrating the effect of statistical skews in easy-to-learn tasks.

Original: After that, we’ll present a more precise analysis of the same in a 2D setting in Theorem 5 (for exponential loss) and in Theorem 6 (for logistic loss).
Revised: After that, we’ll present a more precise analysis of the same in a 2D setting in Theorem 5 (for exponential loss) and in Theorem 6 (for logistic loss).

Original: Our result below focuses on any easy-to-learn task and on a corresponding dataset where there are no geometric skews.
Revised: Our result below focuses on any easy-to-learn task and on a corresponding dataset where there are no geometric skews.

Original: Speciﬁcally, we consider a dataset where the invariant features have the same empirical distribution in both the majority subset (where x sp · y > 0 ) and the minority subset (where x sp · y < 0 ).
Revised: Speciﬁcally, we consider a dataset where the invariant features have the same empirical distribution in both the majority subset (where x sp · y > 0 ) and the minority subset (where x sp · y < 0 ).

Original: As a result, in this setting the max-margin classiﬁer would not rely on the spurious feature.
Revised: As a result, in this setting the max-margin classiﬁer would not rely on the spurious feature.

Original: This allows us to focus on a setting where we can isolate and study the effect of statistical skews.
Revised: This allows us to focus on a setting where we can isolate and study the effect of statistical skews.

Original: For the sake of convenience we focus on the exponential loss and under inﬁnitesimal learning rate, and a classiﬁer initialized to the origin.
Revised: For the sake of convenience, we focus on the exponential loss and under inﬁnitesimal learning rate, and a classiﬁer initialized to the origin.

Original: Theorem 4.
Revised: Theorem 4.

Original: (full form of Theorem 2 ) Let H be the set of linear classiﬁers, h ( x ) =
Revised: (full form of Theorem 2 ) Let H be the set of linear classiﬁers, h ( x ) =

Original: w inv x inv + w sp x sp .
Revised: w inv x inv + w sp x sp .

Original: Consider any task that satisﬁes all the constraints in Section 3.1.
Revised: Consider any task that satisﬁes all the constraints in Section 3.1.

Original: Consider a dataset S drawn from D such that the empirical distribution of x inv given x sp · y > 0 is identical to the empirical distribution of x inv given x sp · y < 0 .
Revised: Consider a dataset S drawn from D such that the empirical distribution of x inv given x sp · y > 0 is identical to the empirical distribution of x inv given x sp · y < 0 .

Original: If w inv ( t ) x inv + w sp ( t ) x sp is initialized to the origin, and trained with an inﬁnitesimal rate to minimize the exponential loss on a dataset S  , we have:
Revised: Let w inv ( t ) x inv + w sp ( t ) x sp be initialized to the origin, and trained with an inﬁnitesimal rate to minimize the exponential loss on a dataset S . Then, for any ( x , y ) ∈ S , we have:

Original: [Equation]
Revised: [Equation]

Original: where:
Revised: where:

Original: • p denotes the empirical level of spurious correlation, p = 1 | S | (cid:80) ( x ,y ) ∈ S 1 [ x sp · y > 0] which without generality is assumed to satisfy p ∈ [0 . 5 , 1) .
Revised: • p denotes the empirical level of spurious correlation, p = 1 | S | (cid:80) ( x ,y ) ∈ S 1 [ x sp · y > 0] which without generality is assumed to satisfy p ∈ [0 . 5 , 1) .

Original: • M denotes the maximum value of the margin of the max-margin classiﬁer on S i.e., M = max x ∈ S ˆ w · x
Revised: • M denotes the maximum value of the margin of the max-margin classiﬁer on S i.e., M = max x ∈ S ˆ w · x

Original: where ˆ w is the max-margin classiﬁer on S .
Revised: where ˆ w is the max-margin classiﬁer on S .

Original: • c := 2 B 2 (2 M− 1) 
Revised: • c := 2(2 M− 1) B

Original: Proof.
Revised: Proof.

Original: Throughout the discussion, we’ll denote w inv ( t ) and w sp ( t ) as just w inv and w sp for readability.
Revised: Throughout the discussion, we’ll denote w inv ( t ) and w sp ( t ) as just w inv and w sp for readability.

Original: Let S min and S maj denote the subset of datapoints in S where x sp · y < 0 and x sp · y > 0 respectively.
Revised: Let S min and S maj denote the subset of datapoints in S where x sp · y < 0 and x sp · y > 0 respectively.

Original: Let ˆ D inv denote the uniform distribution over x inv induced by drawing x uniformly from S min .
Revised: Let ˆ D inv denote the uniform distribution over x inv induced by drawing x uniformly from S min .

Original: By the assumption of the theorem, this distribution would be the same if x was drawn uniformly from
Revised: By the assumption of the theorem, this distribution would be the same if x was drawn uniformly from

Original: S maj .
Revised: S maj .

Original: Then, the loss function that is being minimized in this setting corresponds to:
Revised: Then, the loss function that is being minimized in this setting corresponds to:

Original: [Equation]
Revised: [Equation]

Original: where p ∈ [0 . 5 , 1) .
Revised: where p ∈ [0 . 5 , 1) .

Original: Here, the ﬁrst term is the loss on the majority dataset (where x sp = y B ) and the second term is the loss on the minority dataset (where x sp =
Revised: Here, the ﬁrst term is the loss on the majority dataset (where x sp = y B ) and the second term is the loss on the minority dataset (where x sp =

Original: − y B ).
Revised: − y B ).

Original: The update on w sp can be written as:
Revised: The update on w sp can be written as:

Original: [Equation]
Revised: [Equation]

Original: To study the dynamics of this quantity, we ﬁrst bound the value of w inv ( t ) x inv .
Revised: To study the dynamics of this quantity, we ﬁrst bound the value of w inv ( t ) x inv .

Original: Bounds on w inv ( t ) x inv The result from Soudry et al.
Revised: Bounds on w inv ( t ) x inv The result from Soudry et al.

Original: (2018) states that we can write w ( t )
Revised: (2018) states that we can write w ( t )

Original: =
Revised: =

Original: ˆ w ln(1+ t )
Revised: ˆ w ln(1 + t )

Original: + ρ ( t ) where ˆ w is the max-margin classiﬁer and ρ is a residual vector that is bounded as (cid:107) ρ ( t ) (cid:107) 

Revised: + ρ ( t ) where ˆ w is the max-margin classiﬁer and ρ is a residual vector that is bounded as (cid:107) ρ ( t ) (cid:107)

Original: = O
Revised: = O

Original: (ln ln t ) .
Revised: (ln ln t ) .

Original: Since the max-margin classiﬁer here is of the form ˆ w = ( ˆ w inv , 0) (i.e., it only relies on the invariant feature), we can infer from this that w inv ( t ) =
Revised: Since the max-margin classiﬁer here is of the form ˆ w = ( ˆ w inv , 0) (i.e., it only relies on the invariant feature), we can infer from this that w inv ( t ) =

Original: ˆ w inv ln(1 + t ) + ρ † ( t ) where again (cid:107) ρ † ( t ) (cid:107) 

Revised: ˆ w inv ln(1 + t ) + ρ † ( t ) where again (cid:107) ρ † ( t ) (cid:107)

Original: =
Revised: =

Original: O
Revised: O

Original: (ln ln t ) .
Revised: (ln ln t ) .

Original: For a sufﬁciently large t , we can say that ln ln t (cid:28) ln(1 + t ) .
Revised: For a sufﬁciently large t , we can say that ln ln t (cid:28) ln(1 + t ) .

Original: This would then imply that for all x ∈ S ,  w inv ( t ) · x  ∈ [0 .
Revised: This would then imply that for all x ∈ S , | w inv ( t ) · x inv | ∈ [0 .

Original: 5 ˆ w inv x inv ( t ) ln(1 + t ) , 2 ˆ w inv x inv ( t ) ln(1 + t )] .
Revised: 5 ˆ w inv x inv ( t ) ln(1 + t ) , 2 ˆ w inv x inv ( t ) ln(1 + t )] .

Original: Since the max-margin classiﬁer has a margin between 1 and M on the training data, this implies that, for a sufﬁciently large t and for all x ∈ S : 
Revised: Since the max-margin classiﬁer has a margin between 1 and M on the training data, this implies that, for a sufﬁciently large t and for all x ∈ S : | w inv ( t ) · x inv | ∈ [0 .

Original: [Equation]
Revised: 5 ln(1 + t ) , 2 M ln(1 + t )] .

Original: Next, we bound the dynamics of w sp .
Revised: Next, we bound the dynamics of w sp .

Original: Upper bound on w sp .
Revised: Upper bound on w sp .

Original: To upper bound˙ w sp  we  note that ˙ w sp = 0 only when w sp =
Revised: To upper bound w sp , we ﬁrst note that ˙ w sp = 0 only when w sp =

Original: 12 B ln p 1 − p .
Revised: 12 B ln p 1 − p .

Original: Furthermore, ˙ w sp is an increasing function in w sp .
Revised: Furthermore, ˙ w sp is a decreasing function in w sp .

Original: Hence, for any value of w sp that is less than 12 B ln p 1 − p , ˙ w sp ≥ 0 and for any that is greater than this value ˙ w sp ≤ 0 .
Revised: Hence, for any value of w sp that is less than 12 B ln p 1 − p , ˙ w sp ≥ 0 and for any that is greater than this value, ˙ w sp ≤ 0 .

Original: So, we can conclude that when the system is initialized at 0 , it can never cross the point w sp =
Revised: So, we can conclude that when the system is initialized at 0 , it can never cross the point w sp =

Original: 12 B ln p 1 − p .
Revised: 12 B ln p 1 − p .

Original: In other words, for all t , w sp ( t ) ≤ 1 2 B ln p 1 − p .
Revised: In other words, for all t , w sp ( t ) ≤ 12 B ln p 1 − p .

Original: Combining this with the upper bound on w inv ( t ) , we get the desired result.
Revised: Combining this with the lower bound on w inv ( t ) , we get the desired result.

Original: Lower bound on w sp .
Revised: Lower bound on w sp .

Original: We lower bound ˙ w sp via the upper bound on w sp as:
Revised: We lower bound ˙ w sp via the upper bound on w sp as:

Original: [Equation]
Revised: [Equation]

Original: Next, since we have that for all x ∈ S ,  w inv · x inv  ≤ 2 M ln( t + 1) : [Equation] [Equation] pe − w sp B
Revised: Next, since we have that for all x ∈ S , | w inv · x inv | ≤ 2 M ln

Original: [Equation] p [Equation] p
Revised: ( t + 1) :

Original: [Equation]
Revised: [Equation]

Original: Rearranging this and integrating, we get:
Revised: Rearranging this and integrating, we get:

Original: [Equation]
Revised: [Equation]

Original: (Since 2 M ≥ 2 , we can integrate the right hand side as below)
Revised: (Since 2 M ≥ [Equation] we can integrate the right hand side as below)

Original: [Equation]
Revised: [Equation]

Original: since for a sufﬁciently large t
Revised: since for a sufﬁciently large t

Original: [Equation]
Revised: [Equation]

Original: the ﬁnal paranthesis involving t will at least be half,
Revised: the ﬁnal paranthesis involving t will at least be half,

Original: [Equation]
Revised: [Equation]

Original: we can further lower bound the right hand side by applying the inequality x ≥ ln( x +1) for positive x ,
Revised: we can further lower bound the right hand side by applying the inequality x ≥ ln( x +1) for positive x ,

Original: [Equation]
Revised: [Equation]

Original: Taking exponents on both sides and rearranging
Revised: Taking exponents on both sides and rearranging,

Original: [Equation]

Original: Combining this with the upper bound on w inv ( t )

Original: [Equation]
Revised: [Equation]

Original: we get the lower bound on w sp ( t ) /w inv ( t ) .

Original: B. P RECISE ANALYSIS OF STATISTICAL SKEWS FOR A 2D SETTING UNDER EXPONENTIAL LOSS
Revised: B. P RECISE ANALYSIS OF STATISTICAL SKEWS FOR A 2D SETTING UNDER EXPONENTIAL LOSS

Original: We now consider the 2D dataset D 2-dim considered in the main paper, with  B = 1 , and provide a more precise analysis of the dynamics under exponential loss.
Revised: We now consider the 2D dataset D 2-dim considered in the main paper, with the spurious feature scale set as B = 1 , and provide a more precise analysis of the dynamics under exponential loss.

Original: This analysis is provided for the sake of completeness as the proof is self-contained and does not rely on the result of Soudry et al.
Revised: This analysis is provided for the sake of completeness as the proof is self-contained and does not rely on the result of Soudry et al.

Original:  Ji & Telgarsky (2018).
Revised: ; Ji & Telgarsky (2018).

Original: In the next section, we perform a similar analysis for logistic loss.
Revised: In the next section, we perform a similar analysis for logistic loss.

Original: Theorem 5.
Revised: Theorem 5.

Original: Under the exponential loss with inﬁnitesimal learning rate, a linear classiﬁer w inv ( t ) x inv + w sp ( t ) x sp initialized to the origin and trained on D 2-dim with B = 1 satisﬁes:
Revised: Under the exponential loss with inﬁnitesimal learning rate, a linear classiﬁer w inv ( t ) x inv + w sp ( t ) x sp initialized to the origin and trained on D 2-dim with B = 1 satisﬁes:

Original: [Equation]
Revised: [Equation]

Original: where p :=
Revised: where p :=

Original: Pr D 2-dim [ x sp · y > 0] ∈ [0 . 5 , 1] .
Revised: Pr D 2-dim [ x sp · y > 0] ∈ [0 . 5 , 1] .

Original: Proof.
Revised: Proof.

Original: Throughout the proof, we’ll drop the argument t from w inv ( t ) and w sp ( t ) for convenience.
Revised: Throughout the proof, we’ll drop the argument t from w inv ( t ) and w sp ( t ) for convenience.

Original: The loss function that is being minimized in this setting corresponds to:
Revised: The loss function that is being minimized in this setting corresponds to:

Original: [Equation]
Revised: [Equation]

Original: where p ≥ 0 .
Revised: where p ≥ 0 .

Original: Here, the ﬁrst term is the loss on the majority dataset (where x sp = y B ) and the second term is the loss on the minority dataset (where x sp =
Revised: Here, the ﬁrst term is the loss on the majority dataset (where x sp = y B ) and the second term is the loss on the minority dataset (where x sp =

Original: − y B ).
Revised: − y B ).

Original: Now the updates on w inv and w sp are given by:
Revised: Now the updates on w inv and w sp are given by:

Original: [Equation]
Revised: [Equation]

Original:  which means:
Revised: , which means:

Original: [Equation]
Revised: [Equation]

Original: Thus, by rearranging and integrating we get:
Revised: Thus, by rearranging and integrating we get:

Original: [Equation]
Revised: [Equation]

Original: Now let us deﬁne β ( t )
Revised: Now let us deﬁne β ( t )

Original: = w sp /w inv :
Revised: = w sp /w inv :

Original: [Equation]
Revised: [Equation]

Original: To bound this quantity, we’ll consider two cases, t ≥ 1 and t < 1 .
Revised: To bound this quantity, we’ll consider two cases, t ≥ 1 and t < 1 .

Original: First let us consider t ≥ 1 .
Revised: First let us consider t ≥ 1 .

Original: We begin by noting that the numerator w sp ( t ) is increasing with time t .
Revised: We begin by noting that the numerator w sp ( t ) is increasing with time t .

Original: This is because,
Revised: This is because,

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Here, the term 2(2 p − 1) 1 t +2(1 − p ) is increasing with t due to the fact that the numerator is non-negative ( p ≥ 0 . 5 ) and the denominator is decreasing with t .
Revised: Here, the term 2(2 p − 1) 1 t +2(1 − p ) is increasing with t due to the fact that the numerator is non-negative ( p ≥ 0 . 5 ) and the denominator is decreasing with t .

Original: So, given that w sp ( t ) is increasing, we can say that for all t ≥ 1 :
Revised: So, given that w sp ( t ) is increasing, we can say that for all t ≥ 1 :

Original: [Equation]
Revised: [Equation]

Original: Here we have used the fact that the denominator can be upper bounded as ln(1+2(1 − p ) t )+ln(1+ 2 pt ) ≤ ln(1 + 2 t + 4(1 − p ) pt ) ≤ ln(1 + 3 t ) .
Revised: Here we have used the fact that the denominator can be upper bounded as ln(1+2(1 − p ) t )+ln(1+ 2 pt ) ≤ ln(1 + 2 t + 4(1 − p ) pt ) ≤ ln(1 + 3 t ) .

Original: Now, for any t ≤ 1 , we can show that β ( t ) ≥ β (1) =
Revised: Now, for any t ≤ 1 , we can show that β ( t ) ≥ β (1) =

Original: ln1+2 p 3 − 2 p ln(3+4( p − p 2 )) ≥ ln 1+2 p 3 − 2 p ln4 .
Revised: ln

Original: This follows if we can show that β ( t ) is decreasing for t ≥ 0 . Taking its derivative with respect to time, we get:
Revised: [Equation]

Original: [Equation] [Equation] − [Equation]
Revised: p

Original: ) t

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: [Equation]
Revised: [Equation]

Original: − [Equation] [Equation] (1 − p )+ t 
Revised: This follows if we can show that β ( t ) is decreasing for t ≥ 0 .

Original: [Equation] t ))
Revised: Taking its derivative with respect to time, we get:

Original: [Equation]
Revised: [Equation]

Original: The sign of the above quantity is equal to the sign of:
Revised: The sign of the above quantity is equal to the sign of:

Original: [Equation]
Revised: [Equation]

Original: t

Original: [Equation]
Revised: [Equation]

Original: pt )

Original: [Equation]

Original: t (cid:19) =

Original: ln (cid:18)

Original: [Equation]

Original: (1 − p )

Original: + t (cid:19) (cid:18)

Original: [Equation]

Original: (1 − p )

Original: + t (cid:19)

Original: + ln( 2(1 − p ))

Original: (cid:18) 2(1 − p )

Original: + t (cid:19) (cid:124) (cid:123)(cid:122) (cid:125) :

Original: = f ( 1 2(1 − p ) )

Original: − ln (cid:18)2 p + t (cid:19) (cid:18)2 p + t (cid:19) − ln 1 2 p (cid:18)2 p + t (cid:19)

Original: [Equation]

Original: Now, we show that f ( x ) =
Revised: Now, we show that f ( x ) =

Original: ( x + t )
Revised: ( x + t )

Original: ln
Revised: ln

Original: ( x + t ) − ( x + t )
Revised: ( x + t ) − ( x + t )

Original: ln x = ( x + t ) ln (cid:0) 1 + tx (cid:1) is a non-increasing function:
Revised: ln x = ( x + t ) ln (cid:0) 1 + tx (cid:1) is a non-increasing function:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Now since p ≥ 0 . 5 , and f is non-increasing, f (cid:16)
Revised: Now since p ≥ 0 . 5 , and f is non-increasing, f (cid:16)

Original: [Equation]
Revised: [Equation]

Original: (1 − p ) (cid:17) − f (cid:16) 12 p (cid:17) ≤ 0 .

Original: Subsequently, ˙ β ≤ 0 .
Revised: Subsequently, ˙ β ≤ 0 .

Original: Therefore, β ( t ) ≥ β (1) for any t ∈ [0 , 1] .
Revised: Therefore, β ( t ) ≥ β (1) for any t ∈ [0 , 1] .

Original: Upper bound.
Revised: Upper bound.

Original: For an upper bound on β ( t ) , we note that since w sp ( t ) is always increasing w sp ( t ) ≤ limt →∞ w sp ( t ) = ln (cid:16) p 1 − p (cid:17) . On the other hand w inv ( t ) = ln(1 +
Revised: For an upper bound on β ( t ) , we note that since w sp ( t ) is always increasing w sp ( t ) ≤ lim

Original: [Equation]

Original: t +

Original: [Equation]
Revised: [Equation]

Original: p

Original: [Equation]

Original: − p

Original: [Equation] t [Equation] ≥ ln(1 +
Revised: On the other hand w

Original: [Equation]
Revised: [Equation]

Original: Combining these inequalities, we get:
Revised: Combining these inequalities, we get:

Original: [Equation]
Revised: [Equation]

Original: B. A NALYSIS OF STATISTICAL SKEWS FOR A 2D SETTING UNDER LOGISTIC LOSS
Revised: B. A NALYSIS OF STATISTICAL SKEWS FOR A 2D SETTING UNDER LOGISTIC LOSS

Original: While the Theorem 2 and Theorem 5 were concerned with the exponential losses, as noted in Soudry et al.
Revised: While the Theorem 2 and Theorem 5 were concerned with the exponential losses, as noted in Soudry et al.

Original: (2018), the dynamics under logistic loss are similar (although harder to analyze).
Revised: (2018), the dynamics under logistic loss are similar (although harder to analyze).

Original: For the sake of completeness, we show similar results for logistic loss in the same 2D setting as Theorem 5.
Revised: For the sake of completeness, we show similar results for logistic loss in the same 2D setting as Theorem 5.

Original: Theorem 6.
Revised: Theorem 6.

Original: Under the logistic loss with inﬁnitesimal learning rate, a linear classiﬁer w inv ( t ) x inv + w sp ( t ) x sp initialized to the origin and trained on D 2-dim with B = 1 satisﬁes for a sufﬁciently large t (where p :=
Revised: Under the logistic loss with inﬁnitesimal learning rate, a linear classiﬁer w inv ( t ) x inv + w sp ( t ) x sp initialized to the origin and trained on D 2-dim with B = 1 satisﬁes for a sufﬁciently large t (where p :=

Original: Pr D 2-dim [ x sp · y > 0] ∈ [0 . 5 , 1] ):
Revised: Pr D 2-dim [ x sp · y > 0] ∈ [0 . 5 , 1] ):

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Here, the loss function is of the form:
Revised: Here, the loss function is of the form:

Original: [Equation]
Revised: [Equation]

Original: where p ≥ 0 .
Revised: where p ≥ 0 .

Original: Now the updates on w inv and w sp are:
Revised: Now the updates on w inv and w sp are:

Original: [Equation]
Revised: [Equation]

Original: which means:
Revised: which means:

Original: [Equation]
Revised: [Equation]

Original: Solving for this, we get:
Revised: Solving for this, we get:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We ﬁrst derive some useful inequalities.
Revised: We ﬁrst derive some useful inequalities.

Original: First, we argue that forall t ,
Revised: First, we argue that forall t ,

Original: [Equation]
Revised: [Equation]

Original: This is because at the point where w sp ( t ) = 0 , ˙ w sp ( t ) ≥ 2 p − 1 1+ e w inv ≥ 

Revised: This is because at the point where w sp ( t ) = 0 , ˙ w sp ( t ) ≥ 2 p − 1 1+ e w inv ≥

Original: (since p ≥ 0 . 5 ).
Revised: (since p ≥ 0 . 5 ).

Original: Hence, the system can never reach values of w sp < 0 .
Revised: Hence, the system can never reach values of w sp < 0 .

Original: Next, we have for all t ,
Revised: Next, we have for all t ,

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: We can show this by summing up Eq 2 and
Revised: We can show this by summing up Eq 6 and

Original: [Equation]
Revised: [Equation]

Original: and since ˙ w inv ≥ 0 , and w inv ( t ) = 0 , w inv ( t ) ≥ 0 ,
Revised: and since ˙ w inv ≥ 0 , and w inv ( t ) = 0 , w inv ( t ) ≥ 0 ,

Original: [Equation]
Revised: [Equation]

Original: Next, we show:
Revised: Next, we show:

Original: [Equation]
Revised: [Equation]

Original: To show this, we divide Eq 2 by Eq 3, to get: 
Revised: To show this, we divide Eq 6 by Eq 7, to get: [Equation] ⇒

Original: [Equation]
Revised: [Equation]

Original: since by p ≥ 0 .
Revised: since by p ≥ 0 .

Original: 5 , Eq 4 and Eq 5 the ﬁrst two terms are positive, 
Revised: 5 , Eq 8 and Eq 9 the ﬁrst two terms are positive, = ⇒ [Equation] = ⇒

Original: [Equation]
Revised: [Equation]

Original: This proves the ﬁrst inequality.
Revised: This proves the ﬁrst inequality.

Original: The second inequality follows from the fact that 2(1 − p ) t +1 2 pt+ is increasing with t so applying lim t → ∞ gives us an upper bound. Finally
Revised: The second inequality follows from the fact that 2(1 − p ) t +1 2 pt

Original: [Equation]
Revised: [Equation]

Revised: is increasing with t so applying lim t → ∞ gives us an upper bound.

Original:  we rewrite Equation 2 and Equation 3 to get:
Revised: Finally, we rewrite Equation 6 and Equation 7 to get:

Original: [Equation]
Revised: [Equation]

Original: Adding and subtracting  these 
Revised: Adding and subtracting these, we get a different form for the dynamics of these quantities:

Original: [Equation]
Revised: [Equation]

Original: we get a different form for the dynamics of these quantities:

Original: [Equation]
Revised: [Equation]

Original: Lower bound.
Revised: Lower bound.

Original: To prove a lower bound on w sp ( t ) /w inv ( t ) , we’ll ﬁrst lower bound w sp .
Revised: To prove a lower bound on w sp ( t ) /w inv ( t ) , we’ll ﬁrst lower bound w sp .

Original: Observe that:
Revised: Observe that:

Original: [Equation]
Revised: [Equation]

Original: Now, since w sp is upper bounded by a constant (Eq 6), for sufﬁciently large t , the numerator of the second term inside the ln will be positive, and can be lower bounded by (2 p − 1) t .
Revised: Now, since w sp is upper bounded by a constant (Eq 10), for sufﬁciently large t , the numerator of the second term inside the ln will be positive, and can be lower bounded by (2 p − 1) t .

Original: Then, let us consider two scenarios.
Revised: Then, let us consider two scenarios.

Original: Either that w sp > w inv , in which case we already have a lower bound on β ( t ) , or that w sp ≤ w inv .
Revised: Either that w sp > w inv , in which case we already have a lower bound on β ( t ) , or that w sp ≤ w inv .

Original: In the latter case, we can lower bound the above as:
Revised: In the latter case, we can lower bound the above as:

Original: [Equation]
Revised: [Equation]

Original: Since the right hand side is an increasing in t , we can say that for sufﬁciently large t ≥ 1 ,
Revised: Since the right hand side is an increasing in t , we can say that for sufﬁciently large t ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Combining this with Eq 5 we get for sufﬁciently large t , either
Revised: Combining this with Eq 9 we get for sufﬁciently large t , either

Original: [Equation]
Revised: [Equation]

Original: or w sp ( t ) w inv ( t ) ≥ 1 .
Revised: or w sp ( t ) w inv ( t ) ≥ 1 .

Original: Upper bound.
Revised: Upper bound.

Original: To upper bound w sp ( t ) /w inv ( t ) , we’ll lower bound w inv ( t ) : 
Revised: To upper bound w sp ( t ) /w inv ( t ) , we’ll lower bound w inv ( t ) : w

Original: [Equation]
Revised: [Equation]

Original: Since by Eq 5 w inv ( t ) ∈ [0 , ln( t + 1)] and by Eq 6, w sp ( t ) ∈ [0 , 12 ln 1 − pp ] , for a sufﬁciently large t , the linear terms within the ln terms dominate and so, for large t
Revised: Since by Eq 9 w inv ( t ) ∈ [0 , ln( t + 1)] and by Eq 10, w sp ( t ) ∈ [0 , 1 2 ln 1 − p p ] , for a sufﬁciently large t , the linear terms within the ln terms dominate and so, for large t

Original: [Equation]
Revised: [Equation]

Original: Combining this with Eq 6, we get, for large t :
Revised: Combining this with Eq 10, we get, for large t :

Original: [Equation]
Revised: [Equation]

Original: C M ORE ON EXPERIMENTS
Revised: C M ORE ON EXPERIMENTS

Original: Common details: In all our MNIST-based experiments, we consider the Binary-MNIST classiﬁcation task (Arjovsky et al., 2019) where the ﬁrst ﬁve digits ( 0 to 4 ) need to be separated from the rest ( 5 to 9 ).
Revised: Common details: In all our MNIST-based experiments, we consider the Binary-MNIST classiﬁcation task (Arjovsky et al., 2019) where the ﬁrst ﬁve digits ( 0 to 4 ) need to be separated from the rest ( 5 to 9 ).

Original: Unless speciﬁed otherwise, for this we train a fully-connected three-layered ReLU network with a width of 400 and using SGD with learning rate 0 .
Revised: Unless speciﬁed otherwise, for this we train a fully-connected three-layered ReLU network with a width of 400 and using SGD with learning rate 0 .

Original: 1 for 50 epochs.
Revised: 1 for 50 epochs.

Original: In all our CIFAR10-based experiments, unless stated otherwise, we consider the 10-class classiﬁcation problem, and train a
Revised: In all our CIFAR10-based experiments, unless stated otherwise, we consider the 10-class classiﬁcation problem, and train a

Original: ResNetV1 with a depth of 20 for 200 epochs.
Revised: ResNetV1 with a depth of 20 for 200 epochs.

Original: 5 All values are averaged at least over ﬁves runs.
Revised: 6 All values are averaged at least over ﬁves runs.

Original: Finally, when we describe our datasets, we’ll adopt the convention that all pixels lie between 0 and 1 ,
Revised: Finally, when we describe our datasets, we’ll adopt the convention that all pixels lie between 0 and 1 ,

Original: C. R ANDOM FEATURE EXPERIMENTS FROM S ECTION 3. 
Revised: C. R ANDOM FEATURE EXPERIMENTS FROM S ECTION 3.

Original: For the random features based experiment on Binary MNIST in Section 3.1, we consider 50 k random ReLU features i.e., x inv = ReLU ( W x raw ) where W is a 50 k × 784 matrix (and so this is well overparameterized for dataset sizes upto 6400 ).
Revised: For the random features based experiment on Binary MNIST in Section 3.1, we consider 50 k random ReLU features i.e., x inv = ReLU ( W x raw ) where W is a 50 k × 784 matrix (and so this is well overparameterized for dataset sizes upto 6400 ).

Original: Each entry here is drawn from the normal distribution.
Revised: Each entry here is drawn from the normal distribution.

Original: We set the spurious feature support to be {− 100 , 100 } , which is about 1 / 10 th the magnitude of (cid:107) x inv (cid:107) .
Revised: We set the spurious feature support to be {− 100 , 100 } , which is about 1 / 10 th the magnitude of (cid:107) x inv (cid:107) .

Original: We also conduct similar experiments on a two-class CIFAR10 and report similar OoD accuracy drops in Fig 5a.
Revised: We also conduct similar experiments on a two-class CIFAR10 and report similar OoD accuracy drops in Fig 5a.

Original: Here, we use the ﬁrst two classes of CIFAR10, as against grouping ﬁve classes together into each of the classes.
Revised: Here, we use the ﬁrst two classes of CIFAR10, as against grouping ﬁve classes together into each of the classes.

Original: This is because on the latter dataset, the random features representation has poor in-distribution accuracy to begin with.
Revised: This is because on the latter dataset, the random features representation has poor in-distribution accuracy to begin with.

Original: C. I NCREASING NORM EXPERIMENTS FROM S ECTION
Revised: C. I NCREASING NORM EXPERIMENTS FROM S ECTION

Original: The main premise behind our geometric skews argument is that as we increase the number of datapoints, it requires greater norm for the model to ﬁt those points.
Revised: The main premise behind our geometric skews argument is that as we increase the number of datapoints, it requires greater norm for the model to ﬁt those points.

Original: We verify this for random features models on Binary-MNIST and two-class CIFAR10 in Figs 5b, 5c.[Figure]
Revised: We verify this for random features models on Binary-MNIST and two-class CIFAR10 in Figs 5b, 5c.

Original: Fig 5d , Fig 5e , Fig 5f , we plot the distance from initialization of neural network models (presented for the sake of completeness).

Original: While the theory is solely focused on linear classiﬁers, we can verify this premise intuitively for neural network classiﬁers.
Revised: While the theory is solely focused on linear classiﬁers, we can verify this premise intuitively for neural network classiﬁers.

Original: However, for neural networks, the notion of margin is not well-deﬁned.
Revised: However, for neural networks, the notion of margin is not well-deﬁned.

Original: Nevertheless, as a proxy measure, we look at how much distance the weights travel from their initialization in order to classify the dataset completely.
Revised: Nevertheless, as a proxy measure, we look at how much distance the weights travel from their initialization in order to classify the dataset completely.

Original: Such plots have already been considered in Neyshabur et al.
Revised: Such plots have already been considered in Neyshabur et al.

Original: (2017); Nagarajan & Kolter (2017; 2019)
Revised: (2017); Nagarajan & Kolter (2017; 2019)

Original: (although in the completely different context of understanding why deep networks succeed at in-distribution generalization).
Revised: (although in the completely different context of understanding why deep networks succeed at in-distribution generalization).

Original: We present similar plots for completeness.
Revised: We present similar plots for completeness.

Original: Fig 5d shows this for Binary-MNIST on an FNN.
Revised: Fig 5d shows this for Binary-MNIST on an FNN.

Original: For CIFAR10, we conduct two experiments.
Revised: For CIFAR10, we conduct two experiments.

Original: Fig 5f uses a ResNet with Adam and decaying learning rate.
Revised: Fig 5f uses a ResNet with Adam and decaying learning rate.

Original: Here, we observe that the norms saturate after a point, which is because of the learning rate decay.
Revised: Here, we observe that the norms saturate after a point, which is because of the learning rate decay.

Original: Since this does not make a fair comparison between the geometries of larger datasets and smaller datasets, we also plot this for SGD with ﬁxed learning rate in Fig 5e to recover the increasing norms observation.
Revised: Since this does not make a fair comparison between the geometries of larger datasets and smaller datasets, we also plot this for SGD with ﬁxed learning rate in Fig 5e to recover the increasing norms observation.

Original: Here, sometimes the model sometimes saturates at an accuracy of 99% (rather than 100% ); in those cases, we report the value of the weight norm at the ﬁnal epoch (namely, at the 200 th epoch).
Revised: Here, sometimes the model sometimes saturates at an accuracy of 99% (rather than 100% ); in those cases, we report the value of the weight norm at the ﬁnal epoch (namely, at the 200 th epoch).

Original: C. B ROADER EXAMPLES OF GEOMETRIC FAILURE
Revised: C. B ROADER EXAMPLES OF GEOMETRIC FAILURE

Original: We elaborate on the multiple datasets we showcased in the paper as examples where ERM fails due to a geometric skew.
Revised: We elaborate on the multiple datasets we showcased in the paper as examples where ERM fails due to a geometric skew.

Original: We ﬁrst discuss the two CIFAR10 datasets, and then discuss the cats vs. dogs example, and then discuss two Binary-MNIST datasets, one that is similar to the cats vs. dogs example, and another corresponding to high-dimensional spurious features.
Revised: We ﬁrst discuss the two CIFAR10 datasets, and then discuss the cats vs. dogs example, and then discuss two Binary-MNIST datasets, one that is similar to the cats vs. dogs example, and another corresponding to high-dimensional spurious features.

Original: C.3. CIFAR10 EXAMPLE WITH SPURIOUSLY COLORED LINE
Revised: C.3. CIFAR10 EXAMPLE WITH SPURIOUSLY COLORED LINE

Original: Here we provide more details about the CIFAR-10 dataset we presented in Sec 4 and in Fig 1c.
Revised: Here we provide more details about the CIFAR-10 dataset we presented in Sec 4 and in Fig 1c.

Original: This dataset can be thought of as an equivalent of the cow-camel classiﬁcation tasks but for 10 classes.
Revised: This dataset can be thought of as an equivalent of the cow-camel classiﬁcation tasks but for 10 classes.

Original: For this, we use ten different values of the spurious feature, one for each class.
Revised: For this, we use ten different values of the spurious feature, one for each class.

Original: We argue that the[Figure] failure here arises from the fact that the ResNet requires greater norms to ﬁt larger datapoints (see
Revised: We argue that the failure here arises from the fact that the ResNet requires greater norms to ﬁt larger datapoints (see

Original: Fig 5e), and so a similar argument as Theorem 1 should explain failure here.
Revised: Fig 5e), and so a similar argument as Theorem 1 should explain failure here.

Original: Dataset details.
Revised: Dataset details.

Original: To create the ten-valued spurious feature, we consider a vertical line passing through the middle of each channel, and also additionally the horizontal line through the ﬁrst channel.
Revised: To create the ten-valued spurious feature, we consider a vertical line passing through the middle of each channel, and also additionally the horizontal line through the ﬁrst channel.

Original: Next, we let each of these four lines take a constant value of either (0 . 5 ± 0 .
Revised: Next, we let each of these four lines take a constant value of either (0 . 5 ± 0 .

Original: 5 B ) where
Revised: 5 B ) where

Revised: [Figure]

Revised: 5d , Fig 5e , Fig 5f , we plot the distance from initialization of neural network models (presented for the sake of completeness).

Original: B ∈ [ − 1 , 1] denotes a “spurious feature scale”.
Revised: B ∈ [ − 1 , 1] denotes a “spurious feature scale”.

Original: Since each of these lines can take two conﬁgurations, it allows us to instantiate 16 different conﬁgurations.
Revised: Since each of these lines can take two conﬁgurations, it allows us to instantiate 16 different conﬁgurations.

Original: We’ll however use only 10 of these conﬁgurations, and arbitrarily ﬁx a mapping from those conﬁgurations to the 10 classes.
Revised: We’ll however use only 10 of these conﬁgurations, and arbitrarily ﬁx a mapping from those conﬁgurations to the 10 classes.

Original: For convenience let us call these ten conﬁgurations x sp , 1 , . . .
Revised: For convenience let us call these ten conﬁgurations x sp , 1 , . . .

Original: , x sp , 10 .
Revised: , x sp , 10 .

Original: Then, for any datapoint in class i , we denote the probability of the spurious feature taking the value x sp ,j , conditioned on y , as p i,j .
Revised: Then, for any datapoint in class i , we denote the probability of the spurious feature taking the value x sp ,j , conditioned on y , as p i,j .

Original: To induce a spurious correlation, we set p i,i > 0 . 1 , and set all other p i,j :=
Revised: To induce a spurious correlation, we set p i,i > 0 . 1 , and set all other p i,j :=

Original: (1 − p i,i ) / 10 .
Revised: (1 − p i,i ) / 10 .

Original: Thus, every value of the spurious feature x sp ,j is most likely to occur with its corresponding class j .
Revised: Thus, every value of the spurious feature x sp ,j is most likely to occur with its corresponding class j .

Original: Finally, note that to incorporate the spurious pixel, we zero out the original pixels in the image, and replace them with the spurious pixels.
Revised: Finally, note that to incorporate the spurious pixel, we zero out the original pixels in the image, and replace them with the spurious pixels.

Original: For the observation reported in Fig 1c, we use the value of B = 0 .
Revised: For the observation reported in Fig 1c, we use the value of B = 0 .

Original: 5 during training and testing.
Revised: 5 during training and testing.

Original: We set p i,i = 0 .
Revised: We set p i,i = 0 .

Original: 5 for all classes.
Revised: 5 for all classes.

Original: This means that on 50% of the data the spurious feature is aligned with the class (we call this the ‘Majority’ group).
Revised: This means that on 50% of the data the spurious feature is aligned with the class (we call this the ‘Majority’ group).

Original: On the remaining 50% data, the spurious feature takes one of the other 9 values at random (we call this the ‘Minority’ group).
Revised: On the remaining 50% data, the spurious feature takes one of the other 9 values at random (we call this the ‘Minority’ group).

Original: C.3. CIFAR10 EXAMPLE WITH A LINE IN THE THIRD CHANNEL
Revised: C.3. CIFAR10 EXAMPLE WITH A LINE IN THE THIRD CHANNEL

Original: Here, we elaborate on the discussion regarding the dataset in Fig 2d.
Revised: Here, we elaborate on the discussion regarding the dataset in Fig 2d.

Original: In this dataset, we add a line to the last channel of CIFAR10 (regardless of the label), and vary its brightness during testing.
Revised: In this dataset, we add a line to the last channel of CIFAR10 (regardless of the label), and vary its brightness during testing.

Original: We argue that one way to understand the failure is via the fact that the “linear mapping” Constraint 5 is broken.
Revised: We argue that one way to understand the failure is via the fact that the “linear mapping” Constraint 5 is broken.

Original: In particular, if we imagine that each channel contains the same invariant feature x inv (which is almost the case as can be seen in Fig 7), then for simplicity, we can imagine this dataset to be of the form ( x inv , x inv + x sp ) i.e., x inv and x sp are not orthogonal to each other.
Revised: In particular, if we imagine that each channel contains the same invariant feature x inv (which is almost the case as can be seen in Fig 7), then for simplicity, we can imagine this dataset to be of the form ( x inv , x inv + x sp ) i.e., x inv and x sp are not orthogonal to each other.

Original: In this scenario, the second co-ordinate can still be fully predictive of the label, and therefore the max-margin classiﬁer would rely on both the ﬁrst and the second co-ordinate to maximize its margins e.g., w 1 · x inv + w 2 ( x inv + x sp ))
Revised: In this scenario, the second co-ordinate can still be fully predictive of the label, and therefore the max-margin classiﬁer would rely on both the ﬁrst and the second co-ordinate to maximize its margins e.g., w 1 · x inv + w 2 ( x inv + x sp ))

Original: + b where w 1 , w 2 (cid:54) = 0 .
Revised: + b where w 1 , w 2 (cid:54) = 0 .

Original: Crucially, since the classiﬁer has not quite disentangled x sp[Figure] from the invariant part of the second channel, this makes the classiﬁer vulnerable to test-time shifts on x sp .
Revised: Crucially, since the classiﬁer has not quite disentangled x sp from the invariant part of the second channel, this makes the classiﬁer vulnerable to test-time shifts on x sp .

Original: In Sec A we detail this under the failure mode of Constraint 5, and visualize this failure in
Revised: In Sec A we detail this under the failure mode of Constraint 5, and visualize this failure in

Original: Fig 4c, and also connect it to Theorem 1. 
Revised: Fig 4c, and also connect it to Theorem 1. [Figure] [Figure]

Original: Dataset details.
Revised: Dataset details.

Original: In Fig 6b, we visualize this dataset for multiple values of a spurious feature scale parameter, B ∈ [ − 4 , 4] .
Revised: In Fig 6b, we visualize this dataset for multiple values of a spurious feature scale parameter, B ∈ [ − 4 , 4] .

Original: In particular, we take the last channel of CIFAR10 and add B to the middle line of the channel.
Revised: In particular, we take the last channel of CIFAR10 and add B to the middle line of the channel.

Original: Since this can result in negative pixels, we add a value of 4 to all pixels in the third channel, and then divide all those pixels by a value of 

Revised: Since this can result in negative pixels, we add a value of 4 to all pixels in the third channel, and then divide all those pixels by a value of

Original: + 8 so that they lie in between 0 and 1 .
Revised: + 8 so that they lie in between 0 and 1 .

Original: (This normalization is the reason the color of the images differ from the original CIFAR10 dataset; as such this normalization is not crucial to our discussion.)
Revised: (This normalization is the reason the color of the images differ from the original CIFAR10 dataset; as such this normalization is not crucial to our discussion.)

Original: More experimental results.
Revised: More experimental results.

Original: We run two kinds of experiments: one where we add only a vertical line to all images, and another where we add a horizontal line to 50% of the images (essentially simulating data from two different domains).
Revised: We run two kinds of experiments: one where we add only a vertical line to all images, and another where we add a horizontal line to 50% of the images (essentially simulating data from two different domains).

Original: We also run experiments for two different values of B during training, 0 . 2 and 1 . 0 .
Revised: We also run experiments for two different values of B during training, 0 . 2 and 1 . 0 .

Original: As a “control” experiment, we completely fade out the original CIFAR image in the third channel.
Revised: As a “control” experiment, we completely fade out the original CIFAR image in the third channel.

Original: Then, according to our explanation, the model should not fail in this setting as data is of the form ( x inv , x sp ) i.e., the two features are orthogonally decomposed.
Revised: Then, according to our explanation, the model should not fail in this setting as data is of the form ( x inv , x sp ) i.e., the two features are orthogonally decomposed.

Original: We summarize the key observations from these experiments (plotted in Fig 8) here:
Revised: We summarize the key observations from these experiments (plotted in Fig 8) here:

Original: As predicted, we observe

Original: that
Revised: [Figure]

Original:  when trained on the “orthogonal” dataset (where the original CIFAR10 image is zerod out in the third channel), the OoD performance of the neural network is unaffected.
Revised: As predicted, we observe that when trained on the “orthogonal” dataset (where the original CIFAR10 image is zerod out in the third channel), the OoD performance of the neural network is unaffected.

Original: This provides evidence for our explanation of failure in the “non-orthogonal” dataset.
Revised: This provides evidence for our explanation of failure in the “non-orthogonal” dataset.

Original: We observe that training on ERM on the “multidomain dataset” (Figs 8b, 8d) that has both horizontal and vertical lines, makes it more robust to test-time shifts compared to the dataset with purely vertical lines (Figs 8a, 8c).
Revised: We observe that training on ERM on the “multidomain dataset” (Figs 8b, 8d) that has both horizontal and vertical lines, makes it more robust to test-time shifts compared to the dataset with purely vertical lines (Figs 8a, 8c).

Original: Remarkably, even though the third channel has only a very faint copy of the image (Fig 7), the classiﬁer still learns a signiﬁcant enough weight on the channel that makes it susceptible to shifts in the middle line.
Revised: Remarkably, even though the third channel has only a very faint copy of the image (Fig 7), the classiﬁer still learns a signiﬁcant enough weight on the channel that makes it susceptible to shifts in the middle line.

Original: We note that introducing a different kind of test-time shift such as a Gaussian shift, is not powerful enough to cause failure since such a shift does not align with the weights learned, w 2 .
Revised: We note that introducing a different kind of test-time shift such as a Gaussian shift, is not powerful enough to cause failure since such a shift does not align with the weights learned, w 2 .

Original: However, shifts such as the line in this case, are more likely to be aligned with the classiﬁer’s weights, and hence cause a drop in the accuracy.
Revised: However, shifts such as the line in this case, are more likely to be aligned with the classiﬁer’s weights, and hence cause a drop in the accuracy.

Original: C.3. C ATS VS . D OGS EXAMPLE WITH COLORS INDEPENDENT OF LABEL
Revised: C.3. C ATS VS . D OGS EXAMPLE WITH COLORS INDEPENDENT OF LABEL

Original: Recall that the dataset from Fig 1d consists of a scenario where the images of cats vs. dogs (Elson et al., 2007) are colored independently of the label.
Revised: Recall that the dataset from Fig 1d consists of a scenario where the images of cats vs. dogs (Elson et al., 2007) are colored independently of the label.

Original: To generate this dataset, we set the ﬁrst channel to be zero.
Revised: To generate this dataset, we set the ﬁrst channel to be zero.

Original: Then, for a particular color, we pick a value B ∈ [ − 1 , 1] , and then set the second channel of every image to be 0 .
Revised: Then, for a particular color, we pick a value B ∈ [ − 1 , 1] , and then set the second channel of every image to be 0 .

Original: 5 · (1 − B ) times the original ﬁrst channel image, and the second channel to be 0 .
Revised: 5 · (1 − B ) times the original ﬁrst channel image, and the second channel to be 0 .

Original: 5 · (1 + B ) times the original ﬁrst channel image.
Revised: 5 · (1 + B ) times the original ﬁrst channel image.

Original: For the blueish images, we set B = 0 .
Revised: For the blueish images, we set B = 0 .

Original: 90 and[Figure] [Figure] for the greenish images, we set B = − 0 .
Revised: 90 and for the greenish images, we set B = − 0 .

Original: 90 (hence both these kinds of images have non-zero pixels in both the green and blue channels).
Revised: 90 (hence both these kinds of images have non-zero pixels in both the green and blue channels).

Original: We visualize these images in Fig 9.
Revised: We visualize these images in Fig 9.

Original: Then, on the training distribution, we randomly select p fraction of the data to be bluish and 1 − p fraction to be greenish as described above.
Revised: Then, on the training distribution, we randomly select p fraction of the data to be bluish and 1 − p fraction to be greenish as described above.

Original: On the testing distribution, we force all datapoints to be greenish.
Revised: On the testing distribution, we force all datapoints to be greenish.

Original: Finally, note that we randomly split the original cats vs. dogs dataset intopoints for training and use the remaining 5262 datapoints for testing/validation.
Revised: Finally, note that we randomly split the original cats vs. dogs dataset intopoints for training and use the remaining 5262 datapoints for testing/validation.

Original: In Fig 1d, we set 1 − p = 0 .
Revised: In Fig 1d, we set 1 − p = 0 .

Original: 001 , so about ≈ 20 greenish points must be seen during training.
Revised: 001 , so about ≈ 20 greenish points must be seen during training.

Original: We show more detailed results for 1 − p set to 0 .
Revised: We show more detailed results for 1 − p set to 0 .

Original: 001 , 0 . 01 and 0 .
Revised: 001 , 0 . 01 and 0 .

Original: 1 in Fig 10.
Revised: 1 in Fig 10.

Original: We observe that the OoD failure does diminish when 1 − p = 0 .
Revised: We observe that the OoD failure does diminish when 1 − p = 0 .

Original: Explaining this failure via an implicit spurious correlation.
Revised: Explaining this failure via an implicit spurious correlation.

Original: Peculiarly, even though there is no explicit visual spurious correlation between the color and the label here, we can still identify a different kind of non-visual spurious correlation.
Revised: Peculiarly, even though there is no explicit visual spurious correlation between the color and the label here, we can still identify a different kind of non-visual spurious correlation.

Original: To reason about this, ﬁrst observe that, if the two active channels (the blue and the green one) correspond to ( x 1 , x 2 ) , then x 1 + x 2 is a constant across all domains (and is fully informative of the label).
Revised: To reason about this, ﬁrst observe that, if the two active channels (the blue and the green one) correspond to ( x 1 , x 2 ) , then x 1 + x 2 is a constant across all domains (and is fully informative of the label).

Original: Hence x 1 + x 2 can hence be thought of as an invariant feature.
Revised: Hence x 1 + x 2 can hence be thought of as an invariant feature.

Original: On the other hand, consider the feature x diff = x 1 − x 2 .
Revised: On the other hand, consider the feature x diff = x 1 − x 2 .

Original: During training time, this feature would correspond to a version of the original image scaled by a positive factor of 2 B for most datapoints (and scaled by a negative factor of − 2 B for a minority of datapoints).
Revised: During training time, this feature would correspond to a version of the original image scaled by a positive factor of 2 B for most datapoints (and scaled by a negative factor of − 2 B for a minority of datapoints).

Original: A classiﬁer that relies only on x 1 + x 2 to predict the label will work ﬁne on our test distribution; but if it relies on x diff , it is susceptible to fail.
Revised: A classiﬁer that relies only on x 1 + x 2 to predict the label will work ﬁne on our test distribution; but if it relies on x diff , it is susceptible to fail.

Original: Now, we informally argue why an ERM-based classiﬁer would rely on x diff .
Revised: Now, we informally argue why an ERM-based classiﬁer would rely on x diff .

Original: If we were to think of this in terms of a linear classiﬁer, we can say that there must exist a weight vector w diff such thaty · ( w diff · x diff ) > 0 for the majority of the training datapoints. Then, we can imagine a single-
Revised: If we were to think of this in terms of a linear classiﬁer, we can say that there must exist a weight vector w diff such that

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Revised: y · ( w diff · x diff ) > 0 for the majority of the training datapoints.

Original: dimensional spurious feature that corresponds to the component of the data along this direction i.e., x sp := w diff · x diff .
Revised: Then, we can imagine a singledimensional spurious feature that corresponds to the component of the data along this direction i.e., x sp := w diff · x diff .

Original: Notably, for a majority of the datapoints in the training set we have x sp · y >and for a minority of the datapoints this feature does not necessarily correlate align with the label — let’s say x sp · y < 0 for convenience.
Revised: Notably, for a majority of the datapoints in the training set we have x sp · y >and for a minority of the datapoints this feature does not necessarily correlate align with the label — let’s say x sp · y < 0 for convenience.

Original: Then, this setting would effectively boil down to the geometric skew setting considered by Theorem 1.
Revised: Then, this setting would effectively boil down to the geometric skew setting considered by Theorem 1.

Original: In particular, we can say that when the minority group is sufﬁciently small, the classiﬁer would rely on the spurious feature x sp to make its classiﬁcation.
Revised: In particular, we can say that when the minority group is sufﬁciently small, the classiﬁer would rely on the spurious feature x sp to make its classiﬁcation.

Original: We visualize this in Fig 10d.
Revised: We visualize this in Fig 10d.

Original: Thus, even though there is no color-label correlation in this dataset, there is still a spurious correlation, although manifest in a way that does not visually stand out.
Revised: Thus, even though there is no color-label correlation in this dataset, there is still a spurious correlation, although manifest in a way that does not visually stand out.

Original: While this is one such novel kind of spurious correlation, the key message here is that we must not limit ourselves to thinking of spurious correlations as straightforward co-occurences between the label and a spurious object in the image.
Revised: While this is one such novel kind of spurious correlation, the key message here is that we must not limit ourselves to thinking of spurious correlations as straightforward co-occurences between the label and a spurious object in the image.

Original: Remark 3.
Revised: Remark 3.

Original: It is worth distinguishing this spurious correlation with the one in the Colored MNIST dataset (Arjovsky et al., 2019).
Revised: It is worth distinguishing this spurious correlation with the one in the Colored MNIST dataset (Arjovsky et al., 2019).

Original: In Colored MNIST, the color is correlated with the label, and one can again think of x diff as giving rise to the spurious feature.
Revised: In Colored MNIST, the color is correlated with the label, and one can again think of x diff as giving rise to the spurious feature.

Original: However, the manner in which this translates to a spurious feature is different.
Revised: However, the manner in which this translates to a spurious feature is different.

Original: Here, the sign of each co-ordinate in x diff is indicative of the true label (if it is positive, it means the image resides in the ﬁrst channel, and is red, which is correlated with the label).
Revised: Here, the sign of each co-ordinate in x diff is indicative of the true label (if it is positive, it means the image resides in the ﬁrst channel, and is red, which is correlated with the label).

Original: Mathematically, if we deﬁne ˜ w diff to be the vector of all ones, then we can think of ˜ w diff · x diff as a single-dimensional spurious feature here.
Revised: Mathematically, if we deﬁne ˜ w diff to be the vector of all ones, then we can think of ˜ w diff · x diff as a single-dimensional spurious feature here.

Original: In our case however, the vector w diff that yields the single-dimensional spurious feature is different, and is given by the direction of separation between the two classes.
Revised: In our case however, the vector w diff that yields the single-dimensional spurious feature is different, and is given by the direction of separation between the two classes.

Original: Remark 4.
Revised: Remark 4.

Original: We observe that in this non-standard spurious correlation setting, we require the minority group to be much smaller than in standard spurious correlation setting (like CMNIST) to create similar levels of OoD failure.
Revised: We observe that in this non-standard spurious correlation setting, we require the minority group to be much smaller than in standard spurious correlation setting (like CMNIST) to create similar levels of OoD failure.

Original: We argue that this is because the magnitude of the spurious feature | x sp | is much smaller in this non-standard setting.
Revised: We argue that this is because the magnitude of the spurious feature | x sp | is much smaller in this non-standard setting.

Original: Indeed, this effect of the spurious feature magnitude is captured in Theorem 3: the lower bound on the spurious component holds only when the minority group is sufﬁciently small to achieve ˜ κ 2 (cid:46) (cid:112) 1 / 4 − 1 / 2 | x sp | i.e., when the spurious feature magnitude | x sp | is smaller, we need the minority group to be smaller.
Revised: Indeed, this effect of the spurious feature magnitude is captured in Theorem 3: the lower bound on the spurious component holds only when the minority group is sufﬁciently small to achieve ˜ κ 2 (cid:46) (cid:112) 1 / 4 − 1 / 2 | x sp | i.e., when the spurious feature magnitude | x sp | is smaller, we need the minority group to be smaller.

Original: To see why | x sp | differs in magnitude between the standard and non-standard spurious correlation settings, let us To see why | x sp | differs in magnitude between the standard and non-standard spurious correlation settings, recall from the previous remark that in our setting, | x sp |
Revised: To see why | x sp | differs in magnitude between the standard and non-standard spurious correlation settings, let us To see why | x sp | differs in magnitude between the standard and non-standard spurious correlation settings, recall from the previous remark that in our setting, | x sp |

Original: = |
Revised: = |

Original: w diff · x diff | , while in standard spurious correlation settings | x sp | = w diff · x diff .
Revised: w diff · x diff | , while in standard spurious correlation settings | x sp | = w diff · x diff .

Original: Intuitively, ˜ w diff · x diff corresponds to separating all-positive-pixel images from all-negative-pixel images, which are well-separated classes.
Revised: Intuitively, ˜ w diff · x diff corresponds to separating all-positive-pixel images from all-negative-pixel images, which are well-separated classes.

Original: On the other hand, w diff · x diff corresponds to separating images of one real-world class from another, which are harder to separate.
Revised: On the other hand, w diff · x diff corresponds to separating images of one real-world class from another, which are harder to separate.

Original: Therefore, assuming that both weights vectors are scaled to unit norm, we can see that | w diff · x diff | (cid:28) | ˜ w diff · x diff |[Figure]
Revised: Therefore, assuming that both weights vectors are scaled to unit norm, we can see that | w diff · x diff | (cid:28) | ˜ w diff · x diff |

Original: C.3. B INARY -MNIST EXAMPLE WITH COLORS INDEPENDENT OF LABEL
Revised: C.3. B INARY -MNIST EXAMPLE WITH COLORS INDEPENDENT OF LABEL

Original: Similar to the cats vs. dogs dataset, we also consider a Binary-MNIST dataset.
Revised: Similar to the cats vs. dogs dataset, we also consider a Binary-MNIST dataset.

Original: To construct this, for each domain we pick a value B ∈ [ − 1 , 1] , and then set the ﬁrst channel of every image to be 0 .
Revised: To construct this, for each domain we pick a value B ∈ [ − 1 , 1] , and then set the ﬁrst channel of every image to be 0 .

Original: 5 · (1 + B ) times the original MNIST image, and the second channel to be 0 .
Revised: 5 · (1 + B ) times the original MNIST image, and the second channel to be 0 .

Original: 5 · (1 − B ) times the original MNIST image.
Revised: 5 · (1 − B ) times the original MNIST image.

Original: To show greater drops in OoD accuracy, we consider a stronger kind of test-time shift: during training time we set B to have different positive values so that the image is concentrated more towards the ﬁrst channel; during test-time, we ﬂip the mass completely over to the other channel by setting B = − 1 .
Revised: To show greater drops in OoD accuracy, we consider a stronger kind of test-time shift: during training time we set B to have different positive values so that the image is concentrated more towards the ﬁrst channel; during test-time, we ﬂip the mass completely over to the other channel by setting B = − 1 .

Original: Here again, we can visualize the dataset in terms of an invariant feature that corresponds to the sum of the two channels, and a spurious feature that corresponds to w diff · x diff .
Revised: Here again, we can visualize the dataset in terms of an invariant feature that corresponds to the sum of the two channels, and a spurious feature that corresponds to w diff · x diff .

Original: The exact visualization of failure here is slightly different here since we’re considering a stronger kind of shift in this setting.
Revised: The exact visualization of failure here is slightly different here since we’re considering a stronger kind of shift in this setting.

Original: In particular, during training we’d have y · ( w diff · x diff ) > 0 for all the training datapoints in this setting.
Revised: In particular, during training we’d have y · ( w diff · x diff ) > 0 for all the training datapoints in this setting.

Original: Thus, we can think of this as a setting with no minority datapoints in the training set (see Fig 12a).
Revised: Thus, we can think of this as a setting with no minority datapoints in the training set (see Fig 12a).

Original: Then, as a special case of Theorem 1, we can derive a positive lower bound on the component of the classiﬁer along x sp .
Revised: Then, as a special case of Theorem 1, we can derive a positive lower bound on the component of the classiﬁer along x sp .

Original: However, during training time, since x diff is no longer a positively scaled version of the MNIST digits, the value of w diff · x diff would no longer be informative of the label.
Revised: However, during training time, since x diff is no longer a positively scaled version of the MNIST digits, the value of w diff · x diff would no longer be informative of the label.

Original: This leads to an overall drop in the accuracy.
Revised: This leads to an overall drop in the accuracy.

Original: Experimental results.
Revised: Experimental results.

Original: We conduct two sets of experiments, one where we use only a single domain to train and another with two domains (with two unique positive values of B ).
Revised: We conduct two sets of experiments, one where we use only a single domain to train and another with two domains (with two unique positive values of B ).

Original: In both variations, we also consider a control setting where the data is not skewed: in the single-domain experiment, this means that we set B = 0 (both channels have the same mass); in the two-domain experiment this means that we set B to be positive in one domain and negative in another.
Revised: In both variations, we also consider a control setting where the data is not skewed: in the single-domain experiment, this means that we set B = 0 (both channels have the same mass); in the two-domain experiment this means that we set B to be positive in one domain and negative in another.

Original: According to our explanation above, in the single-domain control setting Fig 12, since x inv = 0 , the classiﬁer is likely to not rely on this direction and should be robust to test-time shifts in x diff .
Revised: According to our explanation above, in the single-domain control setting Fig 12, since x inv = 0 , the classiﬁer is likely to not rely on this direction and should be robust to test-time shifts in x diff .

Original: In the two-domain control setting, since the classiﬁer also sees negatively scaled images in x diff , it would be relatively robust to such negative scaling during test-time.
Revised: In the two-domain control setting, since the classiﬁer also sees negatively scaled images in x diff , it would be relatively robust to such negative scaling during test-time.

Original: Indeed, we observe in Fig 12, that the performance on the non-skewed, control datasets are robust.
Revised: Indeed, we observe in Fig 12, that the performance on the non-skewed, control datasets are robust.

Original: On the other hand, on the skewed datasets, we observe greater drops in OoD accuracy when B is ﬂipped, thus validating our hypothesis.
Revised: On the other hand, on the skewed datasets, we observe greater drops in OoD accuracy when B is ﬂipped, thus validating our hypothesis.

Original: C.3. B INARY -MNIST EXAMPLE WITH HIGH - DIMENSIONAL SPURIOUS FEATURES
Revised: C.3. B INARY -MNIST EXAMPLE WITH HIGH - DIMENSIONAL SPURIOUS FEATURES

Original: In this dataset, we consider a two-channel MNIST setting where the second channel is a set of spurious pixels, each independently picked to be either 0 or 0 .
Revised: In this dataset, we consider a two-channel MNIST setting where the second channel is a set of spurious pixels, each independently picked to be either 0 or 0 .

Original: 1 with probability 1 − p and p for[Figure] [Figure] positively labeled points and p and 1 − p for negatively labeled points.
Revised: 1 with probability 1 − p and p for positively labeled points and p and 1 − p for negatively labeled points.

Original: During training we set p = 0 .
Revised: During training we set p = 0 .

Original: 55 and p = 0 .
Revised: 55 and p = 0 .

Original: 60 corresponding to two training domains, and during testing, we ﬂip this to p = 0 . 0 .
Revised: 60 corresponding to two training domains, and during testing, we ﬂip this to p = 0 . 0 .

Original: We visualize this dataset in Fig 11b.
Revised: We visualize this dataset in Fig 11b.

Original: In Fig 13, we observe that the classiﬁer drops toaccuracy during testing.
Revised: In Fig 13, we observe that the classiﬁer drops toaccuracy during testing.

Original: To explain this failure, we can think of the sum of the pixels in the second channel as a spurious feature: since these features are independently picked, with high probability, their sum becomes fully informative of the label.
Revised: To explain this failure, we can think of the sum of the pixels in the second channel as a spurious feature: since these features are independently picked, with high probability, [Figure] [Figure] between the two channels (projected along a direction w diff where they are informative of the label) and the invariant feature as the sum of the two channels.

Revised: Fig 12b and Fig 12c show the OoD performance under different shifts in the scale of the spurious feature B .

Revised: During training time this is set to the value given by ‘Domain1’ and/or ‘Domain2’.

Original:  As discussed in Section C.3.4, this boils down to the setting of Theorem 1 when there are no minority datapoints.
Revised: [Figure] their sum becomes fully informative of the label. As discussed in Section C.3.4, this boils down to the setting of Theorem 1 when there are no minority datapoints.

Original: In Appendix A, we make this argument more formal (see under the discussion of Constraint 4 in that section). 
Revised: In Appendix A, we make this argument more formal (see under the discussion of Constraint 4 in that section). [Figure] feature scale 0. Accuracy

Original: C. E XPERIMENTS FROM S EC 5 ON STATISTICAL SKEWS
Revised: MajorityMinorityAverage [Figure] 2 Spurious feature scale 0. Accuracy

Revised: MajorityMinorityAverage (

Revised: [Figure]

Revised: [Figure]

Revised: 2 Spurious feature scale 0.

Revised: Accuracy

Revised: MajorityMinorityAverage

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: Accuracy

Revised: MajorityMinorityAverage

Revised: [Figure]

Revised: C. E XPERIMENTS FROM S EC 5 ON STATISTICAL SKEWS

Original: Experiment on D 2-dim .
Revised: Experiment on D 2-dim .

Original: For the plot in Fig 3a, we train a linear model with no bias on the logistic loss with a learning rate of 0 .
Revised: For the plot in Fig 3a, we train a linear model with no bias on the logistic loss with a learning rate of 0 .

Original: 001 , batch size of 32 and training set size of 2048 .
Revised: 001 , batch size of 32 and training set size of 2048 .

Original: C.4. B INARY -MNIST EXPERIMENTS VALIDATING THE EFFECT OF STATISTICAL SKEWS .
Revised: C.4. B INARY -MNIST EXPERIMENTS VALIDATING THE EFFECT OF STATISTICAL SKEWS .

Original: For this experiment, we consider a Binary-MNIST dataset where the second channel is set to be either all- 0 or all-constant pixels.
Revised: For this experiment, we consider a Binary-MNIST dataset where the second channel is set to be either all- 0 or all-constant pixels.

Original: We visualize this dataset in Fig 11c.
Revised: We visualize this dataset in Fig 11c.

Original: To generate our control and experimental datasets, we do the following.
Revised: To generate our control and experimental datasets, we do the following.

Original: First, we sample a set of
Revised: First, we sample a set of

Original: Binary-MNIST images S inv and their labels from the original MNIST distribution.
Revised: Binary-MNIST images S inv and their labels from the original MNIST distribution.

Original: Next we create two datasets S maj and S min by taking each of these invariant images, and appending a spurious feature to it.
Revised: Next we create two datasets S maj and S min by taking each of these invariant images, and appending a spurious feature to it.

Original: More precisely, we let S min =
Revised: More precisely, we let S min =

Original: (( x inv , x sp ) , y ) where x sp =
Revised: (( x inv , x sp ) , y ) where x sp =

Original: B ( y + 1) | x inv ∈ S inv } and
Revised: B ( y + 1) | x inv ∈ S inv } and

Original: S maj = { (( x inv , x sp ) , y ) where x sp = B ( − y + 1) | x inv ∈ S inv } .
Revised: S maj = { (( x inv , x sp ) , y ) where x sp = B ( − y + 1) | x inv ∈ S inv } .

Original: We then deﬁne a “control” dataset S con :
Revised: We then deﬁne a “control” dataset S con :

Original: = S maj ∪ S min , which has a 1:1 split between the two groups of points.
Revised: = S maj ∪ S min , which has a 1:1 split between the two groups of points.

Original: Next, we create an experimental “duplicated” dataset S exp := S maj ∪ S min ∪ S dup where
Revised: Next, we create an experimental “duplicated” dataset S exp := S maj ∪ S min ∪ S dup where

Original: S dup is a large dataset consisting of datapoints randomly chosen from S maj , thus creating a spurious correlation between the label and the spurious feature.
Revised: S dup is a large dataset consisting of datapoints randomly chosen from S maj , thus creating a spurious correlation between the label and the spurious feature.

Original: The motivation in creating datasets this way[Figure] [Figure] is that neither S con and S exp have geometric skews; however S exp does have statistical skews, and so any difference in training on these datasets can be attributed to those skews.
Revised: The motivation in creating datasets this way is that neither S con and S exp have geometric skews; however S exp does have statistical skews, and so any difference in training on these datasets can be attributed to those skews.

Original: Observations.
Revised: Observations.

Original: In our experiments, we let S inv be a set of 30 k datapoints, and so S con has 60 k datapoints.
Revised: In our experiments, we let S inv be a set of 30 k datapoints, and so S con has 60 k datapoints.

Original: We duplicate S min nine times so that S exp has 330 k datapoints and has a 10 : 1 ratio between the two groups.
Revised: We duplicate S min nine times so that S exp has 330 k datapoints and has a 10 : 1 ratio between the two groups.

Original: We consider two different settings, one where B = 0 .
Revised: We consider two different settings, one where B = 0 .

Original: 1 during training and
Revised: 1 during training and

Original: B = 1 . 

Revised: B = 1 .

Original: during training.
Revised: during training.

Original: During testing, we report the accuracy under two kinds of shifts: shifting the value of B , and also shifting the correlation completely to one direction (i.e., by concentrating all mass on the minority/majority group).
Revised: During testing, we report the accuracy under two kinds of shifts: shifting the value of B , and also shifting the correlation completely to one direction (i.e., by concentrating all mass on the minority/majority group).

Original: As reported in Fig 14, the statistically skewed dataset does suffer more during test-time when compared to the unskewed dataset.
Revised: As reported in Fig 14, the statistically skewed dataset does suffer more during test-time when compared to the unskewed dataset.

Original: C.4. CIFAR10 EXPERIMENTS VALIDATING THE EFFECT OF STATISTICAL SKEWS .
Revised: C.4. CIFAR10 EXPERIMENTS VALIDATING THE EFFECT OF STATISTICAL SKEWS .

Original: For this experiment, we consider the same CIFAR-10 dataset that we design in Section C.3.1 i.e., we introduce lines in the dataset, that can take 10 different color conﬁgurations each corresponding to one of the 10 different classes.
Revised: For this experiment, we consider the same CIFAR-10 dataset that we design in Section C.3.1 i.e., we introduce lines in the dataset, that can take 10 different color conﬁgurations each corresponding to one of the 10 different classes.

Original: Here, the scale B of the spurious feature varies from [ − 1 , 1] (see
Revised: Here, the scale B of the spurious feature varies from [ − 1 , 1] (see

Original: Section C.3.1 for more details on this).
Revised: Section C.3.1 for more details on this).

Original: The way we construct the control and experimental dataset requires a bit more care here since we have 10 classes.
Revised: The way we construct the control and experimental dataset requires a bit more care here since we have 10 classes.

Original: Speciﬁcally, we replicate S inv ten times, and to each copy, we attach a spurious feature of a particular conﬁguration.
Revised: Speciﬁcally, we replicate S inv ten times, and to each copy, we attach a spurious feature of a particular conﬁguration.

Original: This creates S con which has no geometric or statistical skews.
Revised: This creates S con which has no geometric or statistical skews.

Original: Also, the has a size of | S con | = 10 | S inv | .
Revised: Also, the has a size of | S con | = 10 | S inv | .

Original: Then, we consider the 1 / 10 th fraction of points in S con where the spurious feature has the correct conﬁguration corresponding to the label.
Revised: Then, we consider the 1 / 10 th fraction of points in S con where the spurious feature has the correct conﬁguration corresponding to the label.

Original: We create a large duplicate copy of this subset that is 81 times larger than it (we do this by randomly sampling from that subset).
Revised: We create a large duplicate copy of this subset that is 81 times larger than it (we do this by randomly sampling from that subset).

Original: We add this large duplicate set to S con to get S exp .
Revised: We add this large duplicate set to S con to get S exp .

Original: This gives rise to a dataset where for  any label, there is a 10 : 1 ratio 6 between whether the spurious feature matches the label or where it takes one of the other nine conﬁgurations.
Revised: This gives rise to a dataset where for [Figure] any label, there is a 10 : 1 ratio 7 between whether the spurious feature matches the label or where it takes one of the other nine conﬁgurations.

Original: Observations.
Revised: Observations.

Original: We run experiments by setting | S inv | = 5 k and so | S control | = 50 k and S exp = 455 k .
Revised: We run experiments by setting | S inv | = 5 k and so | S control | = 50 k and S exp = 455 k .

Original: During training we try two different values of B , 0 .
Revised: During training we try two different values of B , 0 .

Original: 1 and 

Revised: 1 and

Original: respectively.
Revised: respectively.

Original: During testing, as in the previous section, we vary both the scale of the spurious feature and also its correlation by evaluating individually on the minority dataset (where the spurious features do not match the label) and majority datasets (where the spurious features match the label).
Revised: During testing, as in the previous section, we vary both the scale of the spurious feature and also its correlation by evaluating individually on the minority dataset (where the spurious features do not match the label) and majority datasets (where the spurious features match the label).

Original: Here, again we observe that the model trained on the non-skewed dataset is less robust, evidently due to the statistical skews.
Revised: Here, again we observe that the model trained on the non-skewed dataset is less robust, evidently due to the statistical skews.

Original: It is worth noting that even though there is no statistical or geometric skew in the control dataset, we observe in Fig 15 (b) that the classiﬁer is not completely robust to shifts in the spurious feature.
Revised: It is worth noting that even though there is no statistical or geometric skew in the control dataset, we observe in Fig 15 (b) that the classiﬁer is not completely robust to shifts in the spurious feature.

Original: We suspect that this may point to other causes of failure speciﬁc to how neural network models train and learn representations.
Revised: We suspect that this may point to other causes of failure speciﬁc to how neural network models train and learn representations.

Original: D S OLUTIONS TO THE O O D PROBLEM
Revised: D S OLUTIONS TO THE O O D PROBLEM

Original: Our discussion regarding geometric and statistical skews tells us about why ERM fails.
Revised: Our discussion regarding geometric and statistical skews tells us about why ERM fails.

Original: A natural follow-up is to ask: how do we ﬁx the effect of these skews to learn a good classiﬁer?
Revised: A natural follow-up is to ask: how do we ﬁx the effect of these skews to learn a good classiﬁer?

Original: Below, we outline some natural solutions inspired by our insights.
Revised: Below, we outline some natural solutions inspired by our insights.

Original: Solution for geometric skews.
Revised: Solution for geometric skews.

Original: Our goal is to learn a margin-based classiﬁer that is not biased by geometric skews, and therefore avoids using the spurious feature.
Revised: Our goal is to learn a margin-based classiﬁer that is not biased by geometric skews, and therefore avoids using the spurious feature.

Original: Recall that we have a majority subset S maj of the training set which corresponds to datapoints where x sp · y > 0 and a minority subset S min which corresponds to datapoints where x sp · y < 0 .
Revised: Recall that we have a majority subset S maj of the training set which corresponds to datapoints where x sp · y > 0 and a minority subset S min which corresponds to datapoints where x sp · y < 0 .

Original: Our insight from the geometric skews setting is that the max-margin classiﬁer fails because it is much “harder” (in terms of (cid:96)norm) to classify the majority dataset S maj using x inv when compared to classifying S min using x inv .
Revised: Our insight from the geometric skews setting is that the max-margin classiﬁer fails because it is much “harder” (in terms of (cid:96)norm) to classify the majority dataset S maj using x inv when compared to classifying S min using x inv .

Original: A natural way to counter this effect would be to somehow bring the difﬁculty levels of these datasets closer.
Revised: A natural way to counter this effect would be to somehow bring the difﬁculty levels of these datasets closer.

Original: In particular, we propose a “balanced” max-margin classiﬁer w bal that does the following:
Revised: In particular, we propose a “balanced” max-margin classiﬁer w bal that does the following:

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original:   where c is a sufﬁciently small constant.
Revised:   where c is a sufﬁciently small constant.

Original: In words, we consider a classiﬁer that maximizes a “balanced” margin on the dataset.
Revised: In words, we consider a classiﬁer that maximizes a “balanced” margin on the dataset.

Original: The balanced margin is computed by scaling down the margins on the minority datapoints, thereby making that subset artiﬁcially harder, and thus diminishing the geometric skew.
Revised: The balanced margin is computed by scaling down the margins on the minority datapoints, thereby making that subset artiﬁcially harder, and thus diminishing the geometric skew.

Original: For an appropriately small choice of c , we can expect the balanced max-margin to minimize its reliance on the spurious feature.
Revised: For an appropriately small choice of c , we can expect the balanced max-margin to minimize its reliance on the spurious feature.

Revised: [Figure]

Original: Note that this sort of an algorithm is applicable only in settings like those in fairness literature where we know which subset of the data corresponds to the minority group and which subset corresponds to the majority group.
Revised: Note that this sort of an algorithm is applicable only in settings like those in fairness literature where we know which subset of the data corresponds to the minority group and which subset corresponds to the majority group.

Original: Solution for statistical skews.
Revised: Solution for statistical skews.