Learning Robust Models by Countering Spurious Correlations

1 INTRODUCTION . Machine learning , especially deep neural networks , has demonstrated remarkable empirical successes over various benchmarks . One promising next step is to extend such empirical achievements beyond i.i.d benchmarks . If we train a model with data from one distribution ( i.e. , the source distribution ) , how can we guarantee the error to be small over other unseen , but related distributions ( i.e. , target distributions ) . Quantifying the generalization error over two arbitrary distributions is not useful , thus , we require the distributions of study similar but different : being similar in the sense that there exists a common function that can achieve zero error over both distributions , while being different in the sense that there exists another different function that can only achieve zero error over the training distribution , but not the test distribution . This problem may not be trivial because the empirical risk minimizer ( ERM ) may lead the model to learn this second function , a topic studied under different terminologies such as spurious correlations ( Vigen , 2015 ) , confounding factors ( McDonald , 2014 ) or dataset bias ( Torralba & Efros , 2011 ) . As a result , small empirical error may not mean the model learns what we expect ( Geirhos et al. , 2019 ; Wang et al. , 2020 ) , thus the model may not be able to perform consistently over other related data . In particular , our view of the challenges in this topic is illustrated with a toy example in Figure 1 where the model is trained on the source domain data to classify triangle vs. circle and tested on the target domain data . However , the color coincides with the shape on the source domain , so the model may learn either the desired function ( relying on shape ) or the spurious function ( relying on color ) . The spurious function will not classify the target domain data correctly while the desired function can , but the ERM can not differentiate them . As one may expect , whether shape or color is considered as desired or spurious is subjective dependent on the task or the data , and in general irrelevant to the statistical nature of the problem . Therefore , our error bound will require the knowledge of the spurious function . While this is a toy example , this scenario surly exists in real world tasks ( e.g. , Jo & Bengio , 2017 ; Geirhos et al. , 2019 ; Wang et al. , 2020 ) . The contributions of this paper are : • We analyze the cross-distribution generalization error bound of a model when the model is trained with a distribution with spuriously correlated features , which is formalized as the main theorem of this paper . • We compare our bound to the widely-accepted domain adaptation one ( Ben-David et al. , 2010 ) and show that our bound can be tighter under assumptions that we consider realistic . • Our main theorem naturally offers principled solutions of this problem , and the solutions are linked to many previous established methods for robustness in a broader context . • As the principled solutions all require some knowledge of the task or the data , our main theorem also leads to a new heuristic absent of the knowledge . This new method may be on a par with the principled solutions , and can outperform the vanilla training empirically . 2 RELATED WORK . There is a rich history of learning robust models . We first discuss works in three topics , all centering around the concept of invariance , where invariance intuitively means the model ’ s prediction preserves under certain shift of the data . We then highlight works related to our theoretical discussion . Cross-domain Generalization This line of works probably originates from domain adaptation ( Ben-David et al. , 2007 ) , which studies the problem of training a model over one distribution and test it over another one . Since ( Ganin et al. , 2016 ) , recent advances along this topic mainly center around the concept of invariance : most techniques leverage different regularizers to learn representations that are invariant to the marginals of these two distributions ( e.g. , Ghifary et al. , 2016 ; Rozantsev et al. , 2018 ) . Further , the community aims beyond the situation that a trained model from domain adaptation may only be applicable to one distribution , and focuses on domain generalization ( Muandet et al. , 2013 ) , which studies the problem of training a model over a collection of distributions and test it with distributions unseen during training . Similarly , most recent methods aim to learn representations invariant to the marginals of the training distributions ( e.g. , Motiian et al. , 2017 ; Li et al. , 2018 ; Carlucci et al. , 2018 ) . Recently , the community extends the study to domain generalization without domain IDs to address the real-world situations that domain IDs are unavailable ( Wang et al. , 2019b ) , which again focuses on learning representations invariant to specifically designed functions . Adversarially Robust Models The study of robustness against adversarial examples was popularized by the empirical observations that small perturbations on image data can significantly alter the model ’ s prediction ( Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ) . This observation initiated a line of works building models invariant to such small perturbations ( the rigorous definitions of “ small perturbations ” will not be discussed in details here ) ( e.g. , Lee et al. , 2017 ; Akhtar et al. , 2018 ) and adversarial training ( Madry et al. , 2018 ) is currently the most widely-accepted method in terms of empirical defense . On the other hand , the community also aims to develop methods that are provably robust to predefined perturbations ( e.g. , Wong & Kolter , 2018 ; Croce & Hein , 2020 ) , which links back to the works of distributional robust models ( e.g. , Abadeh et al. , 2015 ; Sagawa * et al. , 2020 ) , whose central goal is to train models invariant to a predefined shift of distributions . Recent evidence shows key challenges of learning adversarially robust models are spuriously correlated features ( Ilyas et al. , 2019 ; Wang et al. , 2020 ) , connecting adversarial robustness to the next topic . Countering Spurious Correlation Works along this line usually connects the robustness of a model to its ability of ignoring the spurious correlation in the data , which was also studied under the terminologies of confounding factors , or dataset bias . With different concrete definitions of the spurious correlation , methods have been developed for various applications , such as image/video classification ( e.g. , Goyal et al. , 2017 ; Wang et al. , 2019a ; b ; Bahng et al. , 2019 ; Shi et al. , 2020 ) , text classification ( e.g. , He et al. , 2019 ; Clark et al. , 2019 ; Bras et al. , 2020 ; Zhou & Bansal , 2020 ; Ko et al. , 2020 ) , medical diagnosis ( e.g. , Zech et al. , 2018 ; Chaibub Neto et al. , 2019 ; Larrazabal et al. , 2020 ) etc . The key concept is , as expected , to be invariant to the spurious correlated features . Related Theoretical Discussion Out of a rich collection of theoretical discussions in learning robust models , we only focus on the ones for unsupervised domain adaptation , as they will be related to our discussions in the sequel . Popularized by ( Ben-David et al. , 2007 ; 2010 ) , these analyses , although in various forms ( Mansour et al. , 2009 ; Germain et al. , 2016 ; Zhang et al. , 2019 ; Dhouib et al. , 2020 ) , mostly involve two additional terms than standard machine learning generalization bound : one term describes the “ learnable ” nature of the problem and one term quantifies the differences of the two distributions . This second term probably inspired most of the empirical methods forcing invariant representations from distributions . However , the value of invariance is recently challenged ( Wu et al. , 2019 ; Zhao et al. , 2019 ) . For example , Zhao et al . ( 2019 ) argued that “ invariance is not sufficient ” by showing counter examples violating the “ learnable ” nature of the problem and formalized the understanding as that the two distributions have possibly different labeling functions . Key Difference : However , we find the argument of disparity in labeling functions less intuitive , because human will nonetheless be able to agree on the label of an object whichever distribution the object lies in : in the context of this paper , we argue a shared labeling function always exists ( in any task reasonable to human ) , but the ERM model may not have the incentive to learn this function and learns a spurious one instead . As in Figure 1 , we formalize the problem as learning against spurious functions , and argue that the central problem is still invariance , but instead of invariance to marginals , we urge for invariance to the spurious function . Our discussion also applies to more than unsupervised domain adaptation and relates to most of the topics discussed in this section . 3 GENERALIZATION UNDERSTANDING WITH SPURIOUS CORRELATION . 3.1 NOTATIONS & BACKGROUND . We consider a binary classification problem from feature space X ∈ Rp to label space Y ∈ { 0 , 1 } . The distribution over X is denoted as P. A labeling function f : X → Y is a function that maps feature x to its label y . A hypothesis or model θ : X → Y is also a function that maps feature to the label . The difference in naming is only because we want to differentiate whether the function is a natural property of the space or distribution ( thus called a labeling function ) or a function to estimate ( thus called a hypothesis or model ) . The hypothesis space is denoted as Θ . This work concerns with the generalization error across two distributions , namely source and target distribution , denoted as Ps and Pt respectively . As stated previously , we are only interested when these two distributions are similar but different : being similar means there exists a desired labeling function , fd , that maps any x ∈ X to its label ( thus the label y : = fd ( x ) ) ; being different means there exists a spurious labeling function , fp , that for any x ∼ Ps , fp ( x ) = fd ( x ) . This “ similar but different ” property will be reiterated as an assumption ( A2 ) later . We use ( x , y ) to denote a sample , and use ( X , Y ) P to denote a finite dataset if the features are drawn from P. We use P ( θ ) to denote the expected risk of θ over distribution P , and use ·̂ to denote estimated term · ( e.g. , the empirical risk is ̂P ( θ̂ ) ) . We use l ( · , · ) to denote a generic loss function . For a dataset ( X , Y ) P , if we train a model θ̂ = arg min θ∈Θ ∑ ( x , y ) ∈ ( X , Y ) P l ( θ ( x ) , y ) , ( 1 ) previous generalization study suggests that we can expect the error rate to be bounded as P ( θ̂ ) ≤ ̂P ( θ̂ ) + φ ( |Θ| , n , δ ) , ( 2 ) where P ( θ̂ ) and ̂P ( θ̂ ) respectively are P ( θ̂ ) = Ex∼P|θ̂ ( x ) − y| = Ex∼P|θ̂ ( x ) − fd ( x ) | and ̂P ( θ̂ ) = 1 n ∑ ( x , y ) ∈ ( X , Y ) P |θ̂ ( x ) − y| , and φ ( |Θ| , n , δ ) is a function of hypothesis space |Θ| , number of samples n , and δ accounts for the probability when the bound holds . This paper only concerns with this generic form that can subsume many discussions , each with its own assumptions . We refer to these assumptions as A1 . A1 : basic assumptions needed to derived ( 2 ) , formalized with two examples in appendix A.1 .

Summary: This paper studies the problem that spurious features in the training set can cause accuracy drop in the test phase. They formalize a generalization error bound for this setup and provide two solutions, one principled solution with the knowledge of spurious correlated features and one minimal supervision (MS) method without knowing this information, based on their bound. They also have some experimental results demonstrating the effectiveness of their proposed MS method.