Learning Robust Models by Countering Spurious Correlations

1 INTRODUCTION . Machine learning , especially deep neural networks , has demonstrated remarkable empirical successes over various benchmarks . One promising next step is to extend such empirical achievements beyond i.i.d benchmarks . If we train a model with data from one distribution ( i.e. , the source distribution ) , how can we guarantee the error to be small over other unseen , but related distributions ( i.e. , target distributions ) . Quantifying the generalization error over two arbitrary distributions is not useful , thus , we require the distributions of study similar but different : being similar in the sense that there exists a common function that can achieve zero error over both distributions , while being different in the sense that there exists another different function that can only achieve zero error over the training distribution , but not the test distribution . This problem may not be trivial because the empirical risk minimizer ( ERM ) may lead the model to learn this second function , a topic studied under different terminologies such as spurious correlations ( Vigen , 2015 ) , confounding factors ( McDonald , 2014 ) or dataset bias ( Torralba & Efros , 2011 ) . As a result , small empirical error may not mean the model learns what we expect ( Geirhos et al. , 2019 ; Wang et al. , 2020 ) , thus the model may not be able to perform consistently over other related data . In particular , our view of the challenges in this topic is illustrated with a toy example in Figure 1 where the model is trained on the source domain data to classify triangle vs. circle and tested on the target domain data . However , the color coincides with the shape on the source domain , so the model may learn either the desired function ( relying on shape ) or the spurious function ( relying on color ) . The spurious function will not classify the target domain data correctly while the desired function can , but the ERM can not differentiate them . As one may expect , whether shape or color is considered as desired or spurious is subjective dependent on the task or the data , and in general irrelevant to the statistical nature of the problem . Therefore , our error bound will require the knowledge of the spurious function . While this is a toy example , this scenario surly exists in real world tasks ( e.g. , Jo & Bengio , 2017 ; Geirhos et al. , 2019 ; Wang et al. , 2020 ) . The contributions of this paper are : • We analyze the cross-distribution generalization error bound of a model when the model is trained with a distribution with spuriously correlated features , which is formalized as the main theorem of this paper . • We compare our bound to the widely-accepted domain adaptation one ( Ben-David et al. , 2010 ) and show that our bound can be tighter under assumptions that we consider realistic . • Our main theorem naturally offers principled solutions of this problem , and the solutions are linked to many previous established methods for robustness in a broader context . • As the principled solutions all require some knowledge of the task or the data , our main theorem also leads to a new heuristic absent of the knowledge . This new method may be on a par with the principled solutions , and can outperform the vanilla training empirically . 2 RELATED WORK . There is a rich history of learning robust models . We first discuss works in three topics , all centering around the concept of invariance , where invariance intuitively means the model ’ s prediction preserves under certain shift of the data . We then highlight works related to our theoretical discussion . Cross-domain Generalization This line of works probably originates from domain adaptation ( Ben-David et al. , 2007 ) , which studies the problem of training a model over one distribution and test it over another one . Since ( Ganin et al. , 2016 ) , recent advances along this topic mainly center around the concept of invariance : most techniques leverage different regularizers to learn representations that are invariant to the marginals of these two distributions ( e.g. , Ghifary et al. , 2016 ; Rozantsev et al. , 2018 ) . Further , the community aims beyond the situation that a trained model from domain adaptation may only be applicable to one distribution , and focuses on domain generalization ( Muandet et al. , 2013 ) , which studies the problem of training a model over a collection of distributions and test it with distributions unseen during training . Similarly , most recent methods aim to learn representations invariant to the marginals of the training distributions ( e.g. , Motiian et al. , 2017 ; Li et al. , 2018 ; Carlucci et al. , 2018 ) . Recently , the community extends the study to domain generalization without domain IDs to address the real-world situations that domain IDs are unavailable ( Wang et al. , 2019b ) , which again focuses on learning representations invariant to specifically designed functions . Adversarially Robust Models The study of robustness against adversarial examples was popularized by the empirical observations that small perturbations on image data can significantly alter the model ’ s prediction ( Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ) . This observation initiated a line of works building models invariant to such small perturbations ( the rigorous definitions of “ small perturbations ” will not be discussed in details here ) ( e.g. , Lee et al. , 2017 ; Akhtar et al. , 2018 ) and adversarial training ( Madry et al. , 2018 ) is currently the most widely-accepted method in terms of empirical defense . On the other hand , the community also aims to develop methods that are provably robust to predefined perturbations ( e.g. , Wong & Kolter , 2018 ; Croce & Hein , 2020 ) , which links back to the works of distributional robust models ( e.g. , Abadeh et al. , 2015 ; Sagawa * et al. , 2020 ) , whose central goal is to train models invariant to a predefined shift of distributions . Recent evidence shows key challenges of learning adversarially robust models are spuriously correlated features ( Ilyas et al. , 2019 ; Wang et al. , 2020 ) , connecting adversarial robustness to the next topic . Countering Spurious Correlation Works along this line usually connects the robustness of a model to its ability of ignoring the spurious correlation in the data , which was also studied under the terminologies of confounding factors , or dataset bias . With different concrete definitions of the spurious correlation , methods have been developed for various applications , such as image/video classification ( e.g. , Goyal et al. , 2017 ; Wang et al. , 2019a ; b ; Bahng et al. , 2019 ; Shi et al. , 2020 ) , text classification ( e.g. , He et al. , 2019 ; Clark et al. , 2019 ; Bras et al. , 2020 ; Zhou & Bansal , 2020 ; Ko et al. , 2020 ) , medical diagnosis ( e.g. , Zech et al. , 2018 ; Chaibub Neto et al. , 2019 ; Larrazabal et al. , 2020 ) etc . The key concept is , as expected , to be invariant to the spurious correlated features . Related Theoretical Discussion Out of a rich collection of theoretical discussions in learning robust models , we only focus on the ones for unsupervised domain adaptation , as they will be related to our discussions in the sequel . Popularized by ( Ben-David et al. , 2007 ; 2010 ) , these analyses , although in various forms ( Mansour et al. , 2009 ; Germain et al. , 2016 ; Zhang et al. , 2019 ; Dhouib et al. , 2020 ) , mostly involve two additional terms than standard machine learning generalization bound : one term describes the “ learnable ” nature of the problem and one term quantifies the differences of the two distributions . This second term probably inspired most of the empirical methods forcing invariant representations from distributions . However , the value of invariance is recently challenged ( Wu et al. , 2019 ; Zhao et al. , 2019 ) . For example , Zhao et al . ( 2019 ) argued that “ invariance is not sufficient ” by showing counter examples violating the “ learnable ” nature of the problem and formalized the understanding as that the two distributions have possibly different labeling functions . Key Difference : However , we find the argument of disparity in labeling functions less intuitive , because human will nonetheless be able to agree on the label of an object whichever distribution the object lies in : in the context of this paper , we argue a shared labeling function always exists ( in any task reasonable to human ) , but the ERM model may not have the incentive to learn this function and learns a spurious one instead . As in Figure 1 , we formalize the problem as learning against spurious functions , and argue that the central problem is still invariance , but instead of invariance to marginals , we urge for invariance to the spurious function . Our discussion also applies to more than unsupervised domain adaptation and relates to most of the topics discussed in this section . 3 GENERALIZATION UNDERSTANDING WITH SPURIOUS CORRELATION . 3.1 NOTATIONS & BACKGROUND . We consider a binary classification problem from feature space X ∈ Rp to label space Y ∈ { 0 , 1 } . The distribution over X is denoted as P. A labeling function f : X → Y is a function that maps feature x to its label y . A hypothesis or model θ : X → Y is also a function that maps feature to the label . The difference in naming is only because we want to differentiate whether the function is a natural property of the space or distribution ( thus called a labeling function ) or a function to estimate ( thus called a hypothesis or model ) . The hypothesis space is denoted as Θ . This work concerns with the generalization error across two distributions , namely source and target distribution , denoted as Ps and Pt respectively . As stated previously , we are only interested when these two distributions are similar but different : being similar means there exists a desired labeling function , fd , that maps any x ∈ X to its label ( thus the label y : = fd ( x ) ) ; being different means there exists a spurious labeling function , fp , that for any x ∼ Ps , fp ( x ) = fd ( x ) . This “ similar but different ” property will be reiterated as an assumption ( A2 ) later . We use ( x , y ) to denote a sample , and use ( X , Y ) P to denote a finite dataset if the features are drawn from P. We use P ( θ ) to denote the expected risk of θ over distribution P , and use ·̂ to denote estimated term · ( e.g. , the empirical risk is ̂P ( θ̂ ) ) . We use l ( · , · ) to denote a generic loss function . For a dataset ( X , Y ) P , if we train a model θ̂ = arg min θ∈Θ ∑ ( x , y ) ∈ ( X , Y ) P l ( θ ( x ) , y ) , ( 1 ) previous generalization study suggests that we can expect the error rate to be bounded as P ( θ̂ ) ≤ ̂P ( θ̂ ) + φ ( |Θ| , n , δ ) , ( 2 ) where P ( θ̂ ) and ̂P ( θ̂ ) respectively are P ( θ̂ ) = Ex∼P|θ̂ ( x ) − y| = Ex∼P|θ̂ ( x ) − fd ( x ) | and ̂P ( θ̂ ) = 1 n ∑ ( x , y ) ∈ ( X , Y ) P |θ̂ ( x ) − y| , and φ ( |Θ| , n , δ ) is a function of hypothesis space |Θ| , number of samples n , and δ accounts for the probability when the bound holds . This paper only concerns with this generic form that can subsume many discussions , each with its own assumptions . We refer to these assumptions as A1 . A1 : basic assumptions needed to derived ( 2 ) , formalized with two examples in appendix A.1 .

Summary: This paper formalizes a new generalization error bound for the problem of spurious correlation (a.k.a. confounding factors or dataset bias) and shows that it is tighter than the well-established domain adaptation one under realistic assumptions. The analysis leads to a set of solutions linking to established solutions. It further proposes a practical solution that does not require explicit knowledge of spurious correlated features as the established ones need.