Learning Stable Classifiers by Transferring Unstable Features

1 INTRODUCTION . Automatic de-biasing ( Sohoni et al. , 2020 ; Creager et al. , 2021 ; Sanh et al. , 2021 ) has emerged as a promising direction for learning stable classifiers . The key premise here is that no additional annotations for the bias attribute are required . However , bias is a human-defined concept and can vary from task to task . Provided with only input-label pairs , algorithms may not have sufficient information to distinguish stable ( causal ) features from unstable ( spurious ) features . To address this challenge , we note that related tasks are often fraught with similar spurious correlations . For instance , when classifying animals such as camels vs. cows , their backgrounds ( desert vs. grass ) may constitute a spurious correlation ( Beery et al. , 2018 ) . The same bias between the label and the background also persists in other related classification tasks ( such as sheep vs. antelope ) . In the resource-scarce target task , we only have access to the input-label pairs . However , in the source tasks , where training data is sufficient , identifying biases may be easier . For instance , we may have examples collected from multiple environments , in which correlations between bias features and the label are different ( Arjovsky et al. , 2019 ) . These source environments help us define the exact bias features that we want to regulate . One obvious approach to utilize the source task is direct transfer . Specifically , given multiple source environments , we can train an unbiased source classifier and then apply its representation to the target task . However , we empirically demonstrate that while the source classifier is not biased when making its final predictions , its internal continuous representation can still encode information about the unstable features . Figure 1 shows that in Colored MNIST , where the digit label is spuriously correlated with the image color , direct transfer by either re-using or fine-tuning the representation learned on the source task fails in the target task , performing no better than the majority baseline . In this paper , we propose to explicitly inform the target classifier about unstable features from the source data . Specifically , we derive a representation that encodes these unstable features using the source environments . Then we identify distinct subpopulations by clustering examples based on this representation and apply group DRO ( Sagawa et al. , 2019 ) to minimize the worst-case risk over these subpopulations . As a result , we enforce the target classifier to be robust against different values of the unstable features . In the example above , animals would be clustered according to backgrounds , and the classifier should perform well regardless of the clusters ( backgrounds ) . The remaining question is how to compute the unstable feature representation using the source data environments . Following Bao et al . ( 2021 ) , we hypothesize that unstable features are reflected in mistakes observed during classifier transfer across environments . For instance , if the classifier uses the background to distinguish camels from cows , the camel images that are predicted correctly would have a desert background while those predicted incorrectly are likely to have a grass background . More generally , we prove that among examples with the same label value , those with the same prediction outcome will have more similar unstable features than those with different predictions . By forcing examples with the same prediction outcome to stay closer in the feature space , we obtain a representation that encodes these latent unstable features . We evaluate our approach , Transfer OF Unstable features ( TOFU ) , on both synthetic and realworld environments . Our synthetic experiments first confirm our hypothesis that standard transfer approaches fail to learn a stable classifier for the target task . By explicitly transferring the unstable features , our method significantly improves over the best baseline across 12 transfer settings ( 22.9 % in accuracy ) , and reaches the performance of an oracle model that has direct access to the unstable features ( 0.3 % gap ) . Next , we consider a practical setting where environments are defined by an input attribute and our goal is to reduce biases from other unknown attributes . On CelebA , TOFU achieves the best worst-group accuracy across 38 latent attributes , outperforming the best baseline by 18.06 % . Qualitative and quantitative analyses confirm that TOFU is able to identify the unstable features . 2 RELATED WORK . Removing bias via annotations : Due to idiosyncrasies of the data collection process , annotations are often coupled with unwanted biases ( Buolamwini and Gebru , 2018 ; Schuster et al. , 2019 ; McCoy et al. , 2019 ; Yang et al. , 2019 ) . To address this issue and learn robust models , researchers leverage extra information ( Belinkov et al. , 2019 ; Stacey et al. , 2020 ; Hinton , 2002 ; Clark et al. , 2019 ; He et al. , 2019 ; Mahabadi et al. , 2020 ) . One line of work assumes that the bias attributes are known and have been annotated for each example , e.g. , group distributionally robust optimization ( DRO ) ( Hu et al. , 2018 ; Oren et al. , 2019 ; Sagawa et al. , 2020 ) . By defining groups based on these bias attributes , we explicitly specify the distribution family to optimize over . However , identifying the hidden biases is time-consuming and often requires domain knowledge ( Zellers et al. , 2019 ; Sakaguchi et al. , 2020 ) . To address this issue , another line of work ( Peters et al. , 2016 ; Krueger et al. , 2020 ; Chang et al. , 2020 ; Jin et al. , 2020 ; Ahuja et al. , 2020 ; Arjovsky et al. , 2019 ; Bao et al. , 2021 ; Kuang et al. , 2020 ; Shen et al. , 2020 ) only assumes access to a set of data environments . These environments are defined based on readily-available information of the data collection circumstances , such as location and time . The main assumption is that while spurious correlations vary across different environments , the association between the causal features and the label should stay the same . Thus , by learning a representation that is invariant across all environments , they alleviate the dependency on spurious features . In contrast to previous works , we don ’ t have access to any additional information besides the labels in our target task . We show that we can achieve robustness by transferring the unstable features from a related source task . Automatic de-biasing A number of recent approaches focus on a more common setting where the algorithm only has access to the input-label pairs . Sanh et al . ( 2021 ) ; Nam et al . ( 2020 ) ; Utama et al . ( 2020 ) find that weak models are more vulnerable to spurious correlations as they only learn shallow heuristics . By boosting from their mistakes , they obtain a more robust model . Qiao et al . ( 2020 ) uses adversarial learning to augment the biased training data . Creager et al . ( 2021 ) ; Sohoni et al . ( 2020 ) ; Ahmed et al . ( 2020 ) ; Matsuura and Harada ( 2020 ) ; Liu et al . ( 2021 ) propose to identify minority groups by looking at the features produced by a biased model . These automatic approaches are intriguing as they do not require additional annotation . However , we note that bias is a human-centric concept and can vary from tasks to tasks . For models that only have access to the input-label pairs , they have no information to distinguish causal features from bias features . For example , consider the Colored MNIST dataset , where color and digit shape are correlated in the training set but not in the test set . If our task is to predict the digit , then color becomes the spurious bias that we want to remove . Vice versa , if we want to predict the color , then digit shape is spurious . Creager et al . ( 2021 ) ; Nam et al . ( 2020 ) empirically demonstrate that they can learn a color-invariant model for the digit prediction task . However , their approaches will result in the same color-invariant model for the color prediction task , and thus fail at test time , when color and digit are no longer correlated . In this work , we leverage source tasks to define the exact bias that we want to remove for the target task . Transferring robustness across tasks : Prior work has also studied the transferability of adversarial robustness across tasks . For example , Hendrycks et al . ( 2019 ) ; Shafahi et al . ( 2020 ) show that by pre-training the model on a large-scale source task , we can improve the model robustness against adversarial perturbations over l∞ norm . We note that these perturbations measure the smoothness of the classifier , rather than the stability of the classifier against spurious correlations . In fact , our results show that if we directly re-use or fine-tune the pre-trained feature extractor on the target task , the model will quickly over-fit to the unstable correlations present in the data . We propose to address this issue by explicitly inferring the unstable features using the source environments and use this information to guide the target classifier during training . 3 METHOD . Problem formulation We consider the transfer problem from a source task to a target task . For the source task , we assume the standard setting ( Arjovsky et al. , 2019 ) where the training data contain n environments E1 , . . . , En . Within each environment Ei , examples are drawn from the joint distribution Pi ( x , y ) . Following Woodward ( 2005 ) , we define unstable features Z ( x ) as features that are differentially correlated with the label across the environments . We note that Z ( x ) is unknown to the model . For the target task , we only have access to the input-label pairs ( x , y ) ( i.e . no environments ) . We assume that the target label is not causally associated with the above unstable features Z . However , due to collection biases , the target data may contain spurious correlations between the label and Z . Our goal is to transfer the knowledge that Z is unstable in the source task , so that the target classifier will not rely on these spurious features . Overview If the unstable features have been identified for the target task , we can simply apply group DRO to learn a stable classifier . By grouping examples based on the unstable features and minimizing the worst-case risk over these manually-defined groups , we explicitly address the bias from these unstable features ( Hu et al. , 2018 ; Oren et al. , 2019 ; Sagawa et al. , 2020 ) . In our setup , while these unstable features are not accessible , we can leverage the source environments to derive groups over the target data that are informative of these biases . Applying group DRO on these automatically-derived groups , we can eliminate the unstable correlations in the target task . Our overall transfer paradigm is depicted in Figure 2 . It consists of two steps : inferring unstable features from the source task ( Section 3.1 ) and learning stable correlations for the target task ( Section 3.2 ) . First , for the source task we use a classifier trained on one environment to partition data from another environment based on the correctness of its predictions . Starting from the theoretical results in ( Bao et al. , 2021 ) , we show that these partitions reflect the similarity of the examples in terms of their unstable features : among examples with the same label value , those that share the same prediction outcome have more similar unstable features than those with different predictions ( Theorem 1 ) . We can then derive a representation fZ where examples are distributed based on the unstable features Z . Next , we cluster target examples into groups based on the learned unstable feature representation fZ . These automatically-derived groups correspond to different modes of the unstable features , and they act as proxies to the manually-defined groups in the oracle setting where unstable features are explicitly annotated . Finally , we use group DRO to obtain our robust target classifier by minimizing the worst-case risk over these groups .

Summary: The paper is concerned with learning a model in situations when some features have spurious correlations with the label (for example, for classifying sheep vs camel, background can be a spurious feature). The idea is that if a lot of source data is available, these spurious correlation features should be easy to find. To find the unstable features, authors hypothesize that they are related to mistakes that a classifier makes in different environments. Therefore, they learn a model on one environment and run it on a different environment (source data), splitting the data into correctly predicted and not. Then for each partition (correct or not) examples with the same labels are encouraged to be close together, so the embedding learns UNSTABLE features (fz). Then the target data is clustered based on unstable features representation fz, and then DRO (existing method that assumes the existance of correct groupping based on unstable features) is used to train a robust target classifier