Learning Stable Classifiers by Transferring Unstable Features
Authors: Yujia Bao
Shiyu Chang
Regina Barzilay
Venue: ICLR 2022 Submitted (2022)
Keywords: transfer learning
spurious correlation
invariant learning

Abstract: While unbiased machine learning models are essential for many applications, bias is a human-defined concept that can vary across tasks. Given only input-label pairs, algorithms may lack sufficient information to distinguish stable (causal) features from unstable (spurious) features. However, related tasks often share similar biases -- an observation we may leverage to develop stable classifiers in the transfer setting. In this work, we explicitly inform the target classifier about unstable features in the source tasks. Specifically, we derive a representation that encodes the unstable features by contrasting different data environments in the source task. We achieve robustness by clustering data of the target task according to this representation and minimizing the worst-case risk across these clusters. We evaluate our method on both text and image classifications. Empirical results demonstrate that our algorithm is able to maintain robustness on the target task, outperforming the best baseline by 22.9% in absolute accuracy across 12 transfer settings. Our code and data will be publicly available.

Under review as a conference paper at ICLR 2022
LEARNING STABLE CLASSIFIERS BY TRANSFERRING UNSTABLE FEATURES
Anonymous authors Paper under double-blind review
ABSTRACT
While unbiased machine learning models essential for many applications, bias is human-defined concept that can vary across tasks. Given only input-label pairs, algorithms may lack sufficient information to distinguish stable (causal) features from unstable (spurious) features. However, related tasks often share similar biases an observation we may leverage to develop stable classifiers in the transfer setting. In this work, we explicitly inform the target classifier about unstable features in the source tasks. Specifically, we derive representation that encodes the unstable features by contrasting different data environments in the source task. We achieve robustness by clustering data of the target task according to this representation and minimizing the worst-case risk across these clusters. We evaluate our method on both text and image classifications. Empirical results demonstrate that our algorithm is able maintain robustness on the target task for both synthetically generated enviornments and real-world environments. Our code will be publicly available.
INTRODUCTION
Automatic de-biasing (Sohoni et al. 2020 Creager et al. 2021 Sanh et al. 2021 has emerged as a promising direction for learning stable classifiers. The key premise here that no additional annotations for the bias attribute are required. However, bias is a human-defined concept and can vary from task to task. Provided with only input-label pairs, algorithms may not have sufficient information to distinguish stable (causal) features from unstable (spurious) features.
To address this challenge, we note that related tasks are often fraught with similar spurious correlations. For instance, when classifying animals such as camels vs. cows, their backgrounds (desert vs. grass) may constitute a spurious correlation (Beery et al. 2018). The same bias between the label and the background also persists in other related classification tasks (such sheep vs. antelope). In the resource-scarce target task, we only have access to the input-label pairs. However, in the source tasks, where training data is sufficient, identifying biases may be easier. For instance, may have examples collected from multiple environments, in which correlations between bias features and the label are different (Arjovsky al. 2019). These source environments help define the exact bias features that we want to regulate.
One obvious approach to utilize the source task is direct transfer. Specifically, given multiple source environments, we can train an unbiased source classifier and then apply its representation to the target task. However, we empirically demonstrate that while the source classifier not biased when making its final predictions. its internal continuous representation can still encode information about the unstable features. Figure [ shows that in Colored MNIST, where the digit label is spuriously correlated with the image color, direct transfer by either re-using or fine-tuning the representation learned on the source task fails in the target task, performing no better than the majority baseline.
In this paper, we propose to explicitly inform the target classifier about unstable features from the source data. Specifically, we derive a representation that encodes these unstable features using the source environments. Then we identify distinct subpopulations by clustering examples based on this representation and apply group DRO (Sagawa et al. 2019) to minimize the worst-case risk over these subpopulations. As result, we enforce the target classifier to be robust against different values of the unstable features. In the example above, animals would be clustered according to backgrounds, and the classifier should perform well regardless of the clusters (backgrounds).

---

Under review as a conference paper at ICLR 2022
Source tosk; 0 vs
Target taok: vs
TEM
2 20222
2 2
2 2 3
2 2 3 3 3
2 2
DOUSE
2 22272
2 7 3
3 3 3
3
1
3
3 3 3 3
3 Spurious correlation Rips in the testing data.
3
3
Correlations between the spurious color feature and the lahel are different across the two training environments.
Only data environment is available for training.
Figure 1: Transferring across tasks in Colored MNIST (Arjovsky et al. 2019). On the source task, we learn a color-invariant model that achieves oracle performance (given direct access to the unstable features). However, directly transferring this model to the target task, by reusing or fine-tuning its feature extractor, severely overfits the spurious correlation and underperforms the majority baseline (50%) test set where the spurious correlation flips. By explicitly transferring the unstable features, our algorithm TOFU (Transfer OF Unstable features) is able to reach the oracle performance.
The remaining question is how to compute the unstable feature representation using the source data environments. Following [ao et al. (202T), we hypothesize that unstable features are reflected in mistakes observed during classifier transfer across environments. For instance, if the classifier uses the background to distinguish camels from cows, the camel images that are predicted correctly would have a desert background while those predicted incorrectly are likely to have grass background. More generally, we prove that among examples with the same label value, those with the same prediction outcome will have more similar unstable features than those with different predictions. By forcing examples with the same prediction outcome to stay closer in the feature space, we obtain a representation that encodes these latent unstable features.
We evaluate our approach, Transfer OF Unstable features (TOFU), on both synthetic and real- world environments. Our synthetic experiments first confirm our hypothesis that standard transfer approaches fail to learn a stable classifier for the target task. By explicitly transferring the unstable features, our method significantly improves over the best baseline across 12 transfer settings (22.9% in accuracy), and reaches the performance of an oracle model that has direct access to the unstable features (0.3% gap). Next, we consider a practical setting where environments are defined by an input attribute and our goal is to reduce biases from other unknown attributes. On CelebA, TOFU achieves the best worst-group accuracy across 38 latent attributes, outperforming the best baseline by 18. 8.06%. Qualitative and quantitative analyses confirm that TOFU is able to identify the unstable features.
RELATED WORK
Removing bias via annotations: Due to idiosyncrasies of the data collection process, annotations
are often coupled with unwanted biases (Buolamwini and Gebru 2018; Schuster et al., 2019: McCoy
cct al. 2019 Yang et al. 2019). To address this issue and learn robust models, researchers leverage
extra information (Belinkov et al. 2019 Stacey et al. 2020 Hinton 2002 Clark et al. 2019 He jet al.
2019 Mahabadi et al. (020)) One line of work assumes that the bias attributes are known and
have been annotated for each example, e.g., group distributionally robust optimization (DRO) (Hu
et al. 2018 Oren et al. 2019 Sagawa et al.) 2020). By defining groups based on these bias attributes, we explicitly specify the distribution family to optimize over. However, identifying the hidden biases is time-consuming and often requires domain knowledge (Zellers et al. 2019) /aaaauchi et al. 2020).
To address this issue, another line of work (Peters al. [2066] Krueger al.] [220 [0a0| al.
2020: Jin et al. 2020 AAhuja et al. 2020 Arjovsky et al.] 2019 Bao et al.) [022]] Kuang et al. [202
Shen et all 2020) only assumes access to a set of data environments. These environments are defined based on readily-availabl information of the data collection circumstances, such as location and
time. The main assumption is that while spurious correlations vary across different environments, the association between the causal features and the label should stay the same. Thus, by learning representation that is invariant across all environments, they alleviate the dependency on spurious features. In contrast to previous works, we don't 't have access any additional information besides the labels in our target task. We show that we can achieve robustness by transferring the unstable features from related source task.
2

---

Under review as a conference paper at ICLR 2022
Automatic de-biasing A number of recent approaches focus on more common setting where the
algorithm only has access to the input-label pairs. Sanh et al. (202T); ); Nam et al. (2020); Utama et al.
(2020) find that weak models are more vulnerable to spurious correlations
they only learn shallow
heuristics. By boosting from their mistakes, they obtain a more robust model. Qiao et al. (2020) uses
adversarial learning to augment the biased training data. [Creager et al.
(2021 [Sohoni
(2020);
Ahmed et al. (2020); Matsuura and Harada [2020] Liu et al. (2021) propose to identify minority groups by looking at the features produced by a biased model.
These automatic approaches are intriguing as they do not require additional annotation. However, we note that bias is human-centric concept and can vary from tasks to tasks. For models that only have access to the input-label pairs, they have no information to distinguish causal features from bias features. For example, consider the Colored MNIST dataset, where color and digit shape are correlated in the training set but not in the test set. If our task is to predict the digit, then color becomes the spurious bias that we want to remove. Vice versa, we want to predict the color, then digit shape is spurious. Creager et al. ): Nam et al. (2020) empirically demonstrate that they can learn a color-invarian model for the digit prediction task. However, their approaches will result in the same color-ims invariant model for the color prediction task, and thus fail at test time, when color and digit are no longer correlated. In this work, we leverage source tasks to define the exact bias that we want remove for the target task.
Transferring robustness across tasks: Prior work has also studied the transferability of adversarial robustness across tasks. For example, Hendrycks et al. (2019); Shafahi et al. (2020) show that by pre-training the model on large-scale source task, we can improve the model robustness against adversarial perturbations over 1. norm. We note that these perturbations measure the smoothness of the classifier, rather than the stability of the classifier against spurious correlations. In fact, our results show that if we directly re-use or fine-tune the pre-trained feature extractor on the target task, the model will quickly over-fit to the unstable correlations present in the data. We propose to address this issue by explicitly inferring the unstable features using the source environments and use this information to guide the target classifier during training.
3 METHOD
Problem formulation We consider the transfer problem from source task to a target task. For the source task, we assume the standard setting (Arjovsky et al.) 2019) where the training data contain environments Ei...., En- Within each environment E,, examples are drawn from the joint distribution Pa y). Following Woodward (2005), we define unstable features Z(x) as features that are differentially correlated with the label across the environments. We note that Z(x) unknown to the model.
For the target task, we only have access to the input-label pairs (z, y) (i.c. no environments). We assume that the target label is not causally associated with the above unstable features Z. However, due to collection biases, the target data may contain spurious correlations between the label and Z. Our goal is to transfer the knowledge that 2 is unstable in the source task, so that the target classifier will not rely on these spurious features.
Overview If the unstable features have been identified for the target task, we can simply apply group DRO to learn stable classifier. By grouping examples based on the unstable features and minimizing the worst-case risk over these manually-o ddefind groups, we explicitly address the bias from these unstable features (Hu et al. 2018: Oren et al.) 2019 Sagawa et al.) 2020). In our setup, while these unstable features are not accessible, we can leverage the source environments to derive groups over the target data that are informative of these biases. Applying group DRO on these iitomatically-derivve groups, we can climinate the unstable correlations in the target task.
Our overall transfer paradigm is depicted in Figure 2 It consists of two steps: inferring unstable features from the source task (Section 3.1) and learning stable correlations for the target task (Section First, for the source task we use a classifier trained on one environment to partition data from another environment based on the correctness of its predictions. Starting from the theoretical results in (Bao et al. (021), we show that these partitions reflect the similarity of the examples in terms of their unstable features: among examples with the same label value, those that share the same prediction
3.2
3

---

Under review as a conference paper at ICLR 2022
Inferring unstable features from the lusk
souree task label.
unstable fenture: red blue
fz
533
red
f1(x)
blue
Contrasting source enviromments Ei Es
Deriving unstable feature representution examples
S.I Train an environment-go specific classifier on E.,
S.3 For each label value from the sume partition to be close with euch other.
S.2 Partition E, based on the prediction correctness of fs.
Learning stable correlations target
target task label:
unstable fenture: blue
miny max
fz
fz
T.2
T.I cach label value cluster examples based the unstable feature representation fz.
risk the model is robost against differeut unstable feature values.
Minimiie
Figure 2: Our algorithm TOFU 1) infers unstable features from the source task (Section 3.1 and 2) learns stable correlations for the target task (Section 3.2). We create partitions for environment pairs. For ease of illustration, we only depict using fi to partition E2. Best viewed in color.
outcome have more similar unstable features than those with different predictions (Theorem D. We then derive representation where examples distributed based the unstable features Z. Next, we cluster target examples into groups based on the learned unstable feature representation fz. These caaomaticallydderivee groups correspond different modes of the unstable features, and they act as proxies to the manually-defined groups in the oracle setting where unstable features are explicitly annotated. Finally, we use group DRO obtain our robust target classifier by minimizing the worst-case risk over these groups
3.1 INFERRING UNSTABLE FEATURES FROM THE SOURCE TASK
Given the data environments from the source task, we would like to 1) identify the unstable correla- tions across these environments; 2) learn a representation fz (x) that encodes the unstable features Z(x). We achieve the first goal by contrasting the empirical distribution of different environments (Figure S. and Figure S.22) and the second goal by metric learning [figure[[S.3)
Let E, and Ej be two different data environments. ]ao et al. (202T) shows that by training a classifier fi on E, and using it to make predictions on Ej. we can reveal the unstable correlations from its prediction results. Intuitively, the unstable correlations are stronger E,, the classifier fi will overuse these correlations and make mistakes Ej when these stronger correlations do not hold, [
In this work, we connect the prediction results directly to the unstable features. We show that the prediction results of the classifier f. on E, estimate the relative distance of the unstable features.
Theorem (Simplified). Consider examples in Es with label value y. Let X X denote two batches of examples that fs predicted correctly, and let X denote a batch of incorrect predictions. We use represent the mean across given batch. Following the same assumption in (Bao et al. 2027), we have
Z(XY) Z(X <
-
We assume that the distributions of F, and Fg are different enough such that will make mistakes.
4

---

Under review as a conference paper at ICLR 2022
almost surely for large enough batch size
The result makes intuitive sense we would expect example pairs that share the same prediction outcome should be more similar than those with different prediction outcomes. We note that it is critical to look examples with the same label value: otherwise, the unstable features will be coupled with the task-specifer label in the prediction results.
While the value of the unstable features ((x) still not directly accessible, Theorem I enables us to learn a feature representation fz (z) that preserves the distance between the examples in terms of their unstable features. We adopt standard metric learning (Chechik et al. 2010) minimize the following triplet loss:
Cx X x max(0, (
(1)
where is hyper-paramettr. By minimizing Eq equation 0 we encourage examples that have similar unstable features be close in the representation fz. To summarize, inferring unstable features from the source task consists of three steps (Figure[[S)
S.1 For each source environment E,, train environment--peecfiic classifier Si.
S.2 For each pair of environments E; and Ejj, use classifier fi to partition Fyy into two sets: E and E where EY contains examples that Si predicted correctly and E contains those predicted incorrectly.
S.3 Learn an unstable feature representation fz by minimizing Eq equation across all pairs of environments E,, Es and all possible label value
Jz = arg min
E Exyxx E.E
Σ
(x2(X Xx,
where batches Xi, Xy are sampled uniformly from E ly and batch X2 is sampled uniformly from 15 la (fly denotes the subset of with label value y).
3.2 LEARNING STABLE CORRELATIONS FOR THE TARGET TASK
Given the unstable feature representation fz, our goal is learn target classifier that focuses on the stable correlations rather than using unstable features. Inspired by group DRO (Sagawa et al. 2020) we minimize the worst-case risk across groups of examples that are representative different unstable feature values. However, in contrast to DRO, these groups are constructed automatically based on the previously learned representation fzz
For each target label value different clusters (Figure T..)). Since these clusters capture different modes of the unstable features, they are approximations of the typical manually-definee groups when annotations of the unstable features are available. By minimizing the worst-case risk across all clusters, we explicitly enforce the classifier be robust against unstable correlations (Figure cluster within examples of the same label, as opposed to clustering the whole dataset. Otherwise, the cluster assignment may be correlated with the target label.
use the representation Íz to cluster target examples with label y into
We note that it is important to
Concretely, learning stable correlations for the target task has two steps (Figure T).
T.1 For each label value y, apply K-means (12 distance) to cluster examples with label in the feature space fzz We use C ...... Cty denote the resulting cluster assignment, where n. is hyper-parameter.
T.2 Train the target classifier / by minimizing the worst-case risk over all clusters:
arg min max L(CP),
b.
where ((C))) is the empirical risk on cluster CV.
5

---

Under review as a conference paper at ICLR 2022
Table 1: Pearson correlation coefficient between the spurious feature Z and the label Y for each task. The validation environment E follows the same distribution as E We study the transfer problem between different task pairs. For the source task S. the model can access Eqainn (S), E and E (S). For the target task T, the model can access ET (T) and E ET (T)).
Em (S)
p Y)
MNIST
BEER REVIEW
ASK2ME
WATERBIRD
ODD
EVEN
LOOK
AROMA
PALATE
PENE
INCI.
WATER
SEA.
E
0.87
0.87
0.60
0.60
0.60
0.31
0.44
0.36
0.39
0.75
0.75
0.80
0.80
0.80
0.52
0.66
0.63
0.64
E
0.87
0.87
0.60
0.60
0.60
0.31
0.44
0.36
0.39
E
-0.11
-0.11
0.80
-0.80
0.80
0.00
0.00
0.00
0.00
4 EXPERIMENTAL SETUP
4.1 DATASETS AND SETTINGS
Synthetic environments We start with controlled experiments where environments are created based on the spurious correlation. We consider four datasets: MNIST (LeCCn et al., 1998). Beer- Review (McAuley et al., 2012), ASK2ME (Bao et al., (019a)) and Waterbird (Sagawa et al. 2019). In MNIST and BeerReview, we inject spurious feature to the input (background color for MNIST
and pseudo token for BeerReview). In ASK2ME and Waterbird, spurious feature corresponds to an attribute of the input (breast.cance. for ASK2ME and background for Waterbird).
For each dataset, we consider multiple tasks and study the transfer between these tasks. Specifically, for each task, we split its data into four environments: Exrain EEtain E'val, E where spurious correlations vary across the two training environments F.urain, Fegrain For the source task S, the model access both of its training environments Exriin (S), Figmin For the target task T, the model only has access to one training environment (T') We note that the validation set (T)) plays an important role in early-stopping and syper-parameter tuning, especially when the distribution of the data is different between training and testing (Gulrajani and Lopez- Paz (020). In this work, since we don't 't have access to multiple training environments on the target task, we assume that the validation data Eval follows the same distribution the training data Eqain, Table I summarizes the level of the spurious correlations for different tasks. Additional details can be found in Appendix C.
We also consider practical setting where environments are directly defined by a given attribute of the input, and our goal is to reduce model biases from other latent attributes. We study CelebA (Liu et al. 2015a) where each input (an image of a human face) is annotated with 40 binary attributes. The source task is to predict the Eyeglasse attribute and the target task attribute. We use the Young attribute to define two environments: E = {Young = 0} and Fgg = {Young = 1}. In the source task, both environments are available. In the target task, we only have access to environment E1 during training and validation. At test time, we evaluate the robustness of our target classifier against other latent attributes. Specifically, for each unknown attribute such as Male, we partition the testing data into four groups: (Male 1, BlondHair = 0}. [Male = 0, BlondHaii = 0}. (Male = 1, Bl BlondHair = 1}, (Male = 0, BlondHair r = 1]. Following Sagawa et al. 2019 We report the worst-group accuracy and the average-group accuracy.
Natural environments
is to predict the B Ha
4.2 BASELINES
We compare our algorithm against the following baselines. For fair comparison, all methods share the same representation backbone and hyper-paramest search space.
ERM baseline We learn a classifier on the target task from scratch by minimizing the average loss across all examples. Note that this classifier is independent of the source task. Its performance reflects the deviation between the training distribution and the testing distribution of the target task.
"See Appendix B for the full theorem and proof.
'Ill data splits, param-err search spaces are available in the supplementary materials.

---

Under review as a conference paper at ICLR 2022
Table 2: Target task accuracy of different methods. All methods are tuned based on held-out validation set that follows from the same distribution as the target training data. Bottom right: standard deviation across runs. Upper right: source task testing performance (if applicable).
SOURCE
TARGET
ERM
REUSEn
FINETUNES
MULTITASK
TOFU
ORACLE
MNIST
ODD
EVEN
12.3
(7.4 (70.9)
$1.0
11.2
11.6
(69.6)
69.1+1.6
68.7+0.9
BVEN
ODD
9.7
19.2
11.5
10.7
(70
66.8
67.8
LOOK
AROMA
55.5
31.9 (70.))
+1.0
53.7 (70.1)
54.1
(76.0)
4-2.2
75.9
77.3+1.3
BEER REVIEW
LOOK
PALATE
46.9
22.8
42.3
(2..))
73.8
74.0+1.2
AROMA
LOOK
63.9
40.1
13.1
(68.6)
65.2 (08.4)
64.0 (71.5)
80.9
80.1
AROMA
PALATE
46.9.0.3
(4.0
(7.9
50.0
73.5 1.1
74.0
PALATE
LOOK
63.9
40.4
(67.2)
42.88
64.3 42.7
63.1 (7.11
(75.9)
81.0
80.1
PALATE
AROMA
51.5
23.1 13.3
54.5 (58.7)
(T..)
7..
77.3+1.3
ASK.
PENE
INCL.
+9.3
71.7
(72.7)
+0.5
79.3
(71.2)
$0.8
71.1 71.1
1..4
(73.5)
83,2+118
84,8+1.2
INCL
PENE.
71.6
64. 41.5
(2.0
61.9 (1.9)
78.1
78.3
BIRD
WATER
SEA.
81.8+4.3
87.8 87.8
(19.5)
82.0 82.0
(19.3)
44.0
88. 88.0
(99.5)
20.9
93.1
93.7+0.7
SEA.
WATER
75.1
94.6
(8.1))3-1)
(3.5()7.7)
99.0#0.4
98.9
Average
55.2
43.7
55.8
56.4
79.3
79.6
Transfer methods Since the source task contains multiple environments, we can learn a stable model on the source task and transfer it to the target task. We use four algorithms to learn the source task: DANN (Ganin et al. 2016). C-DANN (Li et al.) 2018b), MMD (Li al. 2018a). PI (Bao et al.) We consider three standard methods for transferring the source knowledge:
2021.
REUSE: We directly transfer the feature extractor of the source model to the target task. The feature extractor is fixed when learning the target classifier.
FINETUNE: We update the feature extractor when training the target classifier. (Shafahi et al. 2020 has shown that FINETUNE may improve adversarial robustness of the target task.
MULTITASK: We adopt the standard multi-task learning approach (Caruana 1997) where the source model and the target model share the same feature extractor and are jointly trained together.
Automatic de-biasing methods For the target task, we can also apply de-biasing approaches that do not require environments. We consider the following baselines:
EIIL (Creager et al. 2021 y Based on a pre-trained ERM classifier's prediction logits, we infer an environment assignment that maximally violates the invariant learning principle (Arjovsky al.) 2019) We then apply group DRO to minimize the worst-case loss over all inferred environments.
GEORGE (Sohoni et al. 2020): We use the feature representation of pre-trained ERM classifier to cluster the training data. We then apply group DRO to minimize the worst-case loss over all clusters.
LFF (Nam al. 2020): We train biased classifier together with a de-biased classifier. The biased classifier amplifies its bias by minimizing the generalized cross entropy loss. The de-biased classifier then up-weights examples that are mis-labeled by the biased classifier during training.
M-ADA QQia et al. 2020): : We use a Wasserstein auto-encoder to generate adversarial examples. The de-biased classifier is trained on both the original examples and the generated examples.
DG-MMLD Matsuura and Harada 2020); We iteratively divide target examples into latent domains via clustering, and train the comain-invariinn feature extractor via adversarial learning.
ORACLE For synthetic environments, we can use the spurious feature to define groups and train an oracle model. For example, in task SEABIRD, this oracle model will minimize the worst-case
7

---

Under review as a conference paper at ICLR 2022
Table 3: Worst-group and average-group accuracy on CelebA. The source task is predict Eyeglassed and the target task is to predict BlondHair. We use the attribute Young to define two environments: E1 (Young 0). Eg fYoung 1). Both environments are available in the source task. In the target task, we only have access to Ei during training and validation.. We show the results the first 3 attributes (alphabetical order). The right-most Average" column is computed based on the performance across all 38 attributes. See Appendix for full results.
ArchedEyebrows
Attractive
BagsUnderByes
Average"
METHOD
Worst
Avg.
Worst
Avg.
Worst
Avg.
Worst
Avg.
ERM
75.43
88.52
75.00
88.94
70.91
87.09
61.01
85.07
REUSE
53.71
64.05
52.13
64.85
52.50
66.88
47.58
64.14
REUSEDANN
59,56
72.44
62.03
72.26
64.58
73.83
55.27
72.31
REUSEC-DOM
56.02
67.07
57.78
67.90
57.50
68.33
53.22
68.56
TRANSFER
REUSEMMU
48.91
59.80
48.46
61.51
58.74
63.11
50.61
61.27
FINETUNER
71.86
87.02
72.73
87.34
62.50
84.10
63.07
85.27
FINETUNEBANN
65.38
83.89
63.35
84.98
56.86
81.34
50.60
80.49
FINETUNEC.DANN
73.85
88.90
75.61
89.39
75.86
88.14
62.03
85.57
FINETUNESNN
76.07
88.80
74.33
89.74
78.57
88.61
66.80
86.81
MULTITASK
69.66
86.91
72.73
87.44
70.00
85.21
64.37
85.21
EIIL
64.71
85.12
64.43
85,96
66.67
83.90
57.62
83.22
AUTO-DEBIAS
GEORGE
74.73
87.89
73.66
87.70
77.78
87.97
63.34
85.04
LFF
45.41
60.23
47.67
60.16
42.59
60.72
42.52
62.04
M-ADA
64.61
83.33
67.33
83.59
70.34
85.34
54.55
80.77
DG-MMLD
69.51
87.38
68.42
87.50
63.41
84.78
$5.69
83.51
TOFU
85.66
91.47
88.30
92.76
90.38
92.41
84.86
91.71
blue purple brown
Diambation Male over examples with BlondHalz-00
TOFU
Cluzer V?
EEL
Einttrommnn 8j
25%
24%
71%
75%
Enstroement &
15%
Milu-1
85%
Figure 3: PCA visualization of the stable feature representation fz for ex- amples in MNIST EVEN. fz trained on MNIST ODD. TOFU identifies the hidden spurious color feature by con- trasting different source environments.
Figure 4: Visualization of the unknown attribute Male on CelebA. Left: distributions Male in the training data. Mid: partitions learned by TOFU. Right: partitions learned by EILL TOFU generates partitions that are more informative of the unknown attribute (14% vs. 2%). See Appendix F for results on more attributes.
risk over the following four groups: (seabird in water), (seabird in land] {landbi rdd in water). {landbirg ird in land]. This oracle model helps analyze the performance of our proposed algorithm separately from the inherent limitations (such as model capacity and data size).
RESULTS
Table summariizs our results on synthetic environments. We observe that standard transfer methods fail to improve the ERM baseline. On the other hand, TOFU consistently achieves the best
8

---

Under review as a conference paper at ICLR 2022
Table Quantitative evaluation of the generated clusters against the ground truth unstable features. For comparison, the TRIPLET baseline directly encourages source examples with the same label to stay close to each other in the feature space, from which we generate the clusters. For both methods, we generate two clusters for each target label value and report the average performance across all label values. We observe that the TRIPLET representation, while biased by the spurious correlations, fails to recover the ground truth unstable features for some tasks. By explicitly contrasting the source environments, TOFU derives clusters that are
of the unstable features.
Homogeneity
Completeness
measure
SOURCE
TARGET
TRIPLET
TOFU
TRIPLET
TOFU
TRIPLET
TOFU
MNIST
ODD
EVEN
0.4286
0.6820
0.5811
0.9561
0.4906
0.7960
EVEN
ODD
0.6709
0.6764
0.9362
0.9988
0.7814
0.8055
LOOK
AROMA
0,3321
0.9256
0,2801
0.9218
0,3039
0.9236
BEER REVIEW
LOOK
PALATE
0.3317
0.9095
0,2796
0.8988
0.3034
0,9039
AROMA
LOOK
0.3330
1.0000
0.2811
1.0000
0.3049
1.0000
AROMA
PALATE
0.8252
1.0000
0.7740
1.0000
0.7974
1.0000
PALATE
LOOK 0.8306
0.9816
0.7794
0.9811
0.8028
0.9813
PALATE
AROMA
0.8272
0.9585
0.7764
0.9561
0.7996
0.9573
performance across 12 transfer settings, outperforming the best baseline by 22.9%. While TOFU doesn't have access to the unstable features, by inferring them from the source environments, matches the oracle performance with only 0.30% absolute difference.
Table 3 presents our results on natural environments. This task is very challenging as there are multiple latent spurious attributes in the training data. We observe that most automatic de-biasing methods underperform the ERM baseline. With the help of the source task, FINETUNE and MULTITASK achieve slightly better performance than ERM. TOFU continues to shine in this real-world setting: achieving the best worst-group and average-group performance.
TOFU able to identify the unstable features? Yes. For synthetic environments, we visualize the unstable feature representation produced by fz on MNIST EVEN. Figure [ demonsttatee that while fz only sees source examples (ODD) during training. it can distribute target examples based on their unstable color features. For natural environments, we visualize the distribution of two latent attributes (Male and archedEEybroos over the generated clusters. Figure 4 shows that the distribution gap of the unknown attribute Male across the generated partitions is 2% for EIIL, only marginally better than random splitting (0%). By leveraging information from the source task, TOFU learns partitions that are more informative of the unknown attribute (14%).
How do the generated clusters compare to the oracle groups? We quantitatively evaluate the generated clusters based on three metrics: homogeneity (each cluster contain only examples with the same unstable feature value), completeness (examples with the same unstable feature value belong to the same cluster), and V-measure (the harmonic mean of homogeneity and completeness). From T we see that TOFU is able to derive clusters that resemble the oracle groups on BEER REVIEW. In MNIST, since we generate two clusters for each label value and there are five different colors, is impossible to recover the oracle groups. However, TOFU still achieves almost perfect completeness.
6 CONCLUSION
Reducing model bias is a critical problem for many machine learning applications in the real world. In this paper, we recognize that bias a human-defined concept. Without additional knowledge, automatic de-biasing methods cannot effectively distinguish causal features from spurious features. The main departure of this paper is to identify bias by using related tasks. We demonstrate that when the source task and target task share the same set of biases, we can effectively transfer this knowledge and improve the robustness of the target model without additional human intervention. Compared with 15 baselines across 5 datasets, our approach consistently delivers significant performance gain. Quantitative and qualitative analyses confirm that our method is able to identify the hidden biases. Due to space limitations, we leave further discussions Appendix A

---

Under review as a conference paper at ICLR 2022
REFERENCES
F. Ahmed, Y. Bengio, H. van Seijen, and A. Courville. Systematic generalisation with group invariant predictions. In International Conference on Learning Representations, 2020.
K. Ahuja, K. Shanmugam, K. Varshney, and A. Dhurandhar. Invariant risk minimization games, In International Conference on Machine Learning, pages 145-155. PMLR, 2020.
M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: 1907.02893, 2019.
Y. Bao, Z. Deng. Y. Wang, H. Kim, V. D. Armengol, F. Acevedo, N. Ouardaoui, C. Wang,, G. Parmigiani, R. Barzilay, D. Braun, and K. S. Hughes. Using machine learning and natu- ral language processing review and classify the medical literature cancer susceptibility genes. JCO Clinical Cancer Informatics, (3):1-9, 2019a. doi: 10. 200////.....00000 URL https://dot org/10.......o.........o........ 1200/001111 00042
PMID: 31545655.
Y. Bao, M. Wu, S. Chang, and R. Barzilay. Few-shot text classification with distributional signatures. In International Conference on Learning Representations, 2019b.
Y. Bao, S. Chang, and R. Barzilay. Predict then interpolate: A simple algorithm to learn stable classifiers. International Conference on Muchine Learming (ICML), 2021.
S. Beery, G. Van Horn, and P. Perona. Recognition in terra incognita. In Proceedings of the European Conference on Computer Vision (ECCV), pages 456-473, 2018.
Y. Belinkov, A. Poliak, S. M. Shieber, B. Van Durme, and A. M. Rush. Don't take the premise for granted: Mitigating artifacts in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 877-891, 2019.
J. Buolamwini and T. Gebru, Gender shades: Intersectional accuracy disparities commercial gender classification. In Conference on fairness, accountability and transparency, pages 77-91, 2018.
R. Caruana. Multitask learning. Machine learning. 28(1):41-75, 1997.
S. Chang, Y. Zhang, M. Yu, and T. S. Jaakkola. Invariant rationalization. arXiv preprint arXiv:2003.09722 09772, 2020.
G. Chechik, V. Sharma, U. Shalit, and S. Bengio. Large scale online learning of image similarity through ranking. 2010.
C. Clark, M. Yatskar, and L. Zettlemoyer. Don't take the easy way Ensemble based methods for avoiding known dataset biases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4060-4073, 2019.
E. Creager, J.-H. Jacobsen, and R. Zemel. Environment inference for invariant learning. In Interna- tional Conference on Machine Learning, pages 2189-2200. PMLR, 2021.
de Vries, Misra, Wang, and L. van der Maaten. Does object recognition work for everyone? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 52 59, 2019.
Z. Deng, K. Yin, Y. Bao, V. D. Armengol, C. Wang, A. Tiwari, R. Barzilay, G. Parmigiani, D. Braun, and K. S. Hughes. Validation of a semiautomated natural language processing- procedure for meta-analysis of cancer susceptibility gene penetrance. JCO clinical cancer informatics, 3:1-9, 2019.
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky. aavess-adversariall training neural networks. The journal of machine learning research, 1701)2009-2033, 2016.
10

---

Under review as a conference paper at ICLR 2022
A. Gaut, T. Sun, S. Tang, Y. Huang. J. Qian, M. ElSherief, J. Zhao, D. Mirza, E. Belding, K.-W. Chang, et al. Towards understanding gender bias in relation extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pages 2943-22222222 2443-953,, 2020.
I. Gulrajani and D. Lopez-Paz. In search of lost domain generalization. arXiv preprint arXiv:2007.01433 2020.
H. He, S. Zha, and H. Wang. Unlearn dataset bias in natural language inference by fitting the residual. EMNLP-IJCNLP 20/9, page 132, 2019.
D. Hendrycks, K. Lee, and M. Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning, pages 2712-2721. PMLR, 2019.
G. Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):177-18800 2002.
W. Hu, G. Niu, I. Sato, and M. Sugiyama. Does distributionally robust supervised learning give robust classifiers? International Conference on Machine Learning, pages 2029-2037. PMLR, 2018.
S. Jia, T. Meng, J. Zhao, and K.-W. Chang. Mitigating gender bias amplification in distribution by posterior regularization. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pages
2020.
W. Jin, R. Barzilay, and T. Jaakkola. Enforcing predictive invariance across structured biomedical domains, 2020.
Y. Kim. Convolutional neural networks for sentence classification. Proceedings of the 2014 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 1746-1751, Doha, Qatar, Oct. 2014. Association for Computational Linguistics. doi: 10.31 10.3115/v1/D14- 1181. URL htttst:::: www. a c web. . wwtpg//annolll//...eee..oo
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.690. 2014.
Kiritchenko and S. Mohammad. Examining gender and race bias in two hundred sentiment analysis systems. In Proceedings of the Seventh Joint Conference Lexical and Computational Semantics, pages 43-53, 2018.
D. Krueger, E. Caballero, J.-H. Jacobsen, A. Zhang. J. Binas, D. Zhang, R. L. Priol, and A. Courville. Out-of-distributioo generalization via risk extrapolation (rex), 2020.
K. Kuang, R. Xiong. P. Cui, S. Athey, and B. Li. Stable prediction with model misspecification and agnostic distribution shift. Proceedings of the AAA! Conference on Artificial Intelligence, volume 34, pages 4485-4492. 2020.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, (6(11))2278-233 1998.
T. Lei, R. Barzilay, and T. Jaakkola. Rationalizing neural predictions. In Proceedings of the 2016 Conference Empirical Methods in Natural Language Processing, pages 107-117, 2016.
H. Li, S. J. Pan, S. Wang, and A. C. Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5400-5409, 2018a
Y. Li, M. Gong, X. Tian, T. Liu, and D. Tao. Domain generalization via conditional invariant representations. Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018b.
J. Liu, Z. Hu, P. Cui, B. Li, and Z. Shen. Heterogeneous risk minimization. arXiv preprint arXiv:2105.038/1 2021.
Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference computer vision, pages 3730-3738, 2015a.
11

---

Under review as a conference paper at ICLR 2022
Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015b.
R. K. Mahabadi, Y. Belinkov, and J. Henderson. End-to-end bias mitigation by modelling biases in corpora. ACL, 2020.
T. Matsuura and T. Harada. Domain generalization using mixture of multiple latent domains. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11749-11756. 2020.
J. McAuley, J. Leskovec, and D. Jurafsky. Learning attitudes and attributes from multi aspect reviews. In 2012 IEEE 12th International Conference on Data Mining, pages 1020-1025. IEEE, 2012.
T. McCoy, E. Pavlick, and T. Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Anmual Meeting of the Association for Computational Linguistics, pages 3428-3448, Florence, Italy, July 2019. Association for Computational Linguisties. doi: 10.18653/v1/119- 1334. URL (RR.httts: www www.b..lwwbbrrgg.
anthol oo11119--ypp1-11/ 1334
T. Mikolov, E. Grave, P. Bojanowski, C. Puhrsch, and A. Joulin. Advances in pre-training distributed word representations. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018), 2018.
J. Nam, H. Cha, S. Ahn, J. Lee, and J. Shin. Learning from failure: Training debiased classifier from biased classifier. arXiv preprint arXiv:2007.0233 2000.0266, 2020.
Y. Oren, S. Sagawa, T. Hashimoto, and P. Liang. Distributionally robust language modeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 0218-4228, 2019.
J. H. Park, J. Shin, and P. Fung. Reducing gender bias in abusive language detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799-2804, 2018.
J. Peters, P. Bühlmann, and N. Meinshausen. Causal inference by using invariant prediction: iden- tification and confidence intervals. Journal of the Royal Statistical Society. Series (Statistical Methodology), pages 947-1012, 2016.
F. Qiao, L. Zhao, and X. Peng. Learning to learn single domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12556-12565. 2020.
S. Sagawa, P. W. Koh, T. B. Hashimoto, and Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint 1rX1v:1911.08731, 2019.
S. Sagawa, P. W. Koh, B. Hashimoto, and Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL htttpss://operrreiiee
net//ooumm (--rruuuJJ
K. Sakaguchi, R. Le Bras, C. Bhagavatula, and Y. Choi. Winogrande: An adversarial winograd schema challenge at scale. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8732-8740, 2020.
V. Sanh, Wolf, Y. Belinkov, and A. M. Rush. Learning from others' mistakes: Avoiding dataset biases without modeling them. In International Conference on Learning Representations, 2021. URL
net httpaa/wweeeeeeeeeeooooennoeadaa
T. Schuster, D. Shah, J. Yeo, D. Roberto Filizzola Ortiz, E. Santus, and R. Barzilay. Towards debiasing fact verification models. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing EEMNLP-JCNLP), pages 3410-3416, Hong Kong, China, Nov. 2019. Association for Computational Linguistics. doi: 1/D116 1341. URL https: / www www.wweborr.. anthol Ttchology//D1-1344
12

---

Under review as a conference paper at ICLR 2022
A. Shafahi, P. Saadatpanah, C. Zhu, A. Ghiasi, C. Studer, D. Jacobs, and T. Goldstein. Adversarially robust transfer learning. International Conference on Learning Representations, 2020. URL net/ wttp://orummiiiiee Eorumt1==zyebbD4Y...
btttps://cpenreet
Z. Shen, P. Cui, Liu, T. Zhang, B. Li, and Z. Chen. Stable learning via differentiated variable decorrelation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining. pages 2185-2193, 2020.
N. S. Sohoni, A. Dunnmon, G. Angus, A. Gu, and C. Ré. No subelass left behind: Fine-grained robustness in coarse-grained classification problems. arXiv preprini arXiv:2011. 1.1.:95,1.1 2020.
J. Stacey, P. Minervini, H. Dubossarsky, S. Riedel, and T. Rocktäschel. Avoiding the Hypothesis- Onlly Bias in Natural Language Inference via Ensemble Adversarial Training. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 8281-8291, Online, Nov. 2020. Association for Computational Linguistics. doi: 10. 118633v1/20022 20005/11//2200mmmmmmmmmmim 665. URL httpt:tt///mmm www. acl web. wwg wawpooooyyy///// emn 1p ma in. 665
A. Utama, N. S. Moosavi, and I. Gurevych. Towards debiasing nlu models from unknown biases. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7597-7610, 2020.
Welinder, Branson, Mita, C. Wah, Schroff, S. Belongie, and P. Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001 California Institute of Technology, 2010.
J. Woodward. Making things happen: A theory of causal explanation. Oxford university press, 2005.
K. Yang, K. Swanson, W. Jin, Coley, P. Eiden, H. Gao, A. Guzman Perezz, Hopper, B. Kelley, M. Mathea, et al. Analyzing learned molecular representations for property prediction. Journal of chemical information and modeling, 59(8):3370-3388, 2019.
R. Zellers, A. Holtzman, Bisk, A. Farhadi, and Choi. HellaSwag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics, pages Linguistics. doi: 10.18653/11/19 1472 URL https: /ww. web. org/ anthology/
Florence, Italy, July 2019. Association for Computational
P1 P19-1472
13


Reviews:
[{"review_id": "xPf11zDbJZ6", "reviewer": "Reviewer_J8M5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "(1) Strength: The problem that this paper addressed is important and essential for machine learning. And the whole pipeline is reasonable and achieves good results.\n\t(2) Weakness: \n\t\t(a) Method: The proposed method infers the unstable features on the source task and then learns the stable correlation on the target leveraging the knowledge of unstable features. As for the methods, it uses the Group-DRO as the backbone and clusters the available testing data to produce the group labels. However, I think the method is quite ad-hoc and naïve, and it lacks technical contributions. The idea of clustering data with unstable features is also quite similar to the HRM[1], which further reduces the contributions.\n\t\t(b)Problem Setting: This method requires both labeled data from the source task and target task, which uses much much more information than existing methods for OOD generalization. \n\t\t(c)Theoretical Analysis: There is almost no theoretical analysis for the proposed method. Why it can infer the unstable features? Why this method can generalize to unseen domains? What if the worst-case group also reflects some spurious correlations?\n\t\t(d)Experiments: Since it is both the OOD generalization method and the transfer learning method (setting is the same), there lack many baselines. I think more transfer learning methods should be taken into consideration, as well as some domain generalization methods.\n\t\t(e)Definitions: What do the stable features or unstable features mean in this paper? What is the difference between causal features[2], invariant features[3], or stable features[4][5]? What is the formal definition of stable and unstable? If it is just the same as causal features, why use a totally new name? I think the authors missed many related works here.\n\n[1] Liu, J., Hu, Z., Cui, P., Li, B., & Shen, Z. (2021). Heterogeneous Risk Minimization. ICML2021\n[2] Peters, J., Bühlmann, P., & Meinshausen, N. (2016). Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 947-1012.\n[3 Koyama, M., & Yamaguchi, S. (2020). Out-of-distribution generalization with maximal invariant predictor. arXiv preprint arXiv:2008.01883.]\n[4] Kuang, K., Xiong, R., Cui, P., Athey, S. and Li, B., 2020, April. Stable prediction with model misspecification and agnostic distribution shift. AAAI 2020\n[5] Shen, Z., Cui, P., Liu, J., Zhang, T., Li, B. and Chen, Z., 2020, August. Stable learning via differentiated variable decorrelation. KDD 2020\n", "summary_of_the_paper": "This paper proposes a two-stage method for transfer learning, which firstly infers unstable features from the source task and then learns stable correlations for the target. Experimental results validate the effectiveness of the proposed method.", "summary_of_the_review": "This paper proposes a two-stage method for OOD generalization and transfer learning problems. However, there exist many weaknesses, including method, problem setting, theoretical analysis and experiments. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ajpX0cd9MrX", "reviewer": "Reviewer_L5Kn", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general, the method seems to work and it seems to be justified (both theoretically, albeit in the appendix, and practically). It is really complicated though (involves many steps) and i fail to make a connection between this and say Domain Invariant Representation learning. For example, if i was to solve the problem of domain generalization (the setup seems really similar to me), a reasonable baseline would be to train a source model that learns an invariantt representation between various environments (using DANN, or CMD or MMD or whatever). But this is not in the baselines. Why is that? One difference i \"kinda\" see is that domains are more or less pure - eg domain is really a different background. The way i read it is that environement is more mixed - it can contain multiple backgrounds. It would be nice to draw comparison between your method and DIRL or at least explain why DIRL is not applicable\n\n\nPros:\n- The paper is well written and easy to follow (albeit a lot of helpful info is also in the Appendix)\n- The experimental results seem convincing, there is a study of dependence of (some) hyperparameters on the end result (num of clusters)\n\nCons:\n- DRO needs at least a brief introduction\n- Experiments are only on 2 environements (domains) in train\n- The method seems extremely costly: for a number n of source environements, you train n classifiers, then for n^2 pairs you do partions, then you learn fz representation, then cluster the target data all while tuning and looking for fz dimension and number of clusters. \n-There are hyperparameters that need to be tuned: fz dimension, number of clusters. Tuning all the hyperparameters using limited target data - how does it work? Do you tune the fz dimension (i could not understand it from text, i think u do tune the num of clusters but not sure about fz dimension)\n\n\n\nAdditional comments:\n- For figure 1, source task environments actually look the same to me tbh: the red is mostly correlated with 0 and green is mostly correlated with 1 for both of the environments. If they were flipped between the environments,i would expect Domain invariant representation learning to be able to filter out the color from the embedding layer\n- what if target is also a mix of different environments (you seem to assume that it is all comes from one domain/env \n- How practical is experimental setup you are testing on? For your experiments train was coming from 2 environments that were created artificially\n\nMinor: then we uses=>then we use", "summary_of_the_paper": "The paper is concerned with learning a model in situations when some features have spurious correlations with the label (for example, for classifying sheep vs camel, background can be a spurious feature). The idea is that if a lot of source data is available, these spurious correlation features should be easy to find. To find the unstable features, authors hypothesize that they are related to mistakes that a classifier makes in different environments. Therefore, they learn a model on one environment and run it on a different environment (source data), splitting the data into correctly predicted and not. Then for each partition (correct or not) examples with the same labels are encouraged to be close together, so the embedding learns UNSTABLE features (fz). Then the target data is clustered based on unstable features representation fz, and then DRO (existing method that assumes the existance of correct groupping based on unstable features) is used to train a robust target classifier", "summary_of_the_review": "Please see main review", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "a-rmsUIXJ9t", "reviewer": "Reviewer_RJhJ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n(1) The authors address a critical point that prevents models from generalization, namely spurious correlation.\n(2) The whole pipeline is intuitive and easy to follow, and the empirical results are within expectation.\n\nWeakness:\n(1) A critical limitation of this work is the strong assumption on the transferability of the unstable feature. IMO, such an assumption is restrictive in most settings. For the intra-task transfer (where the classification tasks are the same between source and target), finding the invariant part via invariance learning methods is more realistic. As for the inter-task transfer (a wilder setting),  it is very rare that the unstable features and the way they correlate the outcome are exactly the same across the tasks.\n\n(2) The related work and baselines are not discussed thoroughly. First of all, more transfer learning methods should be discussed, not just REUSE and FINTUNE. Moreover, several automatic de-biasing methods [1,2,3] proposed recently could also be considered to involve.\n\n[1] Qiao, F., Zhao, L., & Peng, X. (2020). Learning to Learn Single Domain Generalization. In CVPR. \n[2] Matsuura, T. and Harada, T., 2020, April. Domain generalization using a mixture of multiple latent domains. AAAI\n[3] J. Liu, Z. Hu, P. Cui, B. Li, and Z. Shen, “Heterogeneous risk minimization”, ICML 2021.", "summary_of_the_paper": "The paper considers a transfer problem when the spurious correlations of source tasks can be applied to the target task. The authors propose to identify the unstable features on source tasks first and then cluster the target data according to these features. Finally, an invariance-based method are incorporated to eliminated the influence of unstable features.", "summary_of_the_review": "This paper addresses the spurious correlation by transferring knowledge from source tasks. Although intuitions are provided and empirical effectiveness is illustrated accordingly, the method is restrictive due to the strong assumption on the transferability of unstable features. Several important related work and baselines are also missing.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Uuw81769svx", "reviewer": "Reviewer_hKdn", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper itself is well written, and does a good job at describing the current research landscape of robust feature learning. Some more thought should be given to the example provided of why a method like TOFU is required, since the current one of using class labels to predict color instead of vice versa seems contrived, and undersells the usefulness of the method. The theoretical claims made throughout the paper are sound, but it is not obvious that they should hold in practice. For example, on datasets where there is heterogeneity in terms of the difficulty in predicting samples, i.e. easy, medium, and ambiguous samples, easy and medium samples might end up in X_1 and X_2, and the ambiguous samples in X_3. Thus, the objective in equation (1) could end up optimizing not for the unstable features in X_1 and X_2 to be closer together than those in X_1 and X_3, but rather for easy samples to be closer together than easy and difficult samples in the learned feature space Z. Additionally, it may be possible to improve S.2 by considering not only correctly predicted vs incorrectly predicted, but also the particular errors that were made. Consider a scenario where images of cars are either misclassified as boats or bicycles. Bicycles are more likely to be on the road than boats, so it may be wise to exclude the cars misclassified as bicycles from X_3, since those contain the correct unstable feature being sought, but violate the assumption of theorem 1. It seems like for TOFU to work, the classifiers trained in step S.1 need to be unstable such that they will make sufficient errors on E_2 in order to have sufficient negative samples for the triplet loss objective in S.3. The limitations of metric learning with a triplet loss should be stated since they could have an influence on model selection done throughout the method.\n \nTOFU has fewer requirements in terms of additional annotations compared to other methods, and domain specific prior knowledge is encoded into how the datasets are split into E_1 and E_2. Quantitative analysis of TOFU reveals that it is significantly superior to the many baselines considered, and that is consistent on all tasks and datasets. This is impressive and rare since many methods in the broader deep learning literature improve on some tasks, but do worse on others. It is also satisfying to see how well TOFU performs in terms of clustering metrics compared to representations learned only for the purpose of classifying the source task. Naturally, the latter representations would contain information encoding the spurious features since they are then leveraged to predict the class label, but they seem to be highly ineffective for the purpose of clustering by unstable feature. Having said that, this analysis is a bit confounded by the fact that ERM uses supervised learning whereas TOFU uses metric learning. Additionally, it would be ideal to have a discussion regarding why the performance improvement of TOFU over ERM is not as significant for the Beer Review data compared to MNIST, even though the clustering scores are much better. In general, the difficulty of the tasks considered for the empirical evaluation should be discussed as it seems to vary quite a bit.\n \nThe novelty of the work is somewhat limited. S.1 to S.3 is nearly identical to the cited work of Bao et al., except that the objective in S.3 is changed to the triplet loss objective. This objective comes from Theorem 1 which is a novel contribution, but is based on theoretical results also from the cited work of Bao et al. It would also be recommended that more real life examples of the transfer learning scenario in question be provided, since it currently seems as if it was created to show the applicability of the method. Overall, the paper is very strong from an empirical perspective, is well structured and reproducible, but ignores any potential limitations and is limited in its discussion.\n\nStrengths\n-Paper is well written and self-contained.\n-Impressive empirical results on many tasks/datasets relevant for this application.\n-Provides an interesting perspective on combining transfer learning and robust feature learning,\n \nWeaknesses\n-Not so clear how the datasets were split to generate different E1 and E2. For example, in the Waterbirds dataset, if the source task is classifying waterfowl, is the only difference between E1 and E2 the percentage of images that have a water background? This is ambiguous in the appendix as well.\n-Conclusion is too short, and is missing a discussion of limitations and future directions.\n-A clear limitation of the work is that the unstable features in the target task have to be the same ones as in the source task\n", "summary_of_the_paper": "The authors introduce a method (TOFU) for learning classifiers that are robust to spurious correlations in a transfer learning setting. They argue that approaches which rely only on (input, label) pairs and use no extra information to identify spurious features could fail to learn a robust model due to insufficient data for the target task. The authors provide scenarios where useful metadata, namely varying environments, could be available for source tasks, ready to be leveraged by the target task classifier for the purpose of identifying a shared bias that might exist across all tasks. To identify unstable/spurious features, the authors follow a 3-step procedure proposed in an existing work. Namely, a classifier is trained on data from environment E1, and is evaluated on data from a second environment E2. The data in E2 is then partitioned according to the correctness of the classifier's predictions on a per-class basis. A metric learning objective using a triplet loss is used to learn a model which embeds samples according to their unstable features with the goal that the unstable features of the correctly predicted samples should be closer together than unstable features between correctly predicted and incorrectly predicted samples. The objective is justified based on prior theoretical work and is illustrated visually. Extensive empirical evaluations are performed, comparing the performance of TOFU against many different baseline methods on both image and text classification tasks.", "summary_of_the_review": "This work introduces a new scenario where current methods for learning robust features are not capable of leveraging all available data. It also provides a solution in the form of a method called TOFU which has few requirements in terms of additional data annotations compared to existing methods. The authors recognize that bias is a human defined concept, and could vary from dataset to dataset, so it is best to have a method that can identify unstable features in an automated way. TOFU is an extension of an existing work, and has some theoretical motivations, but the underlying assumptions may be too strong. TOFU outperforms all other methods considered by a large margin, and the authors investigate a potential source of its success by confirming that it is indeed able to cluster data well according to unstable features: a requirement for later on doing group DRO. Sufficient details are provided to enable reproducibility, and many baselines are compared against, making the experiments comprehensive. However, the method is not particularly novel, nor is it clear if the scenario presented occurs often in real life. Limitations of the method are not considered, even though there are a few clear ones both technical and empirical.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "GU8TviZsIq3", "reviewer": "Reviewer_tPxR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall I like this paper. This paper does a good job of explaining the setup and the writings are mostly clear. This paper builds upon the core idea from Bao et al. 2021 that uses the error of the model in different environments to capture the spurious patterns, and it further extends to the transfer learning setup, which I believe it's a new setup.\n\nMy concerns are as followed:\n1. The assumptions are too strong in my opinions, and I think a discussion section will help readers understand the limitations:\n - A) I am skeptical about the assumption in Bao et al. 2021 that uses the error of the model across environments to capture the spuriousness. There could be many factors behind errors across environments such as mislabeled data, different viewpoints of the image (e.g. rotations), noise, and data distribution shifts. None of it corresponds to the unstable features. If this assumption breaks, then there is no real spurious pattern to be learned here.\n- B) On the data side, this work assumes the source tasks need to have varying degrees of spuriousness and the target task has the same spurious patterns. How do we possibly check these assumptions are real in the real-world setting?\n\n2. Regarding the demonstrated results\n- A) There could be different spurious factors across different classes. And the binarization procedure of correct or incorrect predictions could lead to underspecification since there might be multiple spurious factors but we only treat them as two. Will this method break if there are multiple spurious patterns?\n- B) The multi-stage approach of training could be brittle, and selecting hyperparameters could be difficult since no spurious correlation is known beforehand. Maybe the authors can show if this approach is robust when the assumptions are mildly violated like there is a distribution shift among environments, or when there are multiple spurious patterns in the data, and illustrate how to select hyperparameters.", "summary_of_the_paper": "This paper aims to transfer the knowledge of spurious correlations in a set of source environments to a target environment. They assume the degree of spurious patterns vary among source environments (e.g. color has different correlations with the number in MNIST), and the spurious pattern are the same in both the source and target tasks. Then they aim to transfer the spurious knowledge from source environments to learn a classifier in the target task that ignores the spurious pattern. \n\nThey train the model in 3 stages. First, they train a regular classifier in each environment. E.g. given two source environments E1 and E2, they train the corresponding classifiers f1, f2. Second, they use the error of the f1 on E2 to \"separate\" the E2 examples into two groups, one group for correct predictions and the other group for the wrong predictions under each class. They assume the cause of the error comes from the spurious patterns, so each group will correspond to either high or low spurious patterns learned in f1. Thus they learn a f_Z that outputs an embedding to separate these two groups by triplet loss. Finally, in the target task, they cluster the examples into different groups by the similarity of f_Z that captures examples with similar spurious patterns. Then they apply Distribution Robust Optimization (DRO) on these groups that optimizes the worst performance across all groups. This ignores the spurious patterns based on f_Z in the target task and learns a stable classifier.", "summary_of_the_review": "Pros:\n+ A new setup that transfers the knowledge of spurious patterns to a target task.\n+ The writing is clear. Figures and tables are beautifully produced.\n+ The results are demonstrated in multiple datasets and the number is convincing compared to several recent baselines. The ablation study of group numbers is great.\n\nCons:\n- The hypotheses in my opinion are a bit too strong. It's unclear in what real-world settings this method will work unless in the contrived setup in this paper. Including a limitation in the paper can strengthen the paper.\n- The multi-stage approach of training could be brittle.\n\nI am leaning towards the borderline with marginal acceptance. The experiments are complete with multiple baselines, clear writing and a good ablation study. I feel the assumption is too strong which would need some justifications. And if the authors can show a robust study under different mildly violated assumptions could further strengthen this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The idea of learning unstable features from source tasks to help learn stable features for a target task is interesting and well-motivated. As the proposed method and its theoretical analysis of learning unstable features from tasks are an incremental extension of an existing work [Bao et al. 2021], the technical contributions line in applying the idea of stable and unstable features learning to the setting of transfer learning. Therefore, the evaluation of this work is focused on the effectiveness of the proposed method in the transfer learning setting. 

In transfer learning, one major goal is to make use of knowledge extracted from source tasks to help learn a precise target classifier even with a few or no labeled examples of the target task. It would be more convincing if experiments are conducted to show how the performance of the proposed method changes when the size of labeled data of the target task changes. This is to verify whether the exploitation of unstable features can help to learn a stable classifier for the target tasks more efficiently (i.e., with fewer labeled examples). In addition, as some baseline methods used for comparison do not need to access any labeled data of the target task (like unsupervised domain adaptation or domain generalization approaches), it is not fair to conduct comparison experiments in the setting where there are sufficient labeled examples of the target task since the original designs of such baselines may fail to fully exploit label information in the target task. 

Another concern is whether the proposed method is realistic for real-world transfer learning problems. Though in the rebuttal, the authors provided experimental results on a natural environment (CelebA), the constructed transfer learning problem is more like a toy problem. Indeed, there are many transfer learning benchmark datasets that contain multiple domains/tasks. It would be more convincing if experiments are conducted on those datasets.

By considering the above two concerns, this paper is on the borderline. My recommendation is a weak rejection based on the current form of this paper. Note that as some references listed by reviewers RJhJ and J8M5 are not really related to the proposed research here, the novelty of the proposed method compared with those references is NOT taken into consideration to make this recommendation.
Decision: Reject