Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fix
Authors: Ruocheng Guo
Pengchuan Zhang
Hao Liu
Emre Kiciman
Venue:  (2021)
Keywords: Invariant Risk Minimization
Causal Machine Learning
Out-of-distribution Prediction

Abstract: This work considers the out-of-distribution (OOD) prediction problem where (1)~the training data are from multiple domains and (2)~the test domain is unseen in the training. DNNs fail in OOD prediction because they are prone to pick up spurious correlations. Recently, Invariant Risk Minimization (IRM) is proposed to address this issue. Its effectiveness has been demonstrated in the colored MNIST experiment. Nevertheless, we find that the performance of IRM can be dramatically degraded under \emph{strong $\Lambda$ spuriousness} -- when the spurious correlation between the spurious features and the class label is strong due to the strong causal influence of their common cause, the domain label, on both of them (see Fig. 1). In this work, we try to answer the questions: why does IRM fail in the aforementioned setting? Why does IRM work for the original colored MNIST dataset? Then, we propose a simple and effective approach to fix the problem of IRM. We combine IRM with conditional distribution matching to avoid a specific type of spurious correlation under strong $\Lambda$ spuriousness. Empirically, we design a series of semi synthetic datasets -- the colored MNIST plus, which exposes the problems of IRM and demonstrates the efficacy of the proposed method.

Under review as a conference paper at ICLR 2021
OUT-OF-DISTRIBUTION PREDICTION WITH INVARIANT RISK MINIMIZATION: THE LIMITATION AND AN EF- FECTIVE FIX
Anonymous authors Paper under double-blind review
ABSTRACT
This work considers the out-of-distribution (OOD) prediction problem where (1) the training data are from multiple domains and (2) the test domain is un- seen in the training. DNNs fail in OOD prediction because they are prone to pick up spurious correlations. Recently, Invariant Risk Minimization (IRM) is pro- posed address this issue. Its effectiveness has been demonstrated in the colored MNIST experiment. Nevertheless, we find that the performance of IRM can be dramatically degraded under strong A spuriousness when the spurious correla- tion between the spurious features and the class label is strong due to the strong causal influence of their common cause, the domain label, on both of them (see Fig. D. In this work, we try to answer the questions: why does IRM fail in the aforementioned setting? Why does IRM work for the original colored MNIST dataset? Then, we propose simple and effective approach to fix the problem of IRM. We combine IRM with conditional distribution matching to avoid specific type of spurious correlation under strong A spuriousness. Empirically, we design a series of semi synthetic datasets the colored MNIST plus, which exposes the problems of IRM and demonstrates the efficacy of the proposed method.
INTRODUCTION
Strong empirical results have demonstrated the efficacy of deep neu- ral networks (DNNs) in a variety of areas including computer vision, natural language processing and speech recognition. However, such positive results overwhelmingly rely on the assumption that the train- ing, validation and test data consist of independent identical samples of the same underlying distribution. In contrast, in the setting of out- of-distribution (OOD) prediction where (1) the training data is from multiple domains and (2) the test set has different distribution from the training, the performance DNNs can be dramatically degraded. This is because DNNs prone pick up spurious correlations which do not hold beyond the training data distribution (Beery et al.) 2018 Arjovsky al. 2019). For example, when most camel pictures in a training set have a desert in the background, DNNs will pick up the spurious correlation between desert and the class label, leading failures when camel pictures come with different backgrounds in a test set. Therefore, OOD prediction remains an extremely challeng- ing problem for DNNs.
E
X
X
Y
The causal graph in OOD predic- tion: P(Y X is invariant across domains. The spu- rious correlation P(Y|X*) may vary. A directed (bidi- rected) edge is a causal re- lationship (correlation).
Figure
The invariant causal relationships across different domains turns out to be the key to address the challenge of OOD prediction. The causal graph in Fig. describes the relationships of the variables in the OOD prediction problem. Although spurious correlations learned in one do- main are unreliable another, invariant causal relationships enable DNNs that capture causal rela- tionships to generalize to unseen domains. In practice, it is extremely difficult to know whether an input feature is causal or spurious. Thus, a recipe for training DNNs that capture causal relationships learning causal feature representations that hold invariant causal relationships with the class label.

---

Under review as a conference paper at ICLR 2021
Thus, the main challenge of OOD prediction becomes learning causal feature representations given training data from multiple domains. Invariant Risk Minimization (IRM) (Arjovsky et al. 2019) is proposed to learn causal feature representations for OOD prediction. IRM tormulates the invariance of causal relationships as a constraint. It requires that causal feature representations must result in the same optimal classifier across all domains. IRM is written as the conditional independence, Y IL F ) in (Chang et al. 20200] Zeng et al. 2019) which can be derived from the causal graph in Fig. [ when F(X) is mapping of causal Teatures (X" in Fig. T with no information loss. A
detailed discussion on IRM and Y IL (X) is in Appendix A
Despite IRM's success in the colored MNIST dataset (Arjovsky et al., 2019), in this work, we find an important issue of IRM that has not been discussed. Specifically, we consider the situation where strong spurious correlations among the spurious features, the class label and the domain label only hold for training data. We name this strong A spuriousness using the shape of the structure among X and Y in Fig. Under strong A spuriousness, IRM regularized empirical risk can have low values with spurious feature representations that accurately predict the domain label. This is because, in this setting, picking up spurious features can achieve high accuracy in predicting both domain and class in the training set, but not in the test. However, the colored MNIST dataset cannot expose this issue because the strong similarity between the two training domains makes difficult to pick up the weak spurious correlation between the domain label and the spurious features. To illustrate this problem, we design a new dataset - the colored MNIST plus. As shown in Fig. 2 in this dataset, under strong A spuriousness, the performance of IRM models is significantly degraded.
Moreover, to resolve this issue of IRM, we propose an effective solution, which combines IRM with conditional distribution matching (CDM). The CDM constraint means that the representation distribution of instances from the same class should be invariant across domains. Theoretically, we show that (1) causal feature representations can satisfy CDM and IRM at the same time and (2) CDM can prevent DNNs from learning spurious feature representations that accurately predict the domain label. Empirically, on our newly introduced dataset, the proposed method achieves significant performance improvement over IRM under strong A spuriousness.
2 PRELIMINARIES AND IRM
Notations. We use lowercase (e-g. æ). uppercase (e.g., X) and calligraphic uppercase (e.g., X) letters for values, random variables and spaces. We let X E X, Y E ) and F E & denote raw input, the class label and the domain label where X, y and E are the spaces of input, class labels and domains. A DNN model consists of a feature learning function I' and a classifier G. A feature learning function F : X + IR maps raw input X to its d-dimensional representations F(X). A classifier G: Rd Y' maps feature representation to class label. We denote their parameters by 0F and 0g, respectively. Let = Concat Fr, 0c) denote the concatenation of them.
A domain of ne instances is denoted by De {af yf T= Let Eer and Ele denote the set of training and test domains, in OOD prediction, we have (1) Etr
> 1 and (2) Eir O E = 0.
Problem Statement. Given data from multiple training domains (De)eee.,. We aim to predict the label of of each instance with features x.! from a test domain {2 of ) ELs.
Invariant Risk Minimization. IRM (Arjovsky et al. 2019) is a recently proposed method to im- pose the causal inductive bias: the causal relationships between causal features and the label should be invariant across different domains. It not only aims to address the challenging OOD prediction problem, but also pioneer work that guides causal machine learning research towards the de- velopment of inductive bias imposing causal constraints. The effectiveness of IRM and its variants have been demonstrated across various areas including computer vision (Ahuja et al. 2020), natural language processing (Chang et al.) 2020). CTR prediction (Zeng et al., 2019). reinforcement learn- ing (Zhang et al. (2020a) and financial forecasting (Krueger et al., 2020). AArjossyy et al. (2019) propose the original formulation of IRM as two-stage optimization problem:
arg min 00,0c
Σ E. (G(((()))
eEE.r
(1)
s.t. 0G E mrg (G((((()))
%
2

---

Under review as a conference paper at ICLR 2021
where R denotes the loss function of domain e. Then, they show that the IRM constraint can be imposed by adding a regularizer into the loss function. It turns out the IRM constraint can be written as the conditional independence Y T1 E (XX) (Chang et al., 2020: Zeng et al. 2019). This can be derived from the causal graph in Fig. 1 For brevity, we refer to E F as the IRM constraint.
3 THE LIMITATION OF IRM
In this section, we find that IRM fails under strong A spuriousness when the spurious correlations among spurious features, the domain label and the class label are strong. To show it, we design a new dataset- the colored MNIST plus (CMNIST+) to expose this limitation of IRM. Then, we try to answer two crucial research questions: (I) why does IRM fail in CMNIST+? (2) Why does IRM work for the original colored MNIST (CMNIST) dataset?
3.1
WHY DOES IRM FAIL?
A spurioussesss (p)
0.55
66.32
68.96
69.89
0.55
37.89
37.11
43.48
0.55
60.75
0.6
63.44
68.38
66.40
0.6
35.49
36.24
48.64
0.6
51.73
A usness
0.65
58.42
68.25
66.00
0.65
32.40
39.19
36.92
0.65
40.96
0.7
52.60
62.40
64.12
0.7
36.91
37.82
41.45
0.7
36.14
0.8
37.45
47.01
45.38
0.8
35.80
34.39
31.37
0.8
30.16
0.85
33.04
35.80
37.81
0.85
28.15
31.48
30.36
0.85
27.83
0.9
33.32
31.67
36.31
0.9
26.66
28.37
28.97
0.9
24.61
600
200
400
600
0
Iteration IRM added (Kirm)
Iteration IRM added (Kkm)
Iteration IRM added (Kum)
(a) IRM
(b) IRM with label balancing
(c) ERM
Test accuracy IRM, IRM with balanced classes in each domain, and ERM on CM- NIST+: when the A spuriousness is strong (p 2 0.8), the performance of IRM drops dramatically because, in this situation, the spurious feature representation F(X) E satisfies the IRM con- straint. The naive solution, balancing classes in each domain cannot mitigate this problem of IRM.
Figure 2:
Despite its previous success, results in Fig. 2 show that IRM fails in our newly designed colored MNIST plus dataset, under strong A spuriousness (p > 0.8). Here, show that, under strong A spuriousness, at least two types of feature representations, F(X) = and F(X) = E, satisfy the IRM constraint. When F(X)= E, the feature representations pick up spurious relationships.
Satisfying IRM by Perfectly Predicting the Label. First, we consider a case where there exists a feature representation that perfectly predicts the class label, i.e., F(X) Y. In this case, we know that the feature representation satisfies the IRM constraint because Y IL E|F(X) = Y always holds. This is reasonable because the causal relationship between Y and itself must be invariant across different domains.
Satistying IRM by Perfectly Predicting the Domain Does Not Lead to OOD Generalizable Feature Representations. However, there is another way to satisfy the IRM constraint. If there exists a feature representation F(X)= = E, then the IRM constraint can be satisfied by F(X). This is due to the fact Y TL F(X) E. However, this is not desired behavior of IRM as F(X) = E is type of spurious feature representation. This is because P(Y EE can vary across domains. Using the camel picture example, if domains are the individuals who take the picture, we may find more camel pictures taken by person than from person e' (P(Y camel F = e) > P(Y camell = e'))- E good solution for the original IRM optimization problem (Eq. D This is because F(X) = E also leads to low values of the loss function Rº since the direct causal influence of the domain label E on the class label Y is also strong when A spuriousness is strong. So, with a proper classifier G, we can achieve low values of R with F(X) E. Then, we describe the CMNIST+ dataset and show experimental and theoretical results to support claim.
Why is F(X)
The Colored MNIST Plus (CMNIST+) Dataset. We follow CMNIST (Arjovsky et al. 2019 to create CMNIST+ by resampling and adding colors to instances of MNIST. The digits ( 4 (5 9)
3

---

Under review as a conference paper at ICLR 2021
class Y = 1 = 0). We randomly flip 25% of class labels that spuriousness can be stronger than the causal relationship between the shape S (causal features) and the class label Y. Table Delleridessebbee the dataset. The variable C E {G, B, R) denotes the color, which represents the spurious feature X We explain why we use three colors for CMNIST+ in Appendix [C.2 The parameter p (0.5, 1) controls the strength of the spurious correlations between the color (spurious features) C' and the class label Y through their common cause, the domain label E. The larger the value of p. the stronger the spurious correlations. Intuitively, in the two training domains of CMNIST+, in addition to the strong spurious correlation between the class label Y and color C, we set the spurious correlation between the domain label E and color C' to be strong, too. Thus, the CMNIST+ dataset can expose the problem of IRM: it would not penalize the models that fit the domain variable as the feature representation (F(X) = E). To verify this claim, we show analysis results to support the experimental results in Fig. The data generating process of CMNIST+ can be found in Append C.2
Table 1: Definition of CMNIST+
P(Y 1E)
P(C G|y, E)
P(C B)
P(C
R|Y
E =
0.9
Y ==
p
(1- p)
(1-p)/2
Y =
(1 - (1-p)/2
p)/2
[1 p)/2
E = 2
0.1
Y ==1
(1 p)/2
p
(1 p2p)
Y=
[1 pl/2
p)/2
p
Y ==1
0.1
0.1
0.8
==3
0.5
Y =0
0.4
0.4
0.2
E=1
CMNIST
Between CMNIST and CM NIST+
CMNIST+ (p=0.9)
R
R
A
E=3
Class Y=1
G
B
6
B
G
Figure 3: We visualize the differences between CMNIST and CMNIST+ (p = 0.9) in terms of P(C 1, E). Each large triangle represents the space of P(C|Y, E). Each small triangle shows the values of P(C = c/Y = 1, F. = e), E {R, G, B},e e E (1,2,3).
Analysis Results. Here, we show the analysis results on the CMNIST+ to answer two questions: what features will be learned by ERM and IRM under under different A spuriousness? What is the expected test accuracy of these models? For simplicity, we assume that the classifier is deterministic, which always predicts the majority class given the feature. If the number of instances from each class is the same, then it would predict a random label. We analyze three types of spurious feature representations: (1) those that fit color, (2) those that fit the domain label and (3) those that use a combination of them for prediction. The first case mimics the behavior of ERM that picks up the spurious correlation between color and the class label. The second one represents model satisfying IRM by F(X)= = E. The third one stands for model satisfying IRM by F(X)
Concat E, C). One may argue that IRM can be fixed if simply balance the two classes in each domain. Theoretical analysis shows that this is an invalid solution. We summarize results in Table 2 where É is the domain label predicted by deterministic classifier using color as the feature. This is because in an unseen test domain, we cannot directly use the domain label for prediction. The highlighted numbers show which type of features would be learned by ERM and IRM. As increases, ERM and IRM are more likely to fit spurious features. This explains results in Fig. 2 As DNN may still pick up some causal features (shape in CMNIST+) even when p 0.9, the test accuracy of IRM in practice would be greater than 0.35 without label balancing and 0.2 with label balancing. Similarly, the test accuracy of ERM would be greater than 0.2. From Fig. 2 we can see the test accuracy of IRM, IRM with label balancing and ERM are slightly better than the aforementioned lower bounds. The derivations can be found in Appendix B
4

---

Under review as a conference paper at ICLR 2021
Experimental Setup. We use L (eCCun et al. 1998) instead of a three-layer MLP. To take input with three colors, we modify the first CNN layer to have three channels. LeNet-5 has more predictive power such that it can pick up the three types of spurious correlations (color domain, color class, and domain - class) or the causal relationship (shape - class). We randomly split the instances from the training domains into 80% training and 20% validation. The model selection is done by picking the one with the lowest validation loss in each run. We report the average test accuracy of the selected models in 10 runs. It is crucial to ensure only data from training domains are used in model selection, since the test domain should be unseen during training and validation. During training, we begin applying the IRM penalty at iteration KIRM, set 200, 400, or 600. By varying K KIRM, we aim to examine the following hypothesis: IRM works by pushing the spurious features out of the representations learned by the standard ERM training before the IRM penalty is applied (Krueger et al. 2020). More details on the setup can be found in Appendix
Experimental Results. Fig. 2 shows the performance of IRM and IRM with balanced classes in each domain. We make the following observations: first, the test accuracy of IRM drops dramatically under strong A spuriousness. Second, we show that a naive fix for IRM, balancing the two classes in each domain by oversampling the minority class, does not lead to improvement in performance.
Table 2:
Theoretical validation/test accuracy with the three spurious feature representations (C: color, E: domain and E+C) and the causal feature representation (S: shape) on CMNIST+. The ones with the best validation accuracy in each setting highlighted, which show the type of feature representations would be learned by ERM and IRM.
Validation/ Test Accuracy
withoat label balancing
with label balancing
C(Y
F(Y[E)
P(Y C)
P(Y
P(Y[C)
P(Y
P(Y E, C)
P(Y
0.662/1,2
0.646/0.35
0.662A0.35
0.75/0.75
0,6220.22
0.500.5
0.662N122
0.75NI.75
0.6
0.7/0.2
0.68/1,3
0,370.35
0.75/0.75
0,7/0.2
0.540.
0,770.2
0.75N0.75
0.65
0.738/0.2
0.714/0.35
0.738/0.35
0.75/0.75
0.738/0.2
0.510.5
0.737/0.2
0.75/0.75
0.7
0.775/0.2
0.74880.35
0.775/0.3
0.75/0.75
0.775A0.2
0.500.5
0.775/02
0.75/0.75
0.8
0.85/0.2
0.81550.35
0.81550.35
0.75/9.75
0.85/0.2
0.540.5
0.85/0.2
0,75/0.75
0.85
0.888/0.2
0.849M0.35
0.84910.2
0.75A1.75
0.888/0.2
0.540.5
0.888/012
0.75AU.75
0.9
0,925/0.2
0.88370.35
0.883/0.2
0.75/0775
0.925/0.2
0.310.5
0.925/0.2
0.75/0.75
3.2
WHY DOES IRM WORK FOR THE ORIGINAL COLORED MNIST?
0.3 0.6 0.4 0.2 0.0
70.03
74.91
57.21
0.6 0.4 0.2 0.0
61.96
64.95
57.80
53.40
47.49
56.59
52.10
47.40
48.12
Wolus
Wotus
34.57
46.78
46.54
39.87
47.82
42.86
40.66
41.85
50.62
34.67
45.17
43.82
38.09
39.87
37.94
0.8
37.59
44.68
38.72
0.5
0.7
0.9
0.5
0.7
0.9
P(Y= 1|E
1),
P|
P(Y=
(a) IRM on CMNIST test set
(b) IRM on CMNIST+ test set
Figure 4: Accuracy of IRM trained on datasets between CMNIST and CMNIST+ and tested on the test sets of CMNIST and CMNIST+ (Wpius increases): As the training set becomes more similar to CMNIST+, the performance of IRM gradually drops, on the test sets of CMNIST and CMNIST+.
The Colored MNIST (CMNIST) dataset cannot expose the limitation of IRM under strong A spuri- ousness. This is because its two training domains are quite similar. As shown in the large triangle on the left in Fig, 3 the values of P(C|Y, E') similar for E = 1 and E 2. In addition, the values of P(Y [E) are the same for all E. This makes it difficult to satisfy the IRM constraint by learning feature representations F(X)= E.
Then, we present experiments to show how IRM gradually goes from working on CMNIST to failing on CMNIST+. In Fig. 3 Left and Right, observe the differences between between CMNIST and CMNIST+ (p = 0.9). In Fig. 3 Middle, we create various datasets that interpolate between these two datasets, illustrated by the yellow line in the middle triangle. We use parameter Wppus E [0,
5

---

Under review as a conference paper at ICLR 2021
to control P(C|Y, E) of the weights of CMNIST+ (p 0.9) in the interpolated dataset:
Pemnist (ClY, E) Wppiss + Pemmisa (ClY, E1 - Wpines)
P(C|Y,E) F.
(2)
Eo(P (ClY, ElUepius - Pemmist PemmisuCC|y, (1 - wpluss)
where Pemmistr (CIY, E) and Pammiu (C F) are the values of P(C E) in CMNIST+ and CM-
NIST. When Wylue O and P(Y F = 1) P(Y 0 2) 0.5, the dataset is the same with CMNIST. As Wplus and P(Y = 1E 1) = P(Y = 0 = 2) increase, the dataset becomes more similar to CMNIST+, and becomes the same with CMNIST+ (p 0.9) when
Uplus = and P(Y = E = 1) = P(Y = O 2) = 0.9. For the training sets, we set P(Y 1|E = 1) P(Y 0 2) E 0.7,0.)7 Fig. shows results of IRM the test sets of both CMNIST and CMNIST+. The performance of IRM gradually drops when the values of
P(C|Y, E) becomes more similar to CMNIST+ (when the A spuriousness becomes stronger).
AN EFFECTIVE FIX FOR IRM
In this section, propose simple but effective solution to address the limitation of the IRM training. a series of experiments, we show that the proposed method can improve the performance of IRM even under strong A spuriousness.
Since the IRM constraint is too general, we propose an effective solution combining the condi- tional distribution matching (CDM) constraint and the IRM constraint. The CDM constraint (Li et al. 2018 Long et al., 2018) requires P(F(X)]Y, E) = P(F(X))Y), which means the feature representation distribution given the class label should be invariant across different domains. We first explain why this can be reasonable solution. Then, we propose two types of models that combine the two constraints.
Here, we use the two cases from Section 3.1 to explain why combining IRM with CDM can be an effective fix for IRM.
Perfectly predicting the label satisfies CDM. In this case, F(X) = Y and we already know the IRM constraint satisfied. The CDM condition also holds as P(F(X)(Y,E) = P(Y P(Y]Y) = P(F(X))Y).
Perfectly predicting the domain label violates CDM. this case, F(X) = F. and the IRM con- straint can be satisfied, but the CDM constraint does not hold as (FF())YY,E) P(E = P(F(X) This case implies that we can add CDM to IRM to exclude the undesir- able solution of learning spurious features that accurately predict the domain label.
P(E E) +
One way to enforce the CDM constraint through adversarial training: discriminator tries to infer the source domain from the feature representation F(X) and F(X) tries to adjust itself to fool the discriminator. From this perspective, we can find that the CDM constraint exactly prevents F F(X) from making use of the domain labels to achieve high training accuracy (i.c., low R.).
With the idea of combining IRM and CDM, we can formulate the optimization problem as:
arg min 0p.0c dEEir
Σ E(a.g) (R (G(F(z)), v)]
(3)
s.t. Y IL F)F(()) - -
where Rs denotes the loss function in domain e (e.g., cross entropy loss). To impose the CDM con- straint, we aim minimize certain divergence between feature distributions from different domains, denoted as die(P(F(XX))) E = c)][P(((X))XX, =::== E')), +
We propose two choices of divergences, i.e., Maximum Mean Discrepancy (MMD) and Kullback- Leibler (KL) divergence (through adversarial training), which result in two algorithms: IRM-MMD and IRM-ACDM (i.c., IRM-Adversarial Conditional Distribution Matching).
IRM-MMD. In IRM-MMD, we adopt Maximum Mean Discrepancy (MMD) (Long et al. 2015
Tolstikhin al. 2017 Shalit et al. 2017) as the distribution divergence. The MMD between P and Q. two d-dimensional distributions of feature representations, can be defined as:
(M.Q(PPQ) sup Ez~plf -Ez-gl(()))
(4)
fEH

---

Under review as a conference paper at ICLR 2021
where Z E R R R. maps feature representation to real value, k R R + R denotes the characteristic kernel of f and H. is the RHKS of k. The MMD in Eq is not directly computable. So, we the unbiased estimator of MMD (Gretton et al. 2012a with N samples zi y... , z from P and M samples =i
Q from Q: 2M
N
1 MMDx(P.Q) N(N 1) Elee 2 of)
Σ
2 MN
Q k(z 13
M(M 1)
j=1
(5)
With MMD defined, we can define the loss function of IRM-MMD as:
arg min
Σ
FiRM + B
+BEED E
(MDD)PPECXX).,))PPUUCC ME DED'eEEin e'E
(6)
CEEir
where RM 1 EE (G(F((()) vf) all Vo/w=1. (G( denotes the IRM
Br
regularized loss (Arjovsky al. 2019) for domain hyperparameterss and / control the trade-off between the main loss, the IRM constraint and the CDM constraint.
P() e')) be the Kullback-eeinn (KL) divergence and use adversarial learning to estimate it (Tolstikhin et al. 2017 Song et al. 2020). More precisely, define the loss function of IRM-ACDM as:
IRM-ACDM. In IRM-ACDM. make div(P(((X))X,
arg arg min
Σ
CiRM +
3E E yEV BEES
(KL((P((X))) y, E e) ()) y)), (7)
0
eEEir
where 7/ == P(E= y). In adversarial learning, a conditional discriminator D Rd e,, with parameters On is introduced to predict the domain label of an instance, given its feature representation and class label. As proved in [i et al. (2018); Song et al. (2020), the estimation the KL divergence above can be reformulated the minimax game below:
min max OD
Σ FiRM + Σ DEY cE
Σ
1 (F(()) (F
(8)
where (F(())). = PP(F= e| F()) y) is the predicted probability of the instance for domain e by the discriminator D. In practice, we solve the minimax game above efficiently by the alternative gradient ascent/descent algorithm.
We evaluate IRM-MMD and IRM-ACDM on CMNIST+ show their ef- ficacy under strong A spuriousness. We let the output of the second last layer of LeNet-5 to be the feature representation F(X). For IRM-MMD, we use the multiple kernel MMD (MKMMD) Gret- ton et al. 2012b; Long et al. 5.. For IRM-ACMM, we use a two-layer MLP with |E outputs as the discriminator 1). To show that CDM alone cannot solve OOD prediction under strong A spuri- ousness, we set the weight of the IRM penalty a = 0 in IRM-MMID and IRM-ACDM to obtain the models regularized by MMD and ACDM, respectively. We consider EIIL (Creager et al. 2020) that solves a minimax game. In the max step, it learns soft domain labels for instances s.t. the IRMv1 loss is maximized. In the min step, it minimizes the IRMv1 loss. We also include ERM and oracle. The oracle uses the original LeNet-5 architecture with a single-channel CNN the first layer. It is trained and tested with instances transformed into grayscale.
Experimental Results.
Table. 3 shows the performance of IRM-MMD, IRM-ACDM and the baselines. Under strong A spuriousness, compared to IRM (Fig. 2). we can observe the significant and consistent performance improvement IRM resulting from combining CDM with IRM. IRM-MMD and IRM-ACDM also outperform MMD and ACDM. This verifies that CDM alone cannot solve the OOD prediction problem. EIIL reaches comparable accuracy IRM-MMD and IRM-ACDM when p = 0.85, 0.9. But EIIL has larger standard deviation 10.32%, +13.03%, +13.03% with 0.8, 0.85, 0.9, com- pared IRM-ACDM (#4.34% =4.65%, +3.09 +3.09%) and IRM-MMD (4..6%% +2.5%. +6.97%) Discussion can be found in Appendix C.3
5 RELATED WORK
OOD Prediction. IRM (Arjovsky et al. 2019) formulates causal feature learning as a constraint on the ERM framework (Vapnik 1992). which imposes the causal inductive bias: causal feature
7

---

Under review as a conference paper at ICLR 2021
Table 3: Test accuracy on CMNIST+: by combining CDM with IRM, performance is improved
significantly and consistently under strong A spuriousness (p 2 0.8) compared to IRM.
Method
0.8
0.85
0.9
IRM MMD (ours)
52.91%
40.83%
37,96 %
IRM-ACDM (ours)
57.23%
45.47%
42.85
ENL
43,40%
43.24%
40.93%
IRM
47.01%
37.81%
36.31%
MMD
23.04%
25.22%
24.22%
ACDM
30.41%
29.48%
25.53%
FRM
30.16%
27.83%
24.61%
Oracie
73.10%
73.49%
73.58%
representations lead to the existence of an optimal classifier for all domains. Then it is transformed to regularizer which be minimized along with empirical risks. [Chang et al.. (2020) propose a variant of IRM. They implement IRM by minimizing the difference between two classifiers' out- puts, which take F(X) and Concat (F (F)),, E) as the inputs, respectively. [Ahuja et al. (2020) reformulate the optimization problem of IRM from game theory aspect. IRM is related to robust optimization (Ben- Tal al. 2009). The goal Krueger et al.) (2020) extend robust optimization (Ben-Tai Tal et al., (009) to minimize the empirical risk of the worst domain and maximize that of other domains. They also propose to minimize the variance of domain specific empirical risks along with the empirical risk. Jin et al. (2020) propose to minimize the domain specific risk between two models, one trained on the same domain, the other trained on the other domains. These methods essentially minimize the differences among domain specific risks. Moreover, data augmentation can also improve the generalizability of DNNs from the data perspective (Isse et al.) 2020; van der Wilk et al. 2018). However, it requires prior knowledge on the differences between training and test domains, which are not be available in OOD prediction. Kuang et al. (2018) and Qiao al. (2020) aim handle the case with only one training domain. Zhang et al. a (202bb) propose to make DNNs more robust against test data generated by unseen inter- ventions. They model interventions in training data with generative model and perform test-time inference to catch unseen interventions. Compared to the existing work, this work exposes and fixes the issue of IRM in OOD prediction under strong A spuriousness.
optimize the worst domain specific empirical risk.
Domain Adaptation (DA) assumes that the unlabeled test set can be used during training and vali- dation. From the methodology aspect, distribution matching methods used in DA, such as gradient reversing (Ganin & Lempitsky 2015) and adversarial CDM (Long et al., 2018) are useful for OOD prediction. Peters et al. (2016) realize the invariance of causal relationships can be used for DA. Zhang et al. (2013) propose reweighting and kernel based distribution matching methods to handle three types of DA problems: target shift, conditional shift and generalized target shift. Gong et al. 2016) work on extracting transferable components F(X) that ensure P(F(X) IY) to be invariant across domains with location-scale transformation. Their method can identify how P(Y) changes across domains simultaneously. Different from DA, strictly ensure that the test domain is unseen during training and validation to reflect the scenario of OOD prediction in real-world applications.
6 CONCLUDING REMARKS
This work focuses on the OOD prediction problem under strong A spuriousness. Strong A spuri- ousness means the correlations between spurious features and the class label are strong. We find an important limitation of IRM in OOD prediction under strong A spuriousness: it can be satisfied by spurious feature representations that are predictive of the domain label. To verify it, we design the CMNIST+ dataset which has strong A spuriousness between color (spurious features) the class label through their common cause the domain variable. On CMNIST+, we observe the performance of IRM dramatically drops when the A spuriousness becomes stronger. Based on this observation, we propose simple but an effective fix to mitigate this issue of IRM. The proposed approach com- bines CDM and IRM because CDM can also be satisfied by causal feature representations. At the same time, CDM can prevent DNNs from picking up the aforementioned spurious feature repre- sentations. Experimental results CMNIST+ show significant performance improvement of the proposed method, demonstrating its effectiveness. Interesting future work includes (1) extension of the proposed method to OOD prediction tasks in complex data (e.g., graphs and time series) and (2) development of general causal inductive bias that can impose various conditional independence.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar. Invariant risk min- imization games. arXiv preprint arXiv:2002.04692, 2020.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint (tXXv:190700289000 2019.
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 456-473, 2018.
Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. Robust optimization, volume 28. Princeton University Press, 2009.
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi S Jaakkola. Invariant rationalization. In ICML, 2020.
Elliot Creager, Jörn- -örn-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant learning. In ICML Workshop Uncertainty and Robustness, 2020.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. pp. 1180-112 1189, 2015.
Mingming Gong, Kun Zhang, Tongliang Liu, Dacheng Tao, Clark Glymour, and Bernhard Schölkopf. Domain adaptation with conditional transferable components. International con- ference machine learning, pp. 2839-28222 2848, 2016.
Arthur Gretton, Karsten M Borgwardt, Malte Rasch. Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012a.
Arthur Gretton, Dino Sejdinovic, Heiko Strathmann, Sivaraman Balakrishnan, Massimiliano Pontil, Kenji Fukumizu, and Bharath K Sriperumbudur. Optimal kernel choice for large-scale two-sample tests. In Advances in neural information processing systems, pp. 1205-1213, 2012b.
Maximilian Ilse, Jakub M Tomezak, and Patrick Forré. Designing data augmentation for simulating interventions. arXiv preprini arXiv:2005.01 2020.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Domain extrapolation via regret minimization. arXiv preprint arXiv:2006.03908 2020.
David Krueger, Ethan Caballero, -eerik Jacobsen, Amy Zhang, Jonathan Binas, Remi Le Priol, and Aaron Courville. -ut-of-distribution generalization via risk extrapolation (rex). arXiv preprint arXiv:2003.00666 2020.
Kun Kuang, Peng Cui, Susan Athey, Ruoxuan Xiong, and Bo Li. Stable prediction across unknown environments. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1617-1626. 2018.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-232) 1998.
Ya Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 624 639, 2018.
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International conference on machine learning, pp. 97-105. PMLR, 2015.
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. Advances Neural Information Processing Systems, pp. 1640-1650, 2018.
Judea Pearl and Elias Bareinboim. External validity: From do-calculus to transportability across populations. Statistical Science, pp. 579-595, 2014.

---

Under review as a conference paper at ICLR 2021
Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant pre- diction: identification and confidence intervals. J. R. Stat. Soc. Series B Stat. Methodol., 78(5): 947-1012, 2016.
Fengchun Qiao, Long Zhao, and Xi Peng. Learning to learn single domain generalization. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12556- 12565, 2020.
Uri Shalit, Fredrik Johansson, and David Sontag. Estimating individual treatment effect: gener- alization bounds and algorithms. In International Conference on Machine Learning, pp. 3076- 3085. PMLR, 2017.
Yuhang Song, Wenbo Li, Lei Zhang, Jianwei Yang, Emre Kiciman, Hamid Palangi, Jianfeng Gao, C-C Jay Kuo, and Pengchuan Zhang. Novel human-object interaction detection via adversarial domain generalization. arXiv preprint arXiv:2005. 1406, 2020.
Ilya Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schoelkopf. Wasserstein auto- encoders. arXiv preprint arXiv:1711.0/55% 2017.
Mark van der Wilk, Matthias Bauer, ST John, and James Hensman. Learning invariances using the marginal likelihood. In Advances in Neural Information Processing Systems, pp. 9938-9948, 2018.
Vladimir Vapnik. Principles risk minimization for learning theory. In Advances in neural infor- mation processing systems, pp. 831-838, 1992.
Shuxi Zeng, Pengchuan Zhang, Denis Charles, Eren Manavoglu, and Emre Kiciman. Robust neural networks for causal invariant features extraction. NeurIPS 2019 Workshop, "Do the right thing": machine learning and causal inference for improved decision making, December 2019.
Amy Zhang, Clare Lyle, Shagun Sodhani, Angelos Filos, Marta Kwiatkowska, Joelle Pineau, Yarin Gal, and Doina Precup. Invariant causal prediction for block mdps. arXiv preprint arXiv:2003.06016. 2020a.
Cheng Zhang, Kun Zhang, and Yingzhen Li. A causal view on robustness of neural networks. arXiv preprint arXiv:2005.0100 2020b.
Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In International Conference on Machine Learning, pp. 819-827, 2013.
10

---

Under review as a conference paper at ICLR 2021
A IRM AND THE CONDITIONAL INDEPENDENCE
Under general conditions, the conditional independence IL F is a necessary condition for solutions of the original IRM optimization problem. In (Arjovsky al. 2019), IRM is defined as a two-stage optimization problem. Any solution F(X) to the IRM optimization problem must satisfy Y IL (X). However, some (XX) satisfying Y IL E|F(X) may not be solution to the original IRM problem. For example, generally, F(X) E would not minimize the sum of the domain- -omain-specific risk R However, under strong Á spuriousness, there exist solutions the original IRM problem that still pick up spurious features. Consider the extreme case, in the training data, if Y E, then ( = a solution to the problem.
B ANALYSIS FOR CMNIST+
Fitting Color for Classification. In expectation, the ERM model would learn color as the feature representation for classification. Here, we show the theoretical results for such cases. When the model learns color as the feature representation F(X), we have:
P(Y|C) = Σ P(YIC,
E
P(CIV, (CCY,E)P(VIE) P(EIC)
P(C
(9)
E
P(E)
P(CYYE))P(YI) E)
P(C)
E
The second equality is by Bayes' rule. We know that P(C) = Er
Then, given P(C|Y, E), EEY P(C) and Eq. 9 we can obtain P(Y[C) shown Table
These results imply that the deterministic classifier P(Y = 1 = G) 1, P(Y = 1 = B) = 1,
P(Y = 1|C = R) = 0 for all E [0.55, 0.9] as shown in Table 4 So, the accuracy of the deterministic classifier P!Y on the test set is 0.2. Its accuracy on the training set is Ec P( P(C) where P(C)= Ee Ey P(C|Y, ((C
Fitting Domain Label for Classification. Since the spurious correlation between the domain label and the class label is strong, and IRM cannot penalize models fitting the domain label, P(Y E) can help us understand the expected behavior of the IRM model. Note that the test data is from an unseen domain. So, we analyze the model that first predicts domain by color, then predicts class label by domain. First, analyze P(E(C) as below:
P(E = Σ P(ECCY))((IC)
P(CYY, PICIE.E)(CCI))PYYY)))
P(CIV)
(10)
(CC,E))((E)))PP)
P(C)
P(C|Y, P))(E))) P(C)
where the second and forth qualities are by Bayes' rule. With Eq. 10
list the values of
C(E(C) in Table 6 for E [0.55, 0.9). Thus, we know the deterministic domain prediction results
would be P(E 1 G) 1, P(E 1|C B) 0, and PE 1/C Rt) 0. Since
P(Y E) given in Table we can obtain the deterministic classifier's predictions as:
P 1|C = G) = P(Y = 1|C = B) = 0. P(Y = 1|C = R) = O.
(11)
So, the expected test accuracy of the model would be 0.35. Recall that the model first predicts domain label by color and then predict class label by the predicted domain. By doing this, it would have F(X) R F.. This makes it approximately satisfy the IRM constraint Y II E F(X) since
11

---

Under review as a conference paper at ICLR 2021
Y TT EJE. This implies that the model's performance can be treated as the expected performance of the IRM model in CMNIST+. In terms of the performance of P)Y the training set, can get the results using the same prediction rules as in Eq.
One may argue that the IRM model can perform well if balance the two classes in each domain. Here, we theoretically show this not the case. By setting P(Y|E) = 0.5, we can obtain the values of C) Table Note that practically this can be done by oversampling the minority class of each domain in each mini mini-batch. However, since P(Y () = 0.5, the predictions made by the deterministic classifier P(Y|E) would be just random guess, leading to a test accuracy of 0.5.
Fitting both Domain and Color for Classification. Here, we consider the model that first predicts the domain label by color and then predicts the class label by both the color and the predicted domain label. The first step is the same as the model fitting the domain label. For the second step, we analyze P(Y|C, C, F.) as below:
P(YIC, P(CIY, P(C
(12)
With P(C| Ey P(C|Y, BY and Eq. 12 we obtain values of C(Y|C, F)) as shown in
Table 8 So. given the predicted domains, P(E 1|C G) = 1, PE = 1|C B) = 0. and
P(F 1|C R)
the predictions the class label are P(Y 1 G, =
1,
2) P(Y = 1|C = R, E = 2) = So, the test accuracy is 0.35 when P S 0.8 and 0.2 when p > 0.8.
P(Y 1|C B,
p 0.8. P(Y 1/C B,
2)
p 0.8 and
Similarly, when we make PEY = 0.5 by oversampling the minority class in each domain, can obtain the predictions shown Table 2 So, the deterministic model would make predictions as P(Y = 1|C = G, E = 1) 1, P(Y 1|C = E, = 2) = 1, and P(YY 1|C = R. E) = O. This would lead to test accuracy of 0.2. These results reflect the reasons why the IRM model trained with the balanced classes in each domain (P) = 0.5) has worse performance compared its counterpart trained with the original CMNIST+ data.
Table 4: Analysis results: fitting color
P(Y
1
G)
P(Y
1
B)
P(Y
1 R.
0.55
0.697
0.534
0.29
0.6
0.737
0.545
0.25
0.65
0.775
0.56
0.212
0,811
0.577
0.176
0.8
0.88
0.63
0.111
0.85
0,912
0.67
0.081
0.9
0.942
0.73
0.053
Table Analysis results: fitting color, when P(Y F.)
0.5.
P(Y 1|C G)
P(Y 1|C B)
P(Y 1 R)
0.55
0.633
0.633
0.29
0.6
0.667
0.667
0.25
0.65
0.702
0.702
0.212
0.739
0.739
0.176
0.8
0,818
0.818
0.111
0.85
0.86
0.86
0.081
0.9
0.905
0.905
0.053
C EXPERIMENTAL SETUP AND RESULTS
Here, include details and discussion on experimental setup, datasets and results.
12

---

Under review as a conference paper at ICLR 2021
Table 6: Theoretical analysis results: predicting domain by color.
P(E = 1C
G)
P(E = 1|C
B)
P(E
1CC
R)
0.55
0.697
0.466
0.332
0.6
0.737
0.455
0.65
0.775
0.44
0.27
0.7
0.811
0.423
0.241
0.88
0.37
0.85
0.912
0.33
0.165
0.9
0.942
0.27
0.142
Table 7: Theoretical analysis results: predicting domain by color, when P(Y[E) = 0.5.
P
P(E = G)
P(E = 1|C B)
P(E
1 R)
0.55
0.633
0.367
0.5
0.6
0.667
0.333
0.5
0.65
0,702
0,298
0.5
0.7
0.739
0.261
0.5
0.8
0.818
0.182
0.5
0.85
0.86
0.
0.5
0.9
0.905
0.095
0.5
Table 8; Predicting by both color and domain
P
P(Y 1)
P(Y
1[G,2)
P(Y
1)
P(Y
I1
P(Y = I
P(Y 1|
0.55
0-957
011
09
0.214
0,786
D1043
0.65
0.971
011
0.9
0.202
0.708
DA09
0.7
0.977
0.31
0.639
00023
0.33
0.99
0.1
0.9
0357
0,463
0.01
Table 9: Predicting by both color and domain, class balanced
P
-.G2)
1)
P(Y
1.
1
0.38
0.71
0.5
0.
0.7
0.29
0.29
0.8
0.75
0.75
0.25
022
0,212
0.7
0.824
0.5
0.5
08824
0.176
0.136
0.85
0.19
0.5
03
0.19
0.811
0.081
0.9
0.47
0.5
0.5
0.947
0,033
0,53
C.1 EXPERIMENTAL SETUP
Here, we provide more details on experimental setup. We perform grid search for hyperparameter
tuning. For IRM, IRM-MMD and IRM-ACDM, search the iteration number to plug in the IRM penalty term (KFRM) in 200, 400, 600 and IRM penalty weight a in {1, 10, ...., 10 ). For MMD, ACDM, IRM-MMD, IRM-ACDM, For ACDM and IRM-ACDM, we set the number of steps we train the discriminator D in each iteration 10. For EIIL, we do the same hyperparameter tuning on the IRM penalty weight (a), the iteration to add IRM (K (K)MM) as we do for IRM, IRM-ACDM and IRM-MMD methods. We train the soft environment weight [E Y') for (10, 100, 1000, 10000) steps.
search CDM penalty weight / in (1, 10, ..., 105).
}-
C.2 DATASETS
Here, present more details about the datasets.
CMNIST. CMNIST is introduced by (Arjovsky et al. 2019). We can see the two training domains of CMNIST similar to each other in terms of both P(C E) and P(Y|E) in Table 10 This means CMNIST does not cover the case strong A spurious, since the spurious correlations, color-domain and domain-class, not strong.
13

---

Under review as a conference paper at ICLR 2021
Table 10: Description of CMNIST
F
P(Y = 1
Y
P(C'=
G
P(C
BJV, R)
P(C = RJY. F)
Y ==1
0.9
0.0
0.1
E =
0.5
==0
0.1
0.0
0.9
Y 1
0.8
0.0
0.2
F 2
0.5
Y =0 0
0.2
0.0
0.8
y=1
0.1
0.0
0.9
E ==3
0.5
0.9
0.0
0.1
E-1
CMNIST+ (p=0.5)
CMNIST+ ip=0.5 p=0.9)
CMNIST+ (p=0.9)
E=2
R
E
Class Y-1
B
Figure 5: We visualize CMNIST+ with p [0.5, 0.9) in terms of P(C|Y = 1, E). Each large triangle represents the space of P(C E). Each small triangle shows the values of P(C = c|Y 1, F. e), c E (R. G, B), (1,2,3).
We visualize the CMNIST+ dataset with different values of p in Fig. 5 In addition, we provide a detailed simulation recipe of CMNIST+ and compare it with that of CMNIST. This would also show that CMNIST+ accordance with the causal graph in Fig. [
CMNIST+.
1. We decide the true label Y (without noise) of each instance by its original digit label (0-9).
2. We randomly split the data into test and training.
3. We assign the training instances to the two training domains based on the true label Y* and P(Y|E) in Table 1. This step introduces correlations between and E. In each training environment, we further randomly split the data into training and validation.
4. In each environment, we generate noisy labels Y by randomly flipping them with 25% probability. This means Y Y.
5. Given P(C|Y, E) in Table 1 the noisy label and the domain label, we assign color each instance. This step introduces correlations among X2, Y and E.
Note that there is a difference what causal relationships mean in traditional causal inference and in OOD prediction. In OOD prediction, the definition of causal relationships is different from traditional one. Traditionally. X + Y means the generation of Y is (partially) determined by X does not necessarily mean P(Y]X*) remains the same across domains (Pearl & Bareinboim, 2014). However, in OOD prediction, we say there exists a causal relationship X ->> Y iff P is the same across different domains. We also know in the original MNIST dataset, there exists invariant causal relationship X + Y*. This implies that, from the data generating process of CMNIST+, confirm that (1) there exist causal relationships X + Y + Y. (2) there are correlations among X Y and E. With these two conclusions, we can claim that the causal graph in Fig. is in accordance with CMNIST+ in the OOD prediction problem where the true label Y ignored as is not used in training and evaluation.
Creating Strong A Spuriousness with Two Colors. It is possible to setup strong A spuriousness with two colors for binary classification with two training domains. Here, we use the two colors: red (R) and green (G). To show it is possible, we use an example with the following setup: P(Y =
1 E = 1) 0.9, P(Y = 1|E = 2) 0.1. Let's say for E 1, we set P(C G/Y E,
0.1, P(C = GJY E 2) 0.9 for E = 2. This setup makes strong correlations among the color, the class label and the domain label. Thus, it is possible to create strong spuriousness with just two colors. Our concern with such datasets is that even if strong A spuriousness exists in training
1) = 0.9. G = 0,
= 0.1. Then, we can set P(C = G = 1,
14

---

Under review as a conference paper at ICLR 2021
domains, it is challenge to create test domains that are diverse enough from the training ones. Following the aforementioned setup, for the test domain E = 3. if we set P(Y = = 3) = 0.5, P(C G| 1, E 3) 0.5 and P(C G|Y 0, E 3) 0.5, it would be right in the middle of the two training domains. Unfortunately in this setting, even if IRM fails, it can be difficult observe it with the accuracy. This is because model perfectly fits the color features can reach 0.5 test accuracy. This leads to smaller differences between models fitting causal features and those with spurious features. Thus, becomes more challenging to judge whether model fails in practice, which explains why we use three colors in CMNIST+.
C.3 MORE DISCUSSION ON RESULTS OF EIIL
We evaluate EIIL on CMNIST+. EIIL gets test accuracy 43.40 10.32% 43.24 4 13.03% on CMNIST+ with p 0.8, 0.85, 0.9. Compared to IRM-ACDM 57.23 = 4.34%, 44.83 + 4.65%, 42.85 + 3.09% and IRM-MMD 52.91 + 4.56%, 40.83 + 2.50%, 37.96 = 6.97%, ElIL has comparable mean accuracies when & 0.85, 0.9, but is not stable. There two reasons. (1) EIIL relies on IRM and the soft domain weight q( E X, Y). When q( gEE Y) takes values s.t. strong A spuriousness exists, it makes IRM fail. (2) It has issues in model selection. ElIl can only select models by validation accuracy as ( X, Y) is a scalar qi for instance i. It is learned for the val/test, so validation loss
feasible. Validation accuracy can be high when the model picks up spurious features. While model selection with validation loss considers the regularizers (IRM and CDM), which approximates how well models fit causal features.
15


Reviews:
[{"review_id": "WG0F978s1V", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary.** This paper advances generalizable machine learning via addressing a major limitation of invariant risk minimization (IRM). In particular, the author(s) identified and discussed the issue of strong $\\Lambda$ spurious, where spurious features and class labels are strongly correlated due to common cause, causing unprotected IRM to fail while trying to exclude such non-causal predictors. To avoid this. pitfall, the author(s) proposed to leverage conditional distribution matching (CDM) to regularize the representation, which effectively helps to alleviate this issue. Two empirical solutions, respectively non-adversarial MMD and adversarial KL matching, have been presented and validated. \n\n**Quality & Clarity.** Overall this paper is presented with clarity. The problem is well motivated and carefully discussed. What I found unsatisfactory is the proposed solution needs extra justifications, which is detailed in my weakness section below. \n\n**Originality & Significance.** The author(s) have identified a major weakness of IRM that I also find concerning: while developed from the notation of invariant representations, based on which invariant predictors are defined, IRM does not explicitly regularize the representation in its formulation. This view, provided fully developed, encapsulates sufficient novelties. On the significance side, while this submission indeed addresses a major concern of IRM demonstrated by artificial examples, the author(s) fail to present a concrete real-world example to showcase this concern is a thing that we should actually worry about.  \n\n**Main Weakness.**\nJustification of CDM needs to be strengthened. The author(s) have provided an argument that explains the extreme case, where non-causal features perfectly predict domain. The discussion needs to be substantially enriched. CDM has been proposed for dealing with the label shift in domain adaption, and it relies on assumptions that should be reconciled with those made by IRM. Please clarify. \n\nTheoretical results on pp 4. This is a totally misleading heading. What I have expected is some theoretical discussion, instead, the author(s) have provided a numerical table computed from \"theoretical computations\". I do not consider these as theoretical as they do not generalize beyond this particular example. \n\nInsufficient experimental validation. This is what kills this paper. There is only one experiment performed on a semi-synthetic testbed, which does not serve to evidence the practical utility of this proposal. \n\nThe domain adversarial neural net (DANN) model is highly relevant to the proposal made here and should be carefully discussed and compared. In fact, DANN also regularizes the representation to make it domain-agnostic. \n\n**Minor issues.**\nThe term out-of-distribution (OOD) is a bit misleading, as this phrase is usually associated with the task of anomaly detection, where novel samples that are very different from the training examples are identified.  I would suggest the author(s) replace OOD to avoid confusion. \n\nThe aspect ratio in Fig 1 is off and it. makes readers very uncomfortable. Please redo this figure. And it does not clearly depict what's different compared to the standard scenarios amendable to IRM.", "extra_scores": {}, "extra_text": {}}, {"review_id": "LW-im5tzPuu", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper attacks the problem of OOD learning from the angle of invariant causal feature learning. The key idea is to capture domain invariant causal features and use the extracted causality relation to convey domain-adaptive classification. In this work, domain invariant causal features are learned by IRM, which imposes the consistency constraint between causal features and class labels across different domains. The core idea is to address the existence of spuriousness correlation by introducing the MMD and KL divergence based conditional distribution matching constraint to the IRM learning process. The experimental study based on a crafted MNIST data set demonstrates the superior performances of the regularised IRM learning method in the domain invariant learning task. \n\nIn general, the paper introduces an in-depth discussion of the limitation of the IRM learning mechanism and points out the root cause of failure of IRM (spuriousness correlation). This is interesting and potentially impactful for practical OOD learning tasks. The paper is well-written and the proposed objective is novel to my knowledge. we tend to accept the paper. \n\nStill, our concerns are as follows:\n1. Though the results look promising on the toy data set, it would be better to have a real-world scenario as a testbed for the proposed method. Domain transfer is a popular application. How would this method perform in a domain transfer learning task? \n2. Following the first question, we would expect some discussion about the relation between the proposed method and other transfer learning methods, such as meta-learning methods. Could domain invariant casual feature learning be considered as a way of conducting meta-learning? \n3. A minor issue in Table.1: how many domain labels are there defined in the CMNIST data set? How are they defined? \n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2J3rKotCOx", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary of Paper\n\nThis paper identifies and tries to fix a limitation of the recent work of Invariant Risk Minimization (Arjovsky et al., '19). IRM is a solution framework for the OoD prediction problem, where one has to learn a classifier based on data from multiple domains, hoping to generalize to unseen domains. \n\n1. This paper extends the Colored-MNIST dataset (where MNIST images are spurious colored according to the label) to a new and more difficult dataset called CMNIST+ where they empirically show that IRM fails to generalize to the new domain. The key idea behind the construction of the CMNIST+ dataset is the introduction of a correlation between the spurious features and the _domain_ label during training time (besides the training-time correlation between the spurious features and the class label that is already there in Colored-MNIST). This spurious-feature-domain-label correlation can be introduced by making Domain 1 largely contain points of class 1 and Domain 2 largely contain class 2. They call this sort of correlation as \"$\\Lambda$-spuriousness\". \n\n2. The paper provides an intuitive argument for why IRM does not work on this dataset: the IRM constraint is equivalent to \"class label independent of domain label given feature representation\", such a constraint does not preclude the classifier from learning \"feature representation = domain label\". Such a classifier however would not generalize well to an unseen domain where the domain label is not correlated with the class label. \n\n3. Finally, the paper proposes a fix for IRM which essentially adds a \"conditional distribution matching\" constraint to the IRM constraint. This constraint forces the distribution of the feature representation for any class label to be invariant across domains. By implementing this through (a) an MMD approach and (b) an adversarial learning approach, they show a 10% improvement in OoD accuracy on CMNIST+.\n\n### Strengths\n\n1. The paper is strong on novelty: the problem identified, the explanation provided, the dataset proposed, and the solution proposed are all novel. Overall, the paper piqued my curiosity and I enjoyed reading it.\n\n2. The problem of OoD prediction is practically important. Furthermore, this paper exposes the limitations of an existing solution under a very natural kind of spurious correlation that could occur in real life (i.e., domain-class correlation). \n\n3. The paper also provides a synthetic dataset that will be quite valuable to future work that aims to develop better OoD prediction algorithms. \n\n4. The \"failure\" dataset and the solution proposed are all founded on a solid, intuitive argument. \n\n\n### Weaknesses + important clarification questions\n\n5. I think the paper will benefit greatly from at least one other empirical example both in terms of showing failure of IRM and in terms of showing that IRM-MMD/ACDM improves. This could either be on a synthetic dataset similar to (but not) CMNIST+  (maybe MNIST but with some other spurious feature; or maybe some other dataset with the same spurious feature). Even better, it'd be great (but not absolutely necessary) to verify the performance of IRM-MMD/ACDM on a practical benchmark. \n\n\n6.  I was confused about the definition of $\\Lambda$-spuriousness. The introduction defines this to be the existence of a correlation between label and color during training. Isn't this the same correlation that exists in CMNIST? Is this a typo? I suspect that the $\\Lambda$-spuriousness refers to strong correlation between the class label, the domain label and the spurious features. Or did I completely misunderstand this?\n\n7. Could you explain why you had to resort to using three channels/colors as against just two like in CMNIST? I wonder if this is the point that was addressed by the following line in the paper:\n> The Colored MNIST (CMNIST) dataset cannot expose the limitation of IRM under strong Λ spuriousness. This is because its two training domains are quite similar.\n\n**Update:**   You provided an example 2-color dataset to argue why you can't really see whether IRM fails in the test domain. However, this example doesn't seem to be the correct analog of the example 3-color dataset in your paper? In your 2-color example, domain 1 is mostly Y=1, and most of those datapoints are colored G. Domain 2 is mostly Y=0 and most of those datapoints are colored G too. But the analog of the 3-color dataset would be one where in domain 2 most datapoints have Y=0 and those datapoints are mostly colored B (or R, but not G). Then, consider a test domain where you've an equal proportion of Y=0 and Y=1, but all Y=0 are colored G and Y=1 are colored by  B.  It seems like under this case, if IRM were to use the color, it would have a poorer test accuracy than 0.5. Perhaps I'm missing something here. Nevertheless, it seems like considering a dataset like this, and/or fleshing out a corresponding Table 1 for such a dataset would be critical to substantiate this argument. \n\n\n\n \n\n8. The argument that label balancing does not fix IRM's issue is a crucial argument to justify the fix given here. But I would have appreciated a bit more elaboration on this. The text says \"Theoretical analysis shows that this [label balancing] is an invalid solution\". Could you explain how I can infer this from Table 2? Which column here would correspond to the performance of IRM under label balancing, and why?\n\n\n### Overall opinion\nThe paper identifies a novel gap in an existing algorithm for an important problem, provides an intuitive explanation as to why that gap exists, and also proceeds to provide a reasonable, intuitively-grounded fix for it. Overall, this makes a complete, coherent paper worth publishing.\n\n#### Minor suggestions\n- For completeness, it would be nice to provide a concrete argument and discussion (in the appendix) for the connection between the original IRM constraint and the constraint ``\"Y \\condindep E | F(X)\"\n- In Page 3, there is a footnote against the symbol \"E\" which might be better positioned elsewhere.\n- In Fig 2 (c), $K_{IRM} = 0$ could be misleading (since it can be interpreted as adding the constraint right from the first step). Perhaps $K_{IRM} = -1$ or $K_{IRM} = \\infty$ would be more appropriate.\n\n\n#### References\n\nMartin Arjovsky, Le ́on Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization.\narXiv preprint arXiv:1907.02893, 2019.\n\n\n**Update**: Thanks to the authors for clarifying most of my clarification questions.  I'm not sure I was able to fully follow your argument about why these results can't be adapted to a two-color dataset (see above), but to indicate that you've clarified many of my questions I've increased my confidence score to a 4.  Good luck to the authors.\n\n\n**Further updates**: I'd like to elaborate on my thoughts a bit more with the hope that the authors may find it useful for future versions of the paper. \n\n- First, I really appreciate the authors for performing additional experiments with EIIL during the response phase. I wish to emphasize that, after a long discussion with R3, I've some strong disagreements with their review regarding the \"simple fix\":\n   -   I personally think EIIL is out-of-scope as it is a recent algorithm. It doesn't sound like a \"simple fix\" to me. However, it's great that the authors were able to show that their algorithm works better. \n   -  I don't think pooling all datapoints and splitting it will work. You'll end up with a dataset with label-color correlation in both domains, and both domains will be identical. So IRM won't work here.\n  - One can always come up with some hacks like \"pool all environments and carefully split them back\" that work under the assumption that there's $\\Lambda$-spurious correlation. But those hacks would be sub-optimal if there were no $\\Lambda$-spurious correlation.  Therefore, this is an unfairly powerful \"overfit\" hack, and does not make a good baseline. You want an elegant solution that works whether or not there's $\\Lambda$-spurious correlation as you won't know whether that sort of a spurious correlation exists in practice.\n  - As a side note, in light of the above point, I think it's important that the authors also demonstrate that the CDM constraint added preserves the performance of IRM on the original CMNIST dataset. \n  - Hopefully the authors can keep the EIIL results for future versions of the paper as it only makes the paper stronger.\n\n- I don't think the paper should be heavily penalized for the lack of a realistic dataset, because it's hard to verify $\\Lambda$ spuriousness on realistic benchmarks. However:\n  - I'd strongly encourage that you consider trying similar experiments on a dataset like say Rotated-MNIST (or Rotated-MNIST+ to be more precise, if at all possible). \n  - Even better, you could consider whether similar experiments can be done with Celeb-A where you have access to image attributes like hair color etc., (See Fig 2 https://arxiv.org/abs/2005.04345) and you could try creating different environments by sampling differently in each.\n  - If you think that's it's impossible to create a $\\Lambda$-spurious dataset, you might want to explain in future versions of the paper as to why that's not possible.\n\n- I understand R3's main concern which is that the algorithm in this paper requires that the distribution of the causal features $X_{causal} | Y$ to be the similar across all environments. It seems like IRM doesn't expect this sort of invariance, while algorithms prior to IRM do require something of this sort (including CDM, DANN etc.,). I think one actionable way to address this concern would be to show that there are datasets where IRM + CDM does better than CDM (just like how IRM+CDM does better than IRM in CMNIST+). This way we can see why combining IRM and CDM offers something unique.\n\n\n- Finally, I want to appreciate your efforts in trying to clarify all the reviewers' concerns (at least I found them helpful) and also to update the paper accordingly, add experiments etc., ", "extra_scores": {}, "extra_text": {}}, {"review_id": "-CseCzHzPB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nIn this work, the authors focus on the out-of-distribution generalization problem. The input is dataset from multiple environments and the goal is to learn a model that generalizes well to an unseen test environment. The work is based on recent line of works on invariant risk minimization (IRM) (Arjovsky et al.). The authors show that under a certain type of structure for the generative model, where the domain/environment label itself has a strong correlation with the spurious factors and the target label, IRM fails. The authors propose an extension of the colored MNIST dataset to highlight this problem. Finally, the authors build a method that works better than IRM on the extension of colored MNIST dataset. \n\nPros:\n I appreciate the authors have tried to highlight how the IRM directly applied to datasets from multiple environments will not always work and one has to be careful about the environment induced correlations themselves. \n\nCons: \nI divide my concerns into different subsections below. \n\na)\tIncorrect connection between IRM (Arjovsky et al.) and conditional independence made by the authors: \n\nConditional independence (Y \\perp E | F(X)) is a necessary condition but not sufficient for the theory of IRM to work.  Therefore, analyzing any F that satisfies this property is not sufficient. \nSuppose we have two training environments, E=1 and E=2.  Asssume that we are only interested in binary classification for now. The main condition that is assumed in Arjovsky et al. (Page 9 Definition 7) for the success of IRM is that there exists a representation F*(X) such that \n\nP(Y|F*(X),E=0) = P(Y|F*(X), E=1) = P(Y|F*(X))\n\nThe above condition implies that Y \\perp E | F*(X). In the above expression equating conditionals, there is already an assumption made about F*(X), which is that for both environments E=0 and E=1, the support of F*(X)|E=0 and support of F*(X)|E=1 are equal. Suppose the supports were not equal, then the conditionals can only be equated over the intersection of the supports.  \nIn the driving example used in the paper, i.e., F*(X)=E, the support of the two conditionals F*(X)|E=0 -->E=0 and F*(X)|E=1-->E=1 do not intersect.  Therefore, what authors claim is a problem with IRM is not really a problem but a data generating environment for which the theory of IRM is not guaranteed to be successful. The right claim to make is that CMNIST+ does not satisfy the assumptions IRM makes for the method to be successful. However, this is easy to fix as I explain soon. \n\nBefore moving to the next section, I would like to also make another important remark. The authors investigate any predictor that satisfies the conditional independence condition. This is also not correct because IRM and other IRM based methods select one of the invariant predictors and not all (thus there can always be bad invariant predictors, which does not mean that they will be selected). For a complete characterization of invariant predictors in terms of conditional independences please refer to Koyama et al.\n\nUpdate post discussions: This point a) was corrected by the authors.\n\nb)   Why does IRM not work on the CMNIST+?\n\nWe now turn to providing the explanation why IRM did not work on CMNIST+ as the explanation provided by the authors is not accurate. As we explained in the last section, F(X)=E is a representation that does not satisfy the criterion that the theory of IRM requires. One intuitive way to think is that the success of IRM assumes that representation F that we search over have an overlapping support across the environments. The IRM optimization procedure fails because it does not enforce this assumption in any way and F(X)=E can lead to a better predictor than F(X)=S, where S is the true causal feature. \nIn CMNIST+, the authors have created two environments, where the environment label itself is strongly correlated with the label. Say in E=0, the majority of the labels are 0, and in E=1 the majority of the labels are 1.  Suppose the IRM optimization (Arjovsky et al.) is given a representation F(X)=E as input. The support of F(X) gets partitioned into two disjoint sets X0 = F^{-1}(0) and X1 = F^{-1}(1). E=0 learns a predictor over the set X0 and E=1 learns a predictor over the set X1. A predictor that labels all points in X0 as 0 and X1 as 1 actually satisfies the definition of invariant predictor because it simultaneously minimizes the error in the two environments. If the error of this predictor is actually less than the error of the predictor based on causal features, then this predictor can be selected by the IRM optimization, which is exactly the case in the CMNIST+ dataset. \n\nThe main reason IRM was designed was to make the predictors from different environments be compared when the sets X0 and X1 overlap to some extent at least, i.e. the image of F over the feature distributions in the two environments has to overlap. \nIf the image of F over feature distribution does not overlap at all (as is the case in example considered in the paper), a trivial invariant predictor which is not robust to distribution shifts will exist.\n\n\nc)\tThere is a simple alternate fix for the entire problem:\nThe space of problems that authors want to fix abstractly stated are when the environment label (domain label) itself is so strongly correlated with the label that the IRM is encouraged to use environment as a representation.\nThe fix goes as follows:  Mix the data from the two environments. Take the mixed data and divide into two completely new environments. A manual way to construct these new environments is to divide the data in such a way that the proportion of the colors in the two environments marginally different as was the case in colored MNIST. For an algorithmic approach  to construct these new environments  use this approach (paper http://www.gatsby.ucl.ac.uk/~balaji/udl2020/accepted-papers/UDL2020-paper-045.pdf) (code available at https://github.com/ecreager/eiil). The above paper shows that a single dataset can be divided into multiple environments and retains the gains shown in IRM. \nThe two new environments obtained from the algorithm in the above paper or even through a manual division as explained above would lead to a dataset that is very similar to the original CMNIST. Observe that by mixing and creating two new environments, we are automatically ensuring that the support overlap assumptions required by IRM are satisfied.\nI believe that these approaches can bring the performance back to 68 percent level. \n\nAlso, note that the fix I am proposing is not the simple fix based on label balancing that the authors show does not work. By mixing the environments, we are destroying the spurious environment based correlation that exist. \n\nTherefore, whenever there is a strong environment based spurious correlation, i.e., each environment has a stark difference in the marginal distribution of labels, then the prudent thing to do is to mix the environments destroy the spurious correlation and then construct environments for IRM either manually or through the algorithm I shared above. \n\nUpdate post discussions: The authors did more experiments to show their method works on CMNIST+ better than these baselines. However, I have major concerns with the principle proposed by the authors as a search criterion. I believe the authors approach happens to work on CMNIST+ but is not based on the right principle. More on this in my point d) below.\n\nd)\tThe fix proposed by authors has theoretical problems:\nThe authors have proposed to do conditional distribution matching which constraints that the representation learned has to be independent of the environment conditional on the label. For the sake of discussion, let us consider the structural equation model (SEM) used in Theorem 9 in Arjovsky et al. If we assume that the lambda-spuriousness condition holds for the SEM under consideration. Say for the two environments, the support of the feature distributions do not intersect, the support of the label distributions do not intersect. In this case of lambda-spuriousness, IRM continues to be able to recover the ideal invariant predictor and does not fail (as no assumption in Theorem 9 is violated). However, the representation used by the ideal predictor does not have to satisfy the CDM condition stated by the authors. Therefore, the CDM condition proposed by the authors holds in the extreme case of the example discussed by the authors but does not hold in general. \n\nIn other words, author should show how imposing CDM works in a broad range of settings and not just one example that they use. Based on my argument I above, I highly doubt that CDM condition is actually a necessary condition for the success of IRM under lambda-spuriousness.  \n\ne)\tColored MNIST + issues: \nThe authors never explained in detail how the data in the different environments is generated. The Table 1 is an incomplete definition. It only lays down the conditional distributions. Conditional distributions are not sufficient to decipher the underlying structural equation model.  What authors call lambda-spuriousness is not really a lambda structure. In CMNIST+, there has to be an arrow from the label Y to the spurious features Xs as well, which does not appear in lambda structure. \n\nUpdate based on discussion: The authors explained the data generation process. \n\nf)\tA simple comparison with balance in generation time:\nWhy did the authors not compare their method when P(Y=1|E)=0.5 in both environments. It seems for high values of rho and P(Y=1|E)=0.5 a direct application of IRM does not work.  My suspicion is when  P(Y=1|E)=0.5, IRM does not work because we need more than two training environments, as the number of colors that enter the equation of spurious correlations go from two to three. \n\ng) Comparison with Koyama et al.: Koyama et al. had worked with a lambda type structure and introduced an extension of colored MNIST where environment label plays an important role. A comparison with that work would have been useful.\n\nUpdate based on discussion: The authors clarified that this paper was posted on arxiv on Aug 4 and the ICLR policy requires them to compare only until Aug 2. I am ok with authors not including a comparison with this paper.\n\nQuality: Unfortunately, the paper is not good quality. A lot more work is needed to really justify why what they propose is really a problem with IRM and why the simple fixes I propose won't already solve the problem. \n\nSignificance: The area of OoD generalization is quite significant. However, the problem proposed by the authors and the approach taken by the authors is not of much significance. \n\nOriginality: The authors have proposed a new CMNIST dataset and a new algorithm to fix it. The authors should get credit for proposing the dataset but besides that I don't think the algorithm proposed as a fix is needed. \n\n\n**Final update:** The main criterion used by the authors to search invariant predictors is not correct and is in fact not satisfied by the invariant predictors. For this my suggestion to authors is to modify their criterion in a way that it is at least satisfied by the ideal model you want to learn.\n\nReferences:\n\nKoyama et al.   \"Out-of-distribution generalization with maximal invariant predictor.\" arXiv preprint arXiv:2008.01883 (2020).\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Loosely, while IRM aims to find a feature mapping Phi s.t. response Y given Phi(X) is independent of the environment variables E, they suggest that when E is strongly correlated with Y, then it is possible for Phi obtained via IRM to involve environment variables. They motivate this by suggesting that if there exists a feature mapping Phi(X) = E, it would satisfy the IRM aim, but that this is undesirable.

They suggest instead requiring Phi(X)|Y being invariant to the environment.

The reviewers bring up a couple of concerns. The first is that it is not clear outside some simple examples when Y given Phi(X) being independent of E does not suffice. The second is that the authors also do not empirically validate their fix outside a single simple dataset. Moreover, what are the pitfalls of having Phi(X) given Y being independent of E?

Overall, this is an interesting kernel of an idea; it just needs to be fleshed out a bit more.

Decision: Reject