Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: **Summary.** This paper advances generalizable machine learning via addressing a major limitation of invariant risk minimization (IRM). In particular, the author(s) identified and discussed the issue of strong $\Lambda$ spurious, where spurious features and class labels are strongly correlated due to common cause, causing unprotected IRM to fail while trying to exclude such non-causal predictors. To avoid this. pitfall, the author(s) proposed to leverage conditional distribution matching (CDM) to regularize the representation, which effectively helps to alleviate this issue. Two empirical solutions, respectively non-adversarial MMD and adversarial KL matching, have been presented and validated. 

**Quality & Clarity.** Overall this paper is presented with clarity. The problem is well motivated and carefully discussed. What I found unsatisfactory is the proposed solution needs extra justifications, which is detailed in my weakness section below. 

**Originality & Significance.** The author(s) have identified a major weakness of IRM that I also find concerning: while developed from the notation of invariant representations, based on which invariant predictors are defined, IRM does not explicitly regularize the representation in its formulation. This view, provided fully developed, encapsulates sufficient novelties. On the significance side, while this submission indeed addresses a major concern of IRM demonstrated by artificial examples, the author(s) fail to present a concrete real-world example to showcase this concern is a thing that we should actually worry about.  

**Main Weakness.**
Justification of CDM needs to be strengthened. The author(s) have provided an argument that explains the extreme case, where non-causal features perfectly predict domain. The discussion needs to be substantially enriched. CDM has been proposed for dealing with the label shift in domain adaption, and it relies on assumptions that should be reconciled with those made by IRM. Please clarify. 

Theoretical results on pp 4. This is a totally misleading heading. What I have expected is some theoretical discussion, instead, the author(s) have provided a numerical table computed from "theoretical computations". I do not consider these as theoretical as they do not generalize beyond this particular example. 

Insufficient experimental validation. This is what kills this paper. There is only one experiment performed on a semi-synthetic testbed, which does not serve to evidence the practical utility of this proposal. 

The domain adversarial neural net (DANN) model is highly relevant to the proposal made here and should be carefully discussed and compared. In fact, DANN also regularizes the representation to make it domain-agnostic. 

**Minor issues.**
The term out-of-distribution (OOD) is a bit misleading, as this phrase is usually associated with the task of anomaly detection, where novel samples that are very different from the training examples are identified.  I would suggest the author(s) replace OOD to avoid confusion. 

The aspect ratio in Fig 1 is off and it. makes readers very uncomfortable. Please redo this figure. And it does not clearly depict what's different compared to the standard scenarios amendable to IRM.
review_title: Nice touch, but justifications need to be substantially enriched

final_score: rating: 7: Good paper, accept
review_content: ### Summary of Paper

This paper identifies and tries to fix a limitation of the recent work of Invariant Risk Minimization (Arjovsky et al., '19). IRM is a solution framework for the OoD prediction problem, where one has to learn a classifier based on data from multiple domains, hoping to generalize to unseen domains. 

1. This paper extends the Colored-MNIST dataset (where MNIST images are spurious colored according to the label) to a new and more difficult dataset called CMNIST+ where they empirically show that IRM fails to generalize to the new domain. The key idea behind the construction of the CMNIST+ dataset is the introduction of a correlation between the spurious features and the _domain_ label during training time (besides the training-time correlation between the spurious features and the class label that is already there in Colored-MNIST). This spurious-feature-domain-label correlation can be introduced by making Domain 1 largely contain points of class 1 and Domain 2 largely contain class 2. They call this sort of correlation as "$\Lambda$-spuriousness". 

2. The paper provides an intuitive argument for why IRM does not work on this dataset: the IRM constraint is equivalent to "class label independent of domain label given feature representation", such a constraint does not preclude the classifier from learning "feature representation = domain label". Such a classifier however would not generalize well to an unseen domain where the domain label is not correlated with the class label. 

3. Finally, the paper proposes a fix for IRM which essentially adds a "conditional distribution matching" constraint to the IRM constraint. This constraint forces the distribution of the feature representation for any class label to be invariant across domains. By implementing this through (a) an MMD approach and (b) an adversarial learning approach, they show a 10% improvement in OoD accuracy on CMNIST+.

### Strengths

1. The paper is strong on novelty: the problem identified, the explanation provided, the dataset proposed, and the solution proposed are all novel. Overall, the paper piqued my curiosity and I enjoyed reading it.

2. The problem of OoD prediction is practically important. Furthermore, this paper exposes the limitations of an existing solution under a very natural kind of spurious correlation that could occur in real life (i.e., domain-class correlation). 

3. The paper also provides a synthetic dataset that will be quite valuable to future work that aims to develop better OoD prediction algorithms. 

4. The "failure" dataset and the solution proposed are all founded on a solid, intuitive argument. 


### Weaknesses + important clarification questions

5. I think the paper will benefit greatly from at least one other empirical example both in terms of showing failure of IRM and in terms of showing that IRM-MMD/ACDM improves. This could either be on a synthetic dataset similar to (but not) CMNIST+  (maybe MNIST but with some other spurious feature; or maybe some other dataset with the same spurious feature). Even better, it'd be great (but not absolutely necessary) to verify the performance of IRM-MMD/ACDM on a practical benchmark. 


6.  I was confused about the definition of $\Lambda$-spuriousness. The introduction defines this to be the existence of a correlation between label and color during training. Isn't this the same correlation that exists in CMNIST? Is this a typo? I suspect that the $\Lambda$-spuriousness refers to strong correlation between the class label, the domain label and the spurious features. Or did I completely misunderstand this?

7. Could you explain why you had to resort to using three channels/colors as against just two like in CMNIST? I wonder if this is the point that was addressed by the following line in the paper:
> The Colored MNIST (CMNIST) dataset cannot expose the limitation of IRM under strong Λ spuriousness. This is because its two training domains are quite similar.

**Update:**   You provided an example 2-color dataset to argue why you can't really see whether IRM fails in the test domain. However, this example doesn't seem to be the correct analog of the example 3-color dataset in your paper? In your 2-color example, domain 1 is mostly Y=1, and most of those datapoints are colored G. Domain 2 is mostly Y=0 and most of those datapoints are colored G too. But the analog of the 3-color dataset would be one where in domain 2 most datapoints have Y=0 and those datapoints are mostly colored B (or R, but not G). Then, consider a test domain where you've an equal proportion of Y=0 and Y=1, but all Y=0 are colored G and Y=1 are colored by  B.  It seems like under this case, if IRM were to use the color, it would have a poorer test accuracy than 0.5. Perhaps I'm missing something here. Nevertheless, it seems like considering a dataset like this, and/or fleshing out a corresponding Table 1 for such a dataset would be critical to substantiate this argument. 



 

8. The argument that label balancing does not fix IRM's issue is a crucial argument to justify the fix given here. But I would have appreciated a bit more elaboration on this. The text says "Theoretical analysis shows that this [label balancing] is an invalid solution". Could you explain how I can infer this from Table 2? Which column here would correspond to the performance of IRM under label balancing, and why?


### Overall opinion
The paper identifies a novel gap in an existing algorithm for an important problem, provides an intuitive explanation as to why that gap exists, and also proceeds to provide a reasonable, intuitively-grounded fix for it. Overall, this makes a complete, coherent paper worth publishing.

#### Minor suggestions
- For completeness, it would be nice to provide a concrete argument and discussion (in the appendix) for the connection between the original IRM constraint and the constraint ``"Y \condindep E | F(X)"
- In Page 3, there is a footnote against the symbol "E" which might be better positioned elsewhere.
- In Fig 2 (c), $K_{IRM} = 0$ could be misleading (since it can be interpreted as adding the constraint right from the first step). Perhaps $K_{IRM} = -1$ or $K_{IRM} = \infty$ would be more appropriate.


#### References

Martin Arjovsky, Le ́on Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization.
arXiv preprint arXiv:1907.02893, 2019.


**Update**: Thanks to the authors for clarifying most of my clarification questions.  I'm not sure I was able to fully follow your argument about why these results can't be adapted to a two-color dataset (see above), but to indicate that you've clarified many of my questions I've increased my confidence score to a 4.  Good luck to the authors.


**Further updates**: I'd like to elaborate on my thoughts a bit more with the hope that the authors may find it useful for future versions of the paper. 

- First, I really appreciate the authors for performing additional experiments with EIIL during the response phase. I wish to emphasize that, after a long discussion with R3, I've some strong disagreements with their review regarding the "simple fix":
   -   I personally think EIIL is out-of-scope as it is a recent algorithm. It doesn't sound like a "simple fix" to me. However, it's great that the authors were able to show that their algorithm works better. 
   -  I don't think pooling all datapoints and splitting it will work. You'll end up with a dataset with label-color correlation in both domains, and both domains will be identical. So IRM won't work here.
  - One can always come up with some hacks like "pool all environments and carefully split them back" that work under the assumption that there's $\Lambda$-spurious correlation. But those hacks would be sub-optimal if there were no $\Lambda$-spurious correlation.  Therefore, this is an unfairly powerful "overfit" hack, and does not make a good baseline. You want an elegant solution that works whether or not there's $\Lambda$-spurious correlation as you won't know whether that sort of a spurious correlation exists in practice.
  - As a side note, in light of the above point, I think it's important that the authors also demonstrate that the CDM constraint added preserves the performance of IRM on the original CMNIST dataset. 
  - Hopefully the authors can keep the EIIL results for future versions of the paper as it only makes the paper stronger.

- I don't think the paper should be heavily penalized for the lack of a realistic dataset, because it's hard to verify $\Lambda$ spuriousness on realistic benchmarks. However:
  - I'd strongly encourage that you consider trying similar experiments on a dataset like say Rotated-MNIST (or Rotated-MNIST+ to be more precise, if at all possible). 
  - Even better, you could consider whether similar experiments can be done with Celeb-A where you have access to image attributes like hair color etc., (See Fig 2 https://arxiv.org/abs/2005.04345) and you could try creating different environments by sampling differently in each.
  - If you think that's it's impossible to create a $\Lambda$-spurious dataset, you might want to explain in future versions of the paper as to why that's not possible.

- I understand R3's main concern which is that the algorithm in this paper requires that the distribution of the causal features $X_{causal} | Y$ to be the similar across all environments. It seems like IRM doesn't expect this sort of invariance, while algorithms prior to IRM do require something of this sort (including CDM, DANN etc.,). I think one actionable way to address this concern would be to show that there are datasets where IRM + CDM does better than CDM (just like how IRM+CDM does better than IRM in CMNIST+). This way we can see why combining IRM and CDM offers something unique.


- Finally, I want to appreciate your efforts in trying to clarify all the reviewers' concerns (at least I found them helpful) and also to update the paper accordingly, add experiments etc., 
review_title: Valuable study of and improvement over invariant risk minimization

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper attacks the problem of OOD learning from the angle of invariant causal feature learning. The key idea is to capture domain invariant causal features and use the extracted causality relation to convey domain-adaptive classification. In this work, domain invariant causal features are learned by IRM, which imposes the consistency constraint between causal features and class labels across different domains. The core idea is to address the existence of spuriousness correlation by introducing the MMD and KL divergence based conditional distribution matching constraint to the IRM learning process. The experimental study based on a crafted MNIST data set demonstrates the superior performances of the regularised IRM learning method in the domain invariant learning task. 

In general, the paper introduces an in-depth discussion of the limitation of the IRM learning mechanism and points out the root cause of failure of IRM (spuriousness correlation). This is interesting and potentially impactful for practical OOD learning tasks. The paper is well-written and the proposed objective is novel to my knowledge. we tend to accept the paper. 

Still, our concerns are as follows:
1. Though the results look promising on the toy data set, it would be better to have a real-world scenario as a testbed for the proposed method. Domain transfer is a popular application. How would this method perform in a domain transfer learning task? 
2. Following the first question, we would expect some discussion about the relation between the proposed method and other transfer learning methods, such as meta-learning methods. Could domain invariant casual feature learning be considered as a way of conducting meta-learning? 
3. A minor issue in Table.1: how many domain labels are there defined in the CMNIST data set? How are they defined? 




review_title: This paper discusses an interesting problem of invariant causal feature learning applied in OOD learning scenarios. It proposes to use a divergence based regularization term to address the spurious correlation existing in IRM.  

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary:
In this work, the authors focus on the out-of-distribution generalization problem. The input is dataset from multiple environments and the goal is to learn a model that generalizes well to an unseen test environment. The work is based on recent line of works on invariant risk minimization (IRM) (Arjovsky et al.). The authors show that under a certain type of structure for the generative model, where the domain/environment label itself has a strong correlation with the spurious factors and the target label, IRM fails. The authors propose an extension of the colored MNIST dataset to highlight this problem. Finally, the authors build a method that works better than IRM on the extension of colored MNIST dataset. 

Pros:
 I appreciate the authors have tried to highlight how the IRM directly applied to datasets from multiple environments will not always work and one has to be careful about the environment induced correlations themselves. 

Cons: 
I divide my concerns into different subsections below. 

a)	Incorrect connection between IRM (Arjovsky et al.) and conditional independence made by the authors: 

Conditional independence (Y \perp E | F(X)) is a necessary condition but not sufficient for the theory of IRM to work.  Therefore, analyzing any F that satisfies this property is not sufficient. 
Suppose we have two training environments, E=1 and E=2.  Asssume that we are only interested in binary classification for now. The main condition that is assumed in Arjovsky et al. (Page 9 Definition 7) for the success of IRM is that there exists a representation F*(X) such that 

P(Y|F*(X),E=0) = P(Y|F*(X), E=1) = P(Y|F*(X))

The above condition implies that Y \perp E | F*(X). In the above expression equating conditionals, there is already an assumption made about F*(X), which is that for both environments E=0 and E=1, the support of F*(X)|E=0 and support of F*(X)|E=1 are equal. Suppose the supports were not equal, then the conditionals can only be equated over the intersection of the supports.  
In the driving example used in the paper, i.e., F*(X)=E, the support of the two conditionals F*(X)|E=0 -->E=0 and F*(X)|E=1-->E=1 do not intersect.  Therefore, what authors claim is a problem with IRM is not really a problem but a data generating environment for which the theory of IRM is not guaranteed to be successful. The right claim to make is that CMNIST+ does not satisfy the assumptions IRM makes for the method to be successful. However, this is easy to fix as I explain soon. 

Before moving to the next section, I would like to also make another important remark. The authors investigate any predictor that satisfies the conditional independence condition. This is also not correct because IRM and other IRM based methods select one of the invariant predictors and not all (thus there can always be bad invariant predictors, which does not mean that they will be selected). For a complete characterization of invariant predictors in terms of conditional independences please refer to Koyama et al.

Update post discussions: This point a) was corrected by the authors.

b)   Why does IRM not work on the CMNIST+?

We now turn to providing the explanation why IRM did not work on CMNIST+ as the explanation provided by the authors is not accurate. As we explained in the last section, F(X)=E is a representation that does not satisfy the criterion that the theory of IRM requires. One intuitive way to think is that the success of IRM assumes that representation F that we search over have an overlapping support across the environments. The IRM optimization procedure fails because it does not enforce this assumption in any way and F(X)=E can lead to a better predictor than F(X)=S, where S is the true causal feature. 
In CMNIST+, the authors have created two environments, where the environment label itself is strongly correlated with the label. Say in E=0, the majority of the labels are 0, and in E=1 the majority of the labels are 1.  Suppose the IRM optimization (Arjovsky et al.) is given a representation F(X)=E as input. The support of F(X) gets partitioned into two disjoint sets X0 = F^{-1}(0) and X1 = F^{-1}(1). E=0 learns a predictor over the set X0 and E=1 learns a predictor over the set X1. A predictor that labels all points in X0 as 0 and X1 as 1 actually satisfies the definition of invariant predictor because it simultaneously minimizes the error in the two environments. If the error of this predictor is actually less than the error of the predictor based on causal features, then this predictor can be selected by the IRM optimization, which is exactly the case in the CMNIST+ dataset. 

The main reason IRM was designed was to make the predictors from different environments be compared when the sets X0 and X1 overlap to some extent at least, i.e. the image of F over the feature distributions in the two environments has to overlap. 
If the image of F over feature distribution does not overlap at all (as is the case in example considered in the paper), a trivial invariant predictor which is not robust to distribution shifts will exist.


c)	There is a simple alternate fix for the entire problem:
The space of problems that authors want to fix abstractly stated are when the environment label (domain label) itself is so strongly correlated with the label that the IRM is encouraged to use environment as a representation.
The fix goes as follows:  Mix the data from the two environments. Take the mixed data and divide into two completely new environments. A manual way to construct these new environments is to divide the data in such a way that the proportion of the colors in the two environments marginally different as was the case in colored MNIST. For an algorithmic approach  to construct these new environments  use this approach (paper http://www.gatsby.ucl.ac.uk/~balaji/udl2020/accepted-papers/UDL2020-paper-045.pdf) (code available at https://github.com/ecreager/eiil). The above paper shows that a single dataset can be divided into multiple environments and retains the gains shown in IRM. 
The two new environments obtained from the algorithm in the above paper or even through a manual division as explained above would lead to a dataset that is very similar to the original CMNIST. Observe that by mixing and creating two new environments, we are automatically ensuring that the support overlap assumptions required by IRM are satisfied.
I believe that these approaches can bring the performance back to 68 percent level. 

Also, note that the fix I am proposing is not the simple fix based on label balancing that the authors show does not work. By mixing the environments, we are destroying the spurious environment based correlation that exist. 

Therefore, whenever there is a strong environment based spurious correlation, i.e., each environment has a stark difference in the marginal distribution of labels, then the prudent thing to do is to mix the environments destroy the spurious correlation and then construct environments for IRM either manually or through the algorithm I shared above. 

Update post discussions: The authors did more experiments to show their method works on CMNIST+ better than these baselines. However, I have major concerns with the principle proposed by the authors as a search criterion. I believe the authors approach happens to work on CMNIST+ but is not based on the right principle. More on this in my point d) below.

d)	The fix proposed by authors has theoretical problems:
The authors have proposed to do conditional distribution matching which constraints that the representation learned has to be independent of the environment conditional on the label. For the sake of discussion, let us consider the structural equation model (SEM) used in Theorem 9 in Arjovsky et al. If we assume that the lambda-spuriousness condition holds for the SEM under consideration. Say for the two environments, the support of the feature distributions do not intersect, the support of the label distributions do not intersect. In this case of lambda-spuriousness, IRM continues to be able to recover the ideal invariant predictor and does not fail (as no assumption in Theorem 9 is violated). However, the representation used by the ideal predictor does not have to satisfy the CDM condition stated by the authors. Therefore, the CDM condition proposed by the authors holds in the extreme case of the example discussed by the authors but does not hold in general. 

In other words, author should show how imposing CDM works in a broad range of settings and not just one example that they use. Based on my argument I above, I highly doubt that CDM condition is actually a necessary condition for the success of IRM under lambda-spuriousness.  

e)	Colored MNIST + issues: 
The authors never explained in detail how the data in the different environments is generated. The Table 1 is an incomplete definition. It only lays down the conditional distributions. Conditional distributions are not sufficient to decipher the underlying structural equation model.  What authors call lambda-spuriousness is not really a lambda structure. In CMNIST+, there has to be an arrow from the label Y to the spurious features Xs as well, which does not appear in lambda structure. 

Update based on discussion: The authors explained the data generation process. 

f)	A simple comparison with balance in generation time:
Why did the authors not compare their method when P(Y=1|E)=0.5 in both environments. It seems for high values of rho and P(Y=1|E)=0.5 a direct application of IRM does not work.  My suspicion is when  P(Y=1|E)=0.5, IRM does not work because we need more than two training environments, as the number of colors that enter the equation of spurious correlations go from two to three. 

g) Comparison with Koyama et al.: Koyama et al. had worked with a lambda type structure and introduced an extension of colored MNIST where environment label plays an important role. A comparison with that work would have been useful.

Update based on discussion: The authors clarified that this paper was posted on arxiv on Aug 4 and the ICLR policy requires them to compare only until Aug 2. I am ok with authors not including a comparison with this paper.

Quality: Unfortunately, the paper is not good quality. A lot more work is needed to really justify why what they propose is really a problem with IRM and why the simple fixes I propose won't already solve the problem. 

Significance: The area of OoD generalization is quite significant. However, the problem proposed by the authors and the approach taken by the authors is not of much significance. 

Originality: The authors have proposed a new CMNIST dataset and a new algorithm to fix it. The authors should get credit for proposing the dataset but besides that I don't think the algorithm proposed as a fix is needed. 


**Final update:** The main criterion used by the authors to search invariant predictors is not correct and is in fact not satisfied by the invariant predictors. For this my suggestion to authors is to modify their criterion in a way that it is at least satisfied by the ideal model you want to learn.

References:

Koyama et al.   "Out-of-distribution generalization with maximal invariant predictor." arXiv preprint arXiv:2008.01883 (2020).





review_title: Paper proposes a new dataset extending CMNIST to highlight flaws of IRM  and proposes a fix. Existing works that extend IRM already suffice to fix this problem. Several theoretical issues with fix proposed by the authors.

Meta-review: Loosely, while IRM aims to find a feature mapping Phi s.t. response Y given Phi(X) is independent of the environment variables E, they suggest that when E is strongly correlated with Y, then it is possible for Phi obtained via IRM to involve environment variables. They motivate this by suggesting that if there exists a feature mapping Phi(X) = E, it would satisfy the IRM aim, but that this is undesirable.

They suggest instead requiring Phi(X)|Y being invariant to the environment.

The reviewers bring up a couple of concerns. The first is that it is not clear outside some simple examples when Y given Phi(X) being independent of E does not suffice. The second is that the authors also do not empirically validate their fix outside a single simple dataset. Moreover, what are the pitfalls of having Phi(X) given Y being independent of E?

Overall, this is an interesting kernel of an idea; it just needs to be fleshed out a bit more.

Decision: Reject