Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes an invariant representation learning method across multi-domain from the perspective of the anti-causal learning process. They learn invariant and variant representation simultaneously. It is an interesting idea that considers the anti-causal direction in invariant representation learning. Experimental results show the effectiveness of the proposed method on both synthetic data and real-world datasets.
strengths_and_weaknesses: This paper seeks an invariant learning method considering the anti-causal prediction task. They formed a causal graph to describe anti-causal prediction and learn adaption and invariant representation together. Experimental results show the effectiveness of proposed method. However, some parts of the method should have more discussion. 1. What makes this method hold in anti-causal learning 2. Lack of theoretical analysis of identifiability 3. The linear setting is simple.
questions: 1. Consider the case (Example 1) in Invariant Risk Minimization, the difficulty of anti-causal learning is that one can not have a stable prediction function if they use the effect variable to predict the cause variable (use X2 to predict Y in IRM is regarded as anti-causal learning). Therefore if the authors claim that they can achieve invariance across multi-domain, they should better give a simple example to support their motivation. 
2. This algorithm can also be considered a causal learning method. if we change the direction of the edges between (y,z) and (x), the method will also hold. What makes the proposed method better for anti-causal learning should have more discussion. Causal learning and anti-causal learning should be considered separately since the independent causal mechanism(ICM) will not hold in the anti-causal direction. Note that ICM is not conditional independent, that is 'if X is cause, X is independent of the causal mechanism p(Y|X)'. If this issue can not be solved, why this method is better than IRM or other invariant representation learning where variant and invariant representation are learned simultaneously.
3. Authors should add some ablation study about the performance of the proposed method in both causal and anti-causal datasets. 
4. Authors should better provide some theoretical analysis about whether they can actually learn the invariant representation, which can be regarded as the identifiability property.
5. The linear setting is limited, can the method be extended to non-linear settings?
6. Should add more state-of-art causal invariant learning methods as baselines.



limitations: Some parts of the method should have more discussion. 1. What makes this method hold in anti-causal learning 2. Lack of theoretical analysis of identifiability 3. The linear setting is simple. More details are shown in the question part.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The present work address generalization and transfer of learnable representations in the setting of anti-causal domain shifts. To this end, the work formalize the anti-causal domain shift assumption by adopting the notion of counterfactual invariance (Veitch et. al., 21). Based on this formulation, the work proposes a bi-level optimization based objective to learn domain-invariant and domain-adaptive predictors in a multi-distribution training setup.  Eventually, the proposed algorithm is evaluated on a synthetic task, colored-MNIST and some real-world datasets against ERM, IRM, MAML and show promising performance.
strengths_and_weaknesses: **Strenghts**
- Anti-Causal problem setting is well motivated and the work is well written in general
- I appreciate the formulation based on counterfactual invariance to separate stable from unstable knowledge
- Supportive visual analysis in figure 2
- Valid sets of experimental baselines where proposed approach shows promising performance

**Weaknesses / Suggestions for improvements**
- **Evaluation**: I have some concerns about the real-world applicability of the proposed algorithm and would appreciate some discussion on this matter  
  - There is a significant gap between performances on manually crafted tasks (synthetic setting, color MNIST) which match the underlying assumptions of the framework and on the real-world datasets (Camelyon17, PACs, VLCS). I would like to encourage the authors to address this issue and it's implications for applications on real-world datasets in more detail in the main part (especially including the important findings on real-world datasets in the main part of the paper).
   - What conditions need the training environments to fullfill and what dataset characteristics can break the algorithm? From my point of view, it looks like diversity of the training environment is a key property?
   - Would it be possible to show a similar analysis as figure 2 for one of the real-world datasets?
- **Assumptions**
   - The work assumes that $X_{z}^{\perp}$ does not have a causal effect on $X_z$. What are the implications on potential application(especially on structured datasets)? 
questions: - During fine-tuning, are both the invariant features and the non-stable features adapted or only the non-stable features? Did the authors also consider to finetune with different learning rates, in order to allow an even more rapid adaptation?
- Is any regularization technique employed in ERM? 
limitations: Fine
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, a setting is studied where data come from different domains, the features X are causal descendants of Y, and for some part of X, the relation with Y will change between domains. Based on a notion of counterfactual invariance, an optimization objective and an algorithm are then proposed to disentangle the different parts of X. This allows prediction using just the invariant part of X, but also fine-tuning the predictor on a few samples of a new domain to leverage the information in the non-invariant part of X.
strengths_and_weaknesses: The anti-causal setting studied in this paper has not been studied before to my knowledge (except under an additive noise assumption). I think its exploration of what is possible for domain adaptation with such anti-causal variables, is a significant contribution (mainly but not exclusively for enabling future work). The derivation and methods include some elements that are derived from earlier work, but for the main part, the work is novel.

The paper is very clearly written, though I do have some questions below where e.g. a definition doesn't seem to express exactly what is intended. I hope that addressing these points will further improve this paper.
questions: 1. **Definition 1**: I think the crucial aspect of this definition is not that the causal graph is shared, but that $P(X,Y,Z|U)$ is shared between domains, which is now only mentioned implicitly.

2. **counterfactual invariance**: A "counterfactual" notion of invariance is both a stronger condition than you actually need, and also stronger than can be verified from your data. It is in principle possible to detect if the *distribution* of f(X) (jointly with Y?) given z remains invariant when z changes, but not whether the specific value of f(X) for a given individual is invariant in such a way. Since the term "counterfactual" (as the third level of Pearl's hierarchy) is specifically about this latter case, I feel this term is not the best choice in this context. And more specifically, I would disagree that (line 103) learning a stable predictor "means" learning a counterfactually invariant predictor.

3. **latent parts of X**: Could you clarify in the paper what you mean by "part": a subset of the coordinates of a long vector X, or any function of X (that preserves some of the data in X but discards other parts)?

4. **Causal decomposition of X**: In the second paragraph of section 2.2, I think what you want to work towards is that the two parts of X are independent given Y. If you want to phrase this in terms of the causal diagram, I think you also need to assume here that the two parts have no latent confounders between them, and I suppose selection bias could be a problem as well. I'm not sure what is meant with this extra assumption being "baked into the causal compatibility assumption" (line 123), because the two parts of X were not defined yet when Definition 1 was posed.

5. **$W^b$ and $W^e$**: In line 215, the possibility of making $W^b$ a fixed matrix is mentioned. I assume this is also being done for the experiments? Would it also be a good simplification to restrict $W^e$ to be of the form suggested in footnote 2, or would that reduce the expressive power in an undesirable way? Relatedly, I think the $W^b$ in line 237 (about fine-tuning) should read $W^e$.

Minor points:

* it looks like the display above line 196 doesn't use \lVert and \rVert

* In three displays on page 6, summation signs are ambiguous as to which terms are part of the expression being summed.
limitations: n/a
review_title: null

Meta-review: metareview: In this paper, the authors propose an invariant representation learning method across multi-domain from the perspective of the anti-causal learning process. All the reviewers consider this paper is clearly written and novel. 
confidence: Certain
Decision: Accept