Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a Target Conditioned Representation Independence objective for domain generalization. It claims that a domain-invariant representation may not extend to test domains and refine it to a domain-general representation. Based on this, this paper proposed the TCRI objective and evaluate it on synthetic and real-world data.
strength_and_weaknesses: Strengths:
- This paper is well written.
- This paper gives a new attempt to explain and solve domain generalization in the view of causality. 
- This paper contains detailed proof and designs the method based on the analysis.

Weaknesses:
- The method requires a classifier for each domain, and it is evaluated on a three-domain setting {0.1, 0.2, 0.9}. How the results would be if there are more domains? Why choose {0.1, 0.2, 0.9} in the experiment?
- This paper focuses on methodology. Only one real-world dataset is analyzed. It would be better to have more results.
- The method achieves much better results under “-90%” setting, but for the other settings, the results are not as strong as expected, showing the limitation.


clarity,_quality,_novelty_and_reproducibility: See above.
summary_of_the_review: The proposed method is interesting. It lacks evaluation results.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes an objective for domain generalization. The main findings involve two types of representations: domain invariant - the distribution of the target conditioned on such a representation is the same across training environments; and domain domain-general - the representation is domain invariant across all possible environments. The paper proves, by construction that a representation that captures only a subset of causal features could be domain-invariant but not necessarily domain-general. 

A criterion (Target Conditioned Representation Independence - TCRI ) helpful for domain generality is proposed, using two representations, one domain invariant $\Phi(X)$ and one domain-specific $\Psi(X)$. Informally, this criterion is satisfied when the two representations jointly have enough information while being independent when conditioned on the target. The paper proves that if $\Phi(X)$ and $\Psi(X)$ satisfy TCRI and $\Phi(X)$ satisfies domain independence then $\Phi(X)$ is domain-general.

Then, a loss function is proposed to satisfy this criterion and experimentally it is shown that it can achieve better word-case performance. An important aspect is that the loss used for conditional independence (based on HSIC criterion) can be used for model selection using only training domain data.

strength_and_weaknesses: **Strong points**

S1. The paper tackles an important problem in domain generalization and proposes a sound method for it. The presented downsides of using representations that are domain-invariant but not necessary domain-general are valid and the proposed method solves them.

S2. By improving on other invariant predictors, like IRM, the proposed method has a large potential for domain generalisation and represents a good contribution.

S3. In general, model selection for domain generalization is a hard problem, and the proposed method seems to achieve a good selection using only the training data.

**Weak Points**:

W1. In practice, even when the proposed model selection is used, the performance for all environments is quite low. This hints that the constraints could be too hard to achieve and the model is hard to optimize. What is the performance of the model if the model selection is using testing domain data (oracle selection)? 

W2. Some additional explanations in the proofs are needed.  In step iv of the proof of Lemma 5.3, the variables $Z_c^{i′}$ are aggregated in one of the two representations, but how to choose which one? Is it possible that the variables be aggregated into the representation $\Psi(X)$ that is *not* optimized to be domain-invariant? 

W3. Is it necessary to have the two representations? Could an assumption such as 
$I(\Phi(X); Y) = I(Z_c, Z_e; Y)$, together with domain-invariance assumption be enough to get the same result as Lemma 5.3 and Theorem 5.4? These assumptions would be satisfied when optimizing $ \mathcal{L_\Phi} + \lambda \mathcal{L}_{IRMv1’}$, i.e. the usual way of training IRM. 


clarity,_quality,_novelty_and_reproducibility: The work is mainly clear, but some questions remain to be answered (W2, W3). The proposed method is novel and seems to have good results in the worst-case scenario.
summary_of_the_review: The paper was some good observations and it is useful for domain generalization, especially in cases where the worst-case is crucial and model selection is hard.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper provides necessary and sufficient conditions for domain generalization. Experimental results on a few benchmark datasets are provided.
strength_and_weaknesses: Strengths: A theoretical foundation for generating representations that are domain-invariant and domain generalizable are provided.
Weaknesses: Poorly written paper. Experimental results using synthetic data or variations of MNIST are not convincing.
clarity,_quality,_novelty_and_reproducibility: The paper is hard to read. Mathematical jingoism has obscures the presentation.
summary_of_the_review: The paper presents an approach for domain generalization. The writing style is harsh. Experimental results are not convincing.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper tries to learn domain-general representation by achieving the target conditioned representation independence (TCRI). Specifically, the proposed TCRI method not only addresses the domain invariant property but also assumes that the domain-general feature and domain-specific feature are conditionally independent of each other when given class label $Y$ and domain label $e$. By introducing two constraints to the general loss function which ensure 1) a good predictor, and 2) conditional independence, the TCRI method is demonstrated to achieve domain generality compared to other OOD generalization methods. Theoretical analysis and experimental comparisons are provided to validate TCRI.
strength_and_weaknesses: Strength:
- This paper is based on a reasonable SCM to analyze the data-generating process. As a result, the existing method only solves the domain invariant constraints, failing to address the target conditional independence. Hence, TCRI can reasonably achieve better performance than other methods by addressing the target conditional independence.
- This paper has good theoretical support, making TCRI a solid research work.

Weakness:
- It is unclear how the target conditioned representation independence property can be break down into the proposed total-chain-information-criterion $I(\Phi(X), \Psi(X); Y)=I(Z_c, Z_e; Y)$ and $\Phi(X)\indep\Psi(X)|Y \forall e_i$. There is no detailed derivation.
- Furthermore, how the two properties $I(\Phi(X), \Psi(X); Y)=I(Z_c, Z_e; Y)$ and $\Phi(X)\indep\Psi(X)|Y \forall e_i$ can be solved by the proposed $L_{\Phi\oplus\Psi}$ and $L_{\text{CI}}$. Please give more details.
- How to choose the hyper-parameter $\alpha$ and $\beta$ are not discussed. Additionally, there is no experimental analysis of the importance of each loss term.
- The experiments are quite insufficient. All experiments are conducted on coloredMNIST which is a very small dataset synthesized artificially, not a real-world dataset as indicated by the title of Section 6.1. So the realistic applicability is quite doubtful.
clarity,_quality,_novelty_and_reproducibility: This paper is moderate in clarity and quality. The proposed method is somewhat novel. Codes are provided to ensure reproducibility.
summary_of_the_review: I have carefully read the whole paper. This paper is well-motivated, however, there are still some concerns (see weaknesses). If the authors can address my concerns, I will consider raising my score.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper introduces an new objective function for domain generalization. It seeks at learning
a domain-general representation by optimizing the so-called target conditioned representation independence
criterion. The paper proposes a theoretical analysis of the TCRI condition.

While some reviewers are positive about the paper, most of them also pointed out that the experimental
analysis is too weak to make the paper passes the bar of acceptance. At this point, suggestions for
improvements mostly are towards these points (more datasets, for instance the DomainBed benchmark, 
further experimental ablation studies,...)


justification_for_why_not_higher_score: The score could have been increased with a better an in-depth experimental analyses including
ablation studies on the hyperparameter, applying the method to all the datasets in DomainBed,
a better understanding of the compromise between fullfilling constraints and performance)
Decision: Reject