Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors consider domain generalization as a non-random sample selection problem. It is the first paper of kind to utilize this method for DG problem. Via assessment on both simulation and benchmarking data sets, the authors demonstrate the efficacy of our method both theoretically and empirically on simulated data and four challenging benchmarks.
strength_and_weaknesses: The paper proposed a new method and did extensive experiments to demonstrate the effectiveness of the proposed method, which is a strength of the paper.

As the authors stated at the beginning of the paper, DG questions may be categorized into different categories. While the paper shows that the proposed method works in many cases, it is unclear to the reader, what exact scenario the proposed method may work and in which scenarios it will not. If the authors can shed some light on this direction, it will better navigate the readers about future applications of the method.

It will also be good if the authors can describe the pre-assumption of the methods more explicitly. The original Hackman's method has a number of assumptions, including the assumption of normal distribution etc. What are the implications for adoptions in the current scenarios? Hope the authors can add some discussions on such aspects.

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written. The overall quality of paper is very good. 
Based on my knowledge, the adoption of Heckman's methods is a novel idea. 
I was not able to fully assess the reproducibility of the study, since the authors did not provide a code repository for testing.

summary_of_the_review: The research work is well conducted and the paper is clearly written. Besides a couple of weakness, the research results are worth sharing with the broader audiences to raise discussions and potential further research in this direction.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper casts domain generalization as a sample selection problem. It starts from problem formulation which is then disentangled into two terms: sample selection loss and joint function loss. Authors designed a dual-branch network to implement this idea. Experiments are done on WILDs benchmark, showing its effectiveness against its counterparts.
strength_and_weaknesses: ### Strength

1. This paper is well motivated. It is interesting to see that authors formulate this problem as a sample selection problem.
2. The theory part is novel, with solid proof.
3. Experiments show its effectiveness.

### Weakness

1. The computation complexity is not introduced in the paper. From what I see, there are two feature extractors in the method, while comparison methods only adopt one. So, what's the time complexity and number of parameters comparison? Are the experiments fair?
2. The latest comparison method is Fish, which is in 2021. There are other approaches more latest than this: SWAD and L2A-OT, for instance. Authors should cite and compare with them
3. There are no ablation studies to show the actual contribution of each component in the method.
clarity,_quality,_novelty_and_reproducibility: Most of the parts are clear and well-written. No source code available and this approach is not easy to implement.
summary_of_the_review: I think this paper is novel in formulating the DG problem as a sample selection one. It has its own theory and derivation, with working algorithms. 

=== Comments after rebuttal: I decided to increase the score to 8 given that authors addressed my concerns well.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method for domain generalization in which the variable $S$ selecting the domain correlates both the input $X$ and the output variable $Y$. The proposed method models the joint probability distribution of $(X, Y, S)$ and performs a two-step variant of the maximum likelihood estimation (MLE). The authors demonstrate that the proposed method outperforms the ERM as well as other domain generalization methods. Interestingly, the proposed two-step algorithm performs better than jointly performing the MLE.
strength_and_weaknesses: # Strengths
- The approach of exploiting the correlation between the data and the domain is interesting.
- Theorem 1 is also interesting and well supports the proposed approach.
- The experiments show that the proposed method significantly outperforms previous approaches.

# Weaknesses
- In Section 4, "Equation 5 becomes...": I think this is related to the previous point. Eq. (7) does not exactly correspond to Eq. (5) because it ignores $k = K + 1, \dots, L$. But if we ignore those terms, we will go back to Eq. (3). I am not sure if the proposed method is doing what the paper claims it does.
- The two-step algorithm does not exactly solve Eq. (8) but performs better than the more theoretically faithful one-step algorithm, according to Section 7. The paper does not provide a convincing explanation about this. I suspect that the two-step algorithm is solving something different from Eq. (8) but theoretically justifiable.
clarity,_quality,_novelty_and_reproducibility: It was not very easy for me to follow the paper. I barely understood the abstract and the introduction before going to the technical details. In contrast, it was a relatively smooth read after Section 3. I list some specific points below that are unclear to me:
- What does it mean by "non-random"? The proposed method treats all the variables as random variables. It sounds contradictory to me.
- "A naively trained model, optimizing the prediction performance conditional on the selection mechanisms of the domains in the training data, will fail to perform similarly if sample selection of a testing domain is under a different selection mechanism.": I could not understand this sentence.
- "In such cases, we use $P^k_X$ for $k = 1, \dots, K$ as a surrogate, implying that we learn selection models by contrasting one domain to the remaining source domains": This is not very clear to me. It would be nice if there were more explanations since, I suppose, this is part of the proposed method.

summary_of_the_review: The proposed method is interesting and seemingly effective, but it looks like there is a gap between the theory part and the proposed algorithm. I weakly recommend acceptance.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a DG method, termed HeckmanDG, that learns two models. The first model (g) predicts the domain membership of an input, and the second model (f) predicts the output of interest (Y).  The output of the first model is used when learning the second model to correct for domain selection bias.  The validity of the method hinges on some assumptions about the correlation between f and g (bottom of page 5 and A.2).  The effectiveness of the method has been empirically demonstrated, and a related theoretical result is also given.  All the reviewers are positive about this work.  
justification_for_why_not_higher_score: Although the reviewers support this paper strongly, I feel that the assumptions it makes are too strong. 
justification_for_why_not_lower_score: Highest rated paper in my batch.
Decision: Accept: notable-top-25%