Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: 
The paper discusses the problem of shift between train and test data distributions. The authors assume that training data originate from multiple 'environments', each with particular distribution. They propose a new framework (EDNIL) through which they can infer the environments from data (without human labeling) and then train an environment invariant model. The principal assumption is that of the existence of `invariant features` - the distribution of the outputs conditioned on these is the same across all the environments. The loss is a combination of multiple terms motivated through information theoric arguments. The algorithm consists of alternating steps to infer the environments and train the invariant model. Multiple experiments are provided to corroborate the effectiveness of the method.
strengths_and_weaknesses: Train/test distribution shift is an important and fully answered problem, particularly relevant for all practical applications where the standard iid assumptions typically do not hold. As such the paper addresses a significant question for the community.
I found the paper somewhat difficult to parse and follow. Most importantly, I found the description of the whole setup (shift from train to test distributions or a mix of multiple environments in train and test) rather confusing. But I have more question marks (see Questions) indicating a fairly low clarity of the paper. 
In the end the solution is an alternating optimization in which through the first step the environments are inferred though a form of supervised clustering (similarity metric is the model accuracy) and then this info is used to train a prediction model. If my interpretation is correct, this does not seem very novel.
questions: 1) While the intro speaks about a shift between train and test distributions, in the end it seems you address the problem of a mix of multiple distributions in both train and test. That is the train and test are not fundamentally different, they are just both formed through a combination of multiple distributions, is this right?
2) If so, what is the meaning of $\mathcal{E}_{all}$ in your setup?
3) Do I assume correctly that the output and input spaces are aligned across the environments (including the supports of the distributions)?
4) What are $\Phi$ and $\Psi$? Are these some general functions such as small MLPs?
5) In equation (5) softmax you sum across the E_learn invoronments, Does it mean that the number of environments needs to be fixed beforehand?
6) How is the environment decided at test time, when no labels are available? Which head shall be used? 
limitations: Societal impact - not addressed, not directly relevant.
Limitations - invalidity of graphical model assumption is mentioned. Yes, is reasonable.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Paper proposes a new method to infer environment labels that can then be used for invariant learning (such as IRM methods). Essentially, it uses a neural network for clustering data into environments with appropriate loss functions. The method is more scalable and less sensitive to initializations than the prior methods. Experiments show significant improvements for distribution-shifted tasks without environment labels.
strengths_and_weaknesses: Strengths:

1. Paper tackles an important challenge of robust learning with no environment labels and solves the problems with existing works.

2. Experiments in varied tasks, including images and text, show significant performance improvements. Experiments also help answer questions about the effect of initializations, effect of number of environments.


Weaknesses:

1. Clarity can be improved a lot; especially Section 2.2 and Section 3 (Methodology). As a result, the paper’s contributions are muddled with existing work (for example, which of the learning objectives and loss terms are novel?). 
2. Paper can be improved a lot with better justifications for the choices made in the paper. Currently the loss functions seem a little arbitrary; for example, why are Equations (5) and (9) the best ways to achieve their respective goals. 
3. There are no theoretical guarantees provided by the prior works, but this can be future work. 


questions: 1. Problem 2.1 needs to be explained more, discussing the use of Shannon entropy. Also clarify the novelty; I believe Equation (3) was proposed in [1] but Equation (4) is a contribution?
2. In Equation (4), is it enough that LHS is just > 0 while being arbitrarily small? Should the LHS be maximized instead?
3. The proposed method in Equation (5) needs more explanation on why it was chosen to be this way. I believe it is done to ensure that data is divided into environments that maximize label prediction in the individual environments. This is interesting but requires more justification in the text. Related to my previous comment, $H(Y | X_v) - H(Y | X_v, \mathcal{E}_\text{learn})$ might be actually maximized with this objective, and not just >0. 

4. Discussing $L_{IP}$ before discussing the $M_{IL}$ model hampers clarity. It should also be clarified/reiterated that $L_{IP}$ is only used to update the $M_{EI}$ network and not $\Phi$, and that there are two separate invariance preserving objectives (one for $M_{IL}$ and one for $M_{EI}$). 
5. It is very interesting that the number of environments (clusters) need not be specified accurately for EDNIL to work (as seen in Figure 6). More discussion regarding why this is so will be helpful, i.e., which part of the loss ensures this.


Minor:

1. Please use \text{} in math equations for non-variables, such as \text{learn}, \text{LD}, etc. Also, use \exp in Equation (5).
2. Line 19: “… aims at learning causality expected to be robust …” should be “..aims at learning causal features expected to be robust..”.
3. Line 106: should be “HRM equips a joint learning framework”. 

[1] Jiashuo Liu, Zheyuan Hu, Peng Cui, Bo Li, and Zheyan Shen. Heterogeneous risk minimization. In International Conference on Machine Learning (ICML), 2021.
limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a novel environment-free invariant learning method that uses an auxiliary network to learn environment-specific features, from which environment inferences can be derived. These inferred environments labels are then used to train and invariant model.

UPDATE 2022-aug-05: The paper's clarity has been improved so I am increasing my score by one.
strengths_and_weaknesses: I like the idea of the paper, and was impressed by the experimental results showing that the proposed method EDNIL realizes good worst-group performance in a variety of settings, while addressing the sensitivity to initialization that we see in EIIL. In my opinion, the experimental results are sufficient but the presentation should be improved (see below for specific suggestions).

Strengths
* I like the experimental results. The “effortless initialization” result (Fig 7), showing that the proposed method is not sensitive to initial ERM features, which is a known issue with EIIL, is especially impressive.
* The paper does a good job of describing related approaches to environment-free invariant learning, and comparing against these methods empirically.
Weaknesses
* The paper can be hard to read at times. Some parts would benefit from additional exposition. For example the idea of “diversity” of inferred environments is mentioned in passing with a citation, but it is not fully explained what type of diversity is needed, and how it can be measured formally? The cited paper (https://arxiv.org/abs/2004.05007) mentions that the correlation pattern between Y and spurious features should vary across environments…is the outcome we hope for when introducing L_ED?

questions: * I was a little unclear on the motivation for using the environment (in)dependency objective - I can see its utility in the ablation studies, but what was the reason, conceptually speaking, to introduce this in the first place?
* Beyond looking at the value of L_ED during training, can an experiment be formulated to test whether the inferred environments (samples from P(e|X,Y)) are indeed “diverse” in the way the authors are hoping for?
* Section 3 is a bit hard to parse given that the three objectives are introduced in several places (lines 129–133, Fig 1, Eqn 6). I would suggest that the authors take care to discuss these in the same order each time and include the symbols alongside the objective names (e.g. “Label dependency objective” => “Label dependency objective ($L_{LD}$” in line 129). I also found it a bit confusing that some objectives are for the inference net Phi while others are for the invariant learner. While I can see that Fig 1 attempts to make this distinction by placing the loss terms in different parts of the figure, it would be even clearer if the networks (or their parameters) were used as arguments to the losses, e.g. L_ED(Phi, Psi) = L_ED(Psi) + beta L_LD(Psi) + gamma L_IP(Phi) in Eqn 6). I would also order the terms in Eqn 6 in the same order as they appear in the text.
* Since the variance of per-env losses is used as L_IP, it is worth mentioning REx (https://arxiv.org/abs/2003.00688) as a related work, since it uses a similar objective (with known environments).
* Regarding the tabular experiments, there are some limitations of the UCI-Adult dataset (https://openreview.net/forum?id=bYi_2708mKK) that would be relevant/interesting to NeurIPS readers/attendees (perhaps in a footnote).
* Environment-free invariant learning is a very ambitious problem, which could be difficult to solve in some settings. I would be curious to hear about experimental settings where EDNIL fails; I think these would interest the reader and also make the limitations of this method a bit more clear.
* When mentioning how the training/validation splits for Waterbirds are merged [lines 290–291], it would be good to mention that the original validation split contains all four groups in equal proportions, whereas original training split has unequal proportions.
* I find the double negatives needed to parse Table 1 to be slightly confusing.

limitations: No [line 430]. I suggest that the authors add a discussion of possible societal impacts to a future revision. Given the connections of invariant learning to fairness (https://arxiv.org/abs/2010.07249) and the use of a fairness benchmark in the experiments, there should be plenty to discuss.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In the present paper, the authors tackle the challenging problem of learning a supervised ML model capable of identifying and adapting to different environments hidden in the training data. The method, EDNIL, is composed of two jointly learned models, that take care of the environment identification, the learning of the invariant representations and the label predictions, produced by a multi-headed neural network. The proposed model is compared to different alternative models from the literature of the field, in different challenging benchmarks, and the results show that it closely achieves the best possible invariant learning performance. 

AFTER REBUTTAL
The authors have addressed the main concerns raised during the review. I confirm my assessment of this paper as solid and deserving of acceptance in the conference.
strengths_and_weaknesses: The authors introduce a very flexible and effective learning framework, incorporating the main strengths of the previously introduced models and surpassing many of the limitations thereof. The novel method is shown to perform extremely well when compared to its competitors, achieving close to the performance of IRM with access to the oracle environments. 

I think the main weakness of this paper is in the density of the presentation and in the abundance of acronyms that make the read quite challenging, especially for a non-expert.
questions: - In general, the presence of way too many acronyms can confuse the reader. I do understand that presenting all this material in 9 pages is quite a challenge but the density is a bit excessive in some passages of the manuscript.
- lines 74-93 seem a bit hard to follow, especially about the role of Phi and Xv.
- the role of beta and gamma seems important, but the authors only give space in the main text to the discussion of the extreme cases beta=0 and gamma=0. Can a few comments be added about the range of optimal values for these parameters? How hard is it to fine-tune them and how variable are they across different learning problems?
- lines 166-169 (and later figure 7 and its description): what is the intuitive explanation of the reduced dependence on the training stage for the ERM initialization. Is it about the role of the L_IP term, that avoids trivial environment split scenarios?
- line 195 is not clear, probably just needs rewriting.
- lines 227-230 the sentence seems to be missing a verb and becomes a bit obscure.
- line 236: removing the coefficient beta -> in what sense? setting it to 0, or to 1? Probably rephrasing could help.
- In general, it seems to me that this work could be very impactful also for the field of ML fairness, where the different environments could represent protected categories that are misrepresented in the available data and the learning model risks to become biased. Did the authors consider this connection?

limitations: I think the main limitations of the paper are well acknowledged by the authors.
review_title: null

Meta-review: metareview: This work presents a novel environment-free invariant learning method that uses an auxiliary network to learn environment-specific features, from which environment inferences can be derived. The method is composed of two jointly learned models, that take care of the environment identification, the learning of the invariant representations, and the label predictions, produced by a multi-headed neural network. The proposed model is compared to different alternative models from the literature of the field, in different challenging benchmarks, and the results show that it closely achieves the best possible invariant learning performance.

After some initial discussions, all reviewers agreed that this work is ready for publication, as the work addresses an important problem, presents good empirical results, and will be of significant interest to the community. 

confidence: Certain
Decision: Accept