Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary: 
This work is built on top of IRM, aiming to solve an issue in the biomedical domain where there are very few instances in a given environment. The authors first proposed a new scheme, RGM, to apply tighter constraints on the generalization. And then they described a gradient perturbation scheme in the representation space to corrupt the environment information carried in the representation. The authors claimed improved performance over existing methods on several empirical experiments using biomedical benchmarks. 

Strength: 
1) RGM and a perturbation scheme for structured environments with few instances. Some theoretical analyses showing RGM imposes stricter constraints than IRM. 
2) Pretty comprehensive evaluation on several benchmark datasets to show the model performance/generalization  
3) The flow of the paper is clear, not hard to follow

Weakness 
1) The contribution and significance are not clear to me. Both RGM and perturbation (esp. compared to CrossGrad) do not seem very novel or significant. Maybe the authors could elaborate on this more.   
2) While the authors described the hyperparameter search in the appendix, it would be nice to know more about the auxiliary models and losses. For example, how well does the scaffold classifier perform in the experiments as well as in the ablation studies (i.e. fig 5)? 
3) The data split based on (heavy) atom numbers, scaffold molecule weights and #instances in a protein superfamily look a bit weird. Does this truly probe the model generalization? In other words, does a big molecule weight difference necessarily correspond to two very distinct scaffolds?
4) Figures are not in the correct order (fig 2). Understandable this is likely to accommodate the page limits but is causing some confusion. 
review_title: A new regret minimization algorithm applied for structured environments 

final_score: rating: 5: Marginally below acceptance threshold
review_content: In bioinformatics and biochemistry, we may need to generalize the model beyond the training distribution. The authors propose a new method, the regret minimization (RGM) algorithm, to handle such a problem. The method is inspired by invariant risk minimization (IRM). It regulates the empirical loss with the regret regularizer. Such a regularizer encourages the model to generalize to a new unseen environment. The authors further extend the RGM to the structured inputs, leading to structured RGM (SRGM), which can be trained using gradient perturbation. Overall, the manuscript is interesting, which may be suitable for the general audience of ICLR. 

Pro:
1. Interesting problem and a new self-contained solution.
2. Promising performance across different tasks from different domains.
3. The manuscript is easy to follow although I believe it could be improved further.

Concerns:
1. The authors should further proofread the manuscript. I guess on page 5, the authors wanted to say that "the model would not see any change from one example to another as scaffold variation", right? Some symbols in the paper are not defined as well. For example, in Figure 2, I did not find the definition for $B_i$, $B_j$, and $B_e$. I am willing to increase the score if the manuscript is refined further.
2. On page 6, my feeling is that the model is not very difficult to train. Are there any tricks to train such a model? Is it similar to adversarial training?
3. In Table 2, why we have both RGM and SRGM? What's the difference between the two methods within this table? If I understand correctly, all the inputs are structured, right?
4. In Figure 4, wow much would SRGM decrease for the different number of atoms? I mean the absolute value, not the relative value to ERM.
5. For the HOMO, the authors include data from different protein families. How can the method generalize across different families?
6. Regarding the feature extractor $\phi$, how would the model complexity affect the performance of the proposed method? Any suggestions on the selection of the parametric family?
review_title: Interesting work but the authors should proofread the manuscript further

final_score: rating: 4: Ok but not good enough - rejection
review_content: ##########################################################################

Summary:
The authors proposed regret minimization (RGM) and structured RGM (SRGM) algorithms for generalization across biomedical domains. They quantified generalization to held-out environments by using two auxiliary predictors trained with and without each environment. Furthermore, for structured environments, they simultaneously trained an auxiliary scaffold classifier and used it to generate perturbations highlighting the environmental variation. The authors evaluated the proposed method on several tasks from molecular property prediction, protein homology, and stability prediction. They showed significant performance improvement over current state-of-the-art algorithms.

##########################################################################

Major comments:
While the paper has its own merits, unfortunately, it has several issues that need to be addressed.
-	My main concern is that I am not sure the experiment settings are truly realistic. I agree with the authors’ motivation that generalizing beyond training domains and environments holds great importance. However, in my view, the current experiment setting does not quite represent real-world applications. For molecular property prediction, the authors used the number of atoms or molecular weights to determine the scaffold of data. However, in real-world settings, I do not think it is common to have training data with small molecules and test data with larger molecules. Wouldn’t it be more realistic to determine scaffolds based on sharing a subgraph of a molecular graph as described in Figure 1? 
-	In the case of stability prediction, the train/validation/test splits are not based on the topology. As stated by the authors, the test split contains Hamming distance-1 neighbors of around those from the training set. I am not sure whether the dataset is appropriate to be used to evaluate the proposed method for generalization beyond training domains.
-	The key component of SRGM compared to RGM would be the adoption of a scaffold classifier. Then, I think the performance of SRGM should be inevitably dependent on the performance of the scaffold classifier. Can you provide the classification performance of the scaffold classifier? Assuming that the top-1 accuracy for superfamily classification is similar to that in TAPE (which is only about 40%), can you explain how can SRGM provide performance improvement with such an inaccurate classifier? Furthermore, if most scaffolds contain only few examples, wouldn’t the training of the scaffold classifier be unstable, thus, lower the performance of SRGM as well?
-	I think the ablation studies with SRGM-detach showed some related results, can you show how random perturbations instead of using the scaffold classifier would affect the performance of the proposed method? 

##########################################################################
Minor comments:
-	In protein modeling experiments, how did the authors generate sequence-length invariant protein embeddings from the BERT representations? Did you adopt an additional layer to compute attention-weighted mean protein embeddings as used in Rao et al.?
-	I understand that the authors omitted detailed explanations of each task and data due to space limitations. However, to be self-contained, I would like to recommend to include more information in the appendix. For example, the number of labels for the HOMO task is not stated in the paper.
-	Can you provide a more concise definition of standard and structured environments? 
-	It is obvious that SRGM stands for structured RGM, but anyhow it appeared in the manuscript without proper abbreviation explanation.

##########################################################################

review_title: Official Blind Review #2

final_score: rating: 6: Marginally above acceptance threshold
review_content: 

Summary: The paper proposed a new  regret minimization (RGM) algorithm on structured environment. It is based on one existing SOTA work --- invariant risk minimization (IRM), which assumes the predictor is simultaneously optimal across various environments. To relax the restriction, the authors propose to introduce held-out environment to perform domain perturbation. The experiments are conducted on real-world biomedical applications including molecule property prediction and protein homology and stability prediction. The proposed method outperform baseline methods.



Strength

The paper proposed a novel Regret minimization (RGM) algorithm that extends IRM by relaxing the restrictive assumption on IRM, reformulate the problem as a joint optimization problem including regret minimization. The usage of held-out environment to encourage domain generalization looks very interesting. 

The paper has solid theoretical analysis, and also demonstrate that the predictor obtained by RGM satisfies IRM constraints in the general case.  Empirical studies are convincing. Experiments on two real-world biomedical applications are of high value. The selected baselines are thorough and all from very recent paper. The performance gain looks significant, consistently outperforming baseline methods. 


Weakness:

1. The author argues that their methods works in both standard and structured setting in Introduction Section, but only structured prediction is done. 

2. MLDG is very relevant to the proposed method. It would be great if the author can discuss its difference with RGM in details, instead of mentioning it only in Related Work Section. 

3. The presentation of Section 3 could be improved - notations could be better explained.

4. In theoretical analysis, can authors add the discussion about \phi_*? Now all the discussion are based on \phi \in \Phi_{IRM/RGM}. 

5. I am not sure if the assumptions made in Section 3 are restrictive. For example, “new environments we may encounter at test time exhibit similar variability as the training environments”. Can you add some reference? 


review_title: Interesting idea, solid theoretical analysis. 

Meta-review: The paper has some interesting points in extending IRM to regret minimization, and extending to structured environments.  I can see the writing has been improved in the revision.  The main criticism arises from the experiment, which can be improved in several aspects.   The reviews have been quite detailed and helpful.
Decision: Reject