From: NeurIPS.cc/2021/Workshop/DistShift/Program_Chairs
title: Paper Decision
decision: Accept (Poster)
comment: This paper proposes contrastive training methods on ERM predictions to improve robustness against covariate shifts. The empirical approach is interesting, but reviewers have raised important concerns about 1) the use of group labels in ERM model selection, 2) the limited nature of theoretical results, and 3) the validity of empirical results under ablation studies. These are substantive and serious issues and it is imperative that authors address them (particularly issue #1) and explicitly discuss the limitations of the results. 

To my understanding, it appears that the authors are comparing against benchmarks that similarly use group information for validation and model selection (as opposed to training). Presumably, the proposed approach requires a smaller number of group labels than GroupDRO as it only uses them for model selection. From the text—as far as I can tell—it is not clear that group information is used in this manner. The benefits of this approach and its limitations should be candidly discussed.