Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a method to simultaneously infer soft groupings as well as optimizing for worst performance within the group in an iterative minimizing-maximizing optimization routine. The authors show performance gains on datasets with known and unknown spurious correlations as well as do ablation analysis and human-subject mechanical turk like experiments to show that the method performs competitively. 
strength_and_weaknesses: Strengths:

--- As far as I know, this is the first method that discusses automatic groupings as part of the optimizing process, as opposed to assuming known groupings like in some previous works. This is novel and a significant step in the right direction imo.
--- The paper is very-well written, and well-placed in the known literature. The authors discuss the motivation for their method and illustrate the novelty and the impact well. The authors mention that their work is motivated as a combination of two previous known works, and while this is true I think the overall novelty extends beyond that. 
--- The empirical evaluations are adequate. The authors have shown competitive performance across several datasets, and have done useful ablation and human subject evaluations to illustrate the effectiveness of the method. 

Weaknesses:
---  While this is also done in previous works, I am somewhat confused by the goal of optimizing for the "worst-case" loss. If there are outliers, would such an approach not exacerbate the problem? 


clarity,_quality,_novelty_and_reproducibility: I have no qualms about clarity and quality. The approach is clearly novel, and a step in the right direction. 

There are several knobs at play here, and as long as the authors commit to sharing their code online, I have no concerns about reproducibility. 
summary_of_the_review: I think the paper is strong, it makes useful contributions in terms of automatic soft groupings and the corresponding empirical evaluations that ratify the utility of the proposed method. I would recommend accepting the paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper studies the distributionally robust optimization problem. Due to existing methods depending on expensive group annotations, the proposed AGRO method proposes to discover different groups via an adversarial opponent. Specifically, there are two models are designed in the AGRO, namely the task model and the grouper model. The task model minimizes the worst-case risk, while the grouper model maximizes the same risk. As a result, the grouper model is shown to be effective in distinguishing different groups without the group prior. By conducting extensive experiments on image and language datasets, the AGRO is shown to be superior to compared baseline methods.
strength_and_weaknesses: Strength:
- This paper is well-organized and easy to follow. The general structure is quite clear.
- The proposed method does not need the group annotation and can be trained in an end-to-end manner, which has great applicability.
- The adversarial discovery for groups is a novel concept in DRO. Intuitively, the selected worst group can be highly informative for the current loss landscape, thus being helpful for learning a robust model.

Weakness:
- As is well-known that adversarial process is highly unstable and hard to train. This paper incorporative another adversarial round into DRO which already contains an adversarial component. So, I am doubtful for the training stability and its convergence.
- The main contribution of this paper is the adversarial discovery-based grouper. However, there is no effectiveness analysis on the grouper performance. Since the group distribution is modeled by a continuous probability, I am curious about the softmax accuracy of the grouper on clustering the groups. How does the clustering accuracy change along the training process?
- The proposed method employs the error slice discovery technique into the DRO setting. However, the concept is slice between group is not clearly described. From my understanding, each group could contain different slices. For example, blond woman is a group, in which there exists two slices: blond woman with hat and blond woman without hat. So, it is feasible to directly employ the DOMINO method to conduct grouping? It is possible that the group only helps the performance of worst-slice instead of worst-group?
clarity,_quality,_novelty_and_reproducibility: This paper is good in clarity and quality. The proposed method is somewhat novel. Experimental details are provided for reproduction.
summary_of_the_review: I have carefully read the whole paper. This paper makes some contribution to DRO by introducing adversarial discovery, however, there are still some concerns (see weaknesses). If the authors can address my concerns, I will consider raising my score.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The presence of spurious correlations between the labels and features can be an impediment to out-of-sample generalization. A common solution in the literature for this problem is apply a group distributionally robust optimization (G-DRO) to minimize the worst-loss over subgroups in the data. However, prior approaches for group DRO either require the group identities to be known in advance either in the training or validation set, or require some task-specific knowledge.

This paper presents combines the discovery of error-prone groups with the G-DRO procedure into a single end-to-end training procedure by setting up an adversarial game between a G-DRO solver and an adversary who seeks to identify soft group assignments to maximize worst-group error. The adversary is a parameterized neural network that outputs a distribution over training examples for each group.


strength_and_weaknesses: **Pros:**
- The problem of worst-group optimization under unknown groups is important, and tackling this problem with no prior knowledge of the group identities makes the proposed approach practically relevant
- Setting up an adversarial game between a parametrized grouper model and  the DRO subroutine is a natural idea to try
- The authors carry out an extensive experimental comparison with many recent baselines and multiple benchmarks and human eval tasks.

**Cons:**

My main concern is the lack of sufficient explanation or justification for why the algorithm proposed does indeed optimize the objective that the authors seek to optimize. It appears the authors directly borrow an algorithm from Oren et al. (2019)  for their inner G-DRO solver, but its unclear if the DRO objective in Oren et al. is the same as the one in the present paper. Moreover, the details of the algorithm (both in the main text and appendix) are left vague and one needs to read up prior papers to understand the workings of the algorithm.

**Mismatch in objective between algorithm and formulation:**

My understanding is that the inner group DRO in the formulation on page 4 seeks to minimize over $\theta$, the following worst-group objective:

$$
\max_{g \in \mathcal{G}} \mathbb{E}_{(x, y) \sim q_g}\left[ \ell(x, y; \theta) \right],
$$

where if I understand correctly (although the notation is a bit imprecise), $q_g$ is a distribution over instances belonging to group $g$ and is the output of a grouper neural network.

To solve this inner problem, the authors employ an algorithm from Oren et al., which however seeks to optimize a slightly different CVaR-style objective (as elaborated in Zhou et al. (2021)):

$$
\max_{q \in \mathcal{Q}} \mathbb{E}_{g \sim q,~ (x, y) \sim p(x, y|g)}[ \ell(x, y; \theta) ],
$$

where $\mathcal{Q}$ =  { $q: q(g)  \leq p_{train}(g) / \alpha$ } is a specific uncertainty set over groups defined for a particular fraction $\alpha$.

In the present paper, the formulation presented does not construct this uncertainty set, and seeks to directly optimize over all groups in $G$. The parameter $\alpha$ is however present in algorithm and treated as a hyper-parameter in their experiments.

I think it is important that the authors clarify this mismatch between the formulation they present and eventual algorithm they use. 

It would also be helpful if the authors can be *more precise with their notation*, whether it is in formulating the distribution $q_g$ (was not initially clear if this was over examples or groups), or in the description of the individual steps in the algorithm, such as the use of an exponentiated moving average EMA (not entirely clear what "samples of each group in B" means in Alg 1).

**Convergence of Algorithm 1**:

Another smaller concern relates to the convergence of the alternating optimization between the adversary and the G-DRO. Solving min-max problems can be tricky and may lead to oscillations if the updates are not carefully chosen. For example, merely having the max-player perform a full maximization over its parameters and the min-player perform a full-minimization at each round generally is not known to have good convergence behavior. Limiting each player to few minibatch gradient updates, as done in this paper, may be a useful practical trick, but I think it is also important that authors at least provide some citations for why the approach taken would lead to convergence.

Minor question:
In your implementation of G-DRO from Sagawa et al., did you implement the per-group regularized updates in eq (5) (which I think is what they use in their experiments), or the online version in Algorithm 1 in their paper?

Additional related literature: 
Kirichenko et al. Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations. ArXiv:2204.02937.


clarity,_quality,_novelty_and_reproducibility: Novelty: The use of a parametrized grouper model, although not entirely new (Lahoti et al. already do something similar in the context of example re-weighting) is a nice addition to the group DRO literature, and the experimental evaluations are elaborate.

Quality: My main concern is about the technical correctness of the algorithm use for the specific formulation the authors care about. 

Clarity: The notations need to be made more precise. The algorithmic detail need to be explained in a more self-contained manner.
summary_of_the_review: Given the mismatch between the algorithm and the formulation, the paper falls short of the acceptance threshold. I am happy to revisit my score after hearing back from the authors.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper extends the existing distributionally robust optimization toward the "unsupervised" case, where group information is not given in advance. Following the basic formulation of G-DRO (minimization of the loss function on the worst group), the authors incorporate a grouper model to infer the underlying (unrevealed) group for each instance and perform G-DRO. Despite its simplicity, the proposed method (AGRO) successfully improves the classification performance consistently over ERM, while the existing baselines failed.
strength_and_weaknesses: ### Strengths

- **Simple method**: The proposed method simply incorporates a new model to predict the underlying group attribute for each data point into the existing worst-case group DRO. This idea itself is simple yet the experimental results demonstrate that this is indeed effective in practice.
- **Thorough experimental evaluation**: The authors provide experiments following several different scenarios such as the supervised (with having access to the group information) and unsupervised (without access to the group information) cases and OOD test. Moreover, they provide a human experiment to see whether the elicited group attributes align with human judgment. Those experiments support the usefulness of the proposed method.

### Weaknesses

- **Rather complicated technical details**: Though the proposed method looks simple, I feel there are several brittle parts in the method. For example, the original formulation to optimize only the worst-group loss is later related to the optimization of the down-weighted loss of the worst $\\alpha$-fraction groups; hence a new hyperparameter $\\alpha$ is introduced. The method seems to heavily depend on the number of groups $m$. The tweaks to the initialization of $\\phi$ (using DOMINO, instead of random initialization) look fragile and require a certain amount of labor. These factors might make the method not easy to handle.
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and easy to follow.
I have only a few minor comments.

- In the bottom equation in p.4, $\\hat\\theta_\\mathrm{AGRO}$ should not be an argmax with respect to $\\phi$. The argmax cannot be performed on the argmin (which should be min instead). $\\hat{q}_g$ is not defined.
- In the last part of Section 3.2, the authors explain "we adopt a different initialization for $\\phi$, which is explained in the next subsection", but this should be corrected because it is explained in Section 4.
- In Section 3.3, I did not understand why the authors mention "To estimate model errors on training data, we apply K-fold cross-validation ..." at the first sight because the error estimation does not seem to be used in the proposed method. However, later I figured out that it is indeed used because DOMINO is used as initialization of $\\phi$, which is explained later in Section 4. I guess it's nice to give a pointer in Section 3.3 or aggregate these explanations in a single place to avoid confusion.
- In the caption of Table 2, there should be a space between the first and second sentences.
summary_of_the_review: The paper proposes an effective solution to the issue that we usually do not have access to the group information in distributionally robust optimization. The simple approach seems pretty effective, hence I feel like accepting this paper.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Group distributionally robust optimization (GDRO) mitigates degradations in accuracy due to distributional shifts caused by the presence of spurious correlations in the training data by minimizing the worst expected loss over pre-identified groups in the training data. In practice such groups as seldom known a priori; this paper tackles the problem of simultaneously finding these groups and minimizing the loss over them to achieve GDRO by posing the problem as a min-max optimization problem. The reviewers expressed some concern over the lack of discussion and guarantees of convergence of the algorithm given for solving the min-max problem, but overall find the use of a parameterized grouper to be a nice addition to the GDRO literature, and the method to be novel, simple, and empirically effective. 
justification_for_why_not_higher_score: Although the algorithm empirically is effective, it has multiple knobs and no performance guarantees.
justification_for_why_not_lower_score: Reviewers agree the method is a nice addition to the GRO literature.
Decision: Accept: poster