Modeling the Q -Diversity in a Min-max Play Game for Robust Optimization (2023)

Abstract: Models trained with empirical risk minimization (ERM) are revealed to easily rely on spurious correlations, resulting in poor generalization. Group distributionally robust optimization (group DRO) can alleviate this problem by minimizing the worst-case loss over pre-defined groups. While promising, in practice factors like expensive annotations and privacy preclude the availability of group labels. More crucially, when taking a closer look at the failure modes of out-of-distribution generalization, the typical procedure of reweighting in group DRO loses efficiency. Hinged on the limitations, in this work, we reformulate the group DRO framework by proposing Q-Diversity. Characterized by an interactive training mode, Q-Diversity relaxes the group identification from annotation into direct parameterization. Furthermore, a novel mixing strategy across groups is presented to diversify the under-represented groups. In a series of experiments on both synthetic and real-world text classification tasks, results demonstrate that Q-Diversity can consistently improve worst-case accuracy under different distributional shifts, outperforming state-of-the-art alternatives.

## introduction deep learning models trained with empirical risk minimization (erm) often exhibit drops in accuracy when confronted with data from domains that are under-represented in their training data @xcite . distributionally robust optimization (dro) @xcite provides a natural solution to the issue by replacing the expected risk under a single distribution p with the worst expected risk over a predetermined family of distributions q. however, in dro, considering that direct gradient descent is hard to satisfy @xcite , how to model and optimize over q poses a key challenge. in this way, group dro @xcite is emerging as a methodology for constructing a realistic set of possible q under the annotated groups. crucially, robust optimization over worst groups becomes an active area of research. in general, the practical usage of group dro requires that group identities should be fully known. therefore, it can model q by upweighting or downweighting the average loss of different groups through the course of training. nevertheless, a key obstacle is that the under-represented groups are often unlabeled, or even unidentified. this makes even detecting such performance gaps, let alone mitigating them, a challenging problem. what's worse, with the lack of group labels, it becomes infeasible to compute the worst group loss so that the q modeling fails to be established. although, currently, some unsupervised dro methods for worstgroup optimization have been proposed @xcite , their concentration on optimizing high-loss group may discard considerable portion of the samples adversely impacting the overall accuracy. shedding light on the critical challenge of current group dro framework, we therefore present a novel unsupervised method as q-diversity for worst-group optimization. to realize the group identification without any annotations, we propose to parameterize a classifier as the group assigner for the attainment of group labels. in particular, by alternatively training the group assigner and final class predictor, we formalize an interactive training mode that allows the identification procedure feasible. intriguingly, we can treat the classification loss from the predictor as a direct supervision to guide the assigner for better group labeling. with the well-estimated groups, accordingly, the predictor can perform better on the worst group. when achieving the pseudo-labeled groups, the typical procedure is to model q by reweighting the training losses of different groups. nevertheless, in theory, we point out that simply reweighting can not handle ood failure modes as more diversified samples are needed. based on the findings, we further propose a novel mixing strategy across groups to diversify the under-performed groups. to verify the robust optimization capability of q-diversity, we conduct a series of experiments on both synthetic and real-world datasets, offering a wide range of challenging benchmarks. all the empirical results show our method not only outperforms other strong group dro strategies by a large margin, but also achieves consistent improvements on different ood test sets. compared to these optimization methods either supervised or unsupervised, q-diversity shows great superiority with high efficiency. altogether, our contributions can be summarized as follows: ## q-diversity modeling overview. we address two above limitations of group dro by proposing q-diversity. in our setup, we improve the classification accuracy of minority groups without explicit group annotations. the overall paradigm is depicted in figure 3 . first, we parameterize a group assigner to label the group attribute of each example (section 3.1). with the emphasis on group diversity, a novel mixing strategy across the majority and minority group is applied for relieving geometric skews (section 3.2). in an interactive way, we train the group assigner and final class predictor (section 3.3), allowing them to guide each other for better robust accuracy. ## experiments in this section, we conduct experiments on a synthetic sentiment classification task with complete spurious correlations and two real-world text classification tasks. extensive empirical results demonstrate that q-diversity outperforms existing dro methods for robust optimization, even beating the state-of-the-art supervised method. ## analysis and discussion in this section, we present a detailed analysis on the contribution of the diversified uncertainty set q to its strong unsupervised performance. furthermore, we explore the robustness of our method under different distributional shifts and random label noise. ## conclusion in this paper, we present q-diversity, an unsupervised method to optimize the worst group for model robustness. the formulation of q-diversity extends the annotations of group dro to an automatic assignment through an interactive training mode. furthermore, under the guarantee of a novel mixing strategy across groups, q-diversity can better counteract the failure modes of ood generalization. superior to previous works that only show the efficiency over the particular dataset, we demonstrate q-diversity promises better general-ization capability to various ood sets. we believe that our work casts light on the limitations of group dro which have been overlooked before, and can be viewed as a cornerstone for future study in the worst-group generalization.