Invariant Learning with Partial Group Labels
Authors: Vishnu Suresh Lokhande
Kihyuk Sohn
Jinsung Yoon
Madeleine Udell
Chen-Yu Lee
Tomas Pfister
Venue: ICLR 2022 Submitted (2022)
Keywords: Distributional Robust Optimization
Invariant Representation Learning
Semi-supervised learning
Dataset bias

Abstract: Learning invariant representations is an important requirement in training machine learning models that are driven by spurious correlations in the datasets. These spurious correlations, between input samples and the target labels, wrongly direct the neural network predictions resulting in poor performance on certain groups, especially the minority groups. Robust training against these spurious correlations requires the knowledge of group membership for every sample. Such a requirement is impractical in situations where the data labelling efforts for minority or rare groups is significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information. On the other hand, the presence of such data collection efforts result in datasets that contain partially labelled group information. Recent works have tackled the fully unsupervised scenario where no labels for groups are available. Thus, we aim to fill the missing gap in the literature by tackling a more realistic setting that can leverage partially available sensitive or group information during training. First, we construct a constraint set and derive a high probability bound for the group assignment to belong to the set. Second, we propose an algorithm that optimizes for the worst-off group assignments from the constraint set. Through experiments on image and tabular datasets, we show improvements in the minority group’s performance while preserving overall aggregate accuracy across groups.

Under review as a conference paper at ICLR 2022
INVARIANT LEARNING WITH PARTIAL GROUP LABELS
Anonymous authors
Paper under double-blind review
ABSTRACT
Learning invariant representations is an important requirement when training ma- chine learning models that are driven by spurious correlations in the datasets. These spurious correlations, between input samples and the target labels, wrongly direct the neural network predictions resulting in poor performance on certain groups, especially the minority groups. Robust training against these spurious correlations requires the knowledge of group membership for every sample. Such a requirement is impractical in situations where the data labelling efforts for mi- nority or rare groups is significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information. On the other hand, the pres- ence of such data collection efforts result in datasets that contain partially labelled group information. Recent works have tackled the fully unsupervised scenario where no labels for groups are available. Thus, we aim to fill the missing gap in the literature by tackling a more realistic setting that can leverage partially avail- able sensitive group information during training. First, we construct constraint set and derive high probability bound for the group assignment to belong to the set. Second, we propose algorithm that optimizes for the worst-off group as- signments from the constraint set. Through experiments on image and tabular datasets, we show improvements in the minority group's performance while pre- serving overall aggregate accuracy across groups
INTRODUCTION
Neural networks being overly biased to certain groups of the data is an increasing concern within the machine learning community [1]. A primary cause for bias against specific groups is the presence of extrancous attributes in the datasets that wrongly direct the model responses [38]. Such extraneous attributes are features that need to be controlled for. For example, in computer vision tasks such as image classification or object detection, an extraneous attribute could correspond to the background in an image or a co-occurring object irrelevant to the task, e.g. a person making speech in a football field could be predicted as playing football [6]. The presence of such extraneous attributes warrant model to derive the predictions by making spurious correlations to extraneous features an image rather than actual object of interest. An inevitable consequence of such correlations extraneous attributes is disparities in performance across different groups within the dataset. Specifically, if cer- tain groups form minority, model can simply cheat by having a high overall aggregate accuracy but poor minority group accuracy [24].
Existing works for this problem [2, 30] operate in the regime where the number of groups, likely to be adversely impacted through spurious correlations, are known apriori. Further, they assume a complete knowledge of the group membership of individual samples in the training dataset. While these methods have been proven effective, it is not realistic to assume access to the group labels for every sample. Consider the scenario where the minority / majority groups could be defined by demographic information such as gender or race. An individual can simply choose not to reveal this information due to privacy considerations [16). Alternatively, in medical image settings label class could contain unrecognized subgroups that demand significant burden on the data labelling efforts [33]. An example of such unrecognized subgroups could be a lung cancer detection problem, where the class label could comprise of groups such as solid/subsolid tumors and central/periphhraa neoplasms [24], with many of these groups naturally forming a minority in the dataset. In this work, consider setting where significant portion of the training data is devoid of group labels. We choose to fill a missing gap in the literature where several works bifurcate into methods that either are fully supervised or fully unsupervised in terms of the groups labels. Knowledge the number

---

Under review as a conference paper at ICLR 2022
class O, group
:clsss 1. group : :class 1. group
C: cless O, group? ?
AA
class O. g
A class 1, group
@ Un libeled
(8) Fullylabeeied
(i) Partislly-bbsieed
(a) Problem settings.
(b) Find the worst group assignment with the marginal distribution constraint.
Figure 1: (a) We introduce Worst-off DRO, an invariant learning algorithm for partial group labeled data (as shown in (iii). This is in contrast with other settings, such as (unsupervised) DRO [14] where (i) no group labels are available at train time, or Group DRO [30] when (ii) group labels for all training examples are required. (b) Worst-off DRO finds the worst-case group assignment for missing data with marginal distribution constraint, which may be given as side information, or estimated from labeled counterparts, and optimizes similarly to that of Group DRO. The constraint set defined by the marginal distribution includes the ground boound-truth group labels with high probability, ensuring the training objective of Worst-off DRO to be an upper bound of that of Group DRO with ground-truth group labels of entire data.
of groups in the data makes it convenient to obtain group labels for a tiny portion of the data or take advantage of an existing labelled samples if available. Hence, we address the following research question: Can we train model that is invariant to group membership using partially labelled data?
We answer the question using a framework of distributionally robust optimization (DRO) [23, 32]. DRO allows for training routine that optimizes for the worst-case training loss over predefined set of groups closely connected to the Rawlsian fairness measure [27]. When the group membership is fully known, the method simply upweights/downweighte average training loss of different groups through the course of training [30]. The application of DRO to the partial group label setting poses significant challenges: (1) the lack of group label makes it infeasible to compute the worst-off group loss; and (2) optimizing only for the high-loss samples, by considering them as a worst-off group, discards considerable portion of the training data adversely impacting the overall accuracy.
In light of these challenges, we make the following contributions in this paper. We construct a constraint set that can with high probability encompass the group labels of the unlabeled portion of the data. We optimize for the a worst-off soft group assignment from the constraint set which upper bounds the DRO objective with true group membership. Approximately, high loss samples are assigned to groups with higher weight and lower marginal probability improving the worst- off group accuracies. Further, the low loss samples are not discarded thus preserving the aggregate accuracy across groups We show experiments on two images datasets and one tabular datasets and contrast the performance against several baselines.
2 RELATED WORK
Distributionally robust optimization [4, 11, 23] has been studied as a way to train robust ML models across multiple groups and environments.
Group Robust Optimization. When group information is known at train time, Group DRO [15, 22, 25, 30, 40] or Invariant Risk Minimization (IRM) [2] could be employed improve the performance over multiple groups. Specifically, Group DRO proceeds by minimizing the loss of the group with the largest loss, while IRM enforces a shared predictor across multiple environments be optimal in the form of a Lagrangian multiplier.
Robust Optimization without Demographics. As the group information may not be always avail- able reliably, several studies have been focusing developing methods that remove or reduce their dependence on the group information. [14] has developed a method based on the distributionally robust optimization that minimizes the loss of the samples with losses larger than certain thresh- old. [18] has proposed to reweight the samples in an adversarial way so that the high loss sample could receive more attention over the course of training. Moreover, [20] has proposed simple yet effective two-stage approach called Just- Train- that trains a model by upweighting samples with high losses from the initial ERM model.
While above methods are based on optimizing high loss samples based on sample reweighting, GEORGE [33] and EIII. [7] are two-stage optimization methods, where in the first stage the group
2

---

Under review as a conference paper at ICLR 2022
environment labels are inferred and in the second stage Group DRO or IRM optimization ployed with inferred group labels from the first stage. Our method shares a similar idea to GEORGE or EIIL in that infer group labels for unlabeled data and apply (variant of) Group DRO for optimization. However, we infer group labels a way that the objective becomes the lower bound of that of Group DRO with true group labels with high probability. Such theoretical guarantee is important for safety-critical ML applications.
METHODOLOGY
We introduce robust optimization framework, Worst-off DRO, with partial group information. We revisit the GroupDRO in Section 3.1 and detail our method in Section 3.2 and 3.3. In Section 3.4, describe practical method for optimization.
3.1 PRELIMINARY: GROUPDRO
Let E X' C Idd be data descriptors, E y C [0, be target labels, and g C [1,.... M, be group labels. We assume training neural network parameterized by the weights w that corresponds to a per-sample loss [(x, y; w). Given data triplets {(I.. Vs. (.) h we seek to optimize w for the Rawlsian criterion [14, 27, 39], which minimizes the loss of the worst-off group, as follows:
min max B( y; w)]9] SEg
(1)
[30] proposed practical algorithm optimize Equation I, called Group DRO. This method opti- mizes a weighted expected loss across all groups These weights over the groups, denoted by are drawn from a simplex A The objective function is as described below,
19: = -.](,,,, N
LGDRO min max TEAM
Σ
4j
Σ 1 =
3.2 WORST-OFF DRO
In this work, we are interested in training distributionally robust neural network when group labels are only partially available in the entire dataset. That is, our training dataset constitutes of the fully- labeled dataset [xx; ys: g. } i= and the task-labeled dataset {(r,, Vs. - -) where - indicates the missing group labels.
As noted in Equation 2, the Group DRO requires group labels of entire dataset. When some of them are missing, we propose to optimize for the following objective::
Di=1 1(g =i}(( Mi. w)
LWDRO min max max w QEA EG
9j
Σ 1(0, =}}
where C is set of group assignments (ú:) i= satisfying 9J Hi Vi S K. We call the objective in Equation a Worst-off DRO as it optimizes neural network parameters with respect to the worst-off group assignment in some constraint set C.
Observe that the Worst-off DRO objective forms an upper bound to the Group DRO objective when evaluated at the ground-truth group labels. This is rather straightforward consequence from the fact that the ground-truth group assignment {g] L. falls within the constraint set C. However, for safety-critical applications, such as learning a fair classifier, it is important that the optimal objective (i.e.. Group DRO with a ground-trute -oruund--uthhhha group assignment) is bounded by the learning objective used in optimization. This is simply because optimizing the proposed learning objective guarantees that the corresponding lower bound ground-truth Group DRO is also optimized (assuming same parameters for both the methods). Conversely, objectives of methods such EIIL [7] or GEORGE [33], which optimize the Group DRO or IRM objectives using pseudo group labels, would be difficult to compare with LGDRO as it depends on a heuristic obtain single set of pseudo group labels.
3.3 REDUCING CONSTRAINT SET WITH MARGINAL DISTRIBUTION CONSTRAINT
is clear that the constraint set C plays an important role that connects Worst-off DRO to Group DRO. Specifically, the Worst-off DRO objective can be made tighter bound that of Group DRO
3

---

Under review as a conference paper at ICLR 2022
by further constraining C so long it contains the ground-truth group assignment ) In the subsequent paragraph, we describe how we reduce the constraint set while retaining the ground-truth group assignment using marginal distribution constraint. These constraints may be given side information or could be estimated from the small set of partial group labels under certain conditions.
Let Cp.c C C is subset of whose elements
{gi V satisfy the following condition:
- Vi VisA K,
(4)
Σ 1-1 1 [ =) S,
(5)
N
where Equation 4 implies that the group labels assigned whenever available, and Equation implies that the data marginal distribution should be close to the marginal distribution Then, for any marginal distribution p and c O, it is easy to show EwDRo(Cp.e) (WDRO(C) as Cp.e C Moreover, will see in Lemma I that, with high probability, the constraint set Cp', with the true marginal distribution p' contains the true group assignment {g*}-
Lemma 1.
The constraint set Cp' contains the true group labels {g; with high probability:
PlAT Cp' 2e- 2NN
(6)
The proof is in Appendix A.1. As in Equation 6, the probability of the constraint set containing the true group labels gets closer to by allowing larger variance (c) from the true marginal distribution. For fixed > O, the probability gets closer to as we increase the number of unlabeled data (N).
Finally, this implies that LWDRO(Cp :e) is an upper bound to that of Group DRO:
LGDRO < (wDRO(Cp...) S LWDRO(C) w w.h.p
In practice, however, the true marginal distribution p' may not be available. For our setting where group labels are partially available, with an assumption that group labels are missing completely at random (MCAR) [29], the true marginal distribution could estimated from the subset of data with group labels. This again allows us to formulate constraint set that contains the ground-trute group assignment with high probability.
To be more specific, let be the estimate of the marginal distribution from {(T,, Us. KK
Lemma 2. The constraint set Cp,A+e contains the true group labels igt )N with high probubility:
P([o ECp.8+c) 2
2N 22N 2e 2KK -2K6
We provide a proof in Appendix A.1. Here, ò is introduced to take into account the estimation error of the true marginal distribution p". When K, the number of labeled data, is large, the bound in Equation is close to 1.
Assuming MCAR, for large K: LGDRO LWDRO(Cp.ste)
w.h.p
3.4 A PRACTICAL OPTIMIZATION ALGORRTTH
We are interested in solving the optimization problem LWDRO (Cp.). Unfortunately, the inner maxi- mization problem with respect to the group assignments {g) in Equation 3 is challenging variables are discrete and the objective cannot be decomposed due to the marginal distribution constraint. In this section, describe optimization recipe with few approximations.
First, we propose to use a soft group assignments. This not only converts the problem into con- tinuous optimization problem, but also accommodates inherent uncertainties in group assignment

---

Under review as a conference paper at ICLR 2022
Algorithm Worst-off DRO Algorithm
} 1: Imput: Fully-labelled dataset {(ra, Mis P i== task-labeled dataset {(ri, Vis -) i=
2: Initialization:: learning rates The and Tl. Marginal distribution p,
3: Parameters: Group Weights qj, Worst-off DRO group assignments g.
Neural network parameter
4: for = 0, 2, do
5: {Ø'} max (U)CC.. E/=1 M
of
2, Besl(a 2, Bis
where, Cp.c defined in Equation 9.
6: Gradient descent on w:
w²+1 + w² ne V Σ/= 9j
Σ. oig (asmwa))
7: Exponential ascent on
E, oiylea.ee:"
g+ + g² exp (n, V EØ=1 4j
E, E.06 big
8: end for
9: Output: Trained neural network parameters w"
for data with missing group labels. Specifically, for each data, we retain soft group assignment
9. A M, and optimize the Worst-off DRO objective over the constraint set Cp.e as defined below:
Dig Pigl(an (r; Vi. w)
min gEAM ES)EC
45
===1
(8)
1= Dij
where the constraint set is defined as:
EEAM Vi V. Pii) = 1, K E و.j - p,lse
Cp.e
(9)
The first condition ensures assignments the probability simplex, second one ensures assignments are consistent with ground-truth for labeled data, and the third one ensures the data marginal dis- tribution follows the provided distribution. This resulting optimization problem is solved using off- the-shelf CVXPY solver [8]. More details this and an example are provided in the Appendix A.2.
Finally, we alternate optimization over w, g and {0} as Algorithm 1. That is, we first solve inner maximization over {g], and conduct gradient descent and iterate. An exponential ascent on q achieves smaller losses for linear predictors (like q) [17].
and the exponential gradient ascent
EXPERIMENTS
We test the efficacy of our method on image and tabular datasets, each of which consists of samples from mutually exclusive groups environments. These groups indicative of the background or an RGB identification for image datasets, and attributes such as gender or race for tabular datasets. As discussed in Section size and demographics. The presence of minority groups results in possible scenario where the aggregate performance is (falsely) remarkable, because evaluations even though the performance on the minority groups is poor. In our experiments we assume group numbers be known but group labels are missing completely at random at a fixed rate at train time.
or more of the available groups form minority terms of the sample
dominated by larger groups,
In Section 4.1, we outline our baselines for comparison, our metrics of evaluation and the model se- lection strategy. In Section 4.2, we describe each dataset in detail and highlight the differences across the groups within the dataset. All the quantitative results are available in Table 2 and per-group sum- mary statistics are present in Table 1. More analysis our method is provided Section 4.3.
4.1 EXPERIMENTAL SETTINGS
Baselines. We contrast the performance of method with respect few well-known baselines.
ERM:
Empirical Risk Minimization that optimizes aggregate average loss over the samples in the training dataset.
5

---

Under review as a conference paper at ICLR 2022
Dataset # Labeled
# UnLabeled
Total samples
# Groups
# Minority Samples
Majority Samples
Waterbirds 508
4287
4795
4
113
CMNIST
3983
35657
39640
3
276
2138
Adult
1308
11635
12943
823
ColebA
8030
154740
162770
3547
Table 1:
Dataset description. We show sample counts in labelled and unlabelled training sets, as well as counts for majority and minority groups. The number of labelled samples are about 10% of total samples.
2. Unsup DRO [14]:
Samples with losses exceeding threshold v considered group whose average loss is optimized. Since the method doesn't require group labels, it is an unsupervised algorithm.
3. Group DRO [30]: A method that optimizes the Rawlsian criterion by assigning simplex weights to the groups. The group labels for individual samples are assumed to be available, hence this method is fully-supervised in terms of the group label.
4. Group DRO (Partial): We consider another variant of Group DRO that only uses samples with group labels at train time. We call the method Group DRO (Purtial), to contrast with the above baseline, Group DRO (Oracle).
We compare above methods with our proposal, Worst-off DRO. Note that our approach requires marginal probabilities as an input to the algorithm, which are computed from the training dataset in our experiments. For baselines and Worst-off DRO implementations, samples are drawn randomly for every batch ensuring an unbiased comparison to the ERM baseline. This is unlike [30] who adopt weighted sampling procedure which could be noisy when group labels are uncertain.
Evaluation Metrics. We set aside a test set whose group labels are fully available. Since all of our datasets characterize classification task, we evaluate overall accuracies and per-group accuracies in our experiments. Specifically, we highlight the accuracy of the minority group (min) together with the overall (avg) accuracy where individual samples equally weighted regardless of their group.
Model Selection, Model selection plays a crucial role when distributional differences are observed in dataset [13]. In our problem setting, individual groups may differ from each other in the joint dis- tribution over the data and the label space, however, the testing set resembles the training set. That is, there are no out of distribution samples and the focus is to improve robustness over a predefined set of groups common to both training and testing datasets. Consequently, among the recommendations made in [13], a training domain validation set is feasible strategy for our problem. In our algo- rithm, learning rate and weight decay are important hyper-parametery Prior works [30, 31] noted that ERM fails to optimize for the minority group's ' performance under high regularization regime alternative. Hence, we compare our methods in this regime. We also tune for
thus necessitating algorithmic specific hyper-parameters for each baseline. These arper-parameters are the loss thresh- old in Unsup DRO (MJDRO) and the step size for the group weights in Group DRO (MaDRO) and in Worst-off DRO ()WDRO). A list of all hyper-parameter choices used in the experiments is provided in the Appendix A.3. All the numbers reported in the paper were averaged over three random seeds.
We adopt NVP (novel validation procedure) [9] in our experiments. In this procedure, we first search for hyper-parameters with the best overall accuracy. Then, from the top five best performing hyper-parameters, we select the model that achieves the highest minority group accuracy. Such a procedure offers robustness to bypar-parameters in the reported numbers.
4.2
QUANTITATIVE RESULTS
We describe key results on four datasets, Waterbirds [30], CMNIST [2], Adult [10], and CelebA [21].
4.2.1
WATERBIRDS
The dateset, used in [30], comprises of 4795 images of birds from the CUB dataset [37] and the backgrounds taken from the Places dataset [41]. Each image in the dataset has background of land or water. The target labels are either "landbirds" or "waterbirds". The authors in [30] create four groups with each target label and background class considered as group. In this dataset the groups "landbirds" on water and "waterbirds" on land form a minority. Our results in Table firstly shows that the ERM method attains small minority group accuracy of 60%. All the invariant learning baselines, except for Group DRO (Partial), improve the minority group's 's accuracy. Next, observe that in comparison to Group DRO (Partial), our proposed Worst-off DRO improves the

---

Under review as a conference paper at ICLR 2022
Waterbirds
CMNIST
Adult
CclebA
min
avg
min
avg
min
avg
min
avg
Group DRO (Oracle)
83
92
50
75
82
88
80
94
ERM
60
13
79
68
92
45
95
Unsup DRO
65
88
10
80
68
92
39
96
Group DRO (Partial)
44
81
36
76
67
90
40
95
Worst-off DRO
65
89
39
77
71
91
49
95
Table 2: Quantitative Results. For baselines, we consider an ERM, Unsup DRO [14], Group DRO (Partial)
for partly labelled Group DRO [30] method, Group DRO (Oracle) for the fully supervised model. Our method Worst-off DRO improves the minority group's accuracy (min) while maintaining similar overall accuracy (avg) relative to baselines. The accuracies computed on the test set and an average over three random runs. The standard deviations are provided in the Appendix Table 5.
minority group's performance by a significant margin of 21%. Due to this improvement, the all- group accuracy also improves by 8%. Minority group's performance on sully-super method Group DRO (Oracle) is at 83% accuracy with a window of 18% difference from Worst-off DRO.
4.2.2 COLORED MNIST (CMNIST)
CMNIST, derived from an MNIST [19], is digit recognition dataset where each image is colored either red or green. Digits 5/ 5 are considered as label 0/1. We consider three groups in our experiments. In the first two groups label 0 images are predominantly colored red and vice versa. In the third group, which forms minority, predominantly colored red. Specifically, for the first two groups, the color id is sampled by flipping the target label with probabilities 0.2 and 0.1 respectively, while the third group with probability 0.9. Both training and testing sets contain three groups The overall setup for generating a given group is similar [2]. We show the results on CMNIST in Table 2. Similar to the Waterbirds dataset, Worst- off DRO improves the minority group's s accuracy compared to the ERM method. Relative to Group DRO (Partial), Worst-off DRÔ improves the accuracy of the minority group by 4% and 1% in the overall accuracy. The margin between Worst-off DRO and Group DRO (Oracle) is 11% Among all the baseline, Unsup DRO attains lowest minority group accuracy of 10%. A large trade-off between the minority group accuracy and the all-group accuracy was seen for Unsup DRO in this dataset.
switch coloring such that the label 1 images
4.2.3 ADULT DATASET
We use semi-synthetic version of the Adult dataset [10] for this experiment. Similar to [18], we consider race and sex as the four demographic groups. The target label is income > 50KS and is treated label 1. Similar to the CMNIST dataset, each group has different correlation strength to the target label. For the purposes of the experiment, we exaggerate these spurious correlations caused by group membership close to [7]. Particularly, for samples with group label as Afican- American, we undersample examples with probability P(y = 1 group) = 0.06 whereas for the non African-American group labels, oversample examples with probability P(y = 1 group) 0.94. Table indicates a 5% improvement in the minority group's 's accuracy while maintaining the similar overall accuracy of 90% compared to Group DRO (Partial). The Group DRO (Oracle) method reaches an accuracy of 82% for the minority group compared to Worst-off DRO which achieves 71%. Evidently, ERM underperforms in terms of the minority group's 's accuracy and attains about 68% accuracy.
4.2.4 CELEBA DATASET
CelebA i21]is dataset containing about 200k celebrity faces curated from the internet. There are 40 labels available in this dataset which are annotated by a group of paid adult participants [5]. Similar to [30], we aim to predict the target attribute Blond Hair that is spuriously correlated to the Gender attribute. Specifically, having blond hair correlates with the female attribute. The minority group in this dataset are the images with attributes (blond, male). The proportion of samples in the minority and the majority group is show in Table 1. The quantiative results in Table indicate an improvement of 9% over the Group DRO (Partial) method for the proposed Worst-off DRO algorithm. The Group DRO (Oracle) method achieves the highest minority group accuracy of 80%. The minority group performance for the ERM method, with accuracy of 45% is comparable to Group DRO (Partial). All the methods are similar terms of the average group accuracy with values > 90%.
7

---

Under review as a conference paper at ICLR 2022
Worst-off DRO
Gruup DRD
Worsteoff DRC
Group DRO
Worstaff DRC
Group DRO
Wantseff DRC
Group DROo
74-
Kin Acourat
Mir Accurayy
42
42-
Min
42
30-
25
18 Label percent
05
Label percent
Label
Label
(a) Waterbirds
(b) CMNIST
(c) Adult
(d) CelebA
Figure 2: Increasing the labelled samples. Minority group accuracies are plotted different counts of the labelled samples in the training dataset. Both, Group DRO (Partial) and Worst-off DRO algorithms improve the minority group accuracies with more training labels. Also, the Worst-off DRO method has higher accuracy values than Group DRO method. The aggregate group accuracies are shown in the Appendix Figure 5,
4.3 ABLATION STUDIES
In this section, we discuss different components of our algorithm that influences it's performance.
4.3.1 INCREASING THE LABELLED SAMPLES.
Recall that for the quantitative results in Table 2, the number of labelled samples were around 10% of the total training samples. In this section, we investigate the effects of increasing the number of labelled samples provided to the training algorithm. Although obtaining annotations for groups is an arduous task [18], having more labelled groups provides two benefits for the algorithm. Firstly, the standard deviation of errors in estimating the marginal probabilities from the labelled portion of the data reduces [36] ( A vif samples rate). Secondly, labelled groups reinforce an accurate evaluation of the Rawlsian objective in Equation 2 and appropriate weight updates for the groups. The results shown in Figure 2 depict the minority group accuracy at different labelled percent thresholds. The corresponding plots for average group accuracies are provided in the Appendix Figure 5, Worst-off DRO method is compared with Group DRO (Partial). It is observed that for both the methods, the minority group's accuracy increases with more labelled data. Furthermore, the accuracy values for Worst-off DRO method are better than Group DRO at several thresholds. The methods converge at threshold specific to the datasets. Increasing the labelled counts beyond such a threshold saturates the Worst-off DRO performance, however, Group DRO (Partial) consistently improves until Oracle performance is attained.
4.3.2 MINORITY GROUP VS. OVERALL ACCURACY
As discussed in Section 1, several groups in the training dataset, especially the minority groups, could be distributionally different from the majority group samples. Consequently, a mild tradeoff surfaces between the minority group accuracy values and the aggregate accuracies. Addressing this issue, recall that we leverage a robust model selection criterion such as NVP (see Section 4, model selection paragraph) that balances both the minority and aggregate group accuracies. We extend the results in this section, by plotting evaluations at different pppee--rramtter choices for our algorithm. Figure 3 contrasts minority group and overall accuracy across all the datasets. Evidently, the top- right corners are desirable regions for the models to be present with maximum performance across both the metries. The Adult dataset in Figure 3 shows a clear envelope on the Worst-off DRO models that surpass the corresponding Group DRO models. Similar trend exists on the remaining datasets with more offst-off DRO models concentrated in the top-right corner.
4.3.3 PROGRESSION OF GROUP WEIGHTS
Our algorithm 1, proceeds by assigning weights, the valuuess, to every group. These q vavuus are updated through exponential ascent similar to [30]. Noticeably, the updates on q depend on the worstoff group assignments determined from the constraint set Cp.s+e. In this section, we investigate on how these group weights evolve through the several iteration of the proposed algorithm. Figure plots this evolution across different datasets with the number of curves in given plot matching the group count of that dataset. Since the q- weights are initialized uniformly at random, the curves begin at the same value. As the training progresses, it is observed that the weights on the minority
8

---

Under review as a conference paper at ICLR 2022
Worst-off BRC
Group DRD
Worst-of
Group DRO
Worstoff BRC
Group DRo
Wons-off Group DROo
2.55
Nin. Accuracy
25
2.45
Min Rcc Recusaacy
X
0.2
3:35
Nin.
1.30
41-
3.2
12
21-
2.20
0.
3.8
0.
0.4
0.70
2000 0.0
0.34
Accuracy
(a) Waterbirds
(b) CMNIST
(c) Adult
(d) CelebA
Figure 3: Minority Group vs. Average Accuracy. Evaluations for different hyper-paramend
choices are plotted for Worst-off DRO and Group DRO (Partial) methods. Models from Worst- off DRO training are concentrated in the top-right corner of the plots. This is desirable indicating a high accuracies across the two metrics. For model selection from among the possible choices, we adopt the NVP procedure (see Section 4).
26
3.338
91
Gr
0.vales
24-
2.33
4;
3
9
9.2
2.352
62-
01
3230
2300
5283
160
430
350
5033
10528
20000
20205 42000
60000
Train
(a) Waterbirds
(b) CMNIST
(c) Adult
(d) CelebA
Figure 4: Progression of group weights. The evolution of v- (see Algorithm 1) is plotted for each group. The values for the minority groups increases gradually while those of the majority groups reduce. A high value indicates that the corresponding group receives a higher weight relative to other groups In the plots, minority group is indicate by a on q.
groups gradually increase and those on the majority groups reduce. The plots indicate high g- value on the minority groups towards the end of training. This is desirable because the empirical risk on the minority groups getting upweighted relative to the majority groups.
CONCLUSION
We present Worst-off DRO, an invariant learning method across groups when partial group labels are available. The formulation of Worst-off DRO extends that of Group DRO by optimizing the loss against the worst-off group assignments in the constraint set. By reducing the constraint set with the marginal distribution, we reduce the optimization parameter space while keeping the objective to be an upper bound to that of the Group DRO with true group assignments with high probability. By harnessing both labeled and unlabeled data terms of group, we demonstrate experiments that the Worst-off DRO outperforms both ERM, UnsupDRO, which do not make use of available group labels, as well as the Group DRO (Partial), which does not use unlabeled data.
One future direction, when marginal distribution is not available, is to relax our missing completely at random assumption and bring in different but more realistic modeling assumptions on the miss- ingness of group labels, such missing at random (MAR), where missing values depend other observed attributes [12]. In addition, it would be valuable to design a reduced constraint set contain- ing the true group assignment to reduce the performance gap the Group DRO (Oracle).
9

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
Machine Learning (ML) models that perform poorly minority group or environment have raised lot of concerns within the Al community and broader society in recent years. To democratize ML in real world, learning ML models that perform robustly across groups or environments has become an important venue of research. The proposed Worst-off DRO is a versatile method that could be employed to train invariant classifier across groups even when the group information is available only for the portion be missing for various reasons during the data collection. We further emphasize the importance of theoretical result showing the objective of Worst-off DRO being an upper bound to that of Group DRO with complete group information for safety-critical ML applications.
the data. This is a rather practical scenario as the group information could
REPRODUCIBILITY
We write our experimental code from scratch using PyTorch library [26]. Due to its similarity, our implementation may closely follow that of Group DRO [30].' One of the key differentiation of Worst-off DRO is the inner maximization solver for the worst-off group assignments {ő), which we claborate the exact code using CVXPY solver [8] Algorithm 2 of Appendix. Additional imple- mentation details, including the neural network architectures, as well as value for hyperparameters including the learning rate, weight decay, batch size, number of training epochs, and algorithm- specific parameters are summarized in Table 3 of Appendix and Section A.3 and A.5.
REFERENCES
[1] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. In ICML, 2018.
[2] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint arXiv:1907.022 2019. 1,2, 6,
[3] Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. Robust optimization. Princeton university press, 2009.
[4] Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Ren- nen. Robust solutions of optimization problems affected by uncertain probabilities. Manage- ment Science, 59(2)3341-357, 2013. 2
[5] Mare Böhlen, Varun Chandola, and Amol Salunkhe, Server, server in the cloud. who is the fairest in the crowd? arXiv preprint 1rXiv:/711.08880 2017. 7
[6] Jinwoo Choi, Chen Gao, Joseph CE Messou, and Jia-Bin Huang. Why can't i dance in the mall? learning to mitigate scene bias in action recognition. NIPS, 2019.
[7] Elliot Creager, Jörn-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant learning. In ICML, 2021. 2, 7, 16
[8] Steven Diamond and Stephen Boyd. CVXPY: Python-embedded modeling language for convex optimization. JMLR, 17(83):1-5, 2016. 5, 10
[9] Michele Donini, Luca Oneto, Shai Ben-David, John S Shawe-Taylor, and Massimiliano Pon- til. Empirical risk minimization under fairness constraints. S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianche and R. Garnett (eds.), NIPS. Curran Associates,
Inc., 2018. 6
[10] Dheeru Dua, Casey Graff, et al. Uci machine learning repository. 2017. 6, 7, 16
[11] John C Duchi, Peter W Glynn, and Hongseok Namkoong. Statistics of robust optimization: A generalized empirical likelihood approach. Mathematics of Operations Research, 2021. 2
[12] Martinez-Plumed Fernando, Ferri Cèsar, Nieves David, and Hernández-Orallo José. Missing the missing values: The ugly duckling of fairness in machine learning. International Journal of Intelligent Systems, 2021. 9
httpp:://gi thub. com wwwp:/oowww.cooeeeecoceoooo /gropppDROO
10

---

Under review as a conference paper at ICLR 2022
[13] Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In ICLR, 2021.
[14] Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning, pp. 1929-1938. PMLR, 2018. 2, 7,
[15] Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama. Does distributionally robust super- vised learning give robust classifiers? In ICML, 2018. 2
[16] Thomas Kehrenberg. Myles Bartlett, Oliver Thomas, and Novi Quadrianto. Null-sampling for
interpretable and fair representations. In ECCV, 2020. 1
[17] Jyrki Kivinen and Manfred K Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Information and computation, (32(1):1-63 1997.
[18] Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), NeurlPS, 2020. 2, 7, 8
[19] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, (6(11))2278-233 1998.
[20] Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In ICML, 2021. 2
[21] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015. 6, 7, 16
[22] Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. In ICML, 2019. 2
[23] Hongseok Namkoong and John C Duchi. Stochastic gradient methods for distributionally robust optimization with f-divergences. In NIPS, 2016. 2
[24]
Luke Oakden- Rayder, Jared Dunnmon, Gustavo Carneiro, and Christopher Ré, Hidden strati- fication causes clinically meaningful failures in machine learning for medical imaging. In Proc ACM conference on health, inference, and learning, 2020.
[25]
Yonatan Oren, Shiori Sagawa, Tatsunori B Hashimoto, and Percy Liang. Distributionally ro- bust language modeling. EMNLP//JCNLP,, 2019.
[26] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. NIPS, 2019. 10
[27] John Rawls. Justice as fairness: A restatement. Harvard University Press, 2001. 2,
[28] T Mitchell Roddenberry, Florian Frantzen, Michael T Schaub, and Santiago Segarra. Hodgelets: Localized spectral representations of flows on simplicial complexes. arXiv preprint arXiv:2109.0872. 2021.
[29] Donald B Rubin. Inference and missing data. Biometrika, 63(3):581-592, 1976. 4
[30] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In ICLR, 2019. 1, 2, 6, 7, 8, 10, 16
[31] Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameteriizatoon exacerbates spurious correlations. In ICML, 2020. 6
[32] Alexander Shapiro. Darinka Dentcheva, and Andrzej Ruszczynski. Lectures on stochastic programming: modeling and theory. SIAM, 2021. 2
[33] Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No sub- class left behind: Fine-grained robustness in coarse-grained classification problems. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), NeurIPS, 2020. 1.
3.3
[34] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011. 16

---

Under review as a conference paper at ICLR 2022
[35] Serena Wang, Wenshuo Guo, Harikrishna Narasimhan, Andrew Cotter, Maya Gupta, and Michael Jordan. Robust optimization for faimess with noisy protected groups. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), NeurIPS, 2020.
[36] Larry Wasserman. All of statistic:: concise course in statistical inference, volume 26.
Springer, 2004. 8
[37] Welinder, Branson, T Mita, C Wah, F Schroff, S Belongie, and P Perona. Caltech-uesd birds 200. technical report cns-tr-2010-001, California Institute of Technology, 2010. 6
[38] Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig. Controllable invariance through adversarial feature learning. arXiv preprint arXiv: /705. / /22, 2017.
[39] Chongjie Zhang and Julie A Shah. Fairness in multi-agent sequential decision-making In NIPS, 2014. 3
[40] Jingzhao Zhang, Aditya Krishna Menon, Andreas Veit, Srinadh Bhojanapalli, Sanjiv Kumar, and Suvrit Sra. Coping with label shift via distributionally robust optimisation. In ICLR, 2020.
2
[41] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. PAMI, 40(6)):452-144 2017. 6, 16
[42] Xiaojin Zhu and Andrew B Goldherg. Introduction to semi-supervised learning. Synthesis lectures on artificial intelligence and machine learning, 3(1):1-130, 2009.
12

---

Under review as a conference paper at ICLR 2022
Worst-off DRQ
Gruup DRD
Worsteoff DRG
Group DRO
Worstaff DRC
Group DRO
Wanshoff DRC
Group DROO
Avg. Accurscy
Avg. Accursyy
52.5
22
90.0
27.5
Ave
22
20.0
50
98
Label
Label
Label
Label percent
(a) Waterbirds
(b) CMNIST
(c) Adult
(d) CelebA
Figure 5: Inereasing the labelled samples Average Group Accuracy. We plot the average group accuracies as a function of labelled samples. These accuracies remain fairly similar as the count of labelled samples grows.
A
APPENDIX
A.1 PROOF OF LEMMAS
Lemma 1. The constraint set Cp'.e contains the true group labels {97 Jis
Fizi with high probability:
Pl CCpr..) 2 -2 2e -2Nc
The probability of the true group assignment {ot - in the constraint set Cp' .e is written as follows:
Proof.
Plig -Cp.,) P(p 1 N
Σ.-, - se) 2 1 - 2e -2N
(10)
where Equation 10 holds true from Hoeffding's inequality.
Lemma 2. The constraint set Cp.s+e contuins the true group labels {ot J.M with high probability:
P( Cp.otte 2 1- 2Ne 2e -KK
Proof. Using Hoeffding's 's inequality, we can show that the estimation error of the marginal distribu- tion is bounded by 8 with high probability as follows:
P(P) p,l 8) P( -
K
llg = = 56) 2 1 2e 2e--ee22 -2K8
(11)
Furthermore, we show using Hoeffding's 's inequality that
P P(I05 - 1 N 1=1 1 =- ( 2 2e 2N2
N Σ
(12)
Now, the probability of the true group assignment 1 in the constraint set Cp.o+e is written as follows:
P(1g) li=1 E Cp.o+e) (a0)))p((pp)))
N 1=1 19g. = -ol= 5o]
1 Σm
(13)
2 P (f112) |p} p, s 6) {|p} 1 Σ i=1
NN
1fg = 57| s sc))
(14)
N
2 P (1p: - p, < + P(P)
[p}
Σ
Lig = ألز s -)
-1 (15)
N
2 1 20122-220 2K
2c 2N
(16)
where Equation 14 is due that the intersection events in Equation 14 subset of an event in Equation 13, and Equation 15 is derived using union bound.
13

---

Under review as a conference paper at ICLR 2022
Waterbirds
CMNIST
Learning Rate
0.0001, 0.00001, 0.000001
Learning Rate
0.001, 0.0001, 0.00001
Weight Decay
L5, 1.0, 0.1 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.1, 0.01, 0.001
Weight Decay
0,01, 0.001, 0.0001 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3 0.01, 0.001, 0.0001
YJUDRO
TJUDRO
Y/DBRO
TJGDRO
7WDRO
0.1, 0.01, 0.001
TTWRR
0.01, 0.001, 0.0001
Adult
CelchA
Learning Rate
0.001, 0.0001, 0.00001
Learning Rate
0.0001, 0.00001, 0,000001
Weight Decay
0.01, 0.001, 0.0001 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3
Weight Decay
1.0, 0.1, 0.01 0.9, 0.8, 0.7, 0.6, 0.5, 0.4,
YJUDRO
TJUDRO
1/GDRO
0.01, 0.001, 0.0001
T/CDRO
0.1, 0.01, 0.001
7WDRO
0.01, 0.001, 0.0001
1DWRR
0.1, 0.01, 0.001
Grid search for Table 2. The range of values for each hyper-parameter is listed. A grid search conducted to identify the best performing model. Models outside these range over these hypeer-araaeeeers values were observed to be either unstable or not converging. Model selection is done based on NVP (novel validation procedure) where first the models, with higher overall accuracies, are selected. From the top five such performing models, the with the highest minority group accuracy picked.
Table 3:
Dataset
Method
Architecture
Learning
Weight Decay
Batch Size
Eipochs
Other params
Waterbirds
ERM
ResNe150
0.0001
0.1
128
300
Waterbirds
Unsup DRO
ResNe150
0.0001
0.1
128
300
7=0.3
Waterbirds
Group DRO-(Oracle)
ResNe150
0,00001
1.0
128
300
0=0.001
Waterbirds
Group DRO-(Partia)
ResNe150
0.00001
0.1
128
300
0=0.001
Waterbirds
Worst-olf'
ResNe150)
128
0=0.001
CMNIST
ERM
MLP(390.390)
0.001
0.01
500
CMNIST
Unsup DRO
MLP(390,390)
0.00001
0.001
500
0=0.
CMNIST
Group DRO-(Oracle)
MLP(390,390)
0.0001
0.001
500
00000
CMNIST
Group DRO-(Partial)
MLP(390,390)
0.001
0.01
500
000001
CMNIST
Worst-off' DRO
M1P(390,390)
0.0001
0.01
500
q=0.0001
Adult
ERM
MLP(64.32)
0.0001
0.001
128
200
Adult
Unsup DRO
MI MLP((64332)
0.0001
0.001
128
200
g=0.3
Adult
Group DRO-(Oracle)
MLP(64.32)
0,0001
0.001
128
200
000.0001
Adult
Group DRO-(Partial)
MLP(64.32)
0.0001
0,01
128
200
0=0000
Adult
Worst-off DRO
MLP(64.32)
0.00001
0.001
128
200
y
CelebA
ERM
ResNe150
0.0001
0.01
128
CalehA
Unsup DRO
ResNe150
0.0001
0101
128
0=0.6
CelebA
Group DRO-(Orucle)
ResNet50
0.00001
0.1
128
50
7=0.1
CelchA
Group DRO- (Partaaaia
ResNe150
0.01
CelebA
Worst-off DRO
ResNet50)
0.00001
0.1
128
50
0000001
Table 4: Hyperparamter choices for Table 2. We list the hyper-parameter selected using the NVP procedure (see Section 4) after performing grid-search. Learning rate and weight decay are an important of parameters that influences the minority group performance. Each baseline has it's agorithm--specfice typer-parameter such as step-size of the simplex weights in Group DRO (NKDRO). the loss threshold in Unsup DRO (TEDRO) and the step size for the group weights in Worst-off DRO (JnWdRD). The symbol "_" for batchsize in CMNIST experiments indicate the use of full-batch data for training.
A.2 NOTES ON OPTIMIZATION
When using CVXPY to solve for the Worst-off DRO assignments, simplify the problem by replacing the data marginal distribution Σ dig in the denominator Equation 9 Py, thus providing us with a convex optimization problem. The code for the solver is available in Algorithm 2.
Next, Using three sample, we provide an example of the worst-off assignments made by our algo- rithm,
Example 3. Consider three samples with loss values 4 le > la and two predefined groups Assume the marginal probabilities P1 = 0.6 and P2 0.4. Without loss in generality, assume
91 22 Solving for Worst-off DRO objective results in the following group assignments. With
Di
D="
14

---

Under review as a conference paper at ICLR 2022
Algorithm 2 Group Assignment Solver using CVXPY library
import cvxpy import numpy as up
class Solver (abject):
def init init__ntt______________ (self, controls, bsize, marginals, epsilon, labeled-None): **rrou assignment solver.
Argumenta:
n_controls: integer the number groupa.
bsize: An irtager for the batch size. marginals: 20 array fuz the marginal distribution.
epsilon: flost for variance.
van labeled: A tuple for labeled data indices and their value.
self.X cp. Variable ( (baize, n_contzola))
self.1 Parameter (bsize, 1)) self.q
self.p cp qp.Parameter (e_comsrole, 1), value-mazginals)
labeled is not None:
labeled_idx, labeled_vaine labeled
counts sp. axis=0, keandims=True)
ebj [/self.1. sel5.X)
self.p.T) self.g
conscraints [self.X >=
sp.aum/self.A fself.x, exis=l, keepdins-True) -- ap.ones (lasize, 1)),
ap.abs scp. axis-0, keepdime-Iruet baize self.p.
epailor]
i1 labeled is not None:
constraints +- (self.X Xeabeex(aableediidd
labeled_value)
self.prob cp. Pwoblerllmmoom Maximize M (obj), constraints)
def evxsolve Iself, lesses, weights): **sSolver.
Arguments:
losses: A 20 array for loas values. weighta: array group weights g,
Returns: acray for soft group assignments.
selE.1.value losses sel f.l..q.va lue wei s selF.prob. solve return self.x.value
1
0
constraint Cp.e=u, we obtain {ğ'}
0.8
0.2 0 1 )
Here, the tth row indicates the assignment given
to sample 4,
The above example can be derived by identifying {ğ} that satisfies the constraints Σ In1 S
Np1 and Σ Hi2 Np2. where N 3, Pi 0.6 and P1 0.4 and correspondingly maximizes Worst-off DRO objective. The above example informs us that group assignments depend on the magnitude of loss values in addition to the group weights and marginal probabilities. Importantly, we find that high loss samples are assigned to groups with high group weights and low marginal probabilities.
Observe that the marginal constraints form key ingredient of our algorithm per the above example. Without marginal constraints, we would have obtained the group assignments
00 = ( 0 That is, the assignments would have been made independent of the loss val-
ues and sparsely restricted to the group with large 0 value. Pj
A.3 HYPER-PARAMETE TUNING
parpmeerssssters were selected for each algorithm by performing an NVP procedure (see Sec- tion 4). The best performing model was identified the validation set associated with each dataset.
15

---

Under review as a conference paper at ICLR 2022
Waterbirds
CMNIST
Adult
CclebA
min
avg
min
avg
min
avg
min
avg
Group DRO (Oracle)
0.45
0.02
0.57
0.33
0.94
0.87
1.31
0.27
ERM
0.99
0.07
1.47
0.49
1.06
0.28
3.02
0.05
Unsup DRO
0.89
0.12
1.39
0.64
1.73
0.18
1.81
0.02
Group DRO (Partial)
1.25
1.25
0.76
0.20
0.32
0.34
4.54
0.08
Worst-off DRO
0.96
0.17
1.14
0.35
0.21
0.13
2.05
0.10
Quantitative Results SSandard Deviations. The standard deviations over three random runs of Table is provided. For baselines, we consider an ERM, Unsup DRO [14), Group DRO (Partial) for partly labelled Group DRO [30] method, Group DRO (Oracle) for the fully supervised model.
Table 5:
All the measures were computed and averaged over three random runs. A list of all the hyper- parameters that were tuned for are available in Table 3. The final hyper-parameters selected for each method can be viewed from Table 4.
A.4 ADDITIONAL EXPERIMENTAL RESULTS
We provide the following additional results, first, in Figure 5, we show average group accuracies as function of labelled sample counts. The average group accuracies of the Worst-off DRO method are closely similar across various labelled sample counts. The Group DRO method shows a slight increasing trend in the average accuracies the number of labelled samples increase. Next, corre- sponding to the quantitative results of Table the paper, we provide standard deviations those results in Table 5. The standard deviations for all the methods comparable.
A.5
MORE DETAILS ON DATASETS
A.5.1 WATERBIRDS
This dataset first introduced in [30] and has been developed by cropping images of birds from the CUB dataset [34] and pasting them on the backgrounds from the Places dataset [41]. A ResNet50
model, pre-trained with ImageNet weights, has been used for training in experiments on this dataset. No data augmentation has been applied for any of the Algorithms.
A.5.2 CMNIST
CMNIST dataset comprised of two groups of MNIST images each with specific color. As per the description in the main paper, the target label is flipped with specific correlation to the color. Following the implementation [7], the digit images contain two channels and were downsampled to 14 x 14 pixels.
A.5.3 ADULT
The Adult dataset used in the paper was obtained from the UCI repository [10]. It contains 44, 842 samples. The features that were used in the experiments include "age", "workclass". "fnlwgr", "ed- ucation", "education-num" "marital-status.. ""ccupption", "relationship". "race", "sex", "capital- gain", "capital-loss", "hok."" "native-country". "income". positve target label this dataset is indicated by the attribute "income- branket being above 50KS.
A.5.4 CELEBA
For this dataset, the official train-val-ted splits as recommended by [21] has been used. Similar to the Waterbirds experiments, pre-trained ImageNet- based ResNet50 model has been used for the implementations.
A.6 ABLATION STUDY ON INCREASING THE CONSTRAINT SET SIZE.
We conduct experiments on Worst-off DRO method for different values of the parameter in the set (0, 0.001, 0.00,,, 0.1 The test set accuracies the minority group and average group
16

---

Under review as a conference paper at ICLR 2022
0.875
0.000
min accuracy
0.5
< 0.001
0.001
0.01
800
0.01
1
cm
0.750
7000
200
140
280
300
(a) Waterbirds Minority Group Accuracy
(b) Waterbirds Average Group Accuracy
Figure 6: Varying the parameter in the constraints. The marginal constraint is gradually relaxed by increasing the parameter. The accuracies in the plots are computed on the sets. Performance of the models with s 0.01 are similar, however, the accuracies drop when increasing beyond 0.01 threshold.
reported in Figure 6. Increasing the value also increases the constraint size because the marginal constraint is gradually relaxed. Figure 6 shows that the both minority group accuracy and average group accuracy values reduce with increase value beyond 0.1 threshold. The accuracy values for S 0.01 are comparable. A similar trend hold on other datasets as well.
17


Reviews:
[{"review_id": "xAirDfRhxjQ", "reviewer": "Reviewer_Jwqz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Weaknesses:\n- The main criticism of the paper is that the empirical results are not strong. The simple baselines of ERM and Group DRO (partial) Specifically, on Waterbirds and CelebA, many prior methods (such as JTT, GEORGE) that do not require *any* group labels substantially outperform this paper in terms of worst-group performance. Similarly, on CMNIST, EIIL does not require group labels and outperforms the proposed approach (and I suspect JTT and GEORGE would as well if evaluated).\n- While the theoretical results are correct, they are of limited impact. The gap between L_{GDRO} and L_{WDRO}, even the marginal distribution version, is likely to be large because the constraint set quite is large. In particular, for the points without group labels, only the marginal distributions have to approximately match, but given this the assignment of the individual group labels to the individual points is arbitrary, whereas in reality we could use the data features to better predict the probability of a given point having a particular group label. Thus, L_{WDRO} is a fairly pessimistic loss upper bound whose loss landscape may be quite different than that of L_{GDRO}; indeed, this is borne out by the empirical results.\n\nSuggestions:\n- While the current results are not strong, perhaps there are settings in which the relatively pessimistic L_{WDRO} loss is indeed preferable to existing baselines such as those mentioned above. Finding such a dataset would be helpful.\n- It would also be very helpful to construct a synthetic task on which L_{WDRO} can be proven to be preferable to baselines such as ERM and Group DRO (Partial).\n\nStrengths:\n- The ablation studies are well-conducted and provide useful insight.", "summary_of_the_paper": "This paper studies the case of group robustness when only some of the group labels are known. They propose a generalized objective for this setting that considers the worst-case possible group assignment subject to the known group labels, and an alternating optimization algorithm for minimizing this objective.", "summary_of_the_review": "Overall, the paper has some interesting ideas but unfortunately has limitations, and fairly poor performance in practice. My actual score would be 4, but the rating is unavailable.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "d59F-gWubDC", "reviewer": "Reviewer_RXdi", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "PROS:\n- This paper focus on an interesting problem that has both practical and theoretical importance. In general, learning a fairness-aware classifier with weak supervision is an important problem.\n- The extension to Group DRO is intuitive and reasonable. As stated in the paper, it naturally forms an upper bound of Group DRO.\n- The paper is overall well-written and easy to follow. I can get the point authors want to convey without ambiguity.\n\nCONS:\n- How the size of constraint set affects the convergence / efficiency is not stated theoretically / empirically. It is a crucial property for the proposed algorithm to be feasible.\n- Estimation of the true marginal distribution plays an important role in constraining the group assignment set, and this should be emphasized in the paper. Corresponding empirical verification for Lemma 2 is missing. Also, how the algorithm will perform if the estimation is wrong, thus the true p is not in the set?\n", "summary_of_the_paper": "This paper tackles the problem of optimization for the Rawlsian criterion with only partial group labels. They propose to optimize according to the worst case of group assignment and also propose a method to constrain the set of group assignment.\n", "summary_of_the_review": "Due to the weakness written in the main review, I choose to give a weak reject.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "PNeJJ_knLeI", "reviewer": "Reviewer_aqwr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "##########################################\n\nPros:\n\n+ The theories in this paper are solid and are verified to be right.\n\n+ This paper addresses a more realistic setting about fairness, and their solution seems to make sense.\n\n##########################################\n\nCons:\n\n- The research question of this paper is clear. However, after intruducing this question, they directly turn to explain Worst-off DRO in detail. This is too abrupt. They should add at least one paragraph about its motivation.\n\n-  They convert the feasible set to $\\mathcal{C}_{\\bar{p},\\epsilon}$ by adding an extra condition, thus giving an error estimation will be better.\n\n- Pseudo label-based methods serve as important solutions for this research question. However, related baselines are missing.\n\n- If possible, more fairness-related datasets, e.g., Arrest, Violent, German, should be included in experiments. (Not necessary in the rebuttal period)\n\n##########################################\n\nTypo:\n\n1. \"We optimize for the a worstoff soft group assignment\" -- \"the\" or \"a\" ?\n\nand so on...\n\n##########################################\n\nQuestions during rebuttal period:\n\nPlease address and clarify the cons above.\n\n#########################################\n\n", "summary_of_the_paper": "This paper aims to learn invariant representations for group membership using partially group labeled data. Thus, they propose the Worst-off DRO, which extends Group DRO by optimizing the loss against the worst-off group assignments in the constraint set. ", "summary_of_the_review": "This paper works on a more practial fairness scenario and gives a solid solution in principle. However, there still exists some issues about writting and minor technical details. If these issues can be solved well in rebuttal phase, I am willing to support this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0tV0SIRoUGj", "reviewer": "Reviewer_sood", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed method is technically sound, nicely polished, and well presented.\n\nHowever, I have two major concerns:\n\n1. The paper is motivated by real-world scenarios where only partial group information is available, for example when individuals may choose not to reveal this information due to privacy concerns. These realistic situations should usually result in a *biased* distribution of which samples contain the group annotation, typically due to demographics leading to unknown individual cohorts. However, the entire paper is based on the assumption that group information is *missing completely at random (MCAR)*, in both theoretical section and experimental section. I strongly advise that the authors carefully study other types of missingness, and build ways to understanding the impact of group information availability bias on the learning algorithm proposed. For instance, the algorithm depends on estimating the group distribution $\\hat{\\mathbf{p}}$ through the available samples, this estimation can be biased if not MCAR, which may render the Lemma 2 unreliable at least. Empirical evidence is also called for when $\\hat{\\mathbf{p}}$ is misspecified.\n2. The paper is missing *very important baselines* from SOTA unsupervised methods for improving worst-group performance, e.g., EIIL/JTT. Ideally, the proposed Worst-Off DRO should perform between EIIL/JTT (lower anchor) and Group DRO (anchor bound), since it requires *more* information than any unsupervised method (admittedly, JTT actually does require a small annotated validation set). But since the paper did not report these comparisons, I can only judge from the numbers across those papers on the same dataset (e.g., CUB), and it seems that Worst-Off DRO performs quite poorly compared to JTT for instance. If this is the case, the authors should list those baselines, and discuss the pros/cons as to the advantages of the proposed methods compare to EIIL/JTT.", "summary_of_the_paper": "The paper proposes a new variant of DRO, called Worst-Off DRO, to address a ubiquitous real-world setting where group labels are only partially available over the training set. The core idea is to introduce a nested optimization to maximize the worst-off group assignment over the entire training set, given some sensible constraints, hence enabling to alternate the standard DRO optimization. Empirical results are reported to support the proposed methods.", "summary_of_the_review": "The paper is technically sound and well written. However, due to the two major concerns above (unrealistic MCAR assumption and missing important baselines), I can hardly recommend the paper as ready for publication.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a novel method for learning distributional robust machine learning models when only partial group labels are available to improve performance of learning algorithms on minority groups. 

Pros: The paper is well motivated and written.  The ideas are interesting.  Most work on distributional robust optimization (DRO) are in unsupervised settings where group information is not available.  They provide an approach for the semi-supervised setting through a constraint set.

Cons: 
The empirical results do not show better performance over unsupervised baselines as pointed out by reviewers.  

The authors claim one of the benefits of their proposed approach is a one-stage approach, in contrast to competing models that require a two-stage approach; hence, allowing their approach to reduce compute time.  It’ll be helpful to strengthen this point by showing time comparisons.

Missing labels in this case due to participants withholding sensitive information is not an MCAR case, but the proposed work makes an MCAR assumption.  It’ll help to add a discussion and point out such limitations of the approach.

Summary:  This paper has novel and interesting ideas, but still has several issues as pointed out by the reviewers before it is ready for publication.
Decision: Reject