Re-weighting Based Group Fairness Regularization via Classwise Robust Optimization
Authors: Sangwon Jung
Taeeon Park
Sanghyuk Chun
Taesup Moon
Venue: ICLR 2023 poster (2023)
Keywords: Group Fairness
DRO

Abstract: Many existing group fairness-aware training methods aim to achieve the group fairness by either re-weighting underrepresented groups based on certain rules or using weakly approximated surrogates for the fairness metrics in the objective as regularization terms. Although each of the learning schemes has its own strength in terms of applicability or performance, respectively, it is difficult for any method in the either category to be considered as a gold standard since their successful performances are typically limited to specific cases. To that end, we propose a principled method, dubbed as FairDRO, which unifies the two learning schemes by incorporating a well-justified group fairness metric into the training objective using a classwise distributionally robust optimization (DRO) framework. We then develop an iterative optimization algorithm that minimizes the resulting objective by automatically producing the correct re-weights for each group. Our experiments show that FairDRO is scalable and easily adaptable to diverse applications, and consistently achieves the state-of-the-art performance on several benchmark datasets in terms of the accuracy-fairness trade-off, compared to recent strong baselines. 

Published as a conference paper at ICLR 2023
RE-WEIGHTING BASED GROUP FAIRNESS REGULAR- IZATION VIA CLASSWISE ROBUST OPTIMIZATION
Sangwon Jung Taeeon Park
Sanghyuk Chun? Taesup Moon 1,3i
Department of Electrical and Computer Engineering, Seoul National University
2 NAVER AI Lab 3 ASRI/NMC//PAV/IIS, Seoul National University
{s. jung, ptel235, s snu. ac kr, sanghyuk. cenavercorp.com
ABSTRACT
Many existing group fairness-aware training methods aim to achieve the group fairness by either re-weighting underrepresented groups based on certain rules using weakly approximated surrogates for the fairness metrics in the objective as regularization terms. Although each of the learning schemes has its own strength in terms of applicability or performance, respectively, it is difficult for any method in the either category to be considered as a gold standard since their successful performances are typically limited to specific cases. To that end, we propose a principled method, dubbed as FairDRO, which unifies the two learning schemes by incorporating well-justified group fairness metric into the training objective using a classwise distributionally robust optimization (DRO) framework. We then develop iterative optimization algorithm that minimizes the resulting objective by automatically producing the correct re- weights for each group. Our experiments show that FairDRO is scalable and easily adaptable to diverse applications, and con- sistently achieves the state-of-the performance on several benchmark datasets in terms of the accuracy-faine fairness trade-off, compared recent strong baselines.
INTRODUCTION
Machine learning algorithms are increasingly used various decision-making applications that have societal impact; e.g., crime assessment (Julia Angwin & Kirchner 2016). credit estimation (Khandani et al., 2010). facial recognition (Buolamwini & Gebru 20188) Wang et al. 2019), automated filtering social media (Fan al. 2021). Al-assisted hiring (Nguyen & Gatica-Perez (016) and law enforcement (Garvie 2016). A critical issue in such applications is the potential discrepancy of model performance, e.g., accuracy, across different sensitive groups (e.g., race or gender) (Buolamwini & Gebru 2018), which is easily observed in the models trained with a vanilla empirical risk minimization (ERM) Valiant 1984) when the training data has unwanted bias. To address such issues, the fairness- aware learning has recently drawn attention in the AI research community.
One of the objectives of aaire learning to achieve group fairness, which focuses on the statistical parity of the model prediction across sensitive groups The so-called in-processing methods typically employ additional machinery for achieving the group fairness while training. Depending on the type of machinery used, recent in- processing methods can be divided into two categories (Caton) & Haas 2020):: regularization based methods and re- ne-weighting based methods. Regularization based methods incorporate fairness-pront prrnmnss-prrooiing terms their loss functions. They can often achieve good performance by balancing the accuracy and fairness, but be applied only to certain types of model architectures tasks, such as DNNs (e.g., MFD (Jung al.] 2021) FairHSIC (Quadrianto) et al. 2019)) or binary classification tasks (e.g., Cov (Baharlouci et al. (020 ). On the other hand, re-weighting based methods are more flexible and can be applied
wider range of models and tasks by adopting simpler strategy to give higher weights to samples from underrepresented groups However, most of them (e.g.. LBC (Jiang & Nachum 2020), RW (Kamiran & Calders) 2012), and FairBatch (Roh et al. 2020g) lack sound theoretical justifications for enforcing group fairness and may perform poorly on some benchmark datasets.
Equal contribution.
Corresponding author.

---

Published as a conference paper at ICLR 2023
In this paper, we devise a new in in-processing method, dubbed Faimess-awart Distributionally Robust Optimization (FairDRO), which takes the advantages of both regularization and re-weighting based methods. The core of our method is to unify the two learning categories: namely, FairDRO incorporates a well-justified group fairness metric in the training objective as a regularizer, and opti- mizes the resulting objective function using re-weighting based learning method. More specifically, we first present that a group fairness metric, Difference of Conditional Accuracy (DCA) (Berk et al., 202I), which is a natural extension of Equalized Opportunity (Hardt et al. 2016) to the multi-class, multi-group label settings, is equivalent (up to a constant) to the average (over the classes) of the roots of the variances of groupwise 0-1 losses. We then employ the Group DRO formulation, which uses the x diveeregee ball including quasi-prababilities to convert the DCA (or variance) regularized group-balanced empirical risk minimization (ERM) to a more tractable minimax optimization. The inner maximizer in the converted optimization problem is then used as re-weights for the samples in each group, making a unified connection between the re- weighting and eegularization-aseed fairness-awan learning methods. Lastly, we develop efficient iterative optimization algorithm, which automatically produces the correct (sometimes even negative) re- weights during the optimization process, in more principled way than other re-weighting based methods.
the uncertainty set, for each class separately
In our experiments, we empirically show that our FairDRO is scalable and easily adaptable to diverse application scenarios, including tabular (Julia Angwin & Kirchner (2016) Dua et al. (017). vision (Zhang al., 2017). y and natural language text (Koh al.J (221) datasets. We compare with several representative in- in-processing baselines that apply either re-weighting schemes or surrogate regularizations for group fairness, and show that FairDRO consistently achieves the sfate-of-the -frtt---rtttt performance on all datasets in terms of accuracy- fairness trade-off thanks to leveraging the benefits of both kinds of fairness-awe aware learning methods.
RELATED WORKS
2.1 IN-PROCESSING METHODS FOR GROUP FAIRNESS
Regularization based methods add penalty terms in their objective function for promoting the group fairness. Due to con-differentiability of desired group fairness metrics, they use weaker surrogate regularization terms; e.g., Cov Zafar et al.) 2017b), Rényi (Baharlouei et al.) 2020) and FairHSIC (Quadrianto et al. 2 employ covariance approximation, Rényi correlation (Rényi 1959) and Hilbert Schmidt Independence Criterion (HSIC) (Gretton et al. 2005) between the group label and the model as fairness constraints, respectively. Jung et al. (021) devised MFD that uses Maximum Mean Discrepancy (MMD) (Gretton et al.) 2005) regularization for distilling knowledge from teacher model and promoting group fairness at the same time. Although these methods can achieve high performance when hyperparameters for strengths of the regularization terms are well-tuned, they are sensitive to choices of model architectures and task settings as they employ surrogate regularization terms; see Sec. 5 and Appendix C.2 for more details.
Meanwhile, some other works (Agarwal et al. 2018: Cotter et al. 2019) used an equivalent constrained optimization framework enforce group fairness instead of using the regularization formulation. Namely, they consider minimax problem of the Lagrangian function from the given constrained optimization problem and seek find a saddle point through the alternative updates of model parameters and the Lagrangian variables. By doing so, they can successfully control the degree of fairness while maximizing the accuracy. However, their alternating optimization algorithms require severe computational costs due to the repetitive full training of the model. Furthermore, we empirically observed that they fail to find feasible solution when applied to complex tasks in vision or NLP domains. A more detailed discussion is in Appendix B
As alternative in- processsgsng methods, several re- reeieggttnng based methods (Kamiran & Calders 2012, Jiang & Nachum 2020 Roh et al. 2020 Agarwal et al. 2018) have also been proposed to address group fairness. Kamiran & Calders (2012) proposed a re-weighting scheme (RW) based on the ratio of the number of data points per each group. Recently, Label Bias Correction (LBC) (Jiang & Nachum 2020) and FairBatch (Roh et al., 2020) have been developed, which adaptively adjust weights and mini-batch sizes based on the average loss per group, respectively. Perhaps, the most similar to our work is Agarwal et al. (2018). which demonstrates that a Lagrangian formulation of fairnes--constrinned optimization problem can be reduced to cost-sensitive classification problem
2

---

Published as a conference paper at ICLR 2023
and solved through re-weighting learning scheme with relabling of class labels. However, their method is limited to binary classification tasks, whereas our FairDRO, using the DRO framework, can be applied to more complex tasks with non-binary class labels.
2.2 DISTRIBUTIONALLY ROBUST OPTIMIZATION (DRO)
Distributionally robust optimization (DRO) (een- Tal et al. 2009) is a general robust optimization framework that was originally used for learning model that is robust to potential test distribution shift from the training set, by optimizing for the worst-case distribution in an uncertainty set. The DRO framework is now widely adopted in other applications, including algorithmic faimess.
DRO in algorithmic fairness Numerous works in the algorithmic fairness literature have recently utilized the DRO frameworks in various settings. For example, line of works (Jiang al. 2020
Taskesen et al. 2020 Wang et al. 2021 Mandal et al. 2020) embeds the DRO frameworks KS into the fairness-constrainee optimization problem to make fair model robust to distribution shifts of a test dataset. Another related line of works (Wang et al. 2020: Hashimoto et al. 2018) proposed DRO formulations for achieving fairness in more challenging settings such as learning with noisy group labels or sequential learning. In spite of their effectiveness, both of above lines of works do not use DRO as a direct tool for achieving group fairness. Similarly to ours, Rezaei et al. (2020); Yurochkin et al. (2019) adopt DRO frameworks for the purpose of achieving fairesss. However, the key difference is that they either require the group labels at test time or target individual fairness. On the other hand, our FairDRO directly aims to achieve group fairness while not requiring group labels at test time. A more detailed discussion is in Appendix B
DRO in other applications Applications of DRO are actively studied in other contexts, e.g., image classification (Sagawa et al.) (2020), multilingual machine translation (Oren et al. 2019 Zhou et al.) 2021), long-tailed classification (Samuel & Chechik 2021) out-of-distribution generalization (Kuueerr et al. 2021] Xie et al. 2020). For example, Sagawa et al. (2020) proposed Group DRO which constructs the uncertainty set of joint distributions over the class-group label pairs to withstand the spurious correlations between the class and group labels. Although these works are effective in their applications, the notion of statistical parity across groups (i.e., group fairnes)) has not been explicitly pursued as in our FairDRO.
3 NOTATIONS AND FAIRNESS CRITERION
Notations We consider a classification problem where each sample consists of an input E X, class label y y' and group label E A. The group label is defined by sensitive attributes such as gender or race. Given a training dataset 'D with N samples (i.e., D = {( Ue, H.M we aim to find a fair classifier OFAIR : X + y satisfying the given fairness criterion while maintaining the high classification accuracy. Moreover, given a loss function ( (zx,)) y)) θ (X x x)) + R, we IPP E(O, (z;, 3:)). Additionally, D. and DV denote the subsets of D that are confined the samples with group label class-group label pair (y, a), respectively.
denote L( D) - D
Fairness criterion The notion of fairness may depend on the different points of view on how discrimination is defined (Hardt et al. 2016 Dwork al. 2012 Chouldechoval 2017 Berk et al.) 2021). For example, one may argue that a classifier is fair if its prediction is not dependent on (Demographic Parity), while the other can insist that the classifier should be conditionally independent of a given the true class (Equalized Odds). In our work, we focus on the Equalized Conditional Accuracy (ECA) (Berk et al., (021). which can naturally cover the multi-class, multi-group label setting.
classifier 0 satisfies ECA if all of the accuracies among groups are the same for each given class, ie.. Va, a' E A, y E )) P(P = y = a, Y = v) = p( = y| = a', Y = y), where Y is the prediction of 0. We measure the degree of unfaireess with Difference of Conditional Accuracy (DCA), denoted as ADCA, by taking the average of the maximum accuracy gaps among groups over y:
ADCA
1 y
Σ
Ay: 4, max PP v/ a, Y v) P(p v/ d' Y =-)l-- (1)
DEY'
3

---

Published as a conference paper at ICLR 2023
Remark: We note that ECA has close connections with the two popular group faimess criteria, Equalized Odds (EO) and Equal Opportunity (EOpp) (Hardt et al. 2016). EO requires the conditional independence between Y and A given Y, and hence, we can easily observe that ECA is equivalent to EO for the binary classification setting. Furthermore, we argue that ECA is natural extension of EOpp to the multi-class setting; namely, ECA requires equality of TPR among groups for each class label, which resembles the argument for justifying EOpp.
FAIRNESS-AWARY DRO (FAIRDRO)
Our goal is to unify the regularization and re- re-weighting based methods by incorporating the DCA !T) in the objective function for learning. To that end, we first show in Sec. 4.1 that the empirical DCA is equivalent to the average (over the classes) of the roots of the variances of groupwise 0-1 losses. Then, we utilize the previous result (Xie et al. 2020) given in Sec. 4.2 to make the connection between the Group DRO with the uncertainty set of x -iviveegennce ball ball (including quasi-prohabilities) and the variance regularized group-balanced empirical loss minimization. Subsequently, in Sec. S.3. [4.3 we
present our training objective that applies Group DRO separately for each class y, which essentially converts the DCA (or variance) regularized group-class balanced ERM into a more tractable minimax optimization problem. Finally, Sec. 4.4 proposes an efficient iterative algorithm that does alternating Lunimizztion-maximizztooo maximization of our training objective.
4.1 EQUIVALENCE BETWEEN DCA AND VARIANCE OF GROUPWISE LOSSES
Proposition 1 Assume ((0) = 1(0(x)) + y). Then, the following inequalities hold for all y E y,
2 Var
Var (IC(O,D)))ack s Ag - 2 Var ((C(0) (((0, Dā))
in which Ay is the empirical version of Ay and Var({z; 2 Σ, -, of d numbers {ra) Thus, By of a classifier if and only if Var ((C(0, Dae)) = 0.
xz)² is the variance
The proof is in Appendix A By taking the average empirical DCA is equivalent to the average (over y) of the square roots the variances of the groupwise 0-1 losses, up constant. Building this equivalence, we now able to utilize the Group DRO formulation given in the next subsection to make a connection between DCA regularization and robust optimization.
Y in 2 Proposition | implies that the
4.2 PRIOR WORK ON VARIANCE REGULARIZATION VIA GROUP DRO
While ERM merely minimizes the empirical loss, DRO optimizes for the worst-case distributions in an uncertainty set, as mentioned in Sec. 2.2 Since general DRO (Duchi et 2019) may lead to pessimistic model that optimizes for implausible worst-case distributions, Sagawa et al. (2020) recently proposed the Group DRO framework, which performs a group level and can incorporate prior knowledge about the groups" it aims obtain
GDRO A argmin min max Σ (aL(O,D,).
arg
(3)
IEAIAI
aeA
in which A |Al is the [Al-simplex
The applications of Group DRO are actively studied in various contexts, e.g., image classification (Sagawa et al. 2020) or machine translation (Zhou et al. 2021). In particular, Xie et al. (2020) proposed Risk Variance Penalization (RVP), OOD generalization method, using variance regularization with respect to risks of each domain (a formulation first introduced by V-REx (Krueger)
al. 2021)). The author showed the connection between the variance regularization and Group DRO, when the uncertainty set is the following x" -divergence ball with radius p > 0:
e, 2 Q
Σ 4a 1, Do (g(g AEA
all
sp).
(4)
A
In this subsection, the term "group" is used in more inclusive manner. Namely, the group here refers to any hierarchical structure in the data distribution, e.g., domain, environment, and sensitive attribute.
4

---

Published as a conference paper at ICLR 2023
in which (plll) - (:0pp(9/):)) with ((t) (t- 1)2, and E RI-A denotes the uniform distribution. Note (4) does not have the nonnegativity constraint on q. and hence, depending on p. 2. can also include quaas--robuablities, which have negative components as well. Namely, Xie et al. (2020) shows the following lemma that the inner maximization of Group DRO with 2p uncertantt set equivalent to the group
group-balanced balanced empirical loss plus the group- wise variance term as a regularizer.
Lemma
Xie et al.
2020 Proposition 1) For any finite loss E(O, (x, v)) and Qpp. we have
max Eqah(o,D.)) = [A] - c(0,Da) + Var ((C(0, Da))
(5)
q aEA
To be self-contained, the proof of the lemma is given in Appendix A
4.3 TRAINING OBJECTIVE OF FAIRDRO
To make the connection between Proposition and Lemma and realize the DCA regularization for fairness-aware training, we propose to apply the Group DRO separately for each class y. Namely, we define our FairDRO as minimizing the average of the worst-case losses defined for each class y:
FairDRO
arg min
Σ DEY
max Σ (H), E
(6)
D
O
aEA
in which 2p is defined in q" (o") a is quasi-probability vector, and p is a tunable hyperparameter. We stress that while applying Group DRO by treating each pair (y, as a "group" has been considered in [Sagawa et al. (2020). applying Group DRO separately for each class as in FairDRO has not been considered before. Now, present our key result which follows by combining Proposition
| and Lemma [
Corollary Assume E(O) 1(0(x) + y). Then, given a dataset D, there corresponding positive constant Cp the range of [V2,, Al) such that FaaiDRRO also achieves
min
Σ C(0. + CDVPdocA) (g.a)
The corollary implies that when the 0-1 loss used, FairDRO a regularization based method, described in Sec. [2.]] with the exact DCA serving regularizer. We emphasize that we do not solve the minimization in 7 directly. ÂDCA can have high variance when estimated from small mini-batch, particularly when the number of groups is large, and is even hard to be minimized due con-differentiabiiity. We indeed empirically observe Sec. 5.4 that directly solving 7 using approximation of ÂDCA with a differentiable cross-entropy loss performs poorly. Instead, we solve much more tractable minimax optimization in (6) where the maximization is simple linear optimization and the minimization is for the re-weighted group losses.
At this point, is now clear that during solving 6 FairDRO produces the re-weights {ah)'s Ys for each (y, a). which makes it also categorized as a we-weighting based method. The following proposition shows how the re-weights are characterized and controlled by the hyperparameter p.
Proposition 2 For 2p in (4 with 0, each element of the optimum 9ºº for the inmer-maxinization of O0 can be obtained by closed-form solution:
C(0,DD)
Σ ((0,D))
+
P
X
a
(8)
A
AAI
[L,
DX)
Σ
L(0, 2
PPAAI1 [A
A(lAA-11 -1) AA
and is in the range of
A
A
The proof is in Appendix A
Proposition 2 shows that some elements of q can be negative depending on the radius p and the groupwise loss Zíe, Dz). Moreover, it shows that the range of each element of the optimum 9²= is automatically determined by a radius p and the number of groups [A), which
5

---

Published as a conference paper at ICLR 2023
means FairDRO controls the degree of the group penalization by varying p. We will show in Sec. that we can find a proper degree of penalization by tuning so that the performances on various datasets in terms of accuracy fainnesss trade-off are significantly improved.
crucial difference from typical reighighting based methods is that our re-weights are obtained as the solution of an optimization problem opposed those in Kamiran & Calders) (2012): Jiang & Nachum (220)) that are heuristically obtained based on the ratio the number of data points or losses for each group. Also, we allow negative values for of in 2,. which enables penalizing the high accuracy group more aggressively than typical re-weighting methods that only control the positive
weights for each group and class y.
4.4 AN EFFICIENT ITERATIVE OPTIMIZATION FOR FAIRDRO
canonical way of solving the minimax optimization in DRO is to use a primal-dual method (emirovski et al. 2009) in which one alternates between regular gradient descent on O and an exponentiated gradient (EG) ascent (Kivinen Warmuth 1997) on q. However, the EG algorithm widely used when the variables lie in the probability simplex because it is special case of mirror descent when the convex function for the Bregman divergence is the negative Shannon entropy. Thus, EG cannot be applied to solving (6) since our uncertainty set also allows the quasi- muasi-probability for q".
Hence, we employ the smoothed Iterated Best Response (IBR) which is a variant of IBR recently used for solving DRO problem in (Zhou et al. (021). Using standard IBR for the inner maximization step, is updated as the closed-forn solution of the maximization (i.e., (8) instead the gradient ascent of q. It is known that solving the minimax optimization using the standard IBR has convergence guarantee w.r.t 0 under some regularity assumptions (Zhou et al. 2021). However, it does not imply the convergence of q², and we empirically observed in Appendix 1D66tha the qan,t oscillates unstcadily when the loss for each group fluctuates. To that end, we applied the learning rate scheduling of n & with T' being the number of total epochs, which results in updating qu. smoothly. Namely, our overall update rules for O and q" are
gc+1 + 8
ΣΣ VEJ aEA
nl (('L(00 ((O),D))) D.)).
(update
D
quitt1 - (1 nh
+ nag
Vy E V,
(update q")
(10)
in which qontt arg max E'EO, aEA
2hd 1(0+11 DH).
and náé and ni denote the learning rates for O and q², respectively. By applying the smoothed IBR in (TO). we empirically ob- serve improvements in the group fairness and more stable convergence of dD (detailed results are given in Appendix [D6]]. We note that for (7) to hold, we use 0-1 loss in (6). which makes our training objective still non- differentiable with respect to O. Thus, in practice, we solves the cross-entropy loss for the outer minimization loss for the inner maximization (TO). We also note that 0 is updated in mini-batch manner, whereas q is updated after com- puting the group losses for all training data points. Algorithm [[]llustrates the summary of FairDRO optimization scheme.
Algorithm 1: FairDRO Iterative Optimization
Input Dataset [Da]vey,ee radius p. epoch
T, iteration 7, mini-batch size B
Randomly initialize 8
Set qa.0
<- A
for y E 3, in which 1 E RI-A
for t 0 T do
// 0 0puudate
for - 0 0
Sample mini-batch {xz Vj, a,) be() equally from Dy for each (y.a) E y x A.
9 and the 0-1
Update 0 with using the mini-batch
end
// q-update
10
Update qui for each E y with 10
end
Output: 0", ( IBEY
5 EXPERIMENTAL RESULTS
In this section, we demonstrate the versatility of FairDRO by comparing with various baseline methods three modalities of datasets: (1) two tabular datasets, UCI Adult (Dua al. 2017) and ProPublica COMPAS (Julia Angwin Kirchner 2016), (2) two vision datasets, UTKeaae (Zhang al. 2017), CelebA (Liu al. 2015) (in Appendix D.2) and (3) natural language dataset,

---

Published as a conference paper at ICLR 2023
* Scratch RW LBC + Cov Rényi fairBatch EGR PL FaiDRO (ours)
COMPAS
Accuracy (8)
80-
78-
60
-88
14
12
Aock (%6)
The trade-off's between accuracy and DCA on Adult (left) and COMPAS (right). The per- formances of "Seratch" are highlighted by the black dotted lines. Each point is the result corresponding to different hyperparameter of each method. Each line is a convex envelop of points of the corresponding method.
Figure 1:
CivilComments- CommentssWILDS (Koh et al. 2021). We use logistic regression, ResNet18 (He et al. 2016)), and a pre-trained BERT (Devlin et al. 2019 Woll et al. 2019) as the classifier for each modality, respectively. For the evaluation, we plotted the convex envelops of the trade-offs between the accuracy and ADCA for varying hyperparameters, i.e., the pareto frontiers. As a single number evaluation metric, we also reported the best DCA with at least 95% accuracy of the vanilla trained model (Scratch) in the Appendix [.I] Since the most datasets are severely skewed toward a specific group or class, we report the balanced accuracy over all combinations of groups and classes as |Bellamy et al.) (2018) (the issue discussed more in Appendix D.7). Moreover, we give thorough ablation study on our design choice in (6). namely, the classwise DRO formulation and the uncertainty set (4) in Sec. 3.c.5.44nd Appendix D.4 respectively. We further visualize the re- re-weights produced by FairDRO in Sec. 5.4 More implementation details are given in Appendix C
Comparison methods. We compare FairDRO with vanilla training without any fairness constraint (Seratch) and nine in-processing fair-training methods; three re-weighting based methods (RW (Kamiran & Calders] 2012). LBC (Jiang & Nachum, 2020), FairBBtth (Roh et al.) (0200), four
regularization based methods (Cov (Zafar et al. 2017b), FairHSIC (Quadrianto et al., 2019), Rényi
(Baharlouci et al.) 2020). MFD (Jung al. 2021)) and two methods employing constrained optimization (EGR (Agarwal et al. 2018). PL. (Cotter et al. 2019)). Since we consider a group-class balanced accuracy, modified the objective functions of all baselines including Scratch such that they use the group-class balanced ERM loss, not the standard ERM loss P
We note that Cov, EGR and Rényi are task-dependent, and FairHSIC and MFD are model-dependent; namely, Cov and EGR designed only for binary classification tasks, Rényi is for tasks with binary groups, and HSiC and MFD are for DNN-based classifiers. We also note that all baselines except RW were implemented for targeting EO. Considering that ECA is equivalent to EO only in binary classification as we mentioned we also reported the difference of EO (DEO) for multi-class classification tasks in We provide more details of implementations of the baselines in Appendix C
in Section 3
Appendix D.1
5.1 TABULAR DATASETS
Two tabular datasets, UCI Adult (Dua et al. 2017) (Adult) and ProPublica COMPAS (Julia Angwin & Kirchner 2016) (COMPAS), are used for the benchmark. Both tasks are binary classification tasks (predicting whether the income of an individual exceeds $50K per a year and whether a defendant re-offends, respectively) with binary sensitive groups ("Female" and "Male", and "Caucasian" and "Non-Caucasin", respectively). Following pre-processing of |Bellamy et al. 2018). each dataset includes 45K and 5K rows, respectively.
Fig. T shows the accuracy-fairnest trade-offs for varying hyperparameters on Adult and COMPAS. We note that there is only a single score for RW because it has controllable hyperparameter. In the figure, we observe that FairDRO is placed to the most top-right area among methods, showing the best trade-off, and even achieves the lowest level of DCA with slight loss of accuracy. This implies that FairDRO successfully finds better weights (i.e., quasi probabbrob q") than other re-weighting based methods and employs a better regularizer than other regularization based methods. Furthermore, our FairDRO can provide fareto-frontiers in wider range than other re-weighting baselines by introducing negative weights.
TThe results of employing original objective functions are reported in Appendix D.3 From the results, we observe that using the halanced ERM loss improves the balanced accuracy and DCA for the most methods.
7

---

Published as a conference paper at ICLR 2023
Scratch a LBC Cav RibBatch PL FairSSIC MPD FairDR lours)
UTKFace
CivilComments
BD-
80
Accuracy (8)
Accunacy (80
78-
76-
78-
74-
72-
70
Dock (56)
Figure 2:
The trade-offs between accuracy and DCA on UTKFace (left) and CivilComments (right). Details are the same as Fig.
5.2 VISION DATASETS
We also evaluate FairDRO on UTKFace (Zhang et al. multi-group labels. UTKFaae includes more than 20K images with age, gender and ethnicity labels. We divide the age labels into three classes ("O to 19", "20 to 40", and "more than 40"), and set them as the class labels. We also use four ethnic classes ( ("White", "Black", "Asian", and "Indian" ") as the group labels. The description and results on another vision dataset, CelebA (Liu et al. 2015), are
2017). face dataset with multi-class and
given in Appendix D.2
From Fig. 2 (left), we confirm that FairDRO significantly improves the trade-off, compared to most baselines. For MFD, note that it provides competitive pareto- frontier with higher accuracy than FairDRO, thanks to the knowledge distillation effect. However, MFD requires additional computations for training teacher model, and its high performance is sensitive to task domains (refer to the next result on the NLP task). We again observe that the re-weighting based baselines ineffective in achieving fairness UTKFace, while FairDRO achieves strong performance due to its systematic optimization of the group weights. We further note that much lower accuracy of PL implies that its minimax optimization fails to produce good performance due difficulty of finding feasible saddle points. Since EO is not equivalent to ECA for non-binary target task as aforementioned above, we additionally report the difference EO (DEO) for UTKEaace in Appendix D.I and show FairDRO again outperforms other baselines in DEO.
5.3
LANGUAGE DATASET
In order to show the versatility of FairDRO, we additionally consider natural language dataset, CivilComments- WILDS (Koh et al. 202I). CivilComments a large collection comments on online articles taken from the Civil Comments platform. It comprises 450K comments which annotated with identity attributes by 10 crowdworkers and majority votes. The CivilComments task predict whether given comment is toxic or not. We set ethnicity as the sensitive attribute and extract the subset of comments that contain identities w.r.t. ethnicity ( ("Black", "White", "Asian", "Latino" and "Ohters"). Note that the dataset has continuous values for the identities (between 0 and 1), hence, we binarized the continuous group values by setting the maximum as 1 and the others as O, since the existing in processing methods are only available handle discrete group labels.
The results are shown in Fig. 2 (right). Note that since pre-trained BERT is fine-tuned only for small epochs, the update for q" of FairDRO and weights of re-weighting baselines is executed per 100 iterations, instead of every epoch. We empirically observed that FairHSIC fails to converge due to the gradient exploding. Consistent with the previous results, FairDRO achieves the best trade-off on CivilComments, Note that although MFD is very competitive on UTKFace, it is not effective on CivilComments.
5.4 ANALYSIS
Ablation study. Fig. shows the fccuracy-fairness trade-off for each ablated version of FairDRO on each dataset. "FarDRO (w/o DRO)" directly solves (7) by approximating both terms in the objective with the cross-entropy loss; namely, both terms are replaced with the group-class balanced average cross-entropy loss and the average (over y) of the maximum gaps of groupwise cross-entropy losses. The "RVP (w /classwise)" also similar, but uses the average (over y) of the variances of the groupwise cross-entropy losses (as shown in (5) as regularization term instead of the direct approximation of ÂPCA- FairDRO (wo classwise) defines only one uncertainty over the (y, a)
8

---

Published as a conference paper at ICLR 2023
Scratch
FairDRO (w/o DRO)
RVP classwise)
classwise)
FairDRO (ours)
Adult
COMPAS
UTKKaace
CivilCommentss
Accuracy 180
196
80-
Accuracy
78-
Accuracy
79
76
77
14
10
22
20
1.8
Sooe (%)
Apci (%)
Aack (96)
doca (%)
Figure 3: Ablation study of FairDRO. RVP and FairDRO (w/o classwise) for the ablation study of our key components, the DRO formulation, and classwise treatment, respectively. The best performance of each result Tab. T.b.D.4
Group
Group
Scratch
FairDRO
Accuracy (%6
Accuracy (%)
80
80:
70-
70
60
60
70
50-
50
40-
40
50
10
20
30
40
50
60
70
10
20
30
40
50
60
70
10
20
30
40
50
60
70
Training epochs
Training epochs
Training epechs
(a) Scratch (train)
(b) FairDRO (train)
(c) Scratch & FairDRO (test)
Figure 4: Accuracies and loss weights for each group (q) COMPAS. The groupwise training accuracies and loss weights during training Scratch (a) and FairDRO (b) are shown. Groupwise test accuracies are shown in (c). The reported accuracies and weights are only for label 1, while label results are in Appendix D.5
pair (6) for the ablation study of the classwise treatment. In Fig. we note that RVP (w/ classwise) on CivilComments fails to converge by the gradient exploding. Our observations from the figures are as follows. First, we see that FairDRO achieves consistently better pareto-fronticn on datasets than FairDRO (w/o DRO) and RVP (w/classwise), which asserts that solving (7) by re-weighting learning scheme via our classwise DRO formulation is essential for better performance and stable learning. Second, we clearly observe that FairDRO (w/o classwise) shows sub sub-optimal accuracy-DCC trade-offs (except for Adult), indicating that our classwise DRO formulation is critical for the optimal performance since it makes the connection with the DCA regularization more clearly. Further ablation studies for the choice of the uncertainty set are Appendix D.4]
Visualization of q" We visualize the learning dynamics of the group weights (q"), including training accuracies (Fig. (a) and (b)) and test accuracies (Fig. (c)) for Scratch and FairDRO on COMPAS.
For simplicity, report the result for y (for y 0 is in Appendix D.5 In Fig, (a). q² assigns the same value for each group as Scratch employs the balanced ERM loss. At the top of the plot, Scratch shows large discrepancy in accuracy between the two groups throughout training. On the other hand, Fig. 4 (b) shows FairDRO adjusts q' by assigning low values to the high accuracy group (Group 0) and high values to the low accuracy group (Group 1). Note that q² initially fluctuates and even assigns negative values for the high accuracy group but eventually converges via the learning rate scheduling (TO). a result, we clearly observe both the training and test accuracy gaps between the groups smoothly become negligible the training continues. We believe this example clearly shows the effectiveness of introducing the quasi-probabie probability in 2p. and our FairDRO framework in achieving small test accuracy gaps between the groups, i.e., ADCA-
6 CONCLUDING REMARK
We proposed a theoretically well-justified in-processing method for achieving group fairness. To make clear connection between re- re-weighting and regularization based methods, our FairDRO employs the classwise Group DRO framework with >dddivergence ball including quasi-probabillties uncertainty set. Then, the FairDRO training objective directly incorporates DCA regularizer and can be effectively solved by our proposed re- re-weighting optimization scheme. As a result, our DRO formulation enables FairDRO to have strengths of both weighing based and regularization based methods. Indeed, we showed through our experiments that our FairDRO is applicable to various settings and consistently achieves superior accuracy- -airness trade-off benchmark datasets.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work was supported in part by the NRF grant | and ITTP grants 2No. 0-01343, No. 201-0021--,,,,,, No.2022-0-00959, No. No.2022-0-001133 13]] funded by the Korean government, and SNU-NAVER Hyperscale AI Center.
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. reductions approach to fair classification. In Int. Conf. Mach. Learn. (ICML), pp. 60-69. PMLR, 2018.
Sina Baharlouei, Maher Nouiehed, Ahmad Beirami, and Meisam Razaviyayn. Rényi fair inference. In Int. Conf. Learn. Represent. (ICLR), 2020.
Rachel KE Bellamy, Kuntal Dey, Michael Hind, Samuel C Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovie, al. Al Fair- ness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv preprint arX1v:1810001943, 2018.
Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. Robust optimization, volume 28.
Princeton University Press, 2009.
Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. Fairnsss in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50(1):3-44, 2021.
Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conf. Fairness, Accountability and Transparency (FAccT',, pp. 77-91. PMLR, 2018.
Simon Caton and Christian Haas. Fairness machine learning: A survey. arXiv preprint 0xXi::2010004053. 2020.
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2):153-163, 2017.
Ching-Yas Yao Chuang and Youssef Mrouch. Fair mixup: Fairness via interpolation. In Int. Conf. Learn. Represent. (ICLR), 2021.
Andrew Cotter, Heinrich Jiang, Maya R Gupta, Serena Wang, Taman Narayan, Seungil You, and Karthik Sridharan. Optimization with non-differentiable constraints with applications to fairness, recall, churn, and other goals. Journal of Machine Learning Research (JMLR), 20(172):1-55 2019.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 1 (NAACL-HLT (NAAAlLLHIIAAAAAALHLHLIIHI (1)), pp. 4171-4186, 2019.
Ming-Wei
Dheeru Dua, Casey Graff, et al. UCI machine learning repository. http://archive.ics...
edd/ml 2017.
John C Duchi, Tatsunori Hashimoto, and Hongseok Namkoong. Distributionally robust losses against mixture covariate shifts. Under review, 2019.
Sanghamitra Dutta, Dennis Wei, Hazar Yueksel, Pin- Yu Chen, Sijia Liu, and Kush Varshney. Is there a trade-off between faimess and accuracy? A perspective using mismatched hypothesis testing. In Int. Conf. Mach. Learn. (ICMI), pp. 2803-2813. PMLR, 2020.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Conf. Innov. Theor. Compu. Scien. (ITCS), pp. 214-226, 2012.
10

---

Published as a conference paper at ICLR 2023
Hong Fan, Wu Du, Abdelghani Dahou, Ahmed A Ewees, Dalia Yousri, Mohamed Abd Elaziz, Ammar H Elsheikh, Laith Abualigah, and Mohammed AA Al-qaness. Social media toxicity classification using deep learning: Real-world application UK brexit. Electronics, 10(11):1332, 2021.
Clare Garvie. The perpetual line-up: Unregulated police face recognition in America. Georgetown Law, Center Privacy & Technology, 2016.
Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. Measuring statistical dependence with Hilbert-Schmidt norms. In Int. Conf. Algo. Learn. Theory (ALT), pp. 63-77. Springer, 2005.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Adv. Neural Inform. Process. Syst. (NewrIPS), volume 29, 2016
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Perey Liang. Fairness without demographics in repeated loss minimization. In Int. Conf. Mach. Learn. (ICML), pp. 1929-1938. PMLR, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conf. Comput. Vis. Patlern Recog. (CVPR), pp. 770-778, 2016.
Heinrich Jiang and Ofir Nachum. Identifying and correcting label bias in machine learning. In Int. Conf. Artificial Intelligence and Statistics (AISTATS), pp. 702-712. PMLR, 2020.
Ray Jiang, Aldo Pacchiano, Tom Stepleton. Heinrich Jiang, and Silvia Chiappa. Wasserstein fair classification. In Uncertainty in Artificial Intelligence (UAI), pp. 862-872. PMLR, 2020.
Chi Jin, Praneeth Netrapalli, and Michael Jordan. What is local optimality in nonconvex-nonconeee minimax optimization? In Int. Conf. Mach. Learn. (ICML), pp. 4880-4889, 4889. PMLR, 2020.
Surya Mattu Julia Angwin, Jeff Larson and Lauren Kirchner. There's software used across the country to predict future criminals. and its biased against blacks. ProPublica, 2016. URL
httpp:://githe
wwtppppppwwccccppaaaaapppepepeee compas ana compas-analysiss
Sangwon Jung, Donggyu Lee, Taceon Park, and Taesup Moon. Fair feature distillation for visual recognition. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 12115-12124, 2021.
Sangwon Jung, Sanghyuk Chun, and Taesup Moon. Learning fair classifiers with partially annotated group labels. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 11348-10357, 2022.
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimi- nation. Knowledge and Information Systems (KAIS), 33(1):1-33, 2012.
Amir E Khandani, Adlar J Kim, and Andrew W Lo. Consumer credit-risk models via machine-
learning algorithms. Journal of Banking Finance (JBF), 34(11):767--2788, 2010.
Jyrki Kivinen and Manfred K Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Information and Computation, 132(1):1-63 63, 1997.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wiie wild distribution shifts. In Int. Conf. Mach. Learn. (ICML), pp. 5637-5664. PMLR, 2021.
David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distributin generalization via risk extrapola- tion (REx). In Int. Conf. Mach. Learn. (ICML), pp. 5816-5826. PMLR, 2021.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. Int. Conf. Mach. Learn. (ICML), pp. 6781-6792. PMLR, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In Int. Conf. Comput. Vis. (ICCV), pp. 3730-3738, 2015.

---

Published as a conference paper at ICLR 2023
Iyy Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with restarts. In Int. Conf. Learn. Represent. (ICLR),2011
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Int. Conf. Learn. Represent. (ICLR), 2019.
Debmalya Mandal, Samuel Deng, Suman Jana, Jeannette Wing, and Daniel J Hsu. Ensuring fairness beyond the training data. In Adv. Neural Inform. Process. Syst. (NeurIPS), volume 33, 2020.
Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization (SIOPT), 19 (4):1574-1609, 2009.
Laurent Son Nguyen and Daniel Gatica-Perez, Hirability in the wild: Analysis of online conversa- tional video resumes. IEEE Trans. Multimedia, 18(7):142-1137, 2016.
Yonatan Oren, Shiori Sagawa, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust language modeling. In Conf. Empirical Methods in Natural Language Processing (EMNLP), pp. 4227-4237, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, al. PyTorch: An imperative style, high- performance deep learning library. In Adv. Neural Inform. Process. Syst. (NeurIPS), volume 32, 2019.
Novi Quadrianto, Viktoriaa Sharmanska, and Oliver Thomas. Discovering fair representations in the data domain. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 8227-8236, 2019.
Alfréd Rényi. On measures of dependence. Acta Mathematica Hungarica, 10(3-4):441-5111 1959.
Ashkan Rezaei, Rizal Fathony, Omid Memarrast, and Brian Ziebart. Fairness for robust log loss classification. In Proc. of the AAAI Conf. Artificial Intelligence (AAAI), volume 34, pp. 5511-5518 2020.
Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. Fairbatch: Batch selection for model fairness. Int. Conf. Learn. Represent. (ICLR), 2020.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. Int. Conf. Learn. Represent. (ICLR), 2020.
Dvir Samuel and Gal Chechik. Distributional robustness loss for long-tail tail learning. In Int. Conf. Comput. Vis. (ICCV), pp. 9495-9504, 2021.
Xinyue Shen, Steven Diamond, Yuantao Gu, and Stephen Boyd. Disciplined convex-concave programming. In IEEE 55th Conf. Decision and Control (CDC), pp. 1009-1014. IEEE, 2016.
Bahar Taskesen, Viet Anh Nguyen, Daniel Kuhn, and Jose Blanchet. A distributionally robust approach fair classification. arXiv preprint arXiv:2007.09530, 09,500 2020.
Leslie G Valiant. A theory of the learnable. Communications of the ACM, 27(1)): 1142, 1984.
Mei Wang, Weihong Deng, Jiani Hu, Xunqiang Tao, and Yaohai Huang. Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In Inf. Conf. Comput. Vis. (ICCV), pp. 692-702, 2019.
Serena Wang, Wenshuo Guo, Harikrishna Narasimhan, Andrew Cotter, Maya Gupta, and Michael Jordan. Robust optimization for fairness with noisy protected groups. In Adv. Neural Inform. Process. Syst. (NeurIPS), volume 33, 2020.
Yijie Wang, Viet Anh Nguyen, and Grani Hanasusanto. Wasserstein robust classification with fairness constraints. arXiv preprint arXiv:2 10Xiv:2003306828 2021.
12

---

Published as a conference paper at ICLR 2023
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of the-art natural language processing. arXiv preprint arXiv:1910003771, 2019.
Chuanlong Xie, Fei Chen, Yue Liu, and Zhenguo Li. Risk variance penalization: From distributional robustness to causality. arXiv preprint arXiv:2006.07544, 2020.
Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. Training individually fair ML models with sensitive subspace robustness. arXiv preprint arXiv: 00020, 2019.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fair- ness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proc. of the 26th Int. Conf. World Wide Web (WWW), pp. 1171-1180, 2017a.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. In Int. Conf. Artificial Intelligence and Statistics (AISTATS), pp. 962-970. PMLR, 2017b.
Brian Hu Zhang, Blake I Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. AAAI/ACM Conf. AI, Ethics, and Society (AIES), pp. 335-340, 2018.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progressionnreerrsssinn by conditional adversarial autoencoder. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 5810-5818, 2017.
Chunting Zhou, Daniel Levy, Xian Li, Marjan Ghazvininejad, and Graham Neubig. Distribution- ally robust multilingual machine translation. In Conf. Empirical Methods in Natural Language Processing (EMNLP), pp. 5664 5664-5674, 2021.
13

---

Published as a conference paper at ICLR 2023
APPENDIX
We provide additional materials in this document. We include the proof' for Lemma | and Proposition a 2 in Appendix A additional related work Appendix B and implementation details such as optimization, implementations of baselines, and hyperparameter search in Appendix C Finally, we report additional experimental results in Appendix D
A PROOFS
Proof of Proposition
We define L E RlA[x|A| for brevity:
IN A L(0, D() L(O, Dy), s Vi,j S |A].
If we denote A RRA|*
A (L)
L A|,11
LT ..... L" A1122 ... L LAA
then, we have the following chain of equalities:
Var (IC(E,DWaaEA)
LA Σ (C(0. - -[1A] Σ (C(O, D.))
((),DD) DW) [(0, D(
2|
A
2|
Note that as we consider to be zero-one loss function, we have the followings:
A, _ max 0,0/
1
1(0(x;)
+ y)
1
1(0(z.) + #v}
D
D
max
[CO, C(O,D"))
= A b0
Since || = max, A S - || we obtain the second inequality of 2). Note
also that || E, (1 S V., 1 = A |All We thereby
obtain the first inequality of (
for all y E V. Consider- ing variance is always non-negative, we obtain the necessary condition that Ay 0 implies Var ((C(0, DE)) = 0 for all y E y
From we obtain Å, 0 if Var (e))
Proof of Lemma
Suppose that a model parameter 0 is given. As aEA
Σ
5(22 2h (A) and
Σ A
C(O, DH) is constant, we can decompose
Σ C(O, D into
aEA
Σ
1
L(,,
DM)
Σ
L(0, DM)
a IA|
[A
AEA
aEA
Σ
C(O, D
Σ l9
1
(C(0,
C(0, D
--4 05A
Σ
1 C(O, D.))
1
A
A
a
Let vu 2 q" 11,, where 1 E RRAA From EeEA 28 = 1, we have EBEA Uy =0. 0.
Considering Do [al 1) = EnEA + (1A118- -1)2 EnEA |Al(g!
) = En (1g)2.
14

---

Published as a conference paper at ICLR 2023
the following problems are equivalent:
max Σ q'EO, aEA
L, = max uy"
Σ
[A| C(0, D + Σ
[(0 D
Σ
C(0, (().D:))
|A|
aEA
aEA
aEA
subject to aEA
0 = 0, 0,,111112 v
A
Also, we have the following inequalities:
Σ REAA
[(0, (H) - Σ
1
Ci ((B)
Σ
(u8)
((),
Σ
|A| [(0, (().D))
.EAA |A|
aEA
REA
aEA
(A.1)
|A| V |Al x Var ((C(0, Db) )EELA))
P
(A.2)
which
A.I holds by the Cauchy- Schwartz inequality. The second inequality follows from
Var (1C(0, D8) AAEA)
1 [(0,, DH)
Σ
Σ A C(e. C(e.D))
-
||e011 S 'AA'
P
The equality in (A.2) is attained if and only if the vector a satisfies: (i) vdd and the |A| -imeensssio vector whose a-th element is L(0, D2) EnEA Li,, DW) are in the same direction; (ii) | p. This implies that the a-th element of nº should be [
((0,DR)
Σ aEA
[AT (()
X
A
V EoEd (C(0,D))
Σ 5 (X)
aEA
C(0, D&) - Σ J ((),
aEA
X
|AI
V |Al x Var (1C(0, D
Thus, Lemma holds if and only if
C(,,
Σ [Ai C(O, D
1
P
aEA
A
A
CCO
DH)
E A C(0, (H) aEA
for all a E A.
Proof of Proposition 2
From the proof of Lemma the equality in 5 holds if and only if Σ
C(O, (O)
A C(0, D2)
d
1 A
aEA
AAI
[C (H),DDD)
Σ
A C(O, 22)112
aEA
for Va E A. We now consider each element of q²a For given y E Y' define 72 R Al as follows:
((0,DH) - E. C(0, DM)
M
C(0,D%) E. [A C(0, D2)
for any E A. Fix E A b symmetry. Then, have:
Σ v 0, and thus, 7." = Σ Na
(A.3)
aeAA
nEAli
Σ 21 1, and thus, (2/))²
Σ
(12)
(A.4)
BEAV
15

---

Published as a conference paper at ICLR 2023
By the Cauchy- Suuchy-Schwarzaan inequality, hi = (11, ..., 1) 4--1 (
- V -1 / (x/))2 +... (n=-1)² (7 )2 ... + (7(22))2
= Al V 1- (7/),
in which the first and last equality follow from A.3)) and A.4). respectively.
From the above inequality, we have the followings:
(n,)² = (IAA) - 1) (1 (1,)))
|AIG & |A| - 1,
AAl -
16
(A.5)
[A
Considering
A.3 the equality in A.5
attained if and only if
7
A
V
Va E AVi.
|AI(IA 1)"
Thus, can derive the range described in Proposition 2.
B ADDITIONAL RELATED WORKS
B. FAIRNESS-CONSTRAINDD OPTIMIZATION
As mentioned in Sec. 2.1 some works (Agarwal et al. 2018 Cotter et al. 2019) employ constrained optimization framework to enforce the group fairness. They formulate a minimax problem of the Lagrangian function from the given constrained optimization problem and find saddle point by alternating the outer minimization step w.r.t. model parameters and the inner maximization step w.r.t. the Lagrangiam variables. When updating the model parameters, they employ convex upper bounded surrogate of 0-1 loss such as hinge loss cross entropy loss in order to address con-difeerntiaabiiiy of the fairness constraints. Further, Cotter al. (2099) employ a non-zero sum formulation that uses the surrogates for updating the model parameters and use the original (-- loss for updating the Lagrangian dual variables.
Although both Cotter et al. (2019) and our method similarly solve the minimax problems with 0-1 loss terms by using surrogate losses in the minimization step (w.r.t. model parameters) and the original 0-1 loss in the maximization step, we emphasize that Fair DRO address different kind of minimax problem compared to that of Cotter et al. (2019). Namely, their approach solves a minimax problem involving both model parameters and Lagrangian variables, while only focus the minimization w.r.t. model parameters, with fixed (which corresponds to a fixed Lagrangian variable). Instead, based our theoretical results that the minimization problem w.r.. model parameters can be formulated as another minimax problem based on the DRO framework, FairDRO solves such minimax problem by the non-zero formulation.
Furthermore, we argue that our surrogates would be more advantageous in terms of bounding the original objective function. When implementing the algorithm of Cotter et al. (2019), they make surrogates of group fairness constraints by typically using an upper bounded function of 0-1 loss such as the hinge loss or the cross entropy loss. However, the modified fairness constraint terms with such surrogates sometimes are not necessarily sharp upper bounds of the original constraints, e.g., multi-class classification setting. The reason is because the group fairness constraints, such as EO or ECA, are typically defined as the gap of averaged 0- losses for measuring the parity over groups. However, if we use the cross entropy loss as a surrogate in equation (6), we can get valid convex upper bound of (7) whenever q" is maximizer of the inner maximization problem, and all entries of r are positive values. Thus, we expect that our FairDRO can outperform PL by solving the original objective function better, which is shown in experimental results.
16

---

Published as a conference paper at ICLR 2023
B.2 ROBUST OPTIMIZATION IN ALGORITHMIC FAIRNESS
Much more works related to algorithmic fairness have recently utilized the DRO frameworks for their own goals. line of works embeds the DRO frameworks into the fairness-constrantt optimization
problem to make model fair, and robust to distribution shifts of test dataset (Jiang et al. 2020
Taskesen et al. 2020; Wang et al., 202T Mandal et al. 2020). For example, Mandal et al. (2020)
aimed to train fair classifiers to be with respect weighted perturbations in the training distribution
by considering all possible weight set over the training samples. Wang et al. (202T) and Taskesen et al. (2020) proposed DRO objectives with Wasserstein uncertainty sets which are combined by group fairness constraints. Although all aforementioned methods showed improvements in group fairness and robustness, the key differences from FairDRO are that leverage the DRO framework for directly promoting group fairness, not robustness, and consider the uncertainty set over groups, not samples.
Some works proposed DRO formulations for achieving fairness in more challenging settings like learning with imperfect group labels or sequential learning. Wang al. (2020) proposed how to achieve the group fairness successfully given noisy group labels only, by considering the worst-case distribution of group labels. Hashimoto et (2018) devised repeated loss minimization using the DRO-based training objective with the X" d-ivergeence uncertainty set for encouraging long-ter term fairness in a sequential setting where the group populations changes over time. However, they do not also use DRO as direct tool for achieving group fairness.
There are some recent works perhaps closest to our method in terms of using DRO for achieving the fairness. Rezaei et al. (2020) incorporated group faimess constraint into statistic-mathhing based robust log-loss minimization problem for training fair logistic regression models. Although they showed nice theoretical benefits in terms of its convexity and convergence, their method has limitation that group information is needed at the test time. Yurochkin et al. (2019) aims to achieve the fairness of classifier using DRO frameworks. However, they only focus novel individual fairness notion called as distributionally robust fairness (DRF), not group fairness.
B.3 MINIMAX OPTIMIZATION ALGORITHMIC FAIRNESS
Several works adopted min-max optimization techniques for group faimeess (although they not based on DRO frameworks). Zhang et al. (2018) proposed an adversarial debiasing technique that eliminates group information of latent features of a neural network (NN) model through the mini- max game of an adversary and the NN model. Baharlouei et al.] (2020) and [ et al. (2020) proposed novel regularization terms for group fairness using Rényi correlation and Wasserstein distance respectively, which lead to min-max formulations. We emphasize that these methods rely on additional terms adversaries for promoting group fairness, while our method directly uses the DRO-based min-max formulation as a fairness regularization term.
C MORE IMPLEMENTATION DETAILS
We used PyTorch Paszke et al. 2019); Experiments are performed on a server with AMD Ryzen Threadripper PRO 3975WX CPUs and NVIDIA RTX A5000 GPUs. All models are evaluated on separate test sets, and all experiments are repeated with 4 different random seeds. All the reported results are the averaged results.
C.I OPTIMIZATION
For tabular and vision datasets, we train all models with the Adam W optimizer (Loshchilov & Hutter 2019) for 70 epochs. We set the mini-batch size and the weight decay as 128 and 0.001, respectively, The initial learning rate is set as 0.001 and decayed by cosine annealing technique (Loshchilov &
Hutter 2017).
For the language dataset, we fine-tune pre-trained BERT with the AdamW optimizer for epochs. We set the mini-batch size and the weight decay 24 and 0.001, respectively. The initial learning rate is set as 0.00002 and adjusted with a learning rate schedule using a warm-up phase followed by linear decay. All results reported for the model the last epoch.
17

---

Published as a conference paper at ICLR 2023
C.2 IMPLEMENTATION DETAILS OF IN-PROCESSING BASELINES
Here, we describe the implementation details of each in in-processing baseline used for the experiments.
The original LBC (Jiang & Nachum, 2020) requires multiple full-training iterations by alternativeyy re-weighting each group based on the given fairness metric and re-training the full dataset. Since this optimization procedure needs very high computation budget, we modify full-training iterations to epochs and 14 iterations for vision and language datasets.
FairBatch (Roh et al. 2020) and RW (Kamiran & Calders 2012) are implemented in the way they were originally proposed.
Cov (Zafar al. 2017b) utilized a fairness constraint based on Covariance between the group label and the signed distance of the feature vectors from the decision boundary of classifier. Although they used the Disciplined convex-concave program (DCCP) (Shen et al., (016) solver for the constrained problem, we apply our optimization procedure (i.e., AdamW) by setting the covariance- aawrrance-bbssed constraint as a regularization term. We note that extenstion Cov to multi-class classifcation tasks is non-trivial because the signed distance is not obviously defined for multi-class decision boundary.
MFD (Jung al. ) and FairHSIC (Quadrianto al., 2019) use additional fairness-promotiin regularization terms based on MMD) and HSIC. For FairHSIC, we only implement the second term of their decomposition loss (i.e., the HSIC loss between the feature representations and the group labels). For implementing their regularization terms, we use the Gaussian RBF kernel of which the variance parameter is set the mean of squared distance between all data points, following Jung et al.) (2022). Since regularization terms used in both MFD and FairHSIC are applied to feature vectors of DNN model, they are limited to based models.
Baharlouci et al. (2020) attempted to remove the non- linear dependency between the model prediction and the sensitive attribute by using Rényi correlation as a regularization term. The resulting training objective is a minimax optimization problem where the maximization is for computing the Rényi correlation, and the minimization is for learning a model. When the group label binary, we use modified Algorithm 2 in (Baharlouei et al. (2020), where the minimization step (i.e., line 4 in Algorithm 2) is processed a mini-batch manner. Although Algorithm
designed for the non- non-binary group label, applying the algorithm to task with non- non-binary group labels is not straightforward because the second term in the RHS of equation 6 is biased estimator when using the
optimization.
C.3 HYPERPARAMETERS FOR MAIN RESULTS
We perform the grid search on the hyperparameter candidates for every method. The full hyperparam- eter search space is illustrated in Tab. C.I
C.4 MODEL SELECTION RULE FOR SINGLE EVALUATION METRIC
The accuracy-fairness trade-off may exist in many real- world applications DDutta et al. 2020); better accuracy leads to worse fairness, and vice versa. For example, Fig. Issows the trade-off for the Adult and COMPAS datasets. Therefore, when using single evaluation the model should be carefully selected for fair evaluation. To that end, we explore varying hyperparameters (HPs) of each method for controlling the strength between accuracy and fairness. We then select the best model showing the best fairness criterion ADCA while achieving at least 95% of the accuracy of the vanilla-traine model. With this selection rule, report the best performance of each model throughout Appendix [D] The HPs for each method were extensively scarched, but fairly in terms computation budgets. The search range of each hyperparameter is listsed in Tab. C.T
D ADDITIONAL RESULTS
D.1 RESULT TABLES
Following our model selection rule, described in Appendix C.4, we report the best performances of each result in Tab. D.1 and Tab. D.2 The numbers in the parentheses correspond to the standard
18

---

Published as a conference paper at ICLR 2023
Table C.I: Hyperparameter search spaces. We perform the grid search find the best hyperparameters for
each method.
Method
Hyperparameter
Search range
Cov Zafar 2017b)
Covariance strength À
[10 [10-3.10²)
Rényi Baharlouui et
2020
Rényi correlation strength
[10 2,10]
MFD Jung
2021
MMD strength
[10 110",100)
FairHSIC Quadrianto et al.
2019
HSIC strength À
1002,100]
LBC Jiang & Nachum
2020
LR for re- re-weights 1)
[10-3,
FairBatch (Roh et al.
2020
LR for mini-batch size of each group a
[10 1103,.00)
LR for Lagrange multiplier L norm bound B
[10 100
EGR Agaawal
2018
[10 [10"2,10')
LR for Lagrange multiplier n
[10 10
PL (Cotter et al.
2019
Violation upper bound K
[10 110-2,10) 10']
FairDRO
Radius of X diveergge ball
[10 (10-2,103)
Table D.I: The best performances Adult and COMPAS datasets. The number in the parentheses with +
stands for the standard deviation of each metric obtained from several independent runs with different seeds. The target accuracy (higher is hetter) and DCA (T) (lower is better) are shown. We follow the proposed model
selection criterion in Appendix C.4
Adult
COMPAS
Acc. (t)
ADCA (L)
Acc. (7)
Apca (1)
Scratch
81.95 (+0.03)
6.83 (++0.06)
63.29 (=0.00)
14.51 (++0000)
RW (Kumiran Calders
2012
81.98 (1:0.04)
6.84 (1:0.09)
63.29 (=0.15)
14.87 (#0062)
LBC Jiang & Nachum, 2020
81,21 (++0.17)
4.92 (++.277)
62.04 (=0.24)
6.03 (+1.13)
FairBatch (Koh at Al20
81.63 (++0.03)
5.75 (++0.09)
62,38 (=0.07
6.71 1.07)
Cav (Zafar et al. 2017a
78.68 (+:0.12)
5.08 (++0.51)
60.31 (=0.30)
7.58 (+3.711
Rényt (Baharouee el al
2020
77.09 (++021)
4.86 (++0.78)
61.44 (+0.35)
6.77 (++086)
EGR (Agarwal et al. 2018
80.59 (+:0.04)
6.03 (+0007)
60.66 (=0.73)
7.47 (+0.94)
PL [Ctterr erar, 2011
79.49 (1:0.14)
2.55 (1:0.39)
61.95 (=0.35)
6.73 (10.95)
FairDRO
79.23 (++0.11)
1.99 (++038)
61.97 (=0.24)
5.20 (+0.92)
deviation. From Tab. D.I and Tab. D.2 observe FairDRO achieves the best ADCA with moderate variances. For example, in Tab. D.I while the best baselines achieve ADCA of 2.55 and 6.03 on Adult and COMPAS, respectively, our FRO achieves 1.99 and 5.20 for the same datasets, respectively. We also emphasize that although the accuracy of FairDRO is slightly worse than the one of RW and Scratch on COMPAS, FairDRO shows significantly better DCA (5.20) compared to RW (14.87) and Scratch (14.51).
In Tab, D.2 we observe consistent results with Tab. D.I We again note that we marked results of COV and FairHSIC as N/A because COV is designed only for binary classification and FairHSIC fails to converge due to the gradient exploding. We again observe that FairDRO achieves the best level of the fairness metric (DCA) compared to other methods. We also provide difference of Equalized Odds (DEO) on UTKFace, denoted by ADEO, which is defined as follows:
ADEO
max p( = v/A a.Y -P 3/IA= d' -)]
D
a.a"
V.V'ey
Note that ADCA and ADEO the same for binary labels. Although LBC, FairBatch, and FairHSIC were devised for achieving EO, we again show that FairDRO outperforms basclines on UTKFace with respect to DEO.
19

---

Published as a conference paper at ICLR 2023
Table D.2: The best performances UTKeaace and CivilComments. The number in the parentheses with +
stands for standard deviation of cach metric obtained from several independent runs with different seeds.
"N/A" denotes the method is not trainable. All other details are the same as Tab. [
UTKFace
CivilComments
Acc. (1)
ADCA (1)
ADsO (U)
Acc. (t)
ADca (L)
Scratch
78.83 (=:0.3))
19.42 (=1.44)
13.78 (=0095)
78.12 (#1.22)
12.24 (#2.37)
RW (Kamiran & Calders 2012
79.27 (11.10)
17.92 (=0.60)
11.22 (110.00)
76.58 (111.63)
7.78 (111.19)
LBC Jiang & Nachum 2020
75.58 (#1.84)
16.92 (=1.83)
12.89 (#:1.03)
75.85 (122117)
7.04 (1-0,97)
FairBatch (Kon et al. 2020
78.04 (=:0.66
15.92 (=2.47)
12.39 (#0096)
77.18 (#1.30)
18.91 (#3.80)
Cov (Zafar al. 2017a
NIA
76.39 (11.111)
12.18 (24.07)
FairHSIC Quadiaanto ct al. 2019
77.35 (100,91)
17.58 (=1.67)
12.83 (#0.733
N/A
MFD (Jug of al. 2021
81.83 (+0095)
16.08 (=0.76)
11.08 (+:0555)
76.70 (+1.09)
11.36 (#2.78)
EGR (Agarwal al. 2018
N/A
76.40 (+33004)
9.53 (+6.22)
PL (Cotter at. 2019
78.71 (#0067)
16.42 (=0.83)
11.89 (#00522)
78.00 (#1.67)
10.38 (#3.37)
FairDRO
79.48 (+0.559)
14.25 (=1.26)
11.06 1.02)
77.65 (+1.75)
7.07 (+ 1.444
Scratch ARW
FairHSIc Cov
FairBatch XMF
Rényi FairORD (ours)
CelebA
CelebA
Acc. (f)
4DCA (L)
90.0
Scratch
85.80 (+0.55) 14.24 (+0.84)
RW (Kamiran Calders
2012
77.60 (1:0.68) 15.49 (+1141)
87.5
LBC Uiang & Nachum 2020
88.96 (#:0.97) 6.67 (=2.39)
85.0
FairBatch (Roh
2020
83.85 (.10.70)
10.62 (..0.66)
Cov (Zafar ct al. 201
90.10 (+:0.55) 7.57 (=0.84)
82.5
Rényi (Baharlouii et al.
2020
86.22 (:10.99) 10.62 (11.10)
FairHSIC FairHSICC(Qiadraann (Quadraanoo
2019
86.63 (+0.644) 9.10 (-3.19)
MFD (Jung 2021
90.97 (#:0355) 4.44 (=1.13)
77.5
PL Cotter et al. 2019
83.23 (#: 1.06)
1.46 (=0.84)
Apca (%6
FairDRO
87.95 1.61) 2.57 (.822)
Figure D.I: Trade-offs between accuracy and DCA on CelebA. The "Scratch" perfor- mances are highlighted by the dotted lines
Table D.3: The best performances CelebA. Details are the same Tab. D.I
D.2 CELEBA RESULTS
To show consistent improvements, we conducted an additional experiment on the CelebA dataset, which includes about 200K celebrities' face images with 40 annotated binary attributes. Among the attributes, we select "blond hair" as the class label and "gender" as the group label, which is the set of attributes widely used in many previous works (Sagawa al. 2020 Jung al. 2022
Chuang &
Mrouch 2021
Liu et al. 2021)
Tab. D.3 and Fig. D.I show similar result trend to one of UTKFace Sec. 5.2 FairDRO again outperforms the basclines in terms of DCA and achieves a competitive trade-off with MFD. We also observe re- reiieigt based baselines show worse DCA than FairDRO, meaning that FairDRO finds the better weights over groups for lower DCA.
D.3 RESULTS WITH THE STANDARD ERM LOSS
For fair comparison with FairDRO, we used the balanced ERM loss for the training of each baseline mentioned in Sec. 5 Since all baselines were originally designed for employing the standard ERM, we further reported the results for all methods solving the standard ERM loss. Fig. show the best performances and the accuracy-fairness trade-offs on Adult, COMPAD, UTKFace, and CivilComments datasets.
20

---

Published as a conference paper at ICLR 2023
Scratch RW
LBC
Cov
Rényl COMPAS
FairBatch
FairHsic
FGR
MFD
FairbRo (ours)
Adult
UTKKface
ChvllGemments
122
40
Accurscy (86)
Accursicy (86)
Accurscy 060
78
74
72
75
(N)
Figure D.2: The trade-offs between accuracy and DCA. Each method is trained with the standard ERM. Details are the same as Fig.
Table D.4: Ablation study of FairDRO. RVP (w/ classwise) and FairDRO (w/o classwise) are for the ablation
study of our key two components, the DRO formulation and classwise treatment, respectively. We follow the same model selection criterion introduced in Appendix C.4
Adult
COMPAS
UTKFace
CivilCommentt
Acc. (f)
ADCI (4)
Acc (?)
Apca ())
Acc. (t)
ADCA (4)
Acc. (t)
Apcs (L)
Scratch
81.95
6.83
63.29
14.51
78.83
19.42
78.12
12.24
FairDRO (w/o DRO)
80.47
4.86
62,20
8.55
79.21
17.58
78.13
9.79
RVP (w/ classwise)
80.10
4.62
61.99
9.74
77.25
12.33
N/A
FairDRO (wfo classwise)
78.45
1.62
60.72
6.28
78.25
20.75
77.42
8.85
FairDRO
79.23
1.99
61.97
5.20
79.48
14.25
77.65
7.07
D.4 ABLATION STUDY FOR THE CHOICE OF UNCERTAINTY SET
We continue from Sec. 5.4 and provide further study of FairDRO. We analyze the effect of each constraint employed by FaiDDRO. Tab. shows the overall accuracy, ApCa and the worst accuracy over groups for each ablation case. "Classwise", "iv-divergence" and "Quasi-prob" (4 indicate each component added when defining the uncertainty set of Group DRO (i.e., simplex over groups) as described in Sec. 4 For fair comparison, apply the same optimization scheme (described in Sec. 4.4) for DRO variants.
As the supremum in 3 is attained at the vertex of the simplex ADIIAAI, Group DRO minimizes the worst-case loss over the (y, a) pair, thereby it can prevent the lowest accuracy across the group and class from becoming too low. Thus, Group DRO substantially improves and achieves the best worst-case accuracy over groups compared to ERM (i.e., 1st + 2nd row). However, shows the worst DCA among the DRO variants, confirming the mismatch between the criterion Group DRO uses and the group fairness metric. We observe that changing the simplex uncertainty set of Group DRO to the proposed ddvdieergeecc and quasi-prob uncertainty sets (4) (i.e., 2nd - 3rd row) has an impact on the group fairness mostly. In addition, reducing the size of uncertainty sets via classwise treatment (i.e., 3rd + 6th row) further improves the group fairness. Finally, including negative values in the uncertainty set also helps to improve the group faimess (i.e., 5th - 6th row) on most datasets, by more severely penalizing the major groups with high accuracy with negative weights.
21

---

Published as a conference paper at ICLR 2023
Table D.5: Ablation study for uncertainty FairDRO. We show the effects of each component of the proposed FairDRO: GDRO and the constraints on the uncertainty set, including classwise uncertainty set, x -iveergeecee ball, and quasi-probability. All numbers are average results over Adult, COMPAS, FTKe and CivilComments datasets. GDRO is an abbreviation for Group DRO.
Adult
COMPAS
UTKFace
CivilComments
GDRO Classwse -i-div. Q-prob.
Acc.
Auca
Worst acc.
Acc.
Aoea
Worst ncc.
Acc.
Aoca
Worst acc.
Acc.
Áoea
Worst acc.
x
81.95
6.83
76.86
63.29 14.51
50.78
78.83 19.42
61.00
78.12 12.24
61.26
x
x
81.89 6.18
78.62
62.06
6.68
$7.57
79.54 18.17
61.00
80.32
12.63
69.72
/
78.45 1.62
77.46
60:72
6.28
55.94
78.25
20.75
45.75
77.42
8.85
63.70
81.68
5.89
77.67
62.10
6.41
53.59
78.69
14.50
63.75
79.99
9.16
71.52
/
/
/
81.53
5.76
77.38
62.32
6.42
53.91
78.37
13.75
66.50
78.56
7.89
67.24
79.23
1.99
75.60
61.97
5.20
53,91
79.48 14.25
65.75
77.65
7.07
62.85
Group
Group 1
Scratch
FairDRO
Accuracy (96)
80
Accuracy (%)
80
7.-
70
Accuracy (%)
60-
60-
50-
50
1
10
20
30
40
50
60
70
10
20
30
40
50
60
70
20
30
40
50
60
70
Training epochs
Training epochs
Training
(a) Scratch (train)
(b) FairDRO (train)
(c) Scratch & FairDRO (test)
Figure D.3: Accuracies and loss weights for each group (q) on COMPAS (y = 0). Details are the same as Fig.
4
D.5 VISUALIZATION OF dº
We continue from Sec. 5.4 and provide the visualization results including the weights (q"), training accuracy, and test accuracy for y 0. In Fig. [D.3] (a), Scratch shows that the significant accuracy gap between two groups is kept during all training time. On the other hand, we observe again in Fig. D.3 (b) that q" initially fluctuates and eventually converges by our optimization procedure, and result, the large discrepancy of training accuracy is reduced at the end of training. Finally, we show from Fig. D.3 (c) that the test accuracy gap between the groups of FairDRO is smaller than that of Scratch, i.e., FaiDDRO achieves lower DCA.
D.6 ABLATION STUDY OF SMOOTHED IBR
We observe the effectiveness of smoothed IBR updates introduced in Sec. 4.4 Fig. D.4 and Tab. D.6 compares dynamics of q' and performance on COMPAS depending on whether or not our smoothing technique is used for FairDRO optimization. Jim et al. (020) showed a theoretical result that the standard IBR asymptotically converge to an approximate stationary point w.r.t. under regularity assumptions. However, from Fig. D.4 (left), we still observe the instability of standard IBR; iee. oscillates whenever the group loss fluctuates. On the other hand, Fig. D.4 (right) shows that smoothing technique has obvious effect preventing the oscillation of q when using IBR. Furthermore, Tab. D.6 demonstrattss that stable dynamics of q can significantly improve performance in terms of DCA.
D.7 ISSUE OF STANDARD ACCURACY
In the experiments, reported the balanced accuracy for measuring the performance of the models instead the standard accuracy. While the balanced accuracy offers several advantages in measuring performance imbalanced datasets, also has some drawbacks. In general, it may underestimate the performance of certain groups with relatively large numbers. As in most cases, the groups with large numbers often become majority groups The balanced accuracy metric may capture accuracy
22

---

Published as a conference paper at ICLR 2023
Group 0
Group 1
10 20 30
40
50
60
70
0
10 20 30
40 50 60 70 Training epochs
Training epochs
Figure D.4: q COMPAS. The groupwise loss weights during training for label are shown. The reported weights are for FairDRO (wfo smoothing) (left) and FairDRO (right).
Table D.6: The best performances Adult and COMPAS datascts. We
follow the same model selection criterion described Appendix C.4
Adult
COMPAS
Acc. (t)
Apea (4)
Acc. (t)
Apea (L)
Scratch
81.95
6.83
63.29
14.51
FairDRO (w/o smoothing)
81.55
5.76
61.86
5.50
FairDRO
79.23
1.99
61.97
5.20
Table D.7: number of samples in Adult, COMPAS, and CelebA datasets.
Adult
COMPAS a
CelebA
a =
a
0
a =
a
a
13026
20988
2080
1278
17809
67054
2 1669
9539
1987
822
23060
1567
Table D.8: The number of samples in UTKFaae and CivilCommentt datasets.
UTKFace
CivilComments
0
y=
2049
336
1025
638
11485
17632
8154
4048
9823
y
3735
3116
1856
2221
4564
5892
828
545
1286
=
4294
1074
553
1116
reduction in majority group. In real-world applications, there is also concern about a decrease in majority-gree group accuracy.
However, we argue that reporting the vanilla standard accuracy would be problematic when the dataset severely class- imbalaneed...d. For example, Tab. D.7 shows the number of samples in each dataset with respect the class and group labels. For the case of Adult dataset, 75.2% of the samples are labeled as y 0, hence, naive predictor that always predict Y will achieve the standard accuracy of 75.2% and DCA= 0, which are clearly overestimating the (accuracy, fairness) performance of the classifier. Therefore, avoid such issue, we use the group-class balanced accuracy, rather than the standard accuracy.
23


Reviews:
[{"review_id": "tdfF7IKe-s", "reviewer": "Reviewer_cPS3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a method for training \"fair\" models via a classwise variant of Group DRO. The authors show that this is theoretically equivalent to optimizing for Difference of Conditional Accuracy, but they introduce some relaxations to make the problem tractable in practice. Experiments are conducted on two tabular, two image, and two language datasets with comparisons of the proposed method to baselines.\n", "summary_of_the_review": "Overall, this is an interesting and promising work, but it feels of borderline quality. It is nice to see DRO taken in the direction of more practical problems grounded directly in the fairness literature. I think that the main contribution of this work is on the theoretical side; on the empirical side, the results are quite mixed, and it is hard to disentangle the effects of some of the many implementation and experimental decisions on the empirical results. There is also not a clear framework for evaluation, which makes it even harder to say whether this method clearly \"works\" -- or whether it can be added to the pile of theoretically interesting, but practically unhelpful, robust optimization methods. I have other more minor concerns that are detailed above.", "strength_and_weaknesses": "\n## Major Comments\n\n* The authors suggest that their approach exactly optimizes for the DCA. However, in practice, they admit to being unable to do so, and instead optimize for a convex relaxation of it by using cross-entropy (Remark, Section 4.3). While this may be a minor point, the entire framing of the paper is based on *exactly solving* the DCA problem, not a relaxation of it. I think the abstract, introduction, and framing throughout should clarify this.\n\n* The experimental results are not particularly good. They seem somewhat promising on 1/2 tabular datasets (Adult) and 1/2 vision datasets (CC), but on the others there is weak, if any, evidence that the proposed method is better. We have no estimates of statistical uncertainty, so it is hard to tell much from the point estimates -- but even more, there is no clear evaluation metric for the prpposed model. Do we care about \\delta_DCA? If so,the proposed method doesn't clearly outperform baselines. Do we care about tradeoffs/envelopes? If so, the evidence is mixed at best, with baselines still outperforming the proposed method on at least half the datasets as mentioned. The ablation study is similarly mixed.\n\n\tI am particularly concerned about empirical performance because there is already an abundance of theoretically-motivated methods for DRO, as the authors acknowledge in their review, which don't offer much benefit in practice (probably due to being overly pessimistic). If the proposed method doesn't clearly improve, on real tasks according to metrics we as a community agree are important, I don't see a strong justification for publishing yet another robustness method. I'd like to be wrong about this, as I think the authors' direct connection to fairness and accuracy (also an under-investigated metric in the context of robust learning) are important, but the strong theoretical underpinnings should be accompanied by clear empirical results in order to make a case for acceptance.\n\n* It feels somewhat that the authors bury the lede that DCA is equivalent to EO, in the binary case. It would be useful to make this point much earlier, as some of the introduction/setup feels as if it is emphasizing a \"new\" approach to fair/robustness, but it is really a generalization of a very well-known measure.\n\n## Minor Comments\n\n* It is not intuitively clear to my why we ought to, in principle, balance over classes in ECA (eqn 1). Why is this the right thing to do? It seems to me this could be quite harmful in highly imbalanced datasets, and that this might be part of what is driving the subpar empirical results on many of the experiments.\n\n* The authors should probably cite and discuss the following; it seems relevant to e.g. Sec 4.2: Khani, Fereshte, Aditi Raghunathan, and Percy Liang. \"Maximum weighted loss discrepancy.\" arXiv preprint arXiv:1906.03518 (2019).\n\n* It is often unclear what the authors mean by \"directly\" optimizing for group fairness (e.g. \"both...do not use DRO as a direct tool for achieving group fairness\"). Please clarify.\n\n* \"Debiasing\" is mentioned as an application at least twice; I'm not sure this is an application. I believe \"image classification\" would be a better description of the application in the work cited as a debiasing paper.\n\n* How is the \"arg max\" in Equation (11) solved? It is not clear.\n\n* All of the tables are way too small. Please revise.\n\n* Why are the x-axes in Figures 1,2 both reversed?\n\n* The y=0 curve for Fig. 3 are hidden away in Appendix D.4, but those results look qualitatively different -- oscillating wildly between extremes. Please comment on these results.\n\n## Typos etc.\n\nThere are *many* grammatical problems with the paper. I list a few here, but it needs a thorough revision by a native English speaker.\n\nAbstract: \"between the reweighting-\" -> \"between reweighting\"\n\nP1: \"in learning various decision-making\" -> \"in various decision-making\"\n\nP1: \"such application is\" -> \"such applications is\"\n\nP1: \"re-weighting sample weights\" -> \"re-weighting samples\"\n\nP1: \"To address such issue, the fairness-aware\" -> \"To address such issues, fairness-aware\"\n\nP2: \"in algorithmic fairness literature\" -> \"in the...\"\n\nP7 \"finds the better\" -> finds better", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "cqzVF76xMU", "reviewer": "Reviewer_uGcU", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an in-processing based method to design a fair classifier for the metric difference of conditional accuracy (DCA). First, the authors show that empirical DCA can be approximated by the root of empirical group variance. This suggests using the root of empirical group variance as a regularizer in the learning objective. The authors then use a characterization due to [Xie et. al., 2020] to convert such a regularized ERM objective to a group distributional robust optimization.\n\nThe authors provide an iterative method to solve the fair DRO problem where the min player $\\theta$ uses gradient descent and the max player $q$ uses smoothed best response. This algorithm is evaluated on several datasets (tabular, vision, and language) and compared with several reweighing and regularization based methods. The results show that compared to the existing fair classifiers, the proposed method achieves better trade-off between accuracy and DCA.", "summary_of_the_review": "Overall, the authors present an interesting approach to design a fair classifier and the experimental results demonstrate that the proposed method pareto dominates existing fair classifiers on several datasets. However, I wonder if the proposed approach works beyond the fairness metric DCA. Moreover, the main connection with DRO relies on past literature [Xie et. al. 2020] and the proposed method lacks convergence and generalization guarantees.\n\nUpdate after rebuttal: Many thanks to the authors for the detailed response. It seems to me that the connection between fair classification and DRO is indeed non-trivial, as pointed out by other reviewers. Additionally, the new experimental results look convincing to me. Furthermore, the fairness metric DCA covers the more interesting fairness metrics for binary classification. But I hope the authors add some discussion on how generalizable this approach is to other fairness measures, particularly for multi-class classification. Overall I now feel positive about the paper and will increase my score.", "strength_and_weaknesses": "Strengths:\n- The connection between DCA and empirical variance is interesting and I wonder if one can show similar connections for other fairness measures.\n- The experimental results clearly show that the proposed method pareto dominates several other in-processing based methods.\n\nWeaknesses:\n- The proposed approach seems limited to difference of conditional accuracy (DCA) and it is not clear how this approach can be generalized to other fairness metrics. In fact, the main algorithm is motivated by proposition 1 which shows that empirical DCA can be approximated by the square root of empirical variance.\n- The authors claim that the iterative method for solving fair-DRO is efficient. However, I didn't see any convergence guarantees for the proposed method.\n- The authors also emphasized the use of exact regularization in the learning objective. However, this approach cannot guarantee any desired level of fairness in the training/test set. In particular, if one wants to output a classifier with fairness violation at most $\\alpha$, the current approach seems to require a lot of hyperparameter tuning.\n- There is no result on generalization i.e. how does the fairness guarantee and accuracy generalizes to unseen datasets. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "WsAltiPROqa", "reviewer": "Reviewer_QUdb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors investigate the classification problem with the penalization of the difference in conditional accuracy. They provide a tight characterization between the empirical DCA and variance of 0-1 losses. By combining it with the result of Xie et al., they derive the group-wise distributionally robust optimization form, which is more tractable than the original form. They also provide an iterative algorithm for solving the group DRO form of their optimization by utilizing the closed-form solution of the group DRO part. The empirical comparison demonstrates that the proposed algorithm is competitive or outperforms the existing fair classification method for equal opportunity. Also, the ablation study shows the robustness of the proposed mechanisms. ", "summary_of_the_review": "This paper is well-written and easy to follow. The tight characterization between DCA and the standard deviation of the losses is interesting and significant. The developed algorithm is novel and reasonable. Hence, I recommend acceptance at this point. However, the paper lacks a comparison with the crucial related work. I also have a concern about the empirical evaluations. I'd like the authors to address these concerns.", "strength_and_weaknesses": "Strength:\n- Clearly written and easy to follow\n- A tight connection between DCA and variance of losses is interesting and non-trivial.\n- The proposed algorithm is novel and reasonable.\n\nWeakness:\n- A lack of comparison with the reduction-based methods.\n- The empirical evaluations have some concerns.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "T7xtaMr_mAt", "reviewer": "Reviewer_U6MV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper seeks to enforce the group fairness metric \"Difference of  Conditional Accuracy\", a generalization of the equalized odds criterion, and points out an interesting connection between optimizing this metric and (class-wise) distributionally robust optimization (DRO). Specifically, they show that the criterion can be approximately written in terms of variances of group losses, and uses a result from Xie et al. (2010) to show an equivalence to DRO. Experiments on a number of benchmark datasets show case that the use class-wise DRO can indeed achieve better trade-offs between accuracy and DCA. ", "summary_of_the_review": "The paper makes an interesting connection between DCA and DRO, but requires a more elaborate discussion on some related methods and comparing against them in the experiments.", "strength_and_weaknesses": "Pros:\n- The connection between enforcing DCA (which is typically imposed as constraints in the optimization problem) and DRO is pretty interesting. \n- The experimental results are elaborate enough to show case that DRO can indeed be used to achieve competitive DCA performance\n\nCons:\n- The paper seems to emphasize that the prescribed use of DRO allows one to tackle the non-differentiability of DCA. I think as the authors themselves point out, even after the DRO reformulation, the objective is still intractable, and one does have to use surrogates for at least one step of the min-max optimization. In fact, similar \"selective of use of surrogates\" is common even in the constrained optimization literature. For example, the approach of Cotter et al (2019) formulates a constrained optimization problem to enforce constraints such as equalized odds, and does so by using surrogates for only updates on the model parameters and uses the original 0-1 loss for the group weights. So in terms of being able to address the non-differentiability of DCA, I don't this paper offering a solution that is very different what already exists in the literature.\n\n- A important baseline that is missing is the use of constrained optimization in the above manner to enforce DCA. I see that authors include Zafar et al. (2017b), but this method uses covariance-based surrogates, which I doubt are as tight as the more common cross-entropy style losses. I think either Cotter et al. (2019), which uses surrogates in the same way as this paper, but does so through a Lagrangian formulation instead of DRO, is one compelling baseline to include. Agarwal et al (2018) is another compelling baseline, which comes with theoretical guarantees --  this method uses a Lagrangian formulation based on the 0-1 loss to generate example weights, and solves the resulting cost-weighted optimization problem using a standard cost-sensitive learning (where one could employ a surrogate of choice). From what I can tell, the techniques in both Cotter et al. and Agarwal et al. are amenable to metrics like DCA.\n\nRef:\nCotter et al., \"Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals\", JMLR 2019.\n\nMinor questions:\n- In the experiments, is there a reason why the \"Scratch\" method, which if I understand correctly seeks to optimize a classification loss without fairness constraints, doesn't always do the best on accuracy (e.g. Adult, UTK faces, Civil comments)?\n- In discussing prior re-weighting methods, you mention \"they lack sound theoretical justifications for enforcing group fairness\". I don't think this is particularly true of methods like Agarwal et al.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster