G ROUP F AIRNESS R EGULARIZATION VIA C LASSWISE R OBUST O PTIMIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Existing group fairness-aware training methods typically employ some heuristics, such as re-weighting underrepresented groups based on some rules or using weakly approximated surrogates for the exact fairness metrics as regularization terms, which result in models with sub-optimal accuracy-fairness trade-offs.
The reason for using such heuristics is that the fairness metrics are usually non-differentiable or non-convex, and exactly incorporating those metrics in a tractable learning objective is challenging.
To that end, we propose a principled method that indeed can incorporate an exact form of a well-justified group fairness metric, Difference of Conditional Accuracy (DCA), as a regularizer using a classwise distributionally robust optimization (DRO) framework.
Namely, we first show that the DCA is equivalent (up to a constant) to the average (over the classes) of the roots of the variances of group losses, then employ the Group DRO formulation for each class separately to convert the non-differentiable DCA (or variance) regularized groupbalanced empirical risk minimization to a more tractable minimax optimization.
We then develop an efficient iterative optimization algorithm solving our minimax problem with more exact surrogate via non-zero sum game formulation.
We also show that our resulting method, dubbed as FairDRO, makes an interesting connection between re-weighting based and regularization-based fairness-aware learning.
Our experiments show that FairDRO is scalable, easily adaptable to diverse applications, and consistently improves the group fairness on several benchmark datasets in terms of the accuracy-fairness trade-off, compared to recent state-of-the-art baselines.
I NTRODUCTION
Machine learning algorithms are increasingly used in various decision-making applications that have societal impact; e.g. , crime assessment (Julia Angwin & Kirchner, 2016), credit estimation (Khandani et al., 2010), facial recognition (Buolamwini & Gebru, 2018; Wang et al., 2019), automated filtering in social media (Fan et al., 2021), AI-assisted hiring (Nguyen & Gatica-Perez, 2016) and law enforcement (Garvie, 2016).
A critical issue in such applications is the potential discrepancy of model performance, e.g. , accuracy, across different sensitive groups ( e.g. , race or gender)
(Buolamwini &
Gebru, 2018), which is easily observed in the models trained with a vanilla empirical risk minimization (ERM) (Valiant, 1984) when the training data has unwanted bias.
To address such issues, the fairnessaware learning has recently drawn attention in the AI research community.
One of the objectives of fairness-aware learning is to achieve group fairness , which focuses on the statistical parity of the model prediction across sensitive groups.
The so-called in-processing methods typically employ additional machinery for achieving group fairness while training, e.g. , re-weighting samples or adding fairness-promoting regularizations.
However, since exactly enforcing the given group fairness criterion into the training objective typically results in a non-convex optimization problem, most methods boil down to either using heuristic rules for re-weighting underrepresented samples ( e.g. , RW (Kamiran & Calders, 2012) or LBC (Jiang & Nachum, 2020)) or using approximated surrogate loss for group fairness as a regularization term ( e.g. , R´enyi (Baharlouei et al., 2020), or MFD (Jung et al., 2021b)).
To that end, while the in-processing methods have been easily adopted to train fair complex models like DNN on large-scale datasets, such as vision (Jung et al., 2021b;
Quadrianto et al., 2019) datasets, most of them actually lack theoretically clear justifications for achieving group fairness explicitly.
In this paper, we devise a new in-processing method in a more principled way, dubbed as Fairnessaware DRO (FairDRO).
The goal of our method is to incorporate an exact form of a well-justified group fairness metric as a regularizer in the objective function.
We use as the group fairness metric Difference of Conditional Accuracy (DCA)
(Berk et al., 2021), which is an extension of
Equal Opportunity (Hardt et al., 2016) to the multi-class, multi-group label settings.
To this end, we first present that DCA is equivalent (up to a constant) to the average (over the classes) of the roots of the variances of groupwise 0-1 losses.
Then, we employ the Group DRO formulation, which uses the χ 2 -divergence ball including quasi-probabilities as the uncertainty set, for each class separately to convert the non-differentiable DCA (or variance) regularized group-balanced empirical risk minimization (ERM) to a more tractable minimax optimization.
We then develop an efficient iterative optimization to solve the minimax problem with more exact surrogate via non-zero sum game formulation.
We show that our resulting method automatically produces the correct (sometimes even negative) re-weights for the samples in each group during the optimization process, making an interesting unified connection between the re-weighting and regularization-based fairness-aware learning schemes.
In our experiments, we empirically show that our FairDRO is scalable and easily adaptable to diverse application scenarios, including tabular (Julia Angwin & Kirchner, 2016;
Dua et al., 2017), vision (Zhang et al., 2017), and natural language text (Koh et al., 2021) datasets.
We compare with several representative in-processing baselines that apply either re-weighting schemes or surrogate regularizations for group fairness, and show that FairDRO consistently achieves the state-of-the-art performance on all datasets in terms of accuracy-fairness trade-off.
R ELATED WORKS 2. F AIRNESS - AWARE TRAINING
A comprehensive study on group fairness-aware learning has been conducted under the following three categories (Caton & Haas, 2020): pre-processing , in-processing , and post-processing methods.
In this paper, we focus on the in-processing methods, because they can flexibly incorporate the fairness notion during training without losing information from pre-processing the data or being limited by a pre-trained model as in the post-processing methods.
In-processing methods for group fairness
Recently proposed in-processing methods can be categorized into re-weighting based methods (Kamiran & Calders, 2012; Jiang & Nachum, 2020; Roh et al., 2020; Agarwal et al., 2018) and regularization based methods (Zafar et al., 2017b;
Quadrianto et al., 2019; Baharlouei et al., 2020; Jung et al., 2021b; Chuang & Mroueh, 2021).
The former methods typically determine to give higher weights to the samples from the underrepresented groups.
For example, Kamiran & Calders (2012) proposed a re-weighting scheme (RW) based on the ratio of the number of data points per each group.
Label Bias Correction (LBC) (Jiang & Nachum, 2020) and FairBatch (Roh et al., 2020) adaptively adjust weights and mini-batch sizes based on the average loss per group, respectively.
Although they can be easily adapted to various domains due to their simplicity, most of them lack sound theoretical justifications for enforcing group fairness and result in poor performance on some benchmark datasets.
On the other hand, the regularization-based methods apply weakly approximated surrogate constraints to their training objectives; e.g. , Cov (Zafar et al., 2017b) employ as fairness constraints a covariance approximation w.r.t.
a group label and a model prediction.
Jung et al.
(2021b)
devised MFD that uses Maximum Mean Discrepancy (MMD)
(Gretton et al., 2005) regularization for distilling knowledge and promoting group fairness at the same time.
Despite their success, we note that their performance could be sub-optimal due to the surrogates of the fairness metrics during learning.
Meanwhile, some works (Agarwal et al., 2018; Cotter et al., 2019) employ a constrained optimization formulation to enforce the group fairness, rather than giving additional penalty to loss functions like the regularization-based methods.
They formulate a minimax problem of the Lagrangian function from the given constrained optimization problem and seek to find a saddle point through the alternate updates of model parameters and the Lagrangian variable.
When updating the model parameters to address the non-differentiability of the fairness constraints, they employ a convex upper bound surrogate of the 0-1 loss such as hinge loss or cross entropy loss.
However, it turns out that the use of the modified fairness constraints with such surrogates does not necessarily bound the original constraints, leading to sub-optimal results.
A more detailed discussion is in Appendix.
2. D ISTRIBUTIONALLY ROBUST OPTIMIZATION (DRO)
Distributionally robust optimization (DRO) (Ben-Tal et al., 2009) is a general robust optimization framework that was originally used for learning a model that is robust to potential test distribution shift from the training set, by optimizing for the worst-case distribution in an uncertainty set.
The
DRO framework is now widely adopted in other applications, including algorithmic fairness.
DRO in algorithmic fairness Numerous works in the algorithmic fairness literature have recently utilized the DRO frameworks in various settings.
For example, a line of works (Jiang et al., 2020;
Taskesen et al., 2020; Wang et al., 2021; Mandal et al., 2020) embeds the DRO frameworks into the fairness-constrained optimization problem to make a fair model robust to distribution shifts of a test dataset.
Another related line of works (Wang et al., 2020; Hashimoto et al., 2018) proposed DRO formulations for achieving fairness in more challenging settings such as learning with noisy group labels or sequential learning.
In spite of their effectiveness, both of above lines of work do not use
DRO as a direct tool for achieving group fairness.
Perhaps the closest related work to ours would be (Rezaei et al., 2020; Yurochkin et al., 2019), in which DRO frameworks are adopted for the purpose of achieving fairness.
However, the key difference is that they either require the group labels at test time or target individual fairness.
On the other hand, our FairDRO directly aims to achieve group fairness while not requiring group labels at test time.
A more detailed discussion is in Appendix B.
DRO in other applications Applications of DRO are actively studied in other contexts, e.g. , image classification (Sagawa et al., 2020), multilingual machine translation (Oren et al., 2019;
Zhou et al., 2021), long-tailed classification (Samuel & Chechik, 2021) or out-of-distribution generalization (Krueger et al., 2021;
Xie et al., 2020).
For example, Sagawa et al.
(2020) proposed Group DRO which constructs the uncertainty set of joint distributions over the class-group label pairs to capture the spurious correlations between the class and group labels.
Although these works are effective in their applications, however, the notion of statistical parity across groups (group fairness) has not been explicitly pursued as in our FairDRO.
N OTATIONS AND FAIRNESS CRITERION
Notations We consider a classification problem where each sample consists of an input x ∈ X , a class label y ∈ Y and a group label a ∈ A .
The group label is defined by sensitive attributes such as gender or race .
Given a training dataset D with N samples ( i.e. , D = { ( x i , y i , a i ) } Ni =1 ), we aim to find a fair classifier θ FAIR : X → Y satisfying the given fairness criterion while maintaining the classification accuracy.
Moreover, given a loss function ℓ ( θ , ( x , y )) : Θ × ( X × Y ) → R , we denote L ( θ , D ) ≜ 1 |D| (cid:80) |D| i =1 ℓ ( θ , ( x i , y i )) .
Additionally, D a and D ya denote the subsets of D that are confined to the samples with group label a or class-group label pair ( y, a ) , respectively.
Fairness criterion
The notion of fairness may depend on the different points of view on how discrimination is defined (Hardt et al.
; Dwork et al.
(2012); Chouldechova (2017)
; Berk et al.
For example, one may argue that a classifier is fair if its prediction is not dependent on a ( Demographic Parity ), while the other can insist that the classifier should be conditionally independent on a ( Equalized Odds ).
In our work, we focus on the Equalized Conditional Accuracy (ECA) (Berk et al., 2021), which can naturally cover the multi-class, multi-group label setting.
A classifier θ satisfies ECA if all of the accuracies among groups are the same for each given class, i.e. , ∀ a,
[Equation]
[Equation]
= y ) , where (cid:98) Y is the prediction of θ .
We measure the degree of unfairness with Difference of Conditional Accuracy (DCA), denoted as ∆ DCA , by taking the average of the maximum accuracy gaps among groups over y :
[Equation]
Remark:
We note that ECA has a close connection with the two popular group fairness criteria,
Equalized Odds (EO) and Equal Opportunity (EOpp) (Hardt et al.
EO requires the conditional independence between (cid:98) Y and A given Y , hence, we can easily observe that ECA is equivalent to EO for the binary classification setting.
Furthermore, we argue that ECA is a natural extension of EOpp to the multi-class setting; namely, ECA requires equality of TPR among groups for each class label, which resembles the argument for justifying EOpp.
F AIRNESS - AWARE DRO (F AIR DRO)
Our goal is to incorporate the DCA (1) in the objective function.
However, since DCA is nondifferentiable, it is not straightforward to use DCA directly as a regularization term.
To that end, we first show in Sec.
4.1 that DCA is equivalent to the average (over the classes) of the roots of the variances of groupwise 0-1 losses.
Then, we utilize the previous result (Xie et al., 2020) in
Sec.
4.2 to make the connection between the Group DRO with the uncertainty set of χ 2 -divergence ball including the quasi-probability and the variance regularized group-balanced empirical loss minimization.
Subsequently, in Sec. 4.3, we present our training objective that applies Group DRO separately for each class y , which essentially converts the non-differentiable DCA (or variance) regularized group-class balanced ERM to a more tractable minimax optimization problem.
Finally,
Sec.
4.4 proposes an efficient iterative algorithm that does alternating minimization-maximization of our training objective.
4. E QUIVALENCE BETWEEN DCA AND VARIANCE OF GROUPWISE LOSSES
Proposition 1 Assume ℓ ( θ ) = 1 { θ ( x )
̸ = y } .
Then, the following inequality holds for all y ∈ Y ,
[Equation]
[Equation]
in which (cid:98) ∆ y is the empirical version of ∆ y and Var( { x i } di =1 ) ≜ 1 d (cid:80) i
( x i − 1 d (cid:80) i x i ) 2 is the variance of d numbers { x i } di =1 .
Thus, (cid:98) ∆ y of a classifier is 0 if and only if Var (cid:0) {L ( θ , D ya ) } a ∈A (cid:1)
= 0 .
The proof is in Appendix A. By taking the average over y in (2), Proposition 1 implies that DCA is equivalent to the average (over y ) of the square roots of the variances of the groupwise 0-1 losses, up to a constant.
Building this equivalence, we are now able to utilize the Group DRO formulation given in the next subsection to make a connection between DCA regularization and robust optimization.
4. P RIOR WORK ON VARIANCE REGULARIZATION VIA G ROUP DRO
While ERM minimizes the empirical loss, DRO optimizes for the worst-case distributions in an uncertainty set, as mentioned in Sec. 2.2.
Namkoong & Duchi (2017) have shown the connection between general DRO and variance regularization, and Khani et al.
(2019) further associate it to group fairness by introducing a concept of weighted loss discrepancy.
Since general DRO (Duchi et al., 2019) may lead to a pessimistic model that optimizes for implausible worst-case distributions,
Sagawa et al.
(2020) recently proposed the Group DRO framework, which performs at the group level and can incorporate prior knowledge about the groups 1 ; it aims to obtain
[Equation]
in which ∆ |A| is the |A| -simplex.
The applications of Group DRO are actively studied in various contexts, e.g. , image classification (Sagawa et al., 2020) or machine translation (Zhou et al., 2021).
In particular, Xie et al.
(2020) proposed Risk Variance Penalization (RVP), an OOD generalization method, using a variance regularization with respect to risks of each domain (a formulation first introduced by V-REx (Krueger et al., 2021)).
The author showed the connection between the variance regularization and Group
DRO, when the uncertainty set is the following χ 2 -divergence ball with radius ρ >
[Equation]
[Equation]
in which D ϕ ( p ∥ q ) ≜ (cid:80) i q i
ϕ ( p i /q i ) with ϕ ( t ) =
( t − 1) 2 , and 1 |A| 111 ∈ R |A| denotes the uniform distribution.
Note (4) does not have the nonnegativity constraint of q , hence, depending on ρ , Q ρ can also include quasi-probabilities , which has negative components as well.
Namely, Xie et al.
(2020) shows the following lemma that the inner maximization of Group DRO with Q ρ uncertainty set is equivalent to the group-balanced empirical loss plus the group-wise variance term as a regularizer.
Lemma 1 (Xie et al., 2020, Proposition 1) For any finite loss ℓ ( θ , ( x , y )) and with Q ρ in (4), we have
[Equation]
To be self-contained, the proof of the lemma is given in Appendix A.
4. T RAINING OBJECTIVE OF F AIR DRO
To make the connection between Proposition 1 and Lemma 1 and realize the DCA regularization for fairness-aware training, we propose to apply the Group DRO separately for each class y .
Namely, we define our FairDRO as minimizing the average of the worst-case losses defined for each class y :
[Equation]
in which Q ρ is as defined in (4), q y = { q ya } a ∈A is a quasi-probability vector, and ρ is a tunable hyperparameter.
We stress that while applying Group DRO by treating each pair ( y, a ) as a “group” has been considered in Sagawa et al.
(2020), applying Group DRO separately for each class as in
FairDRO has not been considered before.
Now, we present our key result which follows by combining
Proposition 1 and Lemma 1.
Corollary 1 Assume ℓ ( θ ) =
1 { θ ( x ) ̸ = y } .
Then, given a dataset D , there is a corresponding positive constant C D in the range of [ √ 2 , √ 2 |A| ] such that θ FairDRO also achieves
[Equation]
The corollary implies that when the 0-1 loss is used, FairDRO is also a regularization-based method , as described in Sec. 2.1, with the exact DCA serving as a regularizer.
Note we do not solve the minimization in (7) directly; however, since (cid:98) ∆ DCA can be biased when estimated from a small mini-batch, particularly when the number of groups is large, and is hard to minimize due to nondifferentiability.
Instead, we solve a much more tractable minimax optimization in (6) where the maximization is a simple linear optimization and the minimization is simply for the re-weighted group losses, and no estimation of (cid:98) ∆ DCA is involved.
At this point, it is now clear that during solving (6), FairDRO produces the re-weights { q ya } ’s for each ( y, a ) , which makes it also categorized as a re-weighting based method .
The following proposition shows how the re-weights are characterized and controlled by the hyperparameter ρ .
Proposition 2 For Q ρ in (4) with ρ > 0 , each element of the optimum q ∗ y for the inner-maximization of (6) can be obtained by a closed-form solution:
[Equation]
and is in the range of (cid:104) 1 |A|
[Equation]
The proof is in Appendix A. Proposition 2 shows that some elements of q ∗ y can be negative depending on ρ and
the groupwise loss L ( θ , D ya ) .
Moreover, it shows that the range of each element of the optimum q ∗ y is automatically determined by a radius ρ and the number of groups |A| , which means
FairDRO controls the degree of the penalization by varying ρ .
We will show in Sec.
5 that we can find a proper degree of penalization by tuning ρ so that the performances on various datasets in terms of accuracy-fairness trade-off are significantly improved.
A crucial difference from typical re-weighting based methods is that our re-weights are obtained as the solution of an optimization problem as opposed to those in (Kamiran & Calders, 2012;
Jiang
Nachum, 2020)
that are heuristically obtained based on the ratio of the number of data points or losses for each group.
Also, we consider negative values for q ya in Q ρ .
Such negative values for q ya enable penalizing a high accuracy group more aggressively than typical re-weighting methods that only control the positive weights for each group a and class y , to achieve group fairness.
Remark:
For Corollary 1 to hold, we use 0-1 loss in (6), which makes our training objective still non-differentiable with respect to θ .
Thus, when solving (6), we use the cross-entropy loss for the outer minimization and use the 0-1 loss for the inner maximization, that is, a non-zero sum game formulation.
Since the cross-entropy is the convex surrogate for 0-1 loss, our resulting surrogate is a convex upper bound for the original objective function, leading to practically achieve the near-optimal solution of (6) by our strategy.
We again note that the non-differentiable regularization term in (7) is substituted into a more feasible problem through our DRO formulation.
4. A N EFFICIENT ITERATIVE OPTIMIZATION FOR F AIR DRO
A canonical way of solving the minimax optimization in DRO is to use a primal-dual method (Nemirovski et al., 2009) in which one alternates between a regular gradient descent on θ and an exponentiated gradient (EG) ascent (Kivinen & Warmuth, 1997) on q .
However, the EG algorithm is widely used when the variables lie in the probability space because it is a special case of mirror descent when the convex function for the Bregman divergence is the Shannon Entropy.
Thus, EG cannot be applied to solving (6) since our uncertainty set also allows the quasi-probability for q y .
Hence, we employ the smoothed Iterated Best Response (IBR) which is a variant of IBR recently used for solving a DRO problem in (Zhou et al., 2021).
Using the standard IBR, we update q as the closed-form solution of the maximization ( i.e. , (8)), instead of the gradient ascent of q .
It is known that such IBR procedure eventually converges to equilibrium under some mild regularity assumptions (Zhou et al., 2021).
However, we empirically observed the instability during training; i.e. , the q ∗ y,t oscillates when the loss for each group fluctuates, hence, we applied the learning rate scheduling of η t q = 1 − tT with T being the number of total epochs to update q y,t smoothly .
Namely, our update rules are
[Equation]
in which q ∗ y,t ≜ arg max
[Equation]
Input : Dataset {D ya } y ∈Y ,a ∈A , radius ρ , epoch T , iteration I , mini-batch size B 1 Randomly initialize θ2 Set q y, 0 ← 1 |A| 111 for y ∈ Y , in which 111 ∈ R |A| 3 for t = 0 to T − 1 do // θ -update for i =
to I − 1 do
Sample a mini-batch { ( x j , y j , a j ) } j ∈
[ B ] , equally from D ya for each ( y, a ) ∈
Y × A .
Update θ t with (9) using the mini-batch end //
q -update
Update q y,t for each y ∈ Y with (10) 11 end
Output : θ T , { q y,T } y ∈Y and η t θ and η t q denote the learning rates for θ and q y , respectively.
By applying smoothing for update q y as in (10) , we empirically observe improvements in the group fairness and more stable convergence of q y (result given in Appendix D.6).
Algorithm 1 illustrates the summary of FairDRO optimization scheme.
E XPERIMENTAL RESULTS
In this section, we demonstrate the versatility of FairDRO by comparing it with various baseline methods on three modalities of datasets: (1) two tabular datasets, UCI Adult (Dua et al., 2017) and ProPublica COMPAS (Julia Angwin & Kirchner, 2016), (2) two vision datasets, UTKFace (Zhang et al., 2017),
CelebA (Liu et al., 2015) (in Appendix D.2) and (3) a language dataset, CivilComments-WILDS (Koh et al., 2021).
We use logistic regression,
ResNet18 (He et al., 2016), and a pre-trained BERT (Devlin et al., 2019; Wolf et al., 2019) as the classifier for each modality of datasets, respectively.
For the evaluation, we report the trade-offs between the accuracy and ∆ DCA by varying hyperparameters of each method.
We also evaluated
[Figure]
our method in terms of the best DCA with at least 95% accuracy of Scratch in the Appendix D.1.
Since the most datasets are severely skewed toward a specific group or class, we report the balanced accuracy over all combinations of groups and classes (the issue discussed more in Appendix D.7).
We further give a thorough ablation study on our design choice in (6), namely, the classwise DRO formulation and the uncertainty set (4) in Sec.
5.4 and Appendix D.4, respectively.
We further visualize of re-weights produced by FairDRO in Sec. 5.4.
Further implementation details are given in
Appendix C.
Comparison methods .
We compare FairDRO with vanilla training (Scratch) and nine in-processing fair-training methods; Three re-weighting based methods (RW (Kamiran & Calders, 2012), LBC (Jiang & Nachum, 2020), FairBatch (Roh et al., 2020)), four regularization-based methods (Cov (Zafar et al., 2017b), FairHSIC (Quadrianto et al., 2019), R´enyi (Baharlouei et al., 2020), MFD (Jung et al., 2021b)) and two methods employing a constrained optimization (EGR (Agarwal et al., 2018), PL (Cotter et al., 2019)).
Scratch denotes logistic regression model (for Adult and COMPAS),
ResNet18 (for UTKFace), and BERT (for CivilComments) trained without any fairness constraints.
Since we consider a group-class balanced accuracy, we modified the objective functions of all baselines including Scratch such that they include the balanced ERM loss, not a standard ERM loss.
We note that Cov, EGR and R ´ enyi are task-dependent, and FairHSIC and MFD are model-dependent.
Namely, Cov and EGR are designed only for binary classification tasks.
Also, applying R ´ enyi to non-binary group labels is not straightforward with mini-batch training.
Meanwhile, FairHSIC and
MFD are limited to be only applied for DNN architectures with a sound extractor.
We also note that all baselines except RW were implemented for targeting EO.
Considering that ECA is equivalent to EO only in binary classification, we also reported the difference of EO (DEO) for multi-class classification tasks in Appendix D.1.
We provide more details of description and implementation of the baselines in Appendix C.
5. T ABULAR DATASETS
Two tabular datasets, UCI Adult (Dua et al., 2017)
(Adult) and ProPublica COMPAS (Julia Angwin & Kirchner, 2016) (COMPAS), are used for the benchmark.
Both tasks are binary classification tasks (predicting whether the income of an individual exceeds $50 K per a year and whether a defendant re-offends, respectively) with binary sensitive groups (“Female” and “Male”, and “Caucasian” and “Non-Caucasin”, respectively).
Following pre-processing of Bellamy et al.
(2018), each dataset includes 45K and 5K rows, respectively.
Fig.
1 shows the fairness-accuracy trade-offs with different hyperparameters on Adult and COMPAS.
Note that we only report a single score for RW because it has no controllable hyperparameter.
In the figure, FairDRO is placed to the most top-right area among the methods, showing the best trade-off.
We emphasize that on both datasets, FairDRO can achieve the lowest level of DCA with a slight loss of accuracy.
In addition, other re-weighting based methods perform worse than FairDRO on Adult in terms of DCA, which implies that FairDRO successfully finds better weights ( i.e. , quasi-probability q y ) over groups for achieving group fairness.
[Figure]
Scratch
[Figure]
Cov
FairBatch
FairHSIC
MFD
FairDRO(ours)
5. V ISION DATASETS
We also evaluate FairDRO on UTKFace (Zhang et al., 2017), a face dataset with multi-class and multi-group labels.
UTKFace includes more than 20K images with age, gender and ethnicity labels.
We divide the age labels into three classes (“0 to 19”, “20 to 40”, and “more than 40”), and set these as the class labels.
We also use four ethnic classes ( “White”, “Black”, “Asian”, and “Indian” ) as the group labels.
Results on another vision dataset, CelebA
(Liu et al., 2015), are given in Appendix D.2.
Fig.
2 show the accuracy-fairness trade-offs on UTKFace for each method.
We confirm that FairDRO significantly improves the trade-off, compared to most baselines.
Although MFD also achieves competitive trade-offs compared to FairDRO on UTKFace, we note that MFD requires additional computations for training a teacher model for knowledge distillation.
We also observe that the re-weighting based baselines are ineffective in achieving fairness on UTKFace, while FairDRO systematically optimizes the weights for reducing DCA.
The best performances on UTKFace are in
Tab. D.2.
We additionally report the difference of EO (DEO) in Appendix D.1 and show FairDRO again outperforms them in DEO.
5. L ANGUAGE DATASET
In order to show the versatility of FairDRO, we additionally consider a language dataset,
CivilComments-WILDS (Koh et al., 2021).
CivilComments is a large collection of comments on online article taken from the Civil Comments platform.
It comprises 450K comments which are annotated with identity attributes by 10 crowdworkers and majority votes.
The CivilComments task is to predict whether a given comment is toxic or not.
We set ethnicity as the sensitive attribute and extract the subset of comments that contain identities w.r.t.
ethnicity ( “Black”, “White”, “Asian”, “Latino” and “Ohters” ).
Note that the dataset has continuous values for the identities (between 0 and 1 ), hence, we binarized the continuous group values by setting the maximum as 1 and the others as 0, because the existing in-processing methods are only available to handle discrete group labels.
The results are shown in Fig.
Note that since a pre-trained BERT is fine-tuned only for small epochs, the update for q y of FairDRO and weights of re-weighting baselines is executed periterations, instead of every epoch.
We empirically observed that FairHSIC fails to converge due to the gradient exploding.
Consistent with the previous results, FairDRO achieves the best trade-off on
CivilComments.
Although MFD is the best baseline on UTKFace, it does not show effectiveness in achieving fairness on CivilComments.
The best performances on CivilComments are in Tab. D.2.
5. A NALYSIS
Ablation study .
We analyze the effect of our DRO formulation and the classwise treatment employed by FairDRO.
Fig.
3 shows the fairness-accuracy trade-off of each method on each dataset.
For fair comparison, we employ the classwise treatment for RVP (w/ classwise) and study the effect of our DRO formulation.
Namely, RVP (w/ classwise) directly employs the variance of groupwise cross-entropy losses as a regularization term for each class separately, i.e. , the class-separated version of the RHS of (5) with cross-entropy loss.
FairDRO (w/o classwise) defines only one uncertainty set over the ( y, a ) pair in (6) for the ablation study of the classwise treatment.
In Fig.
3, we note that RVP (w/ classwise) on CivilComments fails to converge by the gradient exploding.
[Figure]
[Figure]
Followings are our observations.
First, we see that although RVP (w/ classwise) improves DCA on UTKFace, it sometimes fails to produce a fair model because its variance regularization using the cross entropy loss is a crude surrogate for DCA.
Second, we clearly observe that FairDRO (w/o classwise) achieves sub-optimal accuracy-DCA trade-offs (except for Adult), which implies that our classwise DRO formulation is critical for the optimal performances since it makes the connection to the DCA regularization more clearly.
Further ablations study for the choice of the uncertainty set is in Appendix D.4.
Visualization of q y .
We visualize the learning dynamics of the weights ( q y ), including training accuracies (Fig. 4 (a) and (b)) and test accuracies (Fig.
4 (c)) for Scratch and FairDRO on COMPAS.
For simplicity, we report the result for y = 1 (for y = 0 is in Appendix D.5).
In Fig.
4 (a), q 1 assigns the same value for each group as Scratch employs the balanced ERM loss.
At the top of the plot,
Scratch shows a large discrepancy in accuracy between the two groups throughout training.
On the other hand, Fig. 4 (b) shows FairDRO adjusts q 1 by assigning low values to the high accuracy group (Group 0) and high values to the low accuracy group (Group 1).
Note that q 1 initially fluctuates and even assigns negative values for the high accuracy group but eventually converges via learning rate scheduling in (10).
As a result, we clearly observe both the training and test accuracy gaps between the groups smoothly become negligible as the training continues.
We believe this example clearly shows the effectiveness of introducing the quasi-probability in Q ρ and our FairDRO framework in achieving small test accuracy gaps between the groups, i.e. , ∆ DCA .
C ONCLUDING REMARK
We proposed a theoretically well-justified in-processing method for achieving the group fairness.
In order to make the non-differentiable DCA regularized problem more tractable, our FairDRO employs the classwise Group DRO framework with χ 2 -divergence ball including quasi-probability as an uncertainty set.
As a result, the FairDRO training objective directly incorporates DCA as a regularizer and can be effectively solved by our proposed optimization scheme.
Furthermore, our
DRO formulation enables FairDRO to have the strengths of both re-weighing based and regularization based methods.
Indeed, we showed through our experiments that our FairDRO are applicable to various settings and consistently achieve superior accuracy-fairness trade-off on benchmark datasets.
A PPENDIX
We provide additional materials in this document.
We include our proofs of proposed Lemma and
Propositions in Appendix A, additional related works in Appendix B, and implementation details such as optimization, implementations of baselines
[Equation]
and hyperparameter search in Appendix C.
Finally, we report additional results in Appendix D
A P ROOFS
Proof of Proposition 1 We define L y ∈ R |A|×|A| for brevity:
[Equation]
If we denote Λ y ∈ R |A| 2 as
[Equation]
then
[Equation]
we have the following chain of equalities:
[Equation]
Note that as we consider ℓ to be zero-one loss function
[Equation]
we have the followings:
[Equation]
Since ∥ Λ y ∥ ∞ = max i | Λ yi | ≤ (cid:112)(cid:80) i (Λ yi )
2 = ∥
Λ y ∥ 2 , we obtain the second inequality of (2).
Note also that ∥ Λ y ∥ 2 = (cid:112)(cid:80) i
( Λ yi ) 2 ≤ (cid:112)(cid:80) i ∥ Λ y ∥ 2 ∞ =
(cid:112) |A|
2 ∥ Λ y ∥ 2 ∞ = |A|∥ Λ y ∥ ∞ .
We thereby obtain the first inequality of (2).
From (2), we obtain (cid:98) ∆ y = 0 if Var (cid:0) {L ( θ , D y a ) } a ∈A (cid:1) = 0 for all y ∈ Y .
Considering variance is always non-negative, we obtain the necessary condition that (cid:98) ∆ y = 0 implies
Var (cid:0) {L ( θ , D ya ) } a ∈A (cid:1) = 0 for all y ∈ Y .
Proof of Lemma 1 Suppose that a model parameter θ is given.
As (cid:80) a ∈A (cid:16) q ya − 1 |A| (cid:17) = 0 and (cid:80) a ∈A 1 |A| L ( θ , D ya ) is a constant, we can decompose (cid:80) a ∈A q ya L ( θ , D ya ) into (cid:88) a ∈A
[Equation]
) − (cid:88) a ∈A |A|L ( θ , D ya ) (cid:17) .
Let v y ≜ q y − 1 |A| 111 , where 111 ∈ R |A| .
From (cid:80) a ∈A q ya = 1 , we have (cid:80) a ∈A v ya = 0 .
Considering D ϕ (cid:16) q (cid:12)(cid:12)(cid:12)(cid:12) 1 |A| 1 1 1 (cid:17) =
(cid:80) a ∈A 1 |A| ( |A| q y a − 1)
= (cid:80) a ∈A |A| ( q y a − 1 |A| )
2 = |A| (cid:80) a ∈A ( v y a ) 2 , the following problems are equivalent:
[Equation]
subject to (cid:88)
[Equation]
Also, we have the following inequalities:
[Equation]
in which (A.2) holds by the Cauchy-Schwartz inequality.
The second inequality follows from
[Equation]
The equality in (A.3) is attained if and only if the vector v y satisfies: (i) v y and the |A| -dimensional vector whose a -th element is L ( θ , D ya ) − (cid:80) a ∈A 1 |A| L ( θ , D ya ) are in the same direction; (ii)
||
v y ||
= ρ |A| .
This implies that the a -th element of v y should be
[Equation]
Thus, Lemma 1 holds if and only if
[Equation]
for all a ∈ A .
Proof of Proposition 2 From the proof of Lemma 1, the equality in (5) holds if and only if
[Equation]
for ∀ a ∈ A .
We now consider each element of q ∗ y .
For given y ∈ Y , we define γ y ∈ R |A| as follows:
[Equation]
for any a ∈ A .
Fix i
∈ A by symmetry.
Then, we have: (cid:88) a ∈A γ ya = 0 , and thus, γ yi = − (cid:88)
a ∈A\ i γ ya , (A.6) (cid:88) a ∈A ( γ ya ) 2 = 1 , and thus, ( γ yi ) 2 = 1 − (cid:88)
a ∈A\ i ( γ ya ) 2 .
(A.7)
By the Cauchy-Schwarz inequality,
[Equation]
in which the first and last equality follow from (A.6) and (A.7), respectively.
From the above inequality, we have the followings:
[Equation]
Considering (A.6), the equality in (A.8) is attained if and only if
[Equation]
Thus, we can derive the range described in Proposition 2.
B A DDITIONAL RELATED WORKS
Much more works related to algorithmic fairness have recently utilized the DRO frameworks for their own goals.
A line of works embeds the DRO frameworks into the fairness-constraint optimization problem to make a model fair, and robust to distribution shifts of a test dataset (Jiang et al., 2020;
Taskesen et al., 2020; Wang et al., 2021; Mandal et al., 2020).
For example, Mandal et al.
(2020) aimed to train fair classifiers to be with respect to weighted perturbations in the training distribution by considering all possible weight set over training samples.
Wang et al.
(2021) and Taskesen et al.
(2020) proposed DRO objectives with Wasserstein uncertainty sets which are combined by group fairness constraints.
Although all aforementioned methods showed improvements in group fairness and robustness, the key differences from FairDRO are that we leverage the DRO framework for directly promoting group fairness, not robustness, and consider the uncertainty set over groups, not samples.
Some works proposed DRO formulations for achieving fairness in more challenging settings like learning with imperfect group labels or sequential learning.
Wang et al.
(2020) proposed how to achieve the group fairness successfully given noisy group labels only, by considering the worst-case distribution of group labels.
Hashimoto et al.
(2018) devised repeated loss minimization using the
DRO-based training objective with the χ 2 -divergence uncertainty set for encouraging long-term fairness in a sequential setting where the group populations changes over time.
However, they do not also use DRO as a direct tool for achieving group fairness.
Several works adopted min-max optimization techniques for group fairness (although they are not based on DRO frameworks).
Zhang et al.
(2018) proposed an adversarial debiasing technique that eliminates group information of latent features of a neural network (NN) model through the minimax game of an adversary and the NN model.
Baharlouei et al.
(2020), and Jiang et al.
(2020) proposed novel regularization terms for group fairness using R ´ enyi correlation and Wasserstein distance respectively, which lead to min-max formulations.
We emphasize that these methods rely on additional terms or adversaries for promoting group fairness, while our method directly uses the
DRO-based min-max formulation as a fairness regularization term.
There are some recent works perhaps closest to our method in terms of using DRO for achieving the fairness.
Rezaei et al.
(2020) incorporated a group fairness constraint into statistic-matching based robust log-loss minimization problem for training fair logistic regression models.
Although they showed nice theoretical benefits in terms of its convexity and convergence, their method has a limitation that group information is needed at the test time.
Yurochkin et al.
(2019) aims to achieve the fairness of a classifier using DRO frameworks.
However, they only focus on a novel individual fairness notion called as distributionally robust fairness (DRF), not group fairness.
Similar to our work, Khani et al.
(2019) also discussed the relation between the group fairness and the variance regularization over group losses by introducing a novel notion, which is the maximum weighted loss discrepancy (MWLD).
They showed that MWLD is related to the loss variance and the group fairness varying on a weighting function of MWLD.
Further, they showed through experimental results that minimizing the loss variance has an effect on improving the group fairness.
However, their MWLD has a limitation showing only a weak connection between variance and group fairness.
Meanwhile, some works (Agarwal et al., 2018; Cotter et al., 2019) employ a constrained optimization framework to enforce the group fairness.
They formulate a minimax problem of the Lagrangian function from the given constrained optimization problem and find a saddle point by alternating the outer minimization step w.r.t.
model parameters and the inner maximization step w.r.t.
the Lagrangian variables.
When updating the model parameters, they employ a convex upper bounded surrogate of 0-1 loss such as hinge loss or cross entropy loss in order to address non-differentiability of the fairness constraints.
Further, similar to our algorithm, Cotter et al.
(2019) employs non-zero sum formulation that uses the surrogates for update of the model parameters and use the original 0-1 loss for update of the Lagrangian variables.
However, they focus on using the non-zero sum game formulation for solving the Lagrangian function.
On the other hand, we only focus on a minimization w.r.t.
model parameters, since our ρ is corresponding to the Lagrangian variable.
Thus, based on our theoretical results that the minimization problem w.r.t.
model parameters can be formulated as another minimax problem based on the DRO framework, FairDRO solves such minimax problem by the non-zero sum formulation.
Furthermore, we argue that our surrogates would be more advantageous in terms of bounding the original objective function.
When implementing the algorithm of Cotter et al.
, they make surrogates of group fairness constraints by typically using an upper bounded function of 0loss such as the hinge loss or the cross entropy loss.
However, the modified fairness constraints term with such surrogates are not necessarily an upper bound of the original constraints because a group fairness constraint such as equalized odds is typically defined as the gap of averaged 0losses for measuring the parity over groups.
However, if we use the cross entropy loss as a surrogate in equation (6), we can get the convex upper bound of (7) whenever q is a maximizer of the inner maximization problem, and all entries of q are positive values.
Thus, we expect that our FairDRO can provide a better solution for the minimization step of the model parameters, which is supported by the comparison of FairDRO and the method of Cotter et al.
from our experimental results.
Furthermore, we expect that our FairDRO can be used as a better plug-in method for updates of model parameters in the algorithm of Cotter et al., which will be an interesting future work.
C M ORE IMPLEMENTATION DETAILS
We used PyTorch (Paszke et al., 2019); Experiments are performed on a server with AMD Ryzen
Threadripper PRO 3975WX CPUs and NVIDIA RTX A5000 GPUs.
All models are evaluated on separate test sets, and all experiments are repeated with 4 different random seeds.
All the reported results are the averaged results.
C. O PTIMIZATION
For tabular and vision datasets, we train all models with the AdamW optimizer (Loshchilov & Hutter, 2019) for 70 epochs.
We set the mini-batch size and the weight decay as 128 and 0.001, respectively.
The initial learning rate is set as 0.001 and decayed by cosine annealing technique (Loshchilov &
Hutter, 2017).
For the language dataset, we fine-tune pre-trained BERT with the AdamW optimizer for 3 epochs.
We set the mini-batch size and the weight decay as 24 and 0.001, respectively.
The initial learning rate is set as 0.00002 and adjusted with a learning rate schedule using a warm-up phase followed by a linear decay.
All results are reported for the model at the last epoch.
[Table]
C. I MPLEMENTATION DETAILS OF IN - PROCESSING BASELINES
Here, we describe the implementation details of each in-processing baseline used for the experiments.
The original LBC (Jiang & Nachum, 2020) requires multiple full-training iterations by alternatively re-weighting each group based on the given fairness metric and re-training the full dataset.
Since this optimization procedure needs a very high computation budget, we modify full-training iterations toepochs and 14 iterations for vision and language datasets.
FairBatch (Roh et al., 2020) and RW (Kamiran & Calders, 2012) are implemented in the way they were originally proposed.
Cov (Zafar et al., 2017b) utilized a fairness constraint based on Covariance between the group label and the signed distance of the feature vectors from the decision boundary of a classifier.
Although they used the Disciplined convex-concave program (DCCP) (Shen et al., 2016) solver for the constrained problem, we apply our optimization procedure ( i.e. , AdamW) by setting the covariance-based constraint as a regularization term.
MFD (Jung et al., 2021b) and FairHSIC (Quadrianto et al., 2019) use additional fairness-promoting regularization terms based on MMD and HSIC.
For FairHSIC, we only implement the second term of their decomposition loss ( i.e. , the HSIC loss between the feature representations and the group labels).
For implementing their regularization terms, we use the Gaussian RBF kernel of which the variance parameter is set as the mean of squared distance between all data points, following Jung et al.
(2021a).
Baharlouei et al.
(2020) attempted to remove the non-linear dependency between the model prediction and the sensitive attribute by using R ´ enyi correlation as a regularization term.
The resulting training objective is a mini-max optimization problem where the maximization is for computing the R´enyi correlation, and the minimization is for learning a model.
When the group label is binary, we use modified Algorithm 2 in Baharlouei et al.
(2020) where the minimization step ( i.e. , line 4 in Algorithm 2) is processed in a mini-batch manner.
Although Algorithm 1 is designed for the non-binary group label, applying the algorithm to our experiments with the non-binary group label is not straightforward because the second term in the RHS of equation 6 is a biased estimator so the mini-batch optimization can be significantly inaccurate.
C. H YPERPARAMETERS FOR MAIN RESULTS
We perform the grid search on the hyperparameter candidates for every method.
The full hyperparameter search space is illustrated in Tab.
C.1.
C. M ODEL SELECTION RULE
The fairness-accuracy trade-off may exist in many real-world applications (Dutta et al., 2020); better accuracy leads to worse fairness, and vice versa.
For example, Fig. 1 shows the trade-off for the
[Table] [Table]
[Table]
Adult and COMPAS datasets.
Therefore, the model selection criterion is critical for fair evaluation.
To that end, we explore varying hyperparameters (HPs) of each method for controlling the strength between accuracy and fairness.
We then select the best model showing the best fairness criterion ∆ DCA while achieving at least 95% of the accuracy of the vanilla-trained model.
With this selection rule, we report the best performance of each model throughout in Appendix D.
The HPs for each method were extensively searched, but fairly in terms of computation budgets.
The search range of each hyperparameter is listsed in Tab.
C.1.
D A DDITIONAL RESULTS D. R ESULT TABLES
Following our model selection rule, described in Appendix C.4, we report the best performances of each result in Tab.
D.1 and Tab. D.2.
The number in the parentheses corresponds to the standard deviation.
From Tab.
D.1 and Tab.
D.2, we observe FairDRO achieve the best ∆ DCA with moderate variances.
For example, in Tab.
D.1, while the best baselines achieve ∆ DCA of 4.86 and 6.03 on Adult and COMPAS, respectively, our FairDRO achieves 1.99 and 5.20 for the same datasets, respectively.
We also emphasize that although the accuracy of FairDRO is slightly worse than the one of RW and Scratch on COMPAS, ours show significantly better DCA (5.20) compared to RW (13.45) and
Scratch (15.10).
In Tab.
D.2, we observe the consistent results with Tab.
D.1.
We note that we marked results of COV and FairHSIC as N/A because COV is designed only for binary classification and FairHSIC fails to converge due to the gradient exploding.
We again observe that FairDRO achieves the best level of the fairness metric (DCA) compared to other methods.
We also provide difference of Equalized
[Table] [Table]
[Figure]
(DEO) on UTKFace, denoted by ∆ DEO , which is defined as follows:
[Equation]
Note that ∆ DCA and ∆ DEO are the same for binary labels.
Although LBC, FairBatch, and FairHSIC were devised for achieving EO, we again show that FairDRO outperforms baselines on UTKFace with respect to DEO.
D. C ELEB A RESULTS
To show consistent improvements, we conducted an additional experiment on the CelebA dataset, which includes about 200K celebrities’ face images with 40 annotated binary attributes.
Among the attributes, we select “blond hair” as the class label and “gender” as the group label, which is the set of attributes widely used in many previous works Sagawa et al.
Jung et al.
(2021a)
; Chuang & Mroueh (2021); Liu et al.
Tab.
D.3 and Fig.
D.1 show a similar result trend to one of UTKFace in Sec.
FairDRO again outperforms the baselines in terms of DCA and achieves a competitive trade-off with MFD.
We also observe re-weighting based baselines show worse DCA than FairDRO, meaning that FairDRO finds the better weights over groups for lower DCA.
D. R ESULTS OF EMPLOYING ORIGINAL OBJECTIVE FUNCTIONS
For fair comparison with FairDRO, we used the balanced ERM loss for the training of each baseline as mentioned in Sec. 5.
Since some of each method takes into account the standard ERM loss in their original version, we also include the results of employing the original objective functions.
Tab. D. and Fig.
D.2 show the best performances and the fairness-accuracy trade-offs on Adult and COMPAS.
The results on UTKFace and CivilComments are shown in Tab.
D.5 and Fig.
D.3.
In addition, the corresponding results on CelebA are shown in Tab.
D.6 and Fig.
D.4, respectively.
[Table] [Table]
[Table]
D. A BLATION STUDY FOR THE CHOICE OF UNCERTAINTY SET
We continue form Sec. 5.4 and provide further study of FairDRO.
We analyze the effect of each constraint employed by FairDRO.
Tab.
D.8 shows the overall accuracy, ∆ DCA and the worst accuracy over groups for each ablation case.
“Classwise”, “ χ 2 -divergence” and “Quasi-prob” ((4)) indicate each component added when defining the uncertainty set of Group DRO ( i.e. , simplex over groups) as described in Sec.
For a fair comparison, we apply the same optimization scheme (described in
Sec. 4.4) for all DRO variants.
As the supremum in 3 is attained at the vertex of the simplex ∆ |Y|×|A| , Group DRO minimizes the worst-case loss over the ( y, a ) pair, thereby it can prevent the lowest accuracy across the group and class from becoming too low.
Thus, Group DRO substantially improves and achieves the best worst-case accuracy over groups compared to ERM ( i.e. , 1st −→ 2nd row).
However, it shows the worst DCA among the DRO variants, confirming the mismatch between the criterion Group DRO uses and the group fairness metric.
We observe that changing the simplex uncertainty set of Group
DRO to the proposed χ 2 -divergence and quasi-prob uncertainty sets (4) ( i.e. , 2nd −→ 3rd row) has an impact on the group fairness mostly.
In addition, reducing the size of uncertainty sets via classwise treatment ( i.e. , 3rd −→ 6th row) further improves the group fairness.
Finally, including negative values in the uncertainty set also helps to improve the group fairness ( i.e. , 5th −→ 6th row) on most datasets, by more severely penalizing the major groups with high accuracy with negative weights.
[Figure] [Figure]
CelebA
[Figure]
[Table]
[Figure]
[Table]
Scratch 78.
RW (Kamiran & Calders, 2012) 77.
LBC (Jiang & Nachum, 2020) 80.
FairBatch (Roh et al., 2020) 79.
FairHSIC (Quadrianto et al., 2019) 86.
MFD (Jung et al., 2021b) 90.
FairDRO 89.
[Figure]
D. V ISUALIZATION OF q
We continue from Sec.
5.4 and provide visualization results including the weights ( q y ), training accuracy, and test accuracy for y = 0 .
In Fig.
D.5 (a), Scratch shows that the significant accuracy gap between two groups is kept during all training time.
On the other hand, we observe again in Fig.
D.
(b) that q 0 initially fluctuates and eventually converges by our optimization procedure, and as a result, the large discrepancy of training accuracy is reduced at the end of training.
Finally, we show from
Fig.
D.5 (c) that the test accuracy gap of FairDRO is than that of Scratch, i.e. , FairDRO achieves lower DCA.
D. A BLATION STUDY OF SMOOTHED IBR
We experimentally observe the effectiveness of smoothed IBR updates introduced in Sec.
4.4 on
FairDRO.
Although the IBR procedure is known to converge (Jin et al., 2020; Lin et al., 2020), in our experiment, we observe that q values oscillate when IBR without smoothing was utilized in FairDRO.
Fig.
D.6 visualizes q of FairDRO without smoothed IBR.
The figure shows the large oscillation of FairDRO without smoothing compared to FairDRO.
Tab.
D.9 is the result of FairDRO without smoothed IBR in the same setting as Tab.
D.1.
For the reported result, the same model selection criterion introduced in Appendix C.4 is applied.
Tab.
D.9 shows that the use of smoothed learning rate schedule for FairDRO achieves improvements in terms of DCA.
As we observe that the oscillation leads to performance degrades in terms of DCA, we identify the effectiveness of smoothed IBR on
FairDRO.
[Table] [Table]
D. I SSUE OF STANDARD ACCURACY
In the experiments, we reported the balanced accuracy for measuring the performance of the models instead of the standard accuracy.
While the balanced accuracy offers several advantages in measuring performance on imbalanced datasets, but also has drawbacks.
In general, it may underestimate the performance of certain groups with relatively large numbers.
As in most cases, the groups with large numbers often become majority groups.
The balanced accuracy metric may not capture accuracy reduction in majority group.
In real-world applications, there is also concern about a decrease in majority-group accuracy.
However, we argue that the vanilla standard accuracy would be problematic when the dataset is severely class-imbalanced.
For example, Tab.
D.10 shows the number of samples in each dataset with respect to the class and group labels.
For the case of Adult dataset, 75.2% of the samples are labeled as y = 0 , hence, a naive predictor that always predict ˆ Y = 0 will achieve the standard accuracy of 75.2% and DCA = 0 , which are clearly overestimating the (accuracy, fairness) performance of the classifier.
Therefore, to avoid such issue, we use the group-class balanced accuracy, rather than the standard accuracy.
[Figure] [Figure]
Adult
[Table]
Scratch
[Table]
[Table] [Table]
[Figure]
o
[Figure]
[Table] [Table]
FairDRO 79.
[Table] [Table]
Adult
COMPAS
CelebA a =a =a =a =a =a =y =
[Table] [Table]