Dr-Fairness: Dynamic Data Ratio Adjustment for Fair Training on Real and Generated Data
Authors: Yuji Roh
Weili Nie
De-An Huang
Steven Euijong Whang
Arash Vahdat
Anima Anandkumar
Venue: Accepted by TMLR ()

Abstract: Fair visual recognition has become critical for preventing demographic disparity. A major cause of model unfairness is the imbalanced representation of different groups in training data. Recently, several works aim to alleviate this issue using generated data. However, these approaches often use generated data to obtain similar amounts of data across groups, which is not optimal for achieving high fairness due to different learning difficulties and generated data qualities across groups. To address this issue, we propose a novel adaptive sampling approach that leverages both real and generated data for fairness. We design a bilevel optimization that finds the optimal data sampling ratios among groups and between real and generated data while training a model. The ratios are dynamically adjusted considering both the model's accuracy as well as its fairness. To efficiently solve our non-convex bilevel optimization, we propose a simple approximation to the solution given by the implicit function theorem. Extensive experiments show that our framework achieves state-of-the-art fairness and accuracy on the CelebA and ImageNet People Subtree datasets. We also observe that our method adaptively relies less on the generated data when it has poor quality. Our work shows the importance of using generated data together with real data for improving model fairness.

Published in Transactions on Machine Learning Research (04/2023)
Dr- Dr-Fairness: Dynamic Data Ratio Adjustment for Fair Training on Real and Generated Data
Yuji Roh* KAIST
yuji.roh@kaist.occ
Weili Nic
unie
NVIDIA
De-An Huang
Geahuang@vvidia..o com
NVIDIA
Steven Euijong Whang
suhang oc.kr
KAIST
Arash Vahdat
avahdat@nvidia.com com
NVIDIA
Anima Anandkumar
Sanandkuuar@nvvidion
NVIDIA
Caltech
Reviewed on OpenReview:
https: // openreview. net/ forum? id= TyBd56VK72
Abstract
Fair visual recognition has become critical for preventing demographic disparity. A major cause of model unfairness is the imbalanced representation of different groups in training data. Recently, several works aim to alleviate this issue using generated data. However, these approaches often use generated data to obtain similar amounts of data across groups, which not optimal for achieving high fairness due to difforent learning difficulties and generated data qualities across groups. address this issue, we propose a novel adaptive sampling approach that leverages both real and generated data for fairness. We design a bilevel optimization that finds the optimal data sampling ratios among groups and between real and generated data while training model. The ratios dynamically adjusted considering both the model's accuracy as well as its fairness. To efficiently solve our non-convex bilevel optimization, we propose simple approximation to the solution given by the implicit function thcorem. Extensive experiments show that our framework achieves statc-of-the-ar fairness and accuracy on the CelebA and ImageNet People Subtree datasets. We also observe that our method adaptively relies less on the generated data when has poor quality. Our work shows the importance of using generated data together with real data for improving model fairness.
1 Introduction
Model fairness in visual recognition is becoming essential to prevent discriminatory predictions over demo- graphics. Recently, numerous unfairness issues have been reported [Wang et al. 2020 Najibi 2020), and several fair image classification approaches have been proposed that do not discriminate against specific groups such as gender, age, or skin color (Ramaswamy et al. 2021 Roh al. 2021
Work done during internship NVIDIA.

---

Published in Transactions on Machine Learning Research (04/2023)
Real Gen. data data
Group
Update the data ratios
Accuracy
88.0
Fairness metric
85.0
Classification loss
EO Disp
Training data
Dr-Fairness
Model
0.10
Adaptive sampling
0.00
in each iteration
ratio FairBatch Ours (b) Accuracy and fairness performances.
Original
(a) Framework overview.
Figure 2: (a) Our framework iteratively updates the data ratios among groups and between real and generated data based the fairness and accuracy of the intermediate model. (b) Performances on CelebA, using gender the group attribute and age as the label attribute. Compared to the original model, the 1:1 ratio baseline (Ramaswamy et al. 2021 does not significantly improve group fairness, measured through equalized odds (EO) disparity. FairBatch (Roh et al. 2021) shows high fairness by adaptively selecting real data only, but loses accuracy. In comparison, Dr-Fairness (ours) achieves high fairness, while not sacrificing accuracy.
With the rapid progress deep generative learning (Karras et al. 2020 Dhariwal Nichol 2021 there a new rescarch direction to improve fairness by augmenting training data with generated data. Recent breakthroughs in generative learning make generated data practical enough to use in real-world applications (OpenAl 2022). and many high-quality pre-trained generative models are now open to the public (Rombach et al., 2022), which obviates the need to retrain such models from scratch for new use cases. Thus, generated data is increasingly used to improve model performances, including fairness. From a fairness perspective, gen- erated data complements real data by making it more diverse. For example, if a specific group's data is col- lected from limited data source that does not have the full data distribution, that group may be discrim- inated in model training due to the bias (Mehrabi et al. 2021). In this case, generated data can be used to supplement that underrepresented group- see an example in Figure !
Amount
Real data
Gen data
Light
Dark
Figure 1: Many real-world image datasets have biased rep- resentation of protected groups (Wang al. 2022). For example, ImageNet People Subtree (Yang et al. 2020) con- tains mostly lighter skin color images for anchors. Here, the generated data can compensate for the underrepresented groups, i.e., the darker skin color distributions for anchors.
However, most fair training approaches that use generated data simply generate similar amounts of data across groups (Ramaswamy et al.] 2021 Choi et al.] 2020), which may not be optimal to improve group fairness such as equalized odds (Hardt et al. 2016) and demographic parity (Feldman et al. 2015).. Such suboptimality could originate from 1) the learning difficulty differences across groups and 2) the potential bias (i.e., typically the form of missing modes) issues in the generated data that can hurt the fairness the model under training. Moreover, as the quality of the generated data can differ among different group groups, simply using a similar amount of generated data may result in suboptimal model accuracy and fairness. Therefore, it is essential to find the right mix of generated and real data for the best accuracy and fairness.
In this paper, we harness the potential of both real and generated data via adaptáve sampling to improve group fairness while minimizing accuracy degradation. To this end, we design a new sampling approach called Dr-Fairness (Dynamic Data Ratio Adjustment for Fairness) that adaptively adjusts data ratios among groups and between real and generated data over iterations, as in Figure 2a
In Table I compare the unique properties of Dr-Fairness against two representative methods: 1) an equal ratio base- line (1:1 ratio) (Ramaswamy et al. 2021 that uses gener- ated data and 2) fairness-aware adaptive sampling bascline (FairBatch) (Roh et al.] 2021) that finds the optimal group ratio for fairness only using real data. We can see that Dr-Fairness subsumes the two basclines and improves them by also optimizing the ratio between real and generated data and utilizing accuracy for ratio updates.
Table 1: Functionality comparison of algorithms.
Finds Utilizes
Uses generated the optinal
Finds
optimal accuracy
Method
real & gen. for ratio data matoo updates
data
group ratio
1:1 ratio
X
X
FairBatch
x
X
Dr- Pairnessss
/
2

---

Published in Transactions on Machine Learning Research (04/2023)
To perform adaptive sampling systematically, we design novel bilevel optimization problem along with an efficient algorithm for solving it. Our bilevel optimization consists of 1) an outer optimization that adjusts data sampling ratios considering both fairness and accuracy and 2) an inner optimization that minimizes the standard empirical risk on both real and generated data, given the current sampling ratios. Although various exact algorithus have been proposed to solve bilevel optimizations (Maclaurin el al., 2015), they often scale poorly our scenario with large models and data. We thus propose an approximate algorithm that uses the implicit function theorem (Krantz & Parks, 2002) and identity-matrix approximation (Luketina et al.) 2016) to efficiently compute the gradient of our bilevel optimization. Specifically, instead of computing the expensive inverse Hessian matrix, we approximate it with simple diagonal identity matrix.
Experiments on CelebA (Liu et al. 2015) and ImageNet People Subtree (Yang et al. 2020) show that our approach achieves the thate-of-th-aar fairness and accuracy performances. For instance, Figure 2b highlights our results on CelebA, where our framework largely outperforms FairBatch, which only uses real data and the 1:1 ratio bascline- see Sec. for comparisons using more basclincs and other fairness metrics, which show consistent results. On the ImageNet. People Subtree classification problem, which represents a large-scale real-world scenario, we achieve better accuracies than the best baseline, with an absolute improvement of 5-9%, while obtaining similar fairness scores. We also observe that our framework adaptively relics less on the generated data when it has poor quality.
Summary of Contributions: (1) We propose Dr-Fairness, a novel adaptive sampling framework for fair training that enjoys the potential of both real and generated data. (2) To perform adaptive sampling systematically, we formulate a bilevel optimization to train fair and accurate models on real and generated data. (3) We also design an approximate algorithm based on the implicit function theorem and identity-matrix approximation to efficiently solve our non-convex optimization. (4) We perform extensive experiments on CelebA and ImageNet People Subtree to show that Dr-Fairness achieves the state-of- accuracy and fairness. (5) Finally, our work reveals the importance of using generated data together with real data to improve model fairness.
2 Related Work
Traditional Model Fairness As model fairness becomes indispensable for Trustworthy AI, numerous
works have been recently proposed to better measure fairness and design fairness- aware algorithms (Narayanan) focus on group fairness measures (Hardt et al., 2016 Feldman ct al. 2015), which are widely studied in the fairness literature. The main approaches for satisfying group
2018). Among various fairness definitions,
fairness are: 1) fix the training data to mitigate bias (Kamiran & Calders 2011 Zemel et al.) 2013). 2)
modify the training process to prevent the model from learning bias (Zafar et al. 2017a b) Zhang et al. [0188a] Agarwal et al. 2018 Roh ct al.) 2020 2021), or 3) alter the outputs of the trained model to achieve fairness metrics (Hardt et al. 2016). Unlortunately most of algorithms are not designed to handle large number of groups or labels, and our contribution is to support such large-scale scenarios for real-world applications. There are several recent works that aim to improve fairness for multiple numbers of groups (Shui et al. 2022b) Zhao et al. ) but they only focus on specific fairness definitions (Shui et al. 2022b), or the performance improvement is limited (Zhao et al. 2021 compared to our work.
Among the previous techniques, FairBatch (Roh et al. optimal group ratio for fairness on real data and shows the state-of-the-art fairness performances on various tabular datasets, including COMPAS (Angwin et al. 2016) and AdultIncome (Kohavi 1996). However, FairBatch may suffer from accuracy degradation due to oversampling on very small-size groups, ospecially in vision datasets. In particular, FairBatch aims optimize only the fairness criterion in the absence of any generated data, and it cannot be easily extended to optimize both accuracy and fairness objectives together, nor can it utilize generated data. Also, the theoretical guarantees of FairBatch do not apply in our non-binary setting because the objectives of our optimization problem become non-convex details on the optimization are in Sec. 3 In contrast, Dr-Fairness can minimize the accuracy degradation of fair training by optimally utilizing both real and generated data based on the fairness and accuracy objectives.
2021) is the most relevant to our work as it finds the
In addition, there are other related studies on fair data reweighing Li & Liu 2022 Jiang & Nachum 2020 Krasanakis et al.) 2018), fair augmentation (Chuang & Mrouch 2021 and fair representations (Shui et. al.
3

---

Published in Transactions on Machine Learning Research (04/2023)
2022a]. Compared to our work, these studies only use real data or do not scale to large datasets. For example, applying the existing fair reweighing techniques on large-scale data may lead to significant training times due to multiple re-trainings (Jiang & Nachum 2020; Krasanakis et al. 2018) or performance degradation due to violations of the underlying assumptions Li & Liu 2022). We leave a detailed discussion in Sec. C
Fairness in Visual Recognition There is an emerging line of research for fairness in visual recogni- tion (Najibi 2020; Wang et al., 2020) where using generated data is critical. Many visual recognition tasks involve multiple classes of varying sizes, and only using real data is often insufficient to improve fairness. In response, several works have proposed new algorithms to create balanced datasct by augmenting the biasod real dataset with well-controlled generated data (Sattigeri et al.) 2019 Choi et al. 2020 Ramaswamy el al, 2021). However, simply balancing the data sizes is not enough to achieve high-enough group fairness, as the learning difficulty and generated data quality can differ across groups. Although recent work (Zietlow et al, 2022) suggests an adaptive data augmentation that generating more data for woose-performing groups, it uses heuristies to adjust data ratios without proper optimization and thus has limited fairness performance. In comparison, Dr-Fairness solves a novel optimization problem to find optimal data ratios and thus obtains both high fairness and accuracy.
Other Related Work Although not our immediate focus, there are other important research lines for fairness: 1) fulfilingg other fairness definitions (e-g., individual fairness (Dwork el. al., 2012) and causal
fairness (Kusner et al. (0017). ) 2) handling noisy or missing group labels (Hashimoto et al. 2018 Celis et al. ), and 3) improving fairness in special classification scenarios (c.g., selective classification (Lee et al. We believe that supporting these aspects can be promising future directions.
2021
3
Framework
In this section, we first formulate bilevel optimization problem for optimizing sampling ratios for real and generated data. We then design a new algorithm that efficiently solves the optimization problem. Throughout this paper, we use the following notations and fairness definitions.
Notations Let x E X be the input feature, and let y E Y and ÿ E Y be the true label and the predicted label, respectively. Let z E Z be a sensitive group attribute, c.g., gender, age, or skin color. Let m be the total number of training samples, and My. be the number of samples in the set {i: y; y, y =} with label y and group label z. Similarly, My,a == Ki y; yHl and m.,, == Ki : 2, = 2H|. Let be the model weights, and the overall empirical risk is given by L(w)= = Th Σ, fly,, 8t), where E(-) represents the loss function. Let Ly,
be the empirical risk over samples in the set {i : y. = y, = 2), i.e., Ly. := my.. Eiy. 3.). Finally, let L (-) and Loen(.) be the empirical risks on real data and generated data, respectively.
Fairness Definitions For the method design, we focus on two prominent group fairness definitions: equalized odds (EO) Hardt et al. 2016) and demographic parity (DP) (Feldman et al. 2015 EO is satisfied when the accuracies conditioned on the true label are the same for different groups (i.e. Pr( = yly y, =1) = Pr($- yly = y, z = 22), Vy E =1 E Z). DP is satisfied when the positive prediction rates are the same 21,
for the groups (i.e., Pr($ = 1|z =1) = Pr(f = 1|z 22), V21, 22 E 2), where DP is designed for binary classifications (i.e., y E {0, 1}) with a favorable label class (e.g., "approval" in loan decision). We show how to mcasure the disparities (i.e., unfairness) among groups for each fairness definition in Sec.
Generated Data In general, any synthetic data, including data from deep generative models, can be considered generated data for fair training. Here, the key role of the generated data in algorithmic fairness is supporting the limited subset of the real data. Also, we implicitly assume that the domains of the generated and real data are the same, but the distributions of the two data can be different. For example, if the real data represents human faces, then we assume the generated data also contains human faces. However, generated data may have fairer distribution than real data. In this paper, we assume we can get group-specific generated data by using conditional image generation techniques (Nie et al. 2021 Dhariwal & Nichol 2021 see details in Secs. and B3
4

---

Published in Transactions on Machine Learning Research (04/2023)
3.1 Bilevel Optimization for Fairness with Real and Generated Data
To design adaplive sampling strategy real and generated data, we first formulate a bilevel optimization for training fair and accurate models. The bilevel optimization consists of inner and outer objectives: 1) we maintain the standard empirical risk minimization (ERM) in the inner problem, and 2) we capture the desired fairness properties in the outer problem. The bilevel formulation allows us to support prominent group fairness metrics and utilize generated data for fairness.
We now explain how our optimization improves group fairness and accuracy together by using both real and generated data. The outer objective aims to find the optimal data ratios among sensitive groups and between real and generated data to minimize the fairness and accuracy losses on the real data distribution. Given the current data ratios, the inner objective runs a weighted ERM with both real and generated data. We can support various prominent group fairness metrics by modifying the outer objective and the constraints. As an illustration, we state our bilevel optimization w.r.t, EO as follows (see the DP version in Sec. A.1
max 1 (w(A, wJ) Lyyes ( 1J]l3 + K. Eyer, My.s real (w (A, H)),
min A, yEt 2
m
arg min Eyen My.x Xy,= Muy, 1.g12 (wr) + (1 (y gg (.))): w(A w(),
w
m
s.t. Ae 10 1], Exerdg 1, 1y E Y.
where Ay is the ratio for group in class y, My.z is the ratio for real data in the (y. 2)-class, and À and μ are the sets of all Xy,z and My,s, respectively. In the outer objective, the first term indicates the fairness loss, and second term indicates accuracy loss. The hyperparamcter k tuncs the importance of the two losses. We note that the Xy,z and My, values are the data ratios within one mini-batch. Thus, among all samples in the real and generated data, our framework serves mini- batches according to Λy,s and My.s. Here, we can capture the EO disparity as the maximum of the loss differences in different groups within the same label (i.c., max y (w) - Ly (w)l).
Remark 1. We explain how this loss-based constraint can capture equalized odds. When the loss func- lion 8,) is 1/0-loss (Le., 84) = 1(y, + 9,), where 1(.) is an indicalor funclion), the loss- based constraint equivalent to the probability of the correct predictions in each (y. 2)-class (i.e., Pr(O = vly = y, = 2)). becomes
perfectly express the equalized odds disparity. Specifically, (w) with 1 1/0-loss is
Therefore, our fairness loss constraint (i.e., maxyeZ.a.. (gge (w(A, ()) - -Ly)) the equalized odds metric, which describes the class-conditioned accuracy disparity omong groups (i.e., max 2z Pr(s = gly = y, =2 =1) - = yly = y, z = =2)])) In practice, we can also use other loss functions like cross-entropy loss instead of the 1/0-loss, as other loss functions have been empirically verified as reasonable prozies for capluring group fairness metrics (Roh el al. 2021 Shen el al. 2022).
Through the above formulation, the amount of generated data is automatically adjusted to augment the real data (e.g,, enhancing minority groups in the real data). In particular, proper usage of generated data can reduce the accuracy degradation caused by over-sampling minority groups from real data.
Advantages of Using Bilevel Formulation The bilevel formulation has various advantages solving our problem. First, we can achieve the desired fairness properties while keeping the standard model training process without re-configuring the model architecture or loss functions. Moreover, our bilevel problem can be solved via an efficient algorithm that we propose in the following section, which is suitable to support large number of groups and label classes. We note that when the numbers of groups and label classes increase (i.c., the numbers of Ay.z and Py, increase), naive formulations like grid search using the validation set may fail to find reasonable solutions within a practical time. In Sec. A.5 we discuss more advantages of using bilevel optimization compared to other problem formulation mothods like distributionally robust optimization (Sinha
et al. 2017
3.2 Algorithm
We now design our algorithm to solve the above bilevel optimization. In this section, we first describe how efficiently approximate our optimization by utilizing the implicit function theorem Krantz & Parks 2002
5

---

Published in Transactions on Machine Learning Research (04/2023)
and adapting identity-matrix approximation (Luketina et al 2016) in fairness setting. We then introduce the overall training procedure, and show the validity of our approximate algorithm on synthetic data.
Algorithm Design Solving bilevel optimization is known to be challenging (Liu et al. 2021). especially when the objectives are non-couvex as in our problem. Thus, we resort to stochastic gradient descent to find the optimal parameters of the bilevel optimization gradually. To obtain the gradients, we first convert our optimization into the unconstrained version:
min max A 22
y (w(), (.) Lyge w (A, p1)1 + lo Σyen, m
my.s L (w( (2, 11);
w A 1b) arg min EyeY, Myit (),, ( (w) + (1- (( Lyg (w)),
w
where (y((y,z)= = =xxp,)))==p(>):)))))) Ex, exp()).=:) (Le., the softmax function), and S(yy,)) := 1/(11 + exp(-Mppp (i.e, the sigmoid function). Denoting the outer and inner objectives as fouter (A, n4, w(,, n)) and finner (X, ue w) respectively, the inner optimization w(d, μ)) = arg min funner (A, fe, w) can be solved efficiently using SGD- like algorithms. The main question is how to solve the outer optimization. We can state the gradient of fouter w.r.t. À as follows:
ooouter Ofonter dA
Ofouter 3r((..))
Sw (wr(A, 1L) O%
(1)
0x
Term A
Term
Term C
where Term A and Term B are the direct gradients w.r.t. À and w(x, jt), respectively, and Term C is the best-response Jacobian. Note that w(A, 11) is the best-response of model weights.
In Eq. [ the best-response Jacobian is hard to directly compute. Although various algorithms have been proposed to explicilly find the best-response Jacobian, most of them require propagating the entire history of the gradients (Maclaurin et al., 2015), which is very time-consuming. Instead, we implicitly measure the best-response Jacobian using the implicit function theorem (Krantz & Parks 2002). This approach does not need to investigate the entire gradient history (Rajeswaran et al.) 2019 Lorraine et al 2020) and builds on the assumption that the inner optimization has converged to a local minimum, i.c., Jimeer = 0. We note that Dw
among various methods of solving bilevel optimization problems, the implicit function theorem significantly improves the algorithm efficiency with theoretical evidence.
We now describe the details on how we convert the best-response Jacobian in Eq. 1 Jusin the implicit function theorem. Here, we first state the original implicit function theorem:
Thcorem 2. (Implicit Function Theorem, stated in Krantz & Parks 2002); de Oliveira (004)) Let F R" X R' + R be a continuously differentiable function, where the input of F is (x, 4) R R Assume there is an input point (a, b) that satisfies F(a, b) = O, and aF(a,b) By (i.e., the Jacobian matria) invertible. Then, there exist open sels U c R and V C Rm that contain a and b, respectively, and satisfy the following:
There is a unique continuuusly differentiable function G, where G(a) = b and F(x, G(x)) 0 for all x 2ED U.
We have the Jacobian matriz of partial derivatives of G in U as follows: dG(x) -1 OF(x G(z)) ( - - (F(x, G(z))
dx
dy
dx
We now apply the above theorem in our setting. To get the best-response Jacobian w.r.l. X, we consider (A.w)) as F(x, y) and w(A) as G(æ). Note that when accessing the gradient w.r.t. X, we can ignore μa without loss of generality, and vice versa. Then, we can rewrite Theorem 2 for our scenario as follows:
dw
IR x R. + R be a continuously differentiable function, where the input of Ofame is (A,w) E RD x R. Assume there is an input point (a, b) that satisfies a. O, and 8² (a.b))))) (i.e., the Jacobian matrix) is invertūble. Then, we have the 8ap
Corollary 3. (Implicit Function Theorem in our setting) Let 0f Owe
Jacobian matrix of partial derivatives of w() as follows:
Ow(() 0x
02 iinner (2, w) 1-1 82 finner (2, w)
-1
dwow
OweX
6

---

Published in Transactions on Machine Learning Research (04/2023)
Algorithm 1: Model Training with Dr-Fairness
Algorithm 2: Dr-Fairness
Input: real data (Treal: Freals Freat), generated data
Input: model parameters w, data dreal and dgen: group ratio A, real data ratio μ
(Tgen: Mgen; Freal)
dreat; dgnn & (Ztrain Yrrain; Freat); ; (Zigons Migen Freal)
<- initial model parameters
Calculate Souter and finner according to w, dreal: dyens X, and μ
X, 1 < initialize sampling ratio logits
Get current sampling ratios og S(H))
Get Nenter and Mester as in Eq. 3 du
for each ileralion do
dx
Update À by douter and μ by dx optimizers (e.g., Adam)
dfouter using
Sinibatch-DD-Faiiresss dreal. dgen: X. pp) Update w according to the minibatch (optionally with exponential moving average (EMA))
du
Draw a miníbatch w.r.t. oy(A), S(n)
Output: model parameters
Output: minibatch
Limee = 0, we Thus, with the assumption that the inner optimization has converged to a local minimum, ie., the into the following equation by replacing the best-response Jacobian to the multiplication of
can convert Eq. [
two matrices:
df dfouter
Souter
02 Siner 0% Oun(2. 14) Ounāw
Souter
8 finner
-1
(2)
dX
Owox
However, oblaining the inverse Hessian (i.e., 10p Simer// /owâro) )1) in Eq. 2 is also computationally expensive.
Thus, we consider the identity matrix approximation (Tuketina et al. 2016 Finn et al. 2017 Geng et al.,
2021 that replaces the inverse Hessian with the identity matrix. Despite its simplicity, such approximation
may be valid for neural networks with normalization layers that make the Hessian matrix diagonally dominant (e.g. BatchNorm), and in practice, it often performs on par with other approximation methods in various
applications (Raiko et al. 2012 Pedregosa 2016 Liu et al. 2018 Wilder et al.
2019 Fung et al. 2022
Given this, we can rewrite Eq. simply as:
dfouter
Souter
Sottr x
82 Sinner
dA
0x Ow 1z)
OwoA
where the second term on the right-hand side is efficiently computed via vector-Jacobian product Paaskke et al. 2017). Similarly, we can also approximate the gradient of fouter w.r.t. μ.
Remark 4. In Eq. 3 the second term dominates the computation cost as it performs both the Jacobion computation and vector-Jacobian product. In comparison, the computation cost of the first term is negligible. Here the computational cost increases when the model parameter size (i.e., dimension of w) and the rammer of groups and label classes (i.e., dimension of X and μ1) increase. In our experiments, we update X and μ at every ileration of the training, but we can further reduce the computational cost by updating X and μ less frequently see an empirical analysis for computational overhead in Sec. A.g
Overall Training Process We now describe the overall training process in Algo. 1 We first initialize the model parameters and the data ratios, and for each iteration, then get minibatch from Dr-Fairnest (Algo. 2 Here, we note that Eq. 3 computes the gradients of ratios (i.e., X and μ.), rather than obtaining an analytical solution for them, thus we need to specify initial ratios in the training. In Algo. 2 we first update the data ratios among groups (A) and between real and gencrated data (ą1) by calculating Mouter and douta
du as in Eq. 3. We then draw a minibatch according to (A) and S(z). Note that the batch sampling with 0y (A) and S(u) provides an unbiased estimator of the weighted ERM in our inner optimization (Roh et al.) 2021). Finally, we update the model parameters w with the given minibatch. Here we can optionally use an exponential moving average (EMA) that averages the model parameters w for improving training stability.
d'
Validity of Our Algorithm We empirically verify how close the solutions from our approximation strategy are to the optimal ones. To this end, we follow the synthetic binary setting in Roh et al. (2021). where FairBatch has a theoretical guarantee to find the optimal group ratios, and compare the optimized group ratios of Dr-Fairness and FairBatch soe details on the sctup in Soc. A.3 Note that in this synthetic sctting, we set the fairness metric to equal opportunity (i.e., a relaxed version of EO that focuses on the positive label) and only the real data to optimize the group ratios X, as FairBatch cannot handle μ for the generated
7

---

Published in Transactions on Machine Learning Research (04/2023)
data. Ideally, if the approximation error in our method is small, Dr-Fairness should obtain the same group ratios and performance as FairBatch.
Figure B shows that our algorithm converges to similar group ratios to those in FairBatch, although the key ideas of the two algorithms on updating the group ratio are very different. Also, the two algorithms have the similar fairness scores (0.012 equal opportunity disparity for both). These results imply that our approximations are good enough to find reasonable solutions, which is consistent with the observations in other applications (Lorraine et al., 2020: Luketina et al., 2016). We note that although FairBatch is known to have theoretical guarantees, they only apply to limited settings (e.g., binary groups and labels), so there is room to improve fairness in other settings. In the next section, we will show that Dr-Fairness achieves much higher accuracy with similar or better fairness than FairBatch on real-world datasets, as our algorithm scales to multiple groups and labels and capable of harnessing the potential of both real and generated data.
We also verify that, the above setting, the identity matrix approximation indeed gets almost the same group ratios with the exact inverse Hessian computation. See more details in Sec. A.4
Group Ratio
1.0
Group Ratio
1.0
0.761
0.773
0.5
1.
A,0
00.5
1
1
0.239
0.227
0.0
0.0
0
100
200
300
400
0
100
200
300
Iterations
Iterations
(a) Group ratios Dr- Da-iairssss.
(b) Group ratios FairBatch.
Figure 3: Comparison of group ratios from Dr-Fairness and FairBatch. Both converge to similar ratios.
4 Experiments
We perform various experiments to evaluate our algorithm. We repeal all experiments with three random seeds and measure all performances on a separate test set see more information in Sec. B.1 We use the Adam optimizer (Kingma & Ba 2015).
Datascts We utilize two real-world datasets: 1) CelebA Liu et al. 2015) to compare our algorithms with baselines and perform various analyses, and 2) ImageNNe People Subtree (Yang al. 2020 to further observe the algorithm performances on a large-scale real-world scenario. Here, CelebA contains facial images, and ImageNet People Subtree provides more general form images related to people. For example, the label classes of the ImageNet People Subtree dataset include various occupations (e.g., anchor, astronaut, and tennis player), which requires the model to understand more complicated features (e.g., clothes and backgrounds) than face information to achieve high classification performance. Note that we are using large datasets instead of the traditional smaller tabular benchmarks for fairness because our goal is to make Dr-Fairness work in large-scale real-world applications.
[CelebA) Contains celebrity images, where each image has 40 attributes (e.g., gender, age, and smiling). We choose group and label attributes that are less subjective and traditionally considered for fairness. The group attributes are gender (male and female) and age (young and old). The label attributes are age, haircolor (black, blond, and others), and smiling (smiling and not-smiling). Note that age can be used as either the group or label attribute. The sizes of the training/validationn /awl/vaaaaaion/aaaaaaaaa test sets are 200k/20kk/000 respectively.
[ImageNet People Subtree] Contains 284 label classes and 3 group attributes: gender (male, female, and unsure), skin color (light, medium, and dark), and age (child, adult, middle, and retired). We first filter out classes that are vague, duplicates, or too small with few samples, which leaves us with 112 classes see details in Sec. B.2 These classes contain about 11lk samples, but only 10% of them have group attribute annotations. We split the group-labeled data into 40% 20% 40% for ceaeiiingvvliiationteeee iiliiattioo/tetingg, respectively.
Data generation We create the generated datasets using state-of-the-art generative models that condition- ally synthesize images for each (y. class. For CelebA, we StyleGAN-bbseed controllable generation
8

---

Published in Transactions on Machine Learning Research (04/2023)
method called LACE Nie et al. 2021 For ImageNet People Subtree, we fine-tune a diffusion model (Dhariwal & Nichol 2021 pre- trained ImageNet (Deng et al. 2009), and use classifier guidance (Song et al. 2020 Dhariwal & Nichol 2021) to sample images in each (y, z)-class. Note that the controllable generation for ImageNet People Subtree more challenging due to its large number of (y: 2 2)-clssses and labeling noises. Thus, the resulting generated data has lower quality than the generated data CelebA. More details on data generation are Soc. B.3
Baselines We compare our algorithm with three types of baselines: 1) vanilla (non-fair) baseline, 2) fair pre-processing baselines, and 3) fair in-processing baselines.
For fair pre-processing training, we consider three baselines: simple sampling, pair-augmenting (PairAug) Ra maswamy et al. 2021), ; and pair-augmenting with our generated data (PairAug") For simple sampling, we over- and under-sample the real data to ensure equal ratio among groups. PairAug is fair augmentation technique that uses the generation methods to synthesize balanced images for groups to reduce the correlation between the group and label attributes. For a fair comparison, we also implement an extension of PairAug (denoted by PairAug*); which uses the same balancing ratio in Ramaswamy et al. (2021), but uses our generated data. We nole that we choose PairAug as our main baseline for the fair data augmentation strategy, as it shows satte-oftthe-ar fairness performances among the augmentation techniques see Sec. B.4 for a detailed comparison.
For fair in-processing training, we consider three baselines: faörness constraind Zalar el al. 2017 7a b domain independence (Wang et al. 2020p, and FairBatch (Roh et al. 2021). Fairness constraint adds a fairness penalty term to the loss function to reduce the unfairness. Domain independence trains separate classifiers per each group to reduce the correlation between the group and label attributes. At inference, one can ensemble the outputs of the trained classifiers to get the final predictions. FairBatch adaptively adjusts batch ratios among groups to improve fairness only using real data.
Metrics We focus on two accuracy metrics and three fairness metrics. [Accuracy] We measure the standard accuracy over all samples and the balanced accuracy that averages y-class wise accuracies. [Fairness] We focus on equalized odds (EO) (Hardt et al., 2016), demographic parity (DP) (Feldman et al.) 2015), and bias ampli- fication (Zhao et al. 2017p. For EO and DP, we mcasure the disparitics (i.c., unfairness) among groups: EO disp. = max= Pr(s-y/e-z,y-D y-v) Pr(A-vly=v)), and DP disp. max |Pr(f=1k=z)- Pr(ý=1))- Together with either EO or DP, we measure bias amplification to see how much the data bias is amplified in the model: Bias amp. = maxzey Pr(z=z/8=-) - Pr(z=zly=y), where = arg m Pr(z=>'(ý=y). Here, a good performance is indicated by high accuracy values, low EO disp. and DP disp. values, and close-to or below zero bias amp. values.
Hyperparameters For Dr-Fairness, we choose from a candidate set {0.1, 1, 10, 20} to have the best fairness score while minimizing the accuracy degradation in the validation set. We set the learning rates for A and μ to 0.005. We initialize À the original (y, r)-ratios in the real data. We initialize μ to 0.5 for CelebA (i.e., we start with 50% real and 50% generated data) and 0.99 for ImageNet People Subtree (i.e., 99% real and 1% generated data). We higher (conservative) le initially for ImageNet People Subtree because its generated data has lower quality than that for CelebA. For all baselines, choose the hyperparameters from candidate sets of each baseline to show the best fairness while minimixing the accuracy degradation in the validation set.
4.1
CelebA Experiments
We evaluate Dr-Fairness on CelebA by comparing it with baselines (Sec. 4.1.1 and analyzing the impact of its hyperparameters (Sec. 4.1.2 components (Sec. 4.1.3), and generated data (Sec. 4.1.4).
4.1.1
Accuracy and Fairness
Table shows the accuracy and fairness performances of different algorithms on CelebA when training w.r.t. EO (see results of training w.rst. DP in Sec. B.5 Here we consider two scenarios: 1) binary setting of y
9

---

Published in Transactions on Machine Learning Research (04/2023)
Table 2: Performances on the CelebA test set when training w.r.t, EO on two scenarios: binary y (age) & z (gender)
and non-binary y (haircolor) & (gender, age). We compare Dr-Fairness with three types of baselines: 1) non-fair baseline, 2) fair pre-processing baselines: Simple Sampling, PairAug, and PairAug", and 3) fair in-processing baselines: Fair. Const., Dom. Indep., and FairBatch. We use ResNet50 (He et al. 2016) for all algorithms. Note that Pair Aug and Fair. Const, cannot be trivially extended to the non-binary labels, so we only show their results in the first column.
y: age and gender
y: haircolor and x: (gender, age)
Method
Acc.
Bal. Acc.
EO Disp.
Bias Amp.
Acc.
Bal. Acc.
EO Disp.
Bins Amp.
Non-fair
86.4+oa
76.4+oa
0.173+a.023
0.101+ao22
83.8+aa
82.0-oa
0.535+a.0
0.014+a.00
Simple Sampling
86.8
78.3
0.132 20.020
0.052
83.2
80.9
0.421
0.026
PairAng (Ramaswany al.
2021
85.6.a.a
79.8
0.030 0.124
PairAug" [Ramaswayy al.
2021
86.7
29.3
0.134
0.053
83.0
80.1
0.406
0.0222o.006
Fair. Const. (Zafar et al.
2017b 7b
86.8
79.620.5
0.106go.0:4
0.028
Dom. Indep. [Wang et
2020
85.02a.
72.4
0.034 2.070
81.7204
74.8
0.340
0.026
FairBatch RRoh al. 2021
84.7 go.
72.5g1
0.023
00.443...mm
79.4g15
68.2g
0.096
0.033
Dr-Fairness
87.7
81.02a.2
0.020
-0.026=0.0m
85.0
84.4go.
0.079 20.023
2..79.0.023
0.01 2o.ooa
(age) & z (gender) and 2) non-binary setting of y (haircolor) (gender, age). In Sec. B.6
we show similar
results for the experiments on other group and label combinations. Also, in Sec. B..7 we visually demonstrate the recuracy-fairness tradeoffs of the algorithms.
The fair pre-processing baselines (in rows 2 4) improve the fairness performances compared to the original non-fair baseline, but still perform worse (i.e., higher EO disp. and higher bias amp.) than the fair in in-processing baselines and Dr-Fairness. Thus, simply equalizing the data ratio among groups may not be enough to achieve high group fairness. Note that is not straightforward to get the generated data from the original PairAug work in the non-binary label setting, so we are not able to report the numbers (e.g., the right columns of Table But we expect that the results would be similar PairAug*, as observed in the binary setting. Additionally, FairnessGAN (Sattigeri et al., 2019) is another previous method that aims to generate fair images, but this method has been reported to show worse fairness and accuracy performances than PairAug see Sec. B.4 for a detailed comparison.
The fair in-processing basclines (in rows 5-7) improve fairness (csp. EO), but tend to sacrifice accuracy because they only utilize real data. Here, in the baselines with higher fairness, the decrease in accuracy becomes more significant. For example, FaicBatch adaptively adjusts the group ratio on real data to improve fairness, but we observe that some small-sized groups end up being oversampled, which is detrimental to the accuracy performance the test set.
In comparison, Dr-Fairness achieves high fairness performances while even improving accuracies by adaptively finding optimal data ratios among groups and between real and generated data. There are two takeaways: 1) we can find a better group ratio than the 1:1 ratio for fairness, and 2) an optimal combination of real and generated data can mitigate the accuracy degradation of fair training.
We eaplain when Dr-Fairness can improve both fairness and accuracy compared to other fairness baselines. We believe this phenomenon is related to the optimal securacy-fairnest tradeoff, which is known to be determined by the data distribution [Menon & Williamson 2018 Roh et al.) (003). When the performance of feir algorithm lies on the optimal accurocy-furrnes tradeoff, any other algorithm can only achieve either better fairness or better accuracy, but connot improve both. However, when the fairness algorithms do not achieve the optimal siccrucy-fairness tradeoff in the given data, there is on opportunity to improve the model's performances toward the optimal tradeoff. For example, we observe that Dr- Fr-Fairness improves both accuracy and fairness compared to other baselines in CelebA, implying that the basclines do not achieve the optimal accumcy--oirmen tradeoff in the first place see the tradeoff curve comparisons in Sec. B.7
Remark 5.
4.1.2 Hyperparameter Analysis
We now evaluate Dr- Da-Faisssss by varying its main hyperparameter i used in the bilevel optimization. A larger k puts more weight on the accuracy loss than the fairness loss. Figures 4a and 4b show the accuracy and
10

---

Published in Transactions on Machine Learning Research (04/2023)
0.6
0.1
FairRatch
Accuracy
$0.0
EO Disp.
0.1
E:0 Disp:
0.3
Dr-Fairness
50
70.0
0.1
0.2
20
0.1
60.0
50
3000
1100
1500 20000
500
10000
1500
2000
78.0
80.0
82.0
81.0
86.0
Iterations
Iterations
Accuracy
(a) Accuracy when varying k.
(b) Fairness when varying &.
(c) Tradeoff curves.
Figure 4: The performances of Dr-Fairnes by varying the hyperparameter is when (y, 2)
(haircolor, (gender, age)).
The first two graphs show the performance changes during the training, and the last graph shows the accuracy-fairness tradeoff curves. Compared to FairBatch, Dr-Fairness shows a better tradeoff.
fairness of Fairnsss during the training with the different k values. As expected. increasing & (say 50) results in higher accuracy and lower fairness. By varying k we can also compare the accuracy-fairness tradeoff curves of Dr-Fairness and FairBatch in Figure 4c Dr-Fairness shows a better tradeoff, which is consistent with the results in Sec. 4.1.1
4.1.3 Ablation Study
We perform an ablation study on our framework to evaluate the impact of each component in the optimization on fairness and accuracy. For fair- ness, we conduct two ablations: F1) remove both the fairness loss in the outer objective and yy, in the inner objective, and F2) only remove the fairness loss in the outer objective. For accuracy, we consider three ablations: A1) remove the ac- curacy loss in the outer objective; and Py.s and the generated data loss the inner objective, A2)
Table 3: Ablation study CelebA, where we consider the
setting of non-binary y (haircolor) and binary z (gender). We
mark noticeable performance degradations with underlines.
Method
Acc.
Bal. Acc.
EO Disp.
Fl: w/o fair. loss and Ay.
85.7
83.9
0.432 10.003
P2: w/o fair. loss
85.
84.6gns
0.261 gn
Al: w/o acc. loss, Palss and gen. duta
79.3a1.
66.72sx
0.0902a.oo
A2: w/o Pa.s and gen. data
80.0a1.a
70.8
0.031
A3: w/o acc. loss
71.5am
82.1
0.046
Di-Fairness
$4.5
82.3
0.029
remove My.z and the generated data loss in the inner objective, and A3) only remove the accuracy loss the outer objective. We note that A2 also represents how Dr-Fairness works only with real data when we cannot utilize generated data. Through this sequence of ablations, we observe that each part of our algorithm gradually improves the fairness and accuracy performances.
In Table 3 the fairness ablations (in rows 1 1-2) show worse fairness as the fairness loss and Xy.z are discarded, and the accuracy ablations (in rows 3 5) demonstrate lower accuracy and balanced accuracy as some of the accuracy loss, generated data, and Myys are removod. We thus conclude that all components in our bilevel optimization contribute the overall performances.
4.1.4 Generated Data of Different Qualities
We analyze the robustness of our framework against the generated data quality as shown in Table 4 We vary the quality of generated data by adding random Gaussian noise to the original images. Interestingly, when the generated data quality decreases (i.e., adding more noise to the images), Dr-Fairness automatically reduces the us- age of the generated data, as shown in the second column of Table 4 With this automatic adjust- ment, Fr-Fairness shows robust performances in the last three columns. When the generated data is fully replaced with Gaussian noise (i.c., severe noise), the accuracy and fairness performances become worse than the clean setting as expected, but the
Table 4: Analysis of the behavior of Dr-Fairness when the quality of generated data changes. We add different amounts of Gaussian noise the original (clean) generated data. In the severe case, we fully replace the generated data with the noise. We set y to haircolor (nou-binary) and gender (binary).
Noise level Final data ratio in gen. data
Ace.
Bal. Acc.
EO Disp.
found in Dr-Fairness
Clean
0.221mnam
81.5aa
82.3
0.029
Light noise
0.210
81.4aa
81.6sia
0.033
Mid noise
0.192mma
84.5aa
81.940a
0.039
Severe noise
0.11 4g14
83.1
78.3
g.0
Nou-fair baseline
83.8 1o.
82.0.e4
0.39
11

---

Published in Transactions on Machine Learning Research (04/2023)
fairness score is still much better than the non-fair baseline by reasonably sacrificing the accuracy. These results show that Dr-Fairness is effective even with low-quality generated data.
4.2 ImageNet People Subtree Experiments
We finally perform experiments on ImageNet People Subtroe, which represents largo-scale real-world scenario with 112 label classes and 3 non-binary group attributes, gender, skin color, and age. As only 10% of the data has group annotations, following Zhao al.] (2021), we first pre-train non-fair model the entire training set with y labels and then fine-tune the pre-trained model to improve fairness on the small set with group labels.
Tables 5 and G show the performances of the algorithms on four group scenarios: gender, skin color, age, and all combinations of them. The overall results are consistent with the CelebA experiments, where we can see Dr-Fairness outperforms the baselines in accuracy, fairness, or both. Specifically, our algorithm shows the best or second- best performance on EO disparity and bias amplification in almost all group settings while obtaining better classification accuracies compared to the basclines with similar fairness scores. For example, obtain classification accuracies better than FairBatch, with an absolute improvement of 5 9%, while achieving similar fairness scores.
As ImageNet People Subtree shows a more complicated real-world scenario than CelobA, we have two additional observations. First, when we train the baselines w.r.t. EO, the bias amplification metric occasionally gels worse compared to the original model (e.g., Dom, Indep. on gender and FairBatch on skin color). This result shows that improving EO, which aims to minimize the label-specific accuracy gap between groups, does not necessarily lead to reducing the bias in the model compared to the data. In addition, as domain independence trains separate classifiers per each group, we suspect that the final model may have undesirable results (e.g., worse bias amp.) if some of the classifiers fail. Second, this dataset contains a large number of (y, 2l-csasses where many of them are extremely small-sized. Here the controllable data generation becomes challenging where the generated labels may be noisy, which negatively affects the fair training as well. Nonetheless, Dr-Fairness still shows a clear improvement in fairness compared to the basclines, and we believe more data with clean labels could further improve its performance.
Table 5: Performances the ImageNet People Subtree lest set when training w.r.t. equalized odds (EO) for either gender or skin color. We mark the best and second best performances among the fairness algorithuns with bold and underline, respectively. Note that PairAug and Fair. Const. used in Table 2 cannot be trivially extended to the non-binary lahels, so we do not use them on ImageNet People Subtree. Other settings are identical to Table
2: gender
z: skin color
Method
Acc.
Bal. Acc.
EO Disp.
Bias Amp.
Acc.
Bal. Acc.
EO Disp.
Biss Amp.
Non-fair
61.4gos
61.81os
0.871
0.2562o.oa1
61.4gcs
61.82oa
0.239 0.
ase
Simple Sampling AairAug [Ramaswamy al. 2021
54.8gos
0.8341o.0s 54.920.
0.282
55.5
55.7210
0.849
0.219 sas
54.620.
54.92os
0.821 mo
0.241 aa osi
58.1
58.32e
0.830 on0
0.225
Dom. Indep. (Wang et al. 2020
59.9
60.1 10.1
0.857 20.010
0.38
ao oss
60.2
60.5
0.874
0. 0.204
FairBatch (Roh et al. 202
52.9aan
53.1
0.816go.om
0.191
20.011
51.9
12.
0.81 181 100000
0.346
Dr-Fairness
58 28.2aum
58.3ama
0.817su.u2 20.012
0.220
61.2 a0.8
0.845 ona 0.137 $00000
The above experiments use ResNet50 (He et al. 2016) the model backbone. In addition, we also conduct experiments using ViT (Dosovitskiy et al. 2021) and observe similar results. Specifically, we use DeiT- S (Touvron et al.] 2021), variant of ViT (Dosovitskiy et al. 2021), on the ImageNet People Subtree dataset w.r.t. the age group attribute. Table 7 shows the accuracy and fairness performances of Dr-Fairness and the representative baselines, where we observe similar results to those in Tables 3 and 6 This experiment shows
that Dr-Fairness is applicable to various network architectures, including VIT.
12

---

Published in Transactions on Machine Learning Research (04/2023)
Table 6: Performances on the ImageNet People Subtree test set when training w.r.t. equalized odds (EO) for either age all combinations of groups. Other settings are identical to Table 2
z: age
z: (gender, skin color, age)
Method
Acc.
Bal. Ace.
EO Disp.
Bias Amp.
Acc.
Bal, Ace.
EO Disp.
Bias Amp.
Non-fair
61.4go
61.82os
0.849
0.191 20.
61.4
61.820a
0.270 0.918
107
Simple Sampling Auir [Ramaswany et al. 2021
53.4go0
53.6220
g.
0.20
54.6
54.8g0
0.885 oer
1.181 2:0
55.6212
55.921s
0.8
0.168 o0
57.920s
0.871 ge ons
153 sae
Dom. Indep, [Wang al. 2020
5
60.1
0.838
0.220
50.0ae
50.1
0,254 2.
FairBatch (Roh al. 2021
51.3
51.521a
0.798 20.042
0.198 20.00 ous
50.3
0.185 0.853
Dr-Fairness
59.7.a.1
59.9
0.784
0.149 s0.012
7
59.1
0,866
0.146
Table 7: Performances on the ImageNet People Subtree test set w.r.t. equalized odds (EO). We use DeiT-S Touvron et
2021 a variant of ViT (Dsovitskiy et al. 2021 for all algorithms. Other settings are identical to Table
age
Method
Acc.
Bal. Acc.
EO Disp.
Bias Amp.
Non-fair
64.8ga
65.220
0.890
0. 09198 can
Simple Sampling PairAug" (Ramaswamy al.
55.3
55.7
0.807
0,261 20 0et
2021
59.5
59.92u1
0.824
0.237 20
Dom. Indep [Wang et al.
2020
51.8218
12.0s1.
18817suma
0. 0.634440 pas
BatcBatch (Roh et al.
2021
68.2#ux
58, Gaus
0.817 2
0.162 #0.016
Dr-Fairness
64.4+u.s
64.8+a.
0.774+0.00
0.176
5 Conclusion
We proposed a novel adaptive sampling approach called Dr-Fairness that utilizes both real and generated data for fairness. To perform adaptive sampling systematically, we first formulated bilevel optimization, where the goal is to find the optimal data ratios among sensitive groups and between real and generated data to achieve high group fairness while minimizing accuracy degradation. To solve the bilevel optimization problem, then designed efficient approximate algorithm based on the implicit function theorem and identity-matrix approximation. Extensive experiments on the CelebA and ImageNet People Subtree datasets showed that Dr-Fairness achieves slate-of-the-arl fairness and accuracy performances. In addition, analyzing the performance Dr-Fairness in other types of datasets, including medical images, will be an interesting future direction. We believe Dr-Fairness opens up new opportunitics for effectively using generated data in large-scale real-world scenarios.
Broader Impact Statement
We believe our work can positively impact society by reducing discrimination in AI applications. In particular, our framework shows that generated data can compensate for unfairness issues in real data (e-g., size bias and lack of diversity) to help obtain better accuracy and fairness results that would not have been possible otherwise. As a result, real-world applications have better chance of ensuring fairness without sacrificing accuracy unnecessarily.
We do note that choosing an appropriate fairness metric for each application is essential, as a poor choice may lead to unintended discrimination. Thus, one needs to carefully choose the target fairness metrics based on the social context in each application. Also, in terms of privacy, we did not involve human subjects or use any direct personal identifiers in the experiments, except for the human images in the publicly available benchmark datascts.
13

---

Published in Transactions on Machine Learning Research (04/2023)
References
Alckh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna M. Wallach. A reductions approach to fair classification. In ICML, pp. 60 69, 2018.
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias: There's software used across the country to predict future criminals. And its biased against blacks. ProPublica, 2016.
L. Elisa Celis, Lingxiao Huang, Vijay Keswani, and Nishooth K. Vishnoi. Fair classification with noisy protected attributes: A framework with provable guarantees. In ICML, 2021.
Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, and Stefano Ermon. Fair generative modeling via weak supervision. ICML, 2020.
Ching-Yar Yao Chuang and Youssef Mroueh. Fair mixup: Fairness via interpolation. ICLR, 2021.
Oswaldo de Oliveira. The implicit and the inverse function theorems: easy proofs. Real Analysis Exchange, (9()):207 218, 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pottern recognition, pp. 248 255. Leee, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. NewrIPS, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. Fairness through awareness. In ITCS, pp. 214-226, 2012.
Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkalasubramaniaa. Certifying and removing disparate impact. In SIGKDD, pp. 259-200 1599-268,4 2015.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, pp. 1126 1135. PMLR, 2017.
Sany Wu Fung, Howard Heaton, Qiuwei Li, Daniel McKenzie, Stanley Osher, and Wotao Yin. Jlb: Jacobian- free backpropagation for implicit networks. In AAAI, 2022.
Zhengyang Geng, Meng- Mao Guo, Hongxu Chen, Xia Li, Ke Wei, and Zhouchen Lin. Is attention better than matrix decomposition? In ICLR, 2021.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In NewrIPS, pp- 3315 3323, 2016.
Tatsunori Hashimoto, Megha Srivastava, Hongscok Namkoong, and Percy Liang. Fairness without demo- graphics in repeated loss minimization. In ICML, pp. 1929 1938, 2018.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, pp. 770 778, 2016.
Heinrich Jiang and Ofir Nachum. Identifying and correcting label bias in machine learning. In AISTATS, 2020.
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst., 33(1):1-33 2011.
Tero Karras, Samuli Laine, Miik Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In IEFE CVPR, pp. 8110 8119, 2020.
14

---

Published in Transactions on Machine Learning Research (04/2023)
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and
Yann LeCun (eds.). ICTR, 2015.
Ron Kohavi. Scaling up the accuracy of bayve-bayes classifiers: A decision-tree hybrid. In SIGKDD, 1996.
Sleven George Krantz and Harold R Parks. The implicil function theorem: history, theory, and applications. Springer Science & Business Media, 2002.
Emmanouil Krasanakis, Eleftherios Spyromitros- Appromitros-iioufis, Symeon Papadopoulos, and Yiannis Kompatsiaris, Adaptive sensitive reweighting to mitigate bias in fairness-aware classification. In WWW, pp. 853-862, 2018.
Matt Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In NewrIPS. 2017.
Joshua K Lee, Yuheng Bu, Deepta Rajan, Prasanna Sattigeri, Rameswar Panda, Subhro Das, and Gregory W Wornell. Fair selective classification via sufficiency. In ICML, volume 139, pp. 6076-6086, 2021.
Peizhao Li and Hongfu Liu. Achieving fairness at no utility cost via data reweighing. JCML, 2022.
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Diffcrentiable architecture scarch. In ICLR, 2018.
Rishong Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level optimization for learning and vision from unified perspective: A survey and beyond. IEEE TPAMI, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In IEEE ICCV, pp.
2015.
Jonathan Lorraine, Paul Vicol, and David Duvenand. Optimizing millions of hyperparameters by implicit differentiation. In AISTATS, pp. 1540 1500. PMLR, 2020.
Jelena Luketina, Mathias Berglund, Klaus Greff, and Tapani Raiko. Scalable gradient-based tuning of continuous regularization hyperparamcters. In ICML, pp. 1952-2960. PMIR, 2016.
Dougal Maclaurin, David Duvenaud, and Ryan Adams. Gradient- hyperparameter optimization through reversible learning. In ICML, pp. 2113 2122. PMLR, 2015.
Sébastien Marcel and Yann Rodriguez. Torchvision the machine-vision package of torch. In ACM 1CM, pp. 1485-1488, 2010.
Ninarch Mchrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Compuling Surveys, 54(6):1 35, 2021.
Aditya Krishna Menon and Robert C Williamson. The cost of fairness in binary classification. In FAT*, 2018.
Alex Najibi. Racial discrimination in face recognition technology. Hareard Online: Science Policy and Social Justice, 24, 2020.
Arvind Narayanan. Translation tutorial: 21 fairness definitions and their politics. In ACM FAccT, volume 1170, 2018.
Weili Nie, Arash Vahdat, and Anima Anandkumar. Controllable and compositional generation with latent- space anergy-based models. NewrIPS, 34:13497 13510, 2021.
OpenAl. Dall 2.
2022.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. In NIPS Autodiff Workshop, 2017.
Fabian Pedregosa. Hyperparameter optimization with approximate gradient. In ICML, pp. 737-746. PMLR, 2016.
15

---

Published in Transactions on Machine Learning Research (04/2023)
Tapani Raiko, Harri Valpola, and Yann LeCun. Deep learning made easier by linear transformations in perceptrons. In Artificial intelligence and slatisties, pp. 924-932. PMLRR 2012.
Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. Advances in newral information processing systems, 32, 2019.
Vikram V Ramaswany, Sunnie SY Kim, and Olga Russakovsky. Fair attribute classification through latent space de-biasing. In IEEE CVPR, pp. 9301-9310, 2021.
Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. FR-Train: A mutual information-based approach to fair and robust training. In ICML, pp. 8147 8157, 2020.
Yuji Roh, Kangwook Lec, Steven Euijong Whang, and Changho Suh. FairBatch: Batch selection for model fairness. In ICLR, 2021.
Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. Improving fair training under correlation shifts. In ICML, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Stable diffusion. http:////.comm/////tttaaadddddda -ifffuiinedd 2022.
Prasanna Sattigeri, Samuel C Hoffman, Vijil Chenthamarakshann and Kush R Varshney. Fairness gan: Generating datasets with fairness properties using a generative adversarial network. IBM Journal of Research and Development, (3(4/)):3 1, 2019.
Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. Optimising equal opportunity fairness in model training. ACL, 2022.
Changjian Shui, Qi Chen, Jiaqi Li, Boyu Wang, and Christian Gagné. Fair representation learning through implicit path alignment. ICML, 2022a.
Changjian Shui, Gezheng Xu, Qi CHEN, Jiaqi Li, Charles Ling, Tal Arbel, Boyu Wang, and Christian Gagné. On learning fairness and accuracy on multiple subgroups. In NewrIPS, 2022b.
Aman Sinha, Hongseok Namkoong, and John C. Duchi. Certifying some distributional robustness with principled adversarial training. In ICLR, 2017.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score- based generative modeling through stochastic differential equations. arXiv preprint 2rXiv:2011.33455 2020.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers distillation through attention. In ICLR, 2021.
Angelina Wang, Alexander Liu, Ryan Zhang, Anat Kleiman, Leslie Kim, Dora Zhao, Iroha Shirai, Arvind Narayanan, and Olga Russakovsky. Revise: A tool for measuring and mitigating bias in visual datasets. IJC,, (30(():1790-1810 2022.
Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards fairness in visual recognition: Effective strategies for bias mitigation. In IEEE CVPR, 2020.
Bryan Wilder, Eric Ewing, Bistra Dilkina, and Milind Tambe. End to end learning and optimization on graphs. NeurIPS, 32, 2019.
Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky. Towards fairer datascts: Filtering and balancing the distribution of the people subtree in the imagenet hierarchy. In FAT pp. 547-558, 2020.
Muhammad Bilal Zafar, Isabel Valera, Manuel Comez-Rodrigee and Krishna P. Gummadi. Fairness constraints: Mechanisms for fair classification. In ATSTATS, pp. 962 970, 2017a.
16

---

Published in Transactions on Machine Learning Research (04/2023)
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In WWW, 2017b.
Richard S. Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. Learning fair representations. In ICML, pp. 325-333, 2013.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In AIES, pp. 335-340, 2018a.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk mininization. ICLR, 2018b.
Eric Zhao, De-An Huang, Hao Liu, Zhiding Yu, Anqi Liu, Olga Russakovsky, and Anima Anandkumar. Scaling fair learning to hundreds of intersectional groups. 2021.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In EMNLP, pp. 2979 2989, 2017.
Dominik Zietlow, Michael Lohaus, Guha Balakrishman, Matthäus Kleindessner, Francesco Locatello, Bernhard Schölkopf, and Chris Russell. Leveling down in computer vision: Pareto inefficiencies fair deep classifiers. IEEE CVPR, pp. 10410 10421, 2022.
17

---

Published in Transactions on Machine Learning Research (04/2023)
A Appendix - Optimization and Algorithm
A.1 Bilevel Optimization for Demographic Parity
Continuing from Sec. 3.1 we formulate our bilevel optimization w.r.t. demographic parity (DP) as follows:
min max A,p 22
{1 Myyee /real (wr (A, (4) m..., 29.2
my.e2 L real (w) 1J)l + k Σ My. real ( (w(A, 14), m...2
U.22
VEV,2EZ
m
w (). arg min
My.t
Xy. (My D L (w) + (1 My, ( Lig (w)}
y.s
w
m
s.t. A (011..) [0, 1]. = 1, EY
where Y (0, inspired by Roh et al. (2021), which gives hint formulating DP loss in bilevel optimization. Intuitively, the fractions in the fairness loss make the model reduces the disparity of each prediction ratio across groups, without considering the sizes of true label classes. This strategy can be a sufficient condition for DP, as the goal of DP is to achieve the same positive prediction ratio among groups see more details in Roh et al. 20211)
13. Note that DP designed for binary classification. For designing the fairness loss, we are
A.2 Training Overhead for Computing Gradients
Continuing from Sec. 3.2 we empirically analyze the training overhead when computing gradients in Eq. D Figure 5 shows the relationship between update frequency and training time the synthetic dataset see the details on the dataset in Sec. A.3 We compare Dr-Fairness with vanilla. (non-fair) training, which shows basic training cost, and we use the same total number of training iterations for both algorithms. As a result, we can reduce the computational cost of Dr-Fairness by updating À and l less frequently (i.e., using a longer update period). For example, we update À and at every two itorations, the training time reduces by about half compared to when updating them at every single iteration.
In addition, Figure shows that we can reduce the training overhead without sacrificing the overall accuracy and fairness performances much. Here, when updating the ratios less frequently, performances using the same total iterations by increasing the learning rates used in the optimizers for X and but this may also lead to a larger standard deviation.
achieve comparable
Finally, we observe consistent results in the CelobA dataset, whore we consider the setting of binary y (age) and x (gender). Specifically, we can reduce the training time by about half by updating and l at every two iterations, while the overall accuracy and fairness performances remain similar those when updating the data ratios at every single iteration, as shown in Table g Similar to the above synthetic data experiment, we increase the learning rates used in the optimizers for À and μ when updating them less frequently.
Table 8: Performances on the CelebA lest set when training w.r.t. EO binary y (age) and (gender). We compare the performances when varying the update period of X and j in Dr-Fairness.
Method
Update Period of & P
Acc.
Bal. Ace.
EO Disp.
Bias Amp.
Non-fair
86.42ea
76.42oa
0.1
0.101 20.000
Dr-Fairness
1 iteration
87.7
81.0ro2
0.020
-0.026
Dr-Fnirness
iterations
87 37.6
81.1
0.025
-0.024
A.3 Setting for the Validity Check
Continuing from Sec. 3.2 we describe the synthetic binary setting in Roh et al. (2021). which is used empirically verify how close the solutions from approximation strategy are to the optimal ones.
18

---

Published in Transactions on Machine Learning Research (04/2023)
Total Training Time (sec)
10
Dr-Fairness
8
Vanilla Training
6-
4
2
10
15
20
Update Period (Iteration)
Figure 5: Update-period training-time curves the synthetic dataset. We compare Dr-Fairness with vanilla (non-fair) training, which shows a basic training cost.
0.88
0.10
Unfairness
Accuracy
0.86
Dr-Pairnes
0.05
Vanilla Training
0.81
Fairuess
Training
0.00
5
10
15
20
10 Update Period (Iteration)
15
20
Update Period (Iteration)
(a) Update-period accuracy.
(b) Update-period vs unfairness (equal opportunity).
Figure 6: Performance comparisons when varying the update period in Dr-Fairness on the synthetic dataset. We
compare Dr-Fairness with vanilla (non-fair) training.
For generating the synthetic dataset, we method in Zafar al. (2017a). which produces two input attributes (x1, x2), one binary label attribute y, and one binary group attribute We draw each sample (x1, X2: y) from Gaussian distributions and make follow biased distribution.
In detail, we generate each sample (x1, X2, y) from two Gaussian distríbutions: (x1,x))lx = ([[-2; 2], [10, 1; 3,) and (x1,x))ly = ~ M[22 2]. [5, 1J), Then, we make z follow a biased x
xx)ly = 0) Pr((xi,x6)ls =]] where (x1,x²) (x) cos(T/4) X2 sin(TT/4) () x1 (in( + +x2 cos(=))). This synthetic dataset contains training, validation, and test sets with 2k, 1k, and 1k samples, respectively.
distribution: Pr(z=1)
In this experiment, use logistic regression models for all algorithms, in Roh el al. 2021
A.4 Comparison with Exact Inverse Hessian Computation
Continuing from Sec. 3.2 we compare our identity matrix-based approximation results with the exact inverse Hessian computation results. We use the same setting described in Sec. A.3 where it is tractable to compute the exact inverse Hessian.
Figure 7 shows the group ratios of Dr-Fairness (with the identity matrix approximation) and Dr-Fairness with the exact inverse Hessian computation. We can see that Fr-insinsss, which uses the identity matrix approximation to estimate the inverse Hessian in Eq. 2 converges similar group ratios those in computing the exact inverse Hessian. It implics that our solution is close the exact solution despite the method's simplicity. Another observation is that the data ratios in Figure 7b converge within fewer iterations than
19

---

Published in Transactions on Machine Learning Research (04/2023)
those in Figure 7a We note that although the number of required iterations is fewer when computing the exact inverse Hessian, the training time is much slower if the number of parameters increases. Thus, the approximation used in Dr-Fairness can be a reasonable solution to estimate the inverse Hessian, which is usually intractable for large models and data.
Group Ratio
1.0
Group Ratio
1.0
0.761
0.763
0.5
1
0.5
1,
1
0.239
0.237
0.0
0.0
0
100
200 Iterations
300
400
0
100
200 Iterations
300
400
(a) Group ratios in Dr- Fairness
(b) Group ratios with the exact inverse Hessian.
Figure 7: Comparison of group ratios À from Dr-Fairness (with the identity matrix approximation) and Dr-Fairness with the exact inverse Hessian computation. Both converge to similar ratios.
A.5 Comparison with Other Problem Formulation Methods
Continuing from Sec. 3.1 we discuss the advantages of using bilevel optimization compared to other problem formulation mothods, especially using distributionally robust optimization (DRO) (Sinha et al. 2017).
DRO one of the prominent problem formulation methods machine learning, which can solve a target objective in a min-max formulation, but we believe that our bilevel formulation is more suitable to handle both real and generated data while improving group fairnoss. In scenario, real data and generated data play very different roles in the bilevel objectives, and such roles are difficult to capture via a DRO formulation.
Tn detail, at the test-time evaluation, we only care about the fairness and accuracy losses (in our outer objective) on the real data distribution. Thorefore, the empirical risk for generated data (I,e0) only appears in the inner objective for model parameters update.
Directly applying DRO to empirical risks of real and generated data does not lead to the same effect as our bilevel objective because this ignores the EO-based loss in our outer objective, and more importantly, it is unclear what the benefit of optimizing the real/generated data sampling ratio to maximize the empirical risks. If we consider an example where we have a high loss on generated data and low loss on real data, then DRO should increase the sampling ratio for the generated data to increase the overall loss. However, the high loss generated data could be the result of the low quality of generated data, and increasing its sampling ratio might instead hurt the performance and fairness. On the other hand, as shown in Sec. 4.1.4 Dr-Fairness would decrease the ratio of generated data when its quality is low.
B Appendix - Experiments
B.1
Experimental Settings
Continuing from Sec. provide detailed information on the experimental settings. We use PyTorch for all experiments and utilize the pre-trained ResNet50 (He et al. 2016) provided by PyTorch library (i.e. torchvision (Marcel & Rodriguez 2010)). We change the last fully-connceted layer of cach model with the number of corresponding label classes. When training, we update all model parameters in the pre-trained model. The batch size all experiments is 128. We set the learning rate for updating model parameters 0.0001. For the data ratio (A and μ1) updates Dr-Fairness, we use the Adam optimizer and set the learning rate for the ratio update 0.005 in all experiments. When calculating the gradients w.r.t. model the autograd functionality in PyTorch. We apply the exponential
parameters data ratios in Eq.
20

---

Published in Transactions on Machine Learning Research (04/2023)
moving average when updating the model parameters in Dr-Fairness. To prevent the overfitting, use the validation set when measuring the fairness and accuracy losses in the outer objective of our bilevel optimization. Similarly, we use the validation set in other baselines if they require the computation of additional (fairness) losses in the algorithm.
B.2 Filtering Label Classes in ImageNet People Subtree
Continuing from Sec. we explain how we filter the label classes in the ImageNet People Subtree dataset. Initially, the dataset contains 284 label classes. Among them, we filter out classes that are vague, duplicates, or too small with few samples. First, we filter vague classes like "ex- "ex-president" and "junior", which are hard to classify even for human annotators. To decide whether each class is vague or not, we perform internal crowdsourcing. For each class, we gather 3 expert decisions and do majority vote. Also, we remove classes that are conceptually duplicates of others. Finally, we set the allowed minimum sample size to 50 and ignore the classes with fewer than 50 samples. As a result, 112 classes are used in our experiments.
B.3 Data Generation
Continuing from Sec. we explain the details on data generation.
For experiments on CelebA, we use LACE Nie et al. 2021 to generate data. LACE is a controllable gencration method that uses an energy-basee model (EBM) the latent space of a pro-trained generative model such as StyleGAN2 (Karras et al. 2020). We consider StyleGAN2 pre-trained on the CelebA-HQ dataset our base generative model. In LACE, we first need to train the latent classifiers in the w-space of StyleGAN2, each of which corresponds to energy function for an individual attribute in the EBM formulation (sce Eq. (4) in NNc et al.] 2021)). ). Since we mainly focus on five attributes (i.c., age, gender, smile, glasses, and haircolor) in the CelebA experiments, we end up with five latent classifiers. Next, for each combination of attribute values (e.g., age='young', gender='female" smile='true' glasses='true' and baircolor-'black)) we use the ordinal differential equation (ODE) sampler in the latent space to sample the corresponding images. We repeat the above sampling process until we cover all the combinations of attribute values.
For experiments on ImageNet People Subtree, we use a guided diffusion model (Dhariwal & Nichol 2021 with classifier guidance (Song et al.) 2020 Dhariwal & Nichol 2021) to generate data. Since there exist no ADM checkpoints pre-trained on ImageNet People Subtree, we first fine-tune the
ADM on ImageNet People Subtrec, where the ADM model that we use conditions on the 112 labels. For classifier guidance, we also need to first train three tnme-dependent attribute classifiers, each corresponding to demographic attribute (i.e., gender, skin color, and age), on noisy images produced by the diffusion process. In particular, we finc-tune the noisy image classifier (also pre-trained ImageNet) with three new prediction heads on 10% of the annotated data. Next, for each combination of label and attribute values, we pass the label value as the input of the conditional ADM and use the classifier guidance (i.e., the guidance from the fine-tune noisy image classifier in Eq. (10) of Dhariwal & Nichol (2021)) with the scale s = 15. Similarly, we repeat the above sampling process until we cover all the combinations of label and attribute values.
Here we describe the number of generated samples in each dataset. In CelebA, we consider 5 attributes for the controllable generation: gender (male and female), age (young and old), smile (true and false), glasses (true and false), and haircolor (black, blond, and others). Thus, these 5 attributes yield 48 class combinations (i.c., 24 x 3). We generate a total of 96k samples, where there are 2k samples for cach altribute combination (e.g., 2k samples for (age='young', gender=" smile='true', glasses="Irue; and (aarccooo=bblackk). In ImageNet People Subtree, there are 112 label classes and 3 group attributes: gender (male, female, and unsure), skin color (light, medium, and dark), and age (child, adult, middle, and retired). Thus, we have 4,032 combinations (i.e., 112 x 3 x 3 4) for the controllable generation. We generate 32 samples for each combination, which results in about 129k samples in total.
21

---

Published in Transactions on Machine Learning Research (04/2023)
Figures B and B show examples of the generated images. We note that the controllable generation for ImageNet People Subtree is more challenging due to its large number of (y, z)-classes and labeling noises. Thus, the resulting generated data is noisier than the generated data in CelebA.
AL
age gender female, smile false, glasses false, haircolor black
age = young, female, glasses false, haircolor blond
age old, gender male; smile true, glasses false,
age young, gender smile true, glasses true,
Figure Examples of the generated images CelebA.
Anchor, gender = female, skin color medium, age = middle
Astronaut, gender female, skin color = light, age = adult
Captain gender male, skin color dark age middle
Tennis player, gender male, skin color light, age child
Figure 9: Examples of the generated images on ImageNet. People Subtree.
B.4 Comparison between FairnessGAN and PairAug
Continuing from Sec. 4.1.1 we compare FairnessGAN (Sattigeri et al., 2019) and PairAug (Ramaswamy et al. 2021). Table g shows the accuracy and fairness performances of the two algorithus on CelebA, where they consider the setting of binary label attribute y (attractive) and binary group attribute (gender). We show the numbers that reported in Ramaswamy et al. (2021). As result, PairAug shows better accuracy and equalized odds performances compared to FairnessGAN.
Table 9: Additional comparisons between baselines using generated data. We compare PairAug and FaimessGAN, where the results are from Ramaswamy et al. 2021 The baselines use the attractive attribute as the label and gender attribute as the group.
Method
Aec.
EO Disp.
PairAug Ramaswamy et al. 2021
80.0
0.21
FairnessGAN (Sttigerii al. 2019
71.0
0.25
22

---

Published in Transactions on Machine Learning Research (04/2023)
B.5 Other Results on CelebA w.r.t. Demographic Parity
Contimuing from Sec. 4.1.1 we perform experiments on the CclcbA datasct w.r.t. demographic parity (DP). Table shows the accuracy and fairness performances of the algorithms. Similar to the results in Table 2 Fe-Fainness shows higher fairness than the pre- pre-proeessing (1:1 data ratio) baselines (i.e. simple sampling, PairAug, and PairAug*) and higher accuracy than the in-processing basclines (i.e., fairness constraint, domain independence, and FairBatch). Here, as the pre-processing baselines are not explicitly designed for DP, their fairness performances in terms of DP are sometimes worse than the original ResNet.50 results. Since DP aims to ensure the same positive prediction rates without considering the true labels, the training sometimes needs to overfit on the positive labels for specific groups to improve DP. Thus, simply equalizing the data ratio among groups may not be enough to improve DP compared to the EO case.
Table 10: Performances on the CelebA test set w.r.t. demographic parity (DP) on the binary y (age) and z (gender) scenario. Other settings identical to Table [ algorithms with bold and underline, respectively.
We mark the best and second best performances among the fairness
y: age and z: gender
Method
Acc.
Bal. Acc.
DP Disp.
Bias Amp.
Non-fair
86.4
76.4202
0.141
0.101
Simple Sampling
86.8
78.3
0.132
0.052
PairAug [Ranaswany el al.
2021
85. 35.6go.s
79.8
0.151
t0.007
0.030
PairAug* *
Ramaswamy et al.
2021
86.7
79.3
0.144 0.14420.000
0.053
Fair. Coust. (Zalar et al.
2017a
85
74.3
0.1242n.000 20.009
0.103
Dom. Indep. [Wang al.
2020
85.0zo.
72.4
0.091
t0.004
0.034
FairBatch (Roh et al.
2021
84.9
74.0
0.07 4 to...oo..
-0.034+o.004
Prr-Faineess
86.3
78.2
0.088 009
-0.02820.00
B.6 Other Results on CelebA with Different Settings
Continuing from Sec. 4.1.1 we perform experiments on the CelebA dataset with different group and label combinations. Tables 11 and 12 show the accuracy and fairness performances on the following two settings: (y, z) (smiling, gender) and (y, z) (haircolor, gender). In both cases, we observe consistent results to those in Sec. 4.1.1] where Dr-Fairness achieves high fairness (esp. EO) while not sacrificing accuracy. We note that in these two settings, the bias amplification values of the non-fair baseline are already very small (i.e., good enough), so the fair algorithms may not further improve the bias amplification.
Table 11: Performances on the CelebA test set w.r.t. equalized odds (EO) on the binary y (smiling) and binary z (gender) scenario. We mark the best and second best performances among the fairness algorithus with bold and underline, respectively. Other settings are identical to Table 2
2: age
Method
Acc.
Bal. Acc.
EO Disp.
Bias Amp.
Non-fair
91.7goa
91.7202
0.027
0.006
Simple Sampling
91.8
91.8ta.
0.019 007
0.012
PairAug (Ramaswaany et al.,
2021
91.42o.
91.4ta
0.022 oo1
0.003
PairAug (Ramaswamyy al. 2021
92.0+o.
92.0+a.
0.031
0.003++o.cmm
Fair. Const. (Zafar et al. 2017b
91.32gs
91.3+0.
0.014 o07
0.009
DOm. Indep. [Wann et al 2020
91.2
91.22o.
0.014.0 ooo
0.009 20.00
FairBatch (Roh al." 2021
91.620.
91.61a.
0.01 210.002
0.01820.00
Dr-Fairness
92.7
92.7
0.013
0.009+t..c4
23

---

Published in Transactions on Machine Learning Research (04/2023)
Table 12: Performances on the CelebA test set w.r.t, equalized odds (bO) on the non-binary y (haircolor) and binary (gender) scenario. We mark the best and second best performances among the fairness algorithms with bold and underline, respectively. Other settings are identical Table 2
yi haircolor and gender
Method
Acc.
Bal. Acc.
EO Disp.
Bias Amp.
Non-fair
83.8=os
82.0+a
0.399
0.010
Simple Sampling
83.11oa
78.91ss
0.322
0.021 10.004
PairAug" (Ramaswamy
2021
82.7.oa
80.01ao
0.37410.02
0.018 1o.osa
Dom, Indep. (Wang 2020
83.2=0a2
79.4+a.9
0.308
0..11+0.000
FairBatch (Roh et al. 2021
78.4=14
68.7+1.
0.085
0.072
Dr-Fairness
84.5_..2
82.3
0.029 10.000
0.023
B.7 Accuracy-Fairness Tradeoffs
we visually demonstrate the accuracy-fairness tradeoffs of Dr-Fairness and the baselines on the CelebA and ImageNet People Subtree datasets. Figure I0 shows the accuracy and unfairness performances of the basclincs and Dr-Fairness. Here, being on the lower-right indicates higher accuracy and fairness and is thus desirable. In CelebA, Dr-Pairness achieves both better accuracy and fairness performances compared to all the baselines. In ImageNet People Subtree, 1) the baselines Simple Sampling, PairAug, and Dom. Indep. show strictly worse performances than Dr-Fairness, 2) FairBatch achieves higher fairness than ours, but the accuracy degradation is severe, and 3) the non-fair baseline shows high accuracy, but much worse fairness. Thus, we can conclude that Dr-Fairness achieves the best accuracy-faine fairness tradeoffs in both datasots.
Continuing from Sec. 4
0.6
Non- Stär
0.93
Simple Sumpling
EO Disp.
0.4
PairAug
Indep
EO Disp.
0.90
FairDotch
Dr-Fairmess
0.88
0.2
0.85
80.0
82.0
84.0
50.0
55.0
60.0
Accuracy
Accuracy
(a) Tradeoffss in CelebA
(b) Tradeoffs in ImageNet People Subtree when (gender, skin color, age).
when (y: 2) (haircolor, (gender, age)).
Figure 10: unfirncys graphs to visualize the algorithm performances the CelebA and ImageNet People Subtree datasets, We note that only Dr-Fairness and FairBatch have accuracy-fairner tuning hyperparameters to produce trade-off curves. The other baselines do not have such tuming knobs, they use fixed strategies to train models. We thus add the tradeoff curves for Dr-Fairness and FairBatch, while the performances of the other baselines shown single points. Being the lower right desirable (high accuracy and fairness).
24

---

Published in Transactions on Machine Learning Research (04/2023)
C
Appendix - Related Work
Contimuing from Sec. we discuss more related work.
There are other related studics on 1) fair data reweighing Li & Liu 2022 Jiang & Nachum 2020 Krasanakis
et al. 2018). 2) fair augmentation (Chuang & Mroueh 2021 and 3) lair representations (Shui et al. 2022a).
The data reweighing techniques (Li & Liu 2022 Jiang Nachum 2020
Krasanakis et al.) 2018 are
relevant. to our data sampling framework, they keep finding data weights to improve group fairness.
Compared to work; Jiang & Nachum (2020) and Krasanakis et al. (0188) require multiple re-training of the model, and Li & Liu (2022) uses additional assumptions, including the loss function being twice differentiable and strictly convex in the model parameters. Therefore, applying these reweighing techniques when training models on large-scale data may lead to significant training times due to multiple re-trainings or performance degradation due to violations the assumptions. In comparison, Dr-Fairness works well in large-scale scenarios as in our experiments.
There is another interesting work called FarrMixup CChaagg & Mrouch 2021 which augments training data for fairness using mixup methods (Zhang et al. 2018b). However, the key difference from ours is that FairMixup only augments the data within the original training data distribution. FairMixup also cannot dynamically adjust sampling ratios from different groups explicitly. In contrast, Dr-Fairness can utilize any additional data, which is not limited to the original training (real) data distribution, and also find the optimal sampling ratios among groups and between real and generated data.
Recently, a fair representation paper (Shui et al. 2022a) uses the bilevel optimization formulation with the implicit function thcorem and shows promising results. Although this work also uses a bilevel formulation, targets a different problem from ours, where the goal is to map the input feature X into the latent variable X' for fairness. In comparison, we use the bilevel formulation to adjust the ratio among groups and between real and generated data to improve fairness. Specifically, explicitly separating the group-wise terms and real/generated data terms to adequately apply the data weights. We note that this inner structure is different from Shui et al. (2022a), where they apply the common outputs of the outer objective to all terms in the inner objective that only considers real data. addition, when approximating the inverse Hessian matrix resulted by the implicit function theorem (e.g., Eq. 2. Shui et al. (2022a) use the conjugate gradient (CG) method (Rajeswaran utilize the identity-matrix approximation (Luketina et al. 2016; Geng et al. 2021). We note that the connity--matixx approximation is known to be much more efficient and may achieve on-par sometimes better performances compared to the CG method (Lorraine el al. 2020
design the inner objective by
al. 2019), whereas
25


Reviews:
[{"review_id": "b9UZ5O19BS", "reviewer": "Reviewer_9NWk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Review Summary\n\n[*Disclosure: this reviewer has reviewed this paper in other venues*]\n\nThis paper addresses an important fairness issue in data-augmentation, specifically in rebalancing the training loss between generated and real data. The authors propose a bi-level objective to automatically balance the weights for sensitive attribute and label balancing. The proposed idea seems sound, and the authors conducted empirical evaluations to support their claims. Therefore I support **acceptance**.\n\nOn the negative side, several statements within the paper still require refinements. My comments, based on TMLR guidelines, are as follows.\n\n### Would some individuals in TMLR's audience be interested in the findings of this paper?\n\nYes. This paper addresses an important fairness issue in data-augmentation, specifically in rebalancing the training loss between generated and real data.\n\n### Are the claims made in the submission supported by accurate, convincing and clear evidence?\n\nMost claims are well-supported. I have few additional comments on some parts\n\n1. Sec 3 Fairness Definitions. For example, in this paper, DP is defined as:\n$$P(\\hat{Y}=1|Z=z_1)= P(\\hat{Y}=1|Z=z_2)$$\nA natural question is how could we evaluate the bias such as DP gap, when Z is non-binary? We consider the DP gap for each pair (z_1 and z_2)? \n\n2. Fig 3 seems quite interesting. I was wondering, why not determine the initial ratio with exact inverse Hessian computation? Then the convergence will be much faster.\n\n3.  Remark 4. \n> We believe this phenomenon is related to the optimal accuracy-fairness tradeoff, which is known to be determined by the data distribution.  When the performance of a fair algorithm lies on the optimal accuracy-fairness tradeoff, any other algorithm can only achieve either better fairness or better accuracy, but cannot improve both\n\nAgree, there is a lower bound on this.\n\n> However, when the fairness algorithms do not achieve the optimal accuracy-fairness tradeoff in the given data, there is an opportunity to improve the model’s performances toward the optimal tradeoff.\n\nThis is not convincing enough. First, each method has its own accuracy-fair trade-off curves. For example, by changing different fair constraints, we could visualize such curves. A fair comparison should visualize such a curve for each method, and demonstrate the proposed method controls a better trade-off. For example, extending Fig 8 to different curves for each method.\n\n4. The Real world example in Fig 1 could better be integrated with real-world generated samples in Fig7. I like the motivating example and I think it is better to consider a better story, for example, including the results in Fig.7.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates the issue of unbalanced data generation, resulting in a bias/fairness concern. The goal is to balance the generated data and control the prediction/fairness performance compared to raw data. To address this problem, the paper proposes a bi-level optimization approach that automatically adjusts the balancing coefficient on subgroups and classes (labels). The proposed method is evaluated on the CelebA/ImageNet dataset, and results demonstrate its effectiveness.\t", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In general, this paper is well-written. I would suggest authors to address the points in *Are the claims made in the submission supported by accurate, convincing and clear evidence.*\n\n", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "xy280fKaVd", "reviewer": "Reviewer_C26t", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strong points:\n- The proposed approach is presented with a good level of detail, is well justified, and appears sound to me.\n- The experiments provide strong evidence that the proposed approach is working as expected.\n- The paper is well written and relatively easy to follow.\n- The problem of algorithmic fairness is important given the impacts AI can have on people and society and the risk of reproducing the biases found in the datasets in machine learning models.\n\nWeaknesses:\n- The paper is tackling a topic in a rather incremental way, it builds upon existing results and proposed approaches with some originality, but it would not qualify it as a foundational paper that would open a new field.\n- The proposal is properly justified and follows some intuitions, but is mostly supported through empirical evaluation. Having more developed theoretical justifications would have made the paper even stronger.\n- The experiments are limited to image classification benchmarks over faces, the approach is not tested in order contexts that may involve other types of images, or even other modalities (e.g., text).\n- Some references are missing (more in next sections).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents an approach to adjust sampling ratios over different groups and between real and generated data to ensure fairness in the models learned. It proceeds through a bi-level optimization routine that is considering both the accuracy and fairness at the outer level, while adapting sampling weights by taking into account real and generated data instances at the inner level. It also proposes an approximation of the inverse Hessian (in Eq. 2) that aims at reducing computations while maintaining performances. The proposed approach is evaluated on face image classification datasets, with several labels per image (including protected variables as gender or age), with a comparison with other fairness-enforcing approaches. Results show the capacity of the approach to maintain a fairness level comparable to the other approaches and baselines evaluated experimentally, while improving the accuracy (sometime significantly) over the datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Overall, the paper appears great and relevant to me. The main weakness is in the scope of the experiments, which is quite limited to image classification over face images. I get that these datasets are commonly available and fit for the type of experiments presented in the paper, but I think it would be a good improvement if other types of data can be also evaluated with the approach. It would support the fact that the proposed approach has a broad scope of applicability. Medical imaging datasets might be an interesting domain for that purpose, given that it can remain in the topics of image classification, using the same backbone architecture, and having significant fairness considerations, while dealing with types of images significantly different from faces.\n\nAlso, the following papers are relevant to the topic but not cited in the manuscript, so I suggest adding them to the literature review.\n\nChangjian Shui, Gezheng Xu, Qi Chen, Jiaqi Li, Charles Ling, Tal Arbel, Boyu Wang, and Christian Gagné. \"On learning fairness and accuracy on multiple subgroups.\" NeurIPS 2022. https://arxiv.org/abs/2210.10837\n\nJoshua K. Lee, Yuheng Bu, Deepta Rajan, Prasanna Sattigeri, Rameswar Panda, Subhro Das, and Gregory W. Wornell. \"Fair selective classification via sufficiency.\" In International Conference on Machine Learning (ICML), pp. 6076-6086. PMLR, 2021.\nhttps://proceedings.mlr.press/v139/lee21b.html\n", "broader_impact_concerns": "The ethical implications are fine, this is in fact a work that supports approaches for deployment of fairness-aware AI algorithms.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "EkWPnp6csN", "reviewer": "Reviewer_c8WV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is very well written and is a pleasure to read.\n2. The proposed method is theoretically justified and also effective in practice. The idea of leveraging generated data seems to be novel and reasonable.\n3. The empirical evaluation justifies the claim and extensive ablation studies help to explain the mechanism.\n\nWeaknesses:\nI didn't find major weaknesses in this submission. Disclaimer: I am not actively working in fairness training so I may miss some context or related work. But there are some minor weaknesses.\n1. I found the paper may lack discussion on the training overhead, especially given the extra cost of computing higher-order derivatives in Eqn. (3) and leveraging generative models. I can expect that the training time could be longer than baseline methods, which is acceptable, but this discussion should exist in the paper.\n2. It appears to have no reason to put the comparison between FairnessGAN and PairAug in Appendix B.4. Please discuss the reason or move it to the main text and update the findings accordingly.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provides an effective training method to promote distributional fairness across different groups (in the notions of EO and DP) by dynamic data ratio adjustment (based on gradients from implicit function theorem and identity matrix approximation) and generated data (from diffusion model with classifier guidance). Experiments on CelebA and ImageNet people subtree show superior fairness or accuracy compared to various types of existing methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weaknesses 1 and 2.", "broader_impact_concerns": "The paper contains a Broader Impact Statement that has adequately discussed ethical implications in my opinion.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers commended that the paper presentation is clear and the proposed method provides solid experimental results for justifying its effectiveness.

Reviewers pointed out the approach is not completely novel which is fine according to TMLR review criteria.

Reviewers also raised questions regarding specific technical points (e.g., how does the proposed approach achieve what kind of fairness, and some specific experimental results). Author feedback addressed most of the questions according to the reviewers.

In final camera ready, I suggest the authors to incorporate the suggestions from the reviewers.
Decision: Accept (poster)