FairGrad: Fairness Aware Gradient Descent
Authors: Gaurav Maheshwari
Michaël Perrot
Venue: Submitted to ICLR 2023 (2023)
Keywords: Group Fairness
Optimization

Abstract: We address the problem of group fairness in classification, where the objective is to learn models that do not unjustly discriminate against subgroups of the population. Most existing approaches are limited to simple binary tasks or involve difficult to implement training mechanisms. This reduces their practical applicability. In this paper, we propose FairGrad, a method to enforce fairness based on a reweighting scheme that iteratively learns group specific weights based on whether they are advantaged or not. FairGrad is easy to implement and can accommodate various standard fairness definitions. Furthermore, we show that it is competitive with standard baselines over various datasets including ones used in natural language processing and computer vision.

Under review as a conference paper at ICLR 2023
FAIRGRAD: FAIRNESS AWARE GRADIENT DESCENT
Anonymous authors
Paper under double-blind review
ABSTRACT
We address the problem of group fairness in classification, where the objective is learn models that do not unjustly discriminate against subgroups of the pop- ulation. Most existing approaches are limited to simple binary tasks or involve difficult implement training mechanisms. This reduces their practical applica- bility. In this paper, we propose FairGrad, a method to enforce fairness based on a reweighting scheme that iteratively learns group specific weights based on whether they are advantaged or not. FairGrad is easy to implement and can ac- commodate various standard fairness definitions. Furthermore, we show that it is competitive with standard baselines over various datasets including ones used in natural language processing and computer vision.
INTRODUCTION
Fair Machine Learning addresses the problem of learning models that are free of any discriminatory behavior against a subset of the population. For instance, consider a company that develops a model to predict whether a person would be suitable hire based on their biography. A possible source of discrimination here can be if, in the data available to the company, individuals that are part of subgroup formed based on their gender, ethnicity, other sensitive attributes, are consistently labelled as unsuitable hires regardless of their true competency due to historical bias. This kind of discrimination can be measured by fairness notion called Demographic Parity (Calders et al., 2009). If the data is unbiased, another source of discriminate may stem from the model itself that consistently mislabel the competent individuals of subgroup as unsuitable hires. This can be measured by faimesss notion called Equality of Opportunity (Hardt et al. 2016).
Several such fairness notions have been proposed in the literature as different problems call for different measures. These notions can be divided into two major paradigms, namely (i) Individual Fairness (Dwork et al., 2012; Kusner et al., 2017) where the idea is to treat similar individuals similarly regardless of the sensitive group they belong to, and (ii) Group Fairness (Calders et al.) 2009 Hardt et al., 2016; Zafar et al. 201 Denis et al. 2021) where the underlying idea is that different sensitive groups should not be disadvantaged compared an overall reference population. In this paper, we focus on group fairness in the context of classification.
The existing approaches for group faimess in Machine Learning may be divided into three main paradigms. First, pre-processing methods aim at modifying a dataset to remove any intrinsic unfair- ness that may exist the examples. The underlying idea that a model learned on this modified data is more likely to be fair (Dwork et al. 2012): Kamiran & Calders 2012 Zemel et al. 2013
Feldman et al. 2013 Calmon et al.) 2017p. Then, post-processing approaches modily the predic- tions of an accurate but unfair model so that it becomes fair (Kamiran et al. 2010 Hardt et al.
2016 Woodworth et al. 2017 Iositidis et al., 2019 Chzhen al.) 2019). Finally, in in-processing methods aim at learning model that is fair and accurate in a single step (Calders & Verwer 2010
Kamishima et al. 2012) Goh et al. 2016: Zafar et al.) 201 7a b Donini et 2l.180188 Krasanakis Kraaannaass et al.
2018 Agarwal et al. 2018 Wu al. 2019 Cotter et al. 2019 Iosifidis & Ntoutsi 2019 Jiang &
Nachum 2020 Lohaus et al. 2020 Roh et al. 2020 Ozdayi et al.) 2021I. this paper, we propose
in- in-procesinng approach based a reweighting scheme that may also be used as a kind of
post-processing approach by fine-tuning existing classifiers.
Motivation. In-processing approaches be further divided into several sub-categories Catton & Haas, 2020). Common amongst them are methods that relax the fairness constraints under consid-
eration simplify the learning process (Zatar et al. 2017a Donini et al. 2018 Wu et al.) 2019

---

Under review as a conference paper at ICLR 2023
The library is available as part of the suppl material. from fairgrad.tore import Crosstntropyloss
Same as PyTorch's 's loss with some additional meta data.
A fairness rate of 0.01 is a good rule of thumb for standardized data. criterion = CrossEntropyloss (y_train, s_train,
fairness_measure
fairness_rate=0, 01)
The dataloador and model are defined and used in the standard way, for x, Yr 8 in data_loader:
loss criterion (model (x), , yr loss. baskwaackw)rd)) optimizer.stepi
(model)
Figure A standard training loop where the PyTorch's loss is replaced by FairGrad's loss.
Indeed, standard fairness notions are usually difficult handle as they are often non-convex and non-differentiable. Unfortunately, these relaxations may be far from the actual fairness measures, leading to sub-optimal models (Lohaus et al.) 2020). Similarly, several approaches address the fair- ness problem by designing specific algorithms and solvers. This is, for example, done by reducing the optimization procedure
simpler problem (Agarwal et al.] 2018), altering the underlying solver (Cotter et al., 2019), or using adversarial learning (Raff & Sylvester 2018). However, these approaches are often difficult to adapt to existing systems as they may require special training pro- cedures changes in the model. They are also often limited in the range of problems to which they can be applied (binary classification, two sensitive groups, ...). Furthermore, they may come with several hy perparameters that need to be carefully tuned obtain fair models. The complexity of the existing methods might hinder their deployment in practical settings. Hence, there a need for simpler methods that are straightforward to integrate in existing training loops.
Contributions. In this paper, we present FairGrad, a general purpose approach to enforce fairness for gradient descent based methods. We propose dynamically update the weights of the examples after each gradient descent update to precisely reflect the fairness level of the models obtained at each iteration and guide the optimization process a relevant direction. Hence, the underlying idea is to use lower weights for examples from advantaged groups than those from disadvantaged groups.
Our method inspired by recent reweighting approaches that also propose change the importance of each group while learning a model (Krasanakis et al. 2018 Iositidis & Ntoutsi 2019 Jiang & Nachum 2020 Roh al., 2020 Ozdayi al.) (022)) We discuss these works in Appendix A
A key advantage of FairGrad is that it is straightforward to incorporate into standard gradient based solvers that support examples reweighing like Stochastic Gradient Descent. Hence, we developed a Python library (provided in the supplementary material) where we augmented standard Py PyTorch losses to accommodate our approach. From a practitioner point of view, it means that using FairGrad is as simple replacing their existing loss from PyTorch with custom loss and passing along some meta data, while the rest of the training loop remains identical. This is illustrated in Figure I extra
It is interesting note that FairGrad only brings the usual optimization ones (learning rates, batch size, ...).
the faimess rate, besides
Another advantage of Fairgrad is that, unlike the existing reweighing based approaches which of- ten focus on specific settings, it is compatible with various group fairness notions, including exact and approximate fairness, can handle both multiple sensitive groups and multiclass problems, and can fine tune existing unfair models. Through extensive experiments, we show that, in addition to its versatility, FairGrad competitive with several standard baselines in fairness on both standard datasets as well as complex natural language processing and computer vision tasks.
2 PROBLEM SETTING AND NOTATIONS
In the remainder of this paper, assume that have access a feature space X, finite discrete label space V, and a set of values for the sensitive attribute. We further assume that there exists an unknown distribution D E Dz where Dz is the set of all distributions over Z = X x y x S and
2

---

Under review as a conference paper at ICLR 2023
that we only get to observe finite dataset T ((1,, Mi, 8;)8; of n examples drawn i..d. from D. Our goal is then to learn an accurate model ho E H. with learnable parameters 0 E IR,, such that fair with respect to a given fairness definition that depends on the sensitive attribute. In Section 2.1 we formally define the fairness measures that are compatible with our approach and provide several examples of popular notions that are compatible with our method. Finally, for the ease of presentation, throughout this paper slightly abuse the notation P (E) and use it to represent both the true probability an event and its estimated probability from a finite sample.
hg : X +
2.1 FAIRNESS DEFINITION
We assume that the data may be partitioned into K disjoint groups denoted Ti,..., Tky-... Tr such that 0 Tk T and k Th = Q. These groups highly depend on the fairness notion under consideration. They might correspond to the usual sensitive groups, as in Accuracy Parity (see Example []. or might be subgroups of the usual sensitive groups, as in Equalized Odds (see Example 2 in the appendix). For each group, we assume that we have access to function Fs advantaged and F& < when the group
Dz x H + R such that F& > when the group disadvantaged. Furthermore, we assume that the magnitude of P' represents the degree to which the group (dis)advantaged. Finally, we assume that each F& can be rewritten as follows:
K
FT ho) = C2 +
Σ Pho(x)) + TTTT)) K=1
(1)
where the constants C are group specific and independent of hgg The probabilities P (ho(z) * T(T))) represent the error rates of ha(2) over each group TR with a slight abuse of notation. Below, we show that Accuracy Parity (Zafar et al. (017a) respects this definition. In Appendix B we show that Equality Opportunity (Hardt et al. 2016), Equalized Odds (Hardt et al. 2016), and Demographic Parity (Calders et al. 2009) also respect this definition.
Example (Accuracy Parity (AP) (Zafar et al., 2017a)). model hg is fair for Accuracy Parity when the probability of being correct is independent of the sensitive attribute, that is, Vr E S
IP ( = y s r) IP (ha(x) (x) = =)
means that we need partition the space into K [ groups and, Vr E S, we define F(r) as the fairness level of group (r)
(T, ho) PP (ho (x:) + v) - (ho + r)
(IP (x r) 1)P (ho ky +
Σ ((s ( ()-=r'P(hho(z)) + y/s =r) (+))(()
where the law of total probability was used to obtain the last equality. Thus Accuracy Parity satisfies
all our assumptions with c(+) =P r) 1, ckpo) (s = r') with r' ,' r, and Ci, 0.
3
FAIRGRAD
In this section, we present FairGrad, the main contribution of this paper. We begin by discussing FairGrad for exact fairness and then present an extension to handle e-fairness.
3.1 FAIRGRAD FOR EXACT FAIRNESS
To introduce our method, we first start with the following optimization problem that is standard in fair machine learning (Cotter et al. 2019
arg min IP (ho(x) + y) ROEM
s.t. Vk [K, Fi (T, ho) 0.
(2)
Then, using Lagrange multipliers, denoted λ1,... ÅR. obtain an unconstrained objective that should be minimized for hg E and maximized for Ag....,,AK E IR:
K
L (ho, Aho..,..... ,,,...,Axk = (ho(x) # v) + Σ (T)
(3)
3

---

Under review as a conference paper at ICLR 2023
To solve this problem, propose to use alternating approach where the hypothesis and the multipliers are updated one after the other
Updating the Multipliers. To update A....., Ag. we will use a standard gradient ascent proce- dure. Hence, given that the gradient of Problem is
FT(T ha)
V Aa L (hg. A1. ......,
Fx (T((To1))
we have the following update rule vk [K]]
x = X FA (T,,F((T,,, (T,
where 77 is rate that controls the importance of each update. In the experiments, fairness rate of 0.01 as our initial tests showed that it is a good rule thumb when the data is properly standardized.
constant
Updating the Model. To update the parameters 0 E R of the model hg, we use standard gradient descent approach. However, first, we notice that given the fairness notions considered, Equation 3 can be rewritten
K + Σ
L (1og AK)
Σ (ho( Tk)
((T))
A
+
where ΣK Ag of is independent of he by definition. Hence, iteration the update rule becomes
K
K
0+ = TT - 170
Σ
(
P
- Σ
A (ho(r) # Tx)
1=
k==
where is the usual learning rate that controls the importance of each parameter update. Here, we obtain our group specific weights V4, wy = [P (Tx) + +K= A that depend on the current fairness level of the model through Å1 . the relative size of each group through P (Tk), and the fairness notion under consideration through the constants C. The exact values of these constants are given Section 2.1 and Appendix B for various group fairness notions. Overall, they are such that, each iteration, the weights of the advantaged groups are reduced and the weights of the disadvantaged groups are increased.
The main limitation of the above update rule is that one needs compute the group-wisg wise gradients VoP (z) * v/Tx) = E(a,y)eT, (ha[))(()))))) Here, I(Na(=) is the indicator function, also called the 0 - 1-loss, that is 1 when ha (x) # and O otherwise. Unfortunately, this usually does not provide meaningful optimization directions. To address this issue, we follow the usual trend in machine learning and replace the 0 1-loss with one of its continuous and differentiable surrogates that provides meaningful gradients. For instance, our experiments, we use the cross entropy loss.
3.2 COMPUTATIONAL OVERHEAD OF FAIRGRAD.
We summarize our approach in Algorithm ! We consider batch gradient descent rather than full gradient descent as it is a popular optimization scheme. We empirically investigate the impact of the batch size Section 4.7 We use italic font to highlight the steps inherent FairGrad that do not appear in classic batch gradient descent. The main difference is Step that is the computation of the fairness levels for each group. However, these can be cheaply obtained from the predictions of h on the current batch which are always available since they are also needed to compute the gradient. Hence, the computational overhead FairGrad is very limited.
It is worth noting that, here, we do not have formal duality guarantees and that the problem is not even guaranteed to have fair solution. Nevertheless, the approach seems to work well in practice as can he seen in the experiments.

---

Under review as a conference paper at ICLR 2023
Algorithm FairGrad for Exact Fairness
Input: Groups ......, Tk. Functions Fi...., Fk. Function class H of models he with parameters 0 E RD Learning rates Te,, and Iterator iter that returns batches of examples. Output: fair model h2.
1: Initialize the group specific weights and the model.
2: for B in iter do
3: Compute the predictions of the current model on the batch
4: Compute the group-wise losses using the predictions.
5:
Compute the current fairness level using the predictions and update the group-wise weights.
6: Compute the overall weighted loss using the group-wise weights.
7: Compute the gradients based on the loss and update the model.
8: end for
9: return the trained model h3
3.3 IMPORTANCE OF NEGATIVE WEIGITTS.
key property of FairGrad is that we allow the use of negative weights throughout the optimization process, that is (Ta. E C strict themselves to positive weights (Roh et al. 2020 Iositidis & Ntoutsi 2019: Jiang & Nachum 2020). In this Section, we show that these negative weights are important as they are sometimes necessary to learn fair models. Hence, in the next lemma, we provide sufficient conditions so that negative weights are mandatory if one wants to enforce Accuracy Parity.
may become negative, while existing methods often
Lemma (Negative weights are necessary.). Assume that the fairness notion under consideration is Accuracy Parity (see Example 7}. Let h2 be the most accurate and fair model. Then using negative weights is necessary as long
min max 12 (ha(x) # v(Th) < P (hō(()) $ v).
h Te haunfair
Proof. The proof is provided in Appendix
The previous condition sometimes be verified in practice. As motivating example, assume a binary setting with only two sensitive groups Ti and T-1- Let hi be the model minimizing P (1)(x) # v|T and assume that P (he (x) # y) < IP (he (x) * y|T-1), that is group T-1 is disadvantaged for accuracy parity. Given hn the most accurate and fair model, we have
min max P (ho(x) + yl Th) (hā" (x) # = EH
P (h) + +))
Ts
hgunfair
as otherwise would have a contradiction since the fair model would also be the most accurate model for group T-1 since P (a)( # y) IP (x) * y|T-1) by definition of Accuracy Parity. In other words, a dataset where the most accurate model for a given group still disadvantages this group requires negative weights.
3.4 FAIRGRAD FOR C-FAIRNESS
In the previous section, we mainly considered exact fairness and we showed that this could be achieved by using reweighting approach. In fact, we can extend this procedure to the case of e- fairness where the fairness constraints are relaxed and controlled amount of violations allowed. Usually, is user defined parameter but it can also be set by the law, it is the case with the 80% rule the US. The main difference with the exact fairness case is that each equality constraint in Problem ( is replaced with two inequalities of the form
Yk: E []], FL(T, ho) S
Vk E KJ] FT he) -c.
The main consequence is that need to maintain twice many Lagrange multipliers and that the group-wise weights are slightly different. Since the two procedures are similar, we omit the details here but provide them in Appendix D for the sake of completeness.
5

---

Under review as a conference paper at ICLR 2023
4 EXPERIMENTS
In this section, we present several experiments that demonstrate the competitiveness of FairGrad as a procedure to learn fair models in a classification setting. We begin by presenting results over standard fairness datasets and Natural language Processing dataset in Section 4.4 We then study the behaviour of the e-fairness variant of FairGrad in Section 4.5 Next, we showcase the fine-tuning
ability of FairGrad Computer Vision dataset in Section 4.6 Finally, investigate the impact
of batch size on the learned model in Section 4.7
4.1 DATASETS
In the main paper, consider 4 different datasets and postpone the results another 6 datasets to Appendix E as they follow similar trends. We also postpone the detailed descriptions of these datasets as well the pre-processing steps.
On the one hand, we consider commonly used fairness datasets, namely Adult Income Kohavi 1996) and CelebA (Liu al. 2015). Both are binary classification datasets with binary sensitive attributes (gender). We also consider a variant of the Adult Income dataset where we add a second binary sensitive attribute (race) obtain a dataset with 4 disjoint sensitive groups.
On the other hand, to showcase the wide applicability of FairGrad, we consider the Twitter Senti- ment- Blodgett al. 2016) dataset from the Natural Language Processing community. consists of 2004 tweets with binary sensitive attribute (race) and binary sentiment score. We also employ the UTKFace datasef" (Zhang al., 2017) from the Computer Vision community. It consists of 23, 708 images tagged with race, age, and gender.
4.2 PERFORMANCE MEASURES
For fairness, we consider the four measures introduced in Section 2.1 and Appendix B. namely Equalized Odds (EOdds), Equality of Opportunity (EOpp), Accuracy Parity (AP), and Demographic Parity (DP). For each specific fairness notion, we report the average absolute fairness level of the different groups over the test set, that is & ΣK (T, he)l (lower is better). To assess the utility of the learned models, we use their accuracy levels over the test set, that 1 is better). All the results reported are averaged over 5 independent runs and standard deviations are provided. Note that, in the main paper, we graphically report a subset of the results over the aforementioned datasets. We provide detailed results in Appendix F including the missing pictures as well as complete tables with accuracy levels, fairness levels, and faimess level of the most well-off and oorst-off groups for all the relevant methods.
K
n Σ;= I=f (higher
4.3 METHODS
We compare FairGrad to 6 different baselines, namely (i) Unconstrained, which is oblivious to any fairness measure and trained using a standard batch gradient descent method, (ii) an Adversarial mechanism (Goodfellow et al. 2014) using a gradient reversal layer (Ganin & Lempitsky 2015).
similar to GRAD-Pree (Raff & Sylvester 2018), where an adversary, with an objective to predict the sensitive attribute, is added to the unconstrained model, (iii) BiFair (Ozdayi et al.) (021), (iv) FairBatch (Roh et al., 2020), (v) Constraints (Cotter et al., 2019). non-convex constrained opti- mization method, and (vi) Weighted ERM where each example is reweighed based on the size of the sensitive group the example belongs to.
In all our experiments, we consider two different hypothesis classes. On the one hand, we use linear models implemented in the form of neural networks with no hidden layers. On the other hand, we use more complex, non-linear architecture with three hidden layers of respective sizes 128, 64, and 32. We use ReLU as our activation function with batch norm normalization and dropout set to 0.2. In both cases, we optimize the cross-entropy loss. We provide the exact setup and hyper--aramen tuning details for all the methods in Appendix E.
http:://langllhh.c. www.s.aasssmmmmmmmmm...o...o.oo wwwaaass.sc////ee Adtu//wittrrAA//
https://uusanggggiiuuuuuu wwtp:::/wggggguhtooo/ot///t.

---

Under review as a conference paper at ICLR 2023
In several experiments, we only consider subsets of the baselines due the limitations of the meth- ods. For instance, BiFair was designed to handle binary labels and binary sensitive attributes and thus is not considered for the datasets with Furthermore, we implemented it using the authors code that freely available online but does not include AP fairness measure, thus do not report results related to this measure for BiFair. Similarly, we also implemented FairBatch from the authors code which does not support AP as fairness measure, thus also exclude from the comparison for this measure. For Constraints, we based our implementation on the publicly available authors library but were only able to reliably handle linear models and thus do not consider this baseline for non- non-linear models. Finally, for Adversarial we used our custom made implementation. However, it is only applicable when learn- ing non-linear models since it requires at least one hidden layer to propagate its reversed gradient.
than two sensitive groups more than two labels.
4.4 RESULTS FOR EXACT FAIRNESS
We report the results over the Adult Income dataset using linear model, the Adult Income dataset with multiple groups with a non-line non-linear model, and the Twitter sentiment dataset using both linear and nonlinear models in Figures 2 3 respecsiyy..vll. In these figures, the best methods are closer the bottom right corner. reduced accuracy. Similarly. a method closer to the top right corner has good accuracy but poor fairness, that is is close to the unconstrained model.
method closer the bottom left corner, it has good fairness but
The main take-away from these experiments is that there is no faireess enforcing method that is consistently better than the others. All of them have strengths, that datasets and fairness measures where they obtain good results, and weaknesses, that is datasets and fairness measures for which they are sub-optimal. For instance, FairGrad achieves better fairness levels for EOdds and EOpp over the Adult dataset with linear model. However, it pays price in terms of accuracy in those settings. Similarly, FairBatch induces better accuracy than the other approaches over Adult with linear model and EOdds and only pays small price in terms of fairness. However, it is significantly worse in terms of fairness over the Adult Multigroup dataset with a non-linear model. Finally. BiFair sub-optimal Adult with EOpp. while being comparable the other approaches the Twitter Sentiment dataset. We observed similar trends on the other datasets, available in Appendix with different methods coming on top for different datasets and fairness measures.
Fak-Grad
FaleGind
Fal- Bai-Gra
Accuraty Parity
Weighted
Fai-Batch
Unconstrated
Equal Opportunity
0.0G
Constrinnt
Odds
0.0G
Weighted
B.0G
Weighted
0.94
0.04
BiFair
Conetrints
0.04
BiFair
Constrints
0.03
0.03
0.00
0,02
0,022
11/02
0.00
0.00
0.0L
0.00
0.00
0.00
0.77
0.81
0.83
0.84
0.86
0.77
0.79
0.83
0.84
0.06
0.77
0.81
0.83
0.84
0.86
Accuracy
Accuracy
Accuracy
(a) Linear- AP
(b) Linear -EOdss
(c) Linear EEOp
Figure 2: Results for the Adult dataset using Linear Models.
0.00
Fai-Grad
0.00
Fai-Grrd
0.00
Fai-Grad
Accuracy Parity
Fai-Batch
Equal Opportunity unni
0.00
1..07
Weighted Erm
Adversarial
Oela
(1.06
(1.60'
Adversarial
Unconstrareed
0.00
BLA
Adversarial
Unconstraited
0.05
0.05
Weightod Erm
0.05
Weighed Erm
0.04
0.04
0.04
0,02
IJ02
1.02
0.00
0.00
0.00
0.69
0.86
0.72
0.79
0.82
0.69
0.76
0.79
0.86
Accuracy
Accuracy
Accuracy
(a) Non Linear- AP
(b) Non Linear- FOdds
(c) Non Linear- EOpp
Figure 3: Results for the Adult Multigroup dataset using Non Linear models.
7

---

068 Under review as a conference paper at ICLR 2023
11:30
FabGrad
Fak
Fak FarBatch
Unconstrahrod Weighted
FaiBatch
Equal Ogportunity
Accuracy Parity
0.16
Equal Ordss
11.15
Usconstaaeed
Weighted
0.15
Weighted
Usconstaanee
0.10
0.10
0.10
BiFair
BiFair
0.05
0.05
0.06
0.69
0.71
0.72
0.74
0.75
0.77
0.69
0.71
0.72
0./4
0.75
0.77
0.69
0.71
0.12
0.74
0.75
0.77
Accuracy
Accuracy
Accuracy
(a) Linear- AP
(b) I Linear FOdds
(c) I Linear EOpp
Fai-Grad
Fai
FairGrad
Accuracy Parity
0.15
Adversarial
Weighted Erm
Equal Ddds
Fai-Batch
Uncomttmnned
Adversarial
Equal Opportunity
0.15
Adversarial
Uncomtminee
0.30
0.30
BiFair
Weightod Erm
0.30
Weighted Erm
0.06
11.06
1..0
0.00
0.00
0.00
0.69
0.74
0.69
Accuracy
Accuracy
Accuracy
(d) Non Linear AP
(e) Non Linear
EOdds
(f) Non Linear- EOpp
Figure 4: Results for the Twitter Sentiment dataset for Linear and Non Linear Models.
4.5 ACCURACY FAIRNESS TRADE-OFF
0.14
6.00 0.0200
8.8
0.05
3.0
8.68
1.30
Accurass yuurryy Parity
8.83
44x
3/00
Equal opportunii
6.00
3.53
8.84
3.09
8.0
0.03
5.00
191 1.00
1.0
0.02000
0.07
0.02
00000
0.00
0.85
0.540
11.15
0,550
0.535
0.545
0.545
0.80
0.550
Accuracy
Accuracy
Accuracy
(a) Celeb AP
(b) Celeb EOdds
(c) Celeb EOpp
Figure 5: Results for CelebA with different faimess measure using Linear models. The Uncon- strained Linear model achieves a test accuracy 0.8532 with fairness level of 0.0499 for EOdds, 0.0204 for AP, and 0.0387 for EOpp.
In this second set experiments, we demonstrate the capability FairGrad to support approximate fairness (see Section B.4). In Figure 5 we show the performances, accuracy-fairner pairs, of several models learned on the CelebA dataset by varying the fairness level parameter e. These results suggest that FairGrad respects the constraints well. Indeed, the average absolute fairness level (across all the groups, see Section 4.2) achieved by FairGrrad is either the same or less than the given threshold. It is worth mentioning that FairGrad is designed to enforce c-faimess for each constraint individually which is slightly different from the summarized quantity displayed here. Finally, the fairness constraint relaxed, the accuracy of the model increases, reaching the same performance as the Unconstrained classifier when the fairness level of the latter is below e.
4.6 FAIRGRAD AS FINE- TUNING PROCEDURE
While FairGrad has primarily been designed to learn fair classifiers from scratch, it can also be used to fine-tune an existing classifier to achieve better faimess.. To showcase this possibility, fine-tune the Re 8 (He et al. 2016) model, developed for image recognition, over the UTKFace dataset (Zhang et al. 2017), consisting of human face images tagged with Gender, Age, and Race information. Following the same process as Roh et al. (2020), we use Race as the sensitive attribute and consider two scenarios where either the gender (binary) with Demographic Parity the fairness
8

---

Under review as a conference paper at ICLR 2023
Table 1: Results for the UTKFace dataset where a 8 is fine-tuned using different strategies.
s=Race y=Gender
R=Rc y=Age
Method
Accuracy
DP
Accuracy
EOdds
Unconstrained
0.8691 0.0075 0.0448 + 0.0066
0.6874 = 0.0080 0.0843 0.0089
FairGrad
0.8397 + 0.0085 0.0111 + 0.0064
0.6491 + 0.0082 0.0506 + 0.0059
Table 2: Effect of the batch size the CelebA dataset with Linear Models and EOdds as the fairness measure.
Batch Size
16
32
64
128
256
512
1024
2048
Accuracy
0.8186
0.8234
0,8215
0.8268
0.8273
0.8286
0.8292
0.8289
0.8303
Accuracy Std
0.0013
0.006
0.0028
0.0025
0.0031
0.0008
0.0027
0.0017
0.0031
Fairness
0,0031
0.0091
0.0045
0.0036
0.0051
0.0046
0.004
0.0038
0.0057
Fairness Std
0.0042
0.0062
0.0012
0.0014
0.0025
0.0032
0.0026
0.0019
0.0018
with Equalized Odds as fairness measure used the target label. The results are displayed Table] [ In both settings, FairGrad is able to learn models that are more fair than Unconstrained fine-tuning procedure, albeit at the expense of accuracy.
measure age
4.7 IMPACT OF THE BATCH-SIZE
In this last set of experiment, we evaluate the impact of batch size on the fairness and accuracy level of the learned model. Indeed, each iteration, in order minimize the overhead associated with FairGrad (see Section 3.I. we update the weights using the fairness level of the model estimated solely on the current batch. When these batches are small, these estimates are unreliable and might lead the model astray. This can be observed in Table where we present the performances of several linear models learned with different batch sizes on the CelebA dataset. small batch sizes, the learned models tends to have slightly lower accuracy and larger standard deviation fairness levels. On the other hand, with a sufficiently large batch size, in this case 64 and above, the learned models close to be perfectly fair. Furthermore, they obtain reasonable levels of accuracy since the Unconstrained model has an accuracy of 0.8532 for this problem.
the one hand, for very
CONCLUSION
In this paper, we proposed FairGrad, a fairness aware gradient descent approach based on a reweight- ing scheme. We showed that it can be used to learn fair models for various group fairness definitions and is able to handle multiclass problems as well settings where there is multiple sensitive groups. We empirically showed the competitiveness of our approach against several baselines on standard fairness datasets and on a Natural Language Processing task. We also showed that it can be used to fine-tune an existing model on a Computer Vision task. Finally, since it is based on gradient descent and has a small overhead, we believe that FairGrad could be used for a wide range of applications. even beyond classification.
Limitations and Societal Impact. While appealing, FairGrad also has limitations. It implicitly assumes that set of weights that would lead to a fair model exists but this might be difficult to verify in practice. Thus, even if in our experiments FairGrad seems to behave quite well, a prac- titioner using this approach should actual fairness level of the learned model. On the other hand, we believe that, due to its simplicity and its versatility, FairGrad could be easily deployed various practical contexts and, thus, could contribute to the dissemination of fair models.
trust blindly. It remains important always check the
REFERENCES
Alckh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. re- ductions approach fair classification. In International Conference Machine Learning, pp-

---

Under review as a conference paper at ICLR 2023
60 69. PMLR, 2018.
Su Lin Blodgett, Lisa Green, and Brendan O'Connor. Demographic dialectal variation in social media: case study of African-American English. 2016.
Toon Calders and Sicco Verwer. Three naive bayes approaches for discrimination--fee classification. Data mining and knowledge discovery, 21 2010.
Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building classifiers with independency
constraints. In 2009 IEEE International Conference Data Mining Workshops, pp. 13-18.
IEEE, 2009.
Flavio P Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. Optimized pre- reeprooessing for discrimination prevention. volume 30, 2017.
Simon Caton and Christian Haas. Fairesss in machine learning: A survey. arXiv preprint
arXiv:2010.04053, 2020.
Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimiliano Pontil. Leveraging labeled and unlabeled data for consistent fair binary classification. arXiv preprint 1wXiv::906600002,, 2019.
Andrew Cotter, Heinrich Jiang, and Karthik Sridharan. Two-player games for efficient non-convex constrained optimization. In Algorithmic Learning Theory. pp. 300-332. PMLR, 2019.
Christophe Denis, Romuald Elie, Mohamed Hebiri, and François Hu. Fairness guarantee in multi-
class classification. arXiv preprint arXiv:2109. 13642, 2021.
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt, Retiring adult: New datasets for fair machine learning. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 6478-6400 2021. URL http:///pccccnnssseeessscccppppp 1/hash/ Abstract..ttrt
Michele Donini, Luca Oneto, Shai Ben-David, John Shawe-Taylon Taylor, and Massimiliano Pontil. Em- pirical risk minimization under faimess constraints. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 2796-2806, 2018
Dheeru Dua, Casey Graff, et al. Uci machine learning repository. 2017.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings the 3rd innovations theoretical computer science confer- ence, pp. 214-226, 2012.
Yanai Elazar and Yoav Goldberg. Adversarial removal of demographic attributes from text data. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ichi Tsujii (eds.). Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 November 4, 2018, pp. 11-21. Association for Computational Linguistics, 2018.
Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Martha Palmer, Rebecca Hwa, and Sebastian Riedel (eds.), Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pp. 1615-1625. Association for Computational Linguistics, 2017.
Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubra- manian. Certifying and removing disparate impact. In proceedings of the 2 1th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 259-268, 2015.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference Ma- chine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 1180-1189, Lille, France, 07-09 Jul 2015. PMLR.
10

---

Under review as a conference paper at ICLR 2023
Gabriel Goh, Andrew Cotter, Maya Gupta, and Michael P Friedlander. Satisfying real-world goals with dataset constraints. In Advances in Neural Information Processing Systems, pp. 2016.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farles Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Xudong Han, Timothy Baldwin, and Trevor Cohn. Diverse adversaries for mitigating bias in train- ing. In Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty (eds.), Proceedings of the 16th Confer- ence of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19 23, 2021, pp. 2760-2765. Association for Computational Linguis- tics, 2021.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29:3315-3323,, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Vasileios Iosifidis and Eirini Ntoutsi. Adafair: Cumulative fairness adaptive boosting. In Proceed- ings of the 28th ACM International Conference on Information and Knowledge Management, pp- 781-790, 2019.
Vasileios Iosifidis, Besnik Fetahu, and Eirini Ntoutsi. Fae: A fairness-aware ensemble framework. In 2019 IEEE International Conference on Big Data (Big Data), pp. 1375-1380. IEEE, 2019.
Heinrich Jiang and Ofir Nachum. Identifying and correcting label bias in machine learning. In International Conference on Artificial Intelligence and Statistics, pp. 702-712. PMLR, 2020.
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrim- ination. Knowledge and Information Systems, 33(1):1-33, 2012.
Faisal Kamiran, Toon Calders, and Mykola Pechenizkiy. Discrimination aware decision tree learn- ing. In 2010 IEEE International Conference on Data Mining, pp. 869-874. IEEE, 2010.
Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-aware classifier with prejudice remover regularizer. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 35-50. Springer, 2012.
Ron Kohavi. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In Evangelos Simoudis, Jiawei Han, and Usama M. Fayyad (eds.), Kdd, pp. 202-207, AAAI Press, 1996.
Emmanouil Krasanakis, Eleftherios ppyromitros-Xioufi Symeon Papadopoulos, and Yiannis Kom- patsiaris. Adaptive sensitive reweighting to mitigate bias in fairness-aware classification. In Pro- ceedings of the 2018 World Wide Web Conference, pp- 853-862. 2018.
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual faimesss. Advances in neural information processing systems, 30, 2017.
Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. How we analyzed the compas recidi- vism algorithm. ProPublica (5 2016), 91):3-33, 2016.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face autributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Michael Lohaus, Michaël Perrot, and Ulrike Von Luxburg. Too relaxed to be fair. In International Conference on Machine Learning, pp. 6360-6369. PMLR, 2020.
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6):1-35, 2021.
Mustafa Safa Ozdayi, Murat Kantarcioglu, and Rishabh Iyer. Bifair: Training fair models with bilevel optimization. arXiv preprint arXiv:2106.04457, 2021.

---

Under review as a conference paper at ICLR 2023
Edward Raff and Jared Sylvester. Gradient reversal against discrimination: A fair neural network learning approach. In 2018 IEEE 5th International Conference on Data Science and Advanced Analyties (DSAA), Pp. 198. IEEE, 2018.
Michael Redmond and Alok Baveja. A data-driven software tool for enabling cooperative infor- mation sharing among police departments. European Journal of Operational Research, 141(3): 660-678, 2002.
Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. Fairbatch: Batch selection for model fairness. In International Conference on Learning Representations, 2020.
Blake Woodworth, Suriya Gunasekar, Mesrob Ohannessian, and Nathan Srebro. Learning non- discriminatory predictors. Conference on Learning Theory, pp. 1920-1953, PMLR, 2017.
Yongkai Wu, Lu Zhang, and Xintao Wu. On convexity and bounds of fairesss-awaee classification. In The World Wide Web Conference, pp. 3356-3362, 2019.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna Gummadi. Fair- ness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference on world wide web, pp. 1171
1180, 2017a.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fair- ness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics, pp- 962-970. PMLR, 201
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning, pp. 325-333. PMLR, 2013.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/reerrssion by conditional adversarial autoencoder. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5810-5818, 2017.
Indre Žliobaite, Faisal Kamiran, and Toon Calders. Handling conditional discrimination. 2011 IEEE 11th International Conference Data Mining, pp. 990-1001. IEEE, 2011.
12

---

Under review as a conference paper at ICLR 2023
In this appendix we provide several details that were omitted in the main paper. First, in Section A we review several works related to ours. Then, Section B we show that several well known group fairness measures compatible with FairGrad. In Section prove Lemma 1. Next, in Section[D D we derive the update rules for FrirGrad with e-fairness. Finally, in Section additional experiments.
we provide
A RELATED WORK
The fairness literature extensive and we refer the interested reader to recent surveys (Caton & Haas 2020; Mehrabi al. (021) get an overview of the subject. Here, focus recent works that are more closely related to our approach.
BiFair (Ozdayi et al., 2021). This paper proposes a bilevel optimization scheme for fairness. The idea use an outer optimization scheme that learns weights for each example so that the trade-off between fairness and accuracy is as favorable as possible while an inner optimization scheme learns model that is accurate as possible. One of the limits of this approach that it does not directly optimize the fairness level of the model but rather a relaxation that does not provide any guarantees on the goodness of the learned predictor. Furthermore, it is limited to binary classification with binary sensitive attribute. In this paper, we also learn weights for the examples in an iterative way. However, we use a different update rule. Furthermore, we focus on proper fairness definitions rather than relaxations and our objective is to learn accurate models with given levels of faimess rather than trade-off between two. Finally, our approach is not limited to the binary setting.
FairBatch (Roh et al. 2020). This paper proposes a batch gradient descent approach that can be used to learn fair models. More precisely, the idea is to draw the batch examples from a skewed distribution that favors the disadvantaged groups by oversampling them. In this paper, we propose to use reweighting approach which could also be interpreted as altering the distribution of the examples based on their fairness level if all the weights were positive. However, we allow the use of negative weights, and we prove that they sometimes necessary to achieve fairness. Furthermore, we use a different update rule for the weights.
AdaFair (Tosifidis & Ntoutsi 2019). This paper proposes a boosting based framework to learn fair models. The underlying idea is to modify the weights of the examples depending both the performances the current strong classifier and the group memberships. Hence, examples that belong the disadvantaged group and are incorrectly classified receive higher weights than the examples that belong to the advantaged group and are correctly classified. In this paper, we use a similar high level idea but we use different weights that do not depend on the performance of the model. Furthermore, rather than a boosting based approach, we consider problems that can be solved using gradient descent. Finally, while AdaFair only focuses on Equalized Odds, we show that our approach works with several fairness notions.
Identifying and Correcting Label Bias in Machine Learning (Jiang & Nachum 2020). This paper considers the fairnsss problem from an original point view as it assumes that the observed labels are biased compared to the true labels. The goal is then to learn model with respect to the true labels using only the observed labels. To this end, the paper proposes to use an iterative reweighting procedure where positive weights for the examples and updated models are alternatively learned. In this paper, we also propose a reweighting approach. However, we use different weights that are not necessarily positive. Furthermore, approach is not limited to binary labels and handle multiclass problems.
B REFORMULATION OF VARIOUS GROUP FAIRNESS NOTION
In this section, present several group faimess notions which respect our faimess definition pre- sented in Section
Example (Equalized Odds (EOdds) (Hardt et al. 2016)). A model ho is fair for Equalized Odds when the probability of predicting the correct label is independent of the sensitive attribute,
13

---

Under review as a conference paper at ICLR 2023
that is, AT E y, Vr S
IP (ho( ( 1) =
(ho(a) = -1
It means that need to partition the space into K | S| groups and, AT E 3, Vr 8, we define F(u.)) as
(T. ho)) = IP (ha(a) # 1 = 1) - P (h)(2) # -|s=r,,
Σ
((s= = r = 1) (hg (z) = 1)
=
(L,))
(1 P (s rly P)) (1)) # 1. =))
where the law of total probability was used to obtain the last equation. Thus. Equalized Odds satisfies
all our assumptions with ((,r) = P (s rly 1) cl,r") F (s r' = 1). c 0 with
(1.r)
r' # r and l' + 1, and C0 (I.r) O.
Example 3 (Equality of Opportunity (EOpp) (Hardt et al. 2016)). A model hg is fair for Equality
of Opportunity when the probability of predicting the correct label is independent of the sensitive attribute for given subset )" C 3 of labels called the desirable outcomes, that is, VI E V", Vr E 8
IP (he (z) = [|s = r, = () = P (h)(2) = [1y = 1).
It means that need to partition the space into K {>/ S groups and, AI E V, E S, we we define F(rr)) as
{
(h)(v) = = P (ho(s)) -1y==
(T. ha)
4(,r) y
v(lr) E y x
which can then be rewritten in the correct form in the same way as Equalized Odds, the only differ- ence being that C(a,, = 0, v(1,r) E y x S )" x S.
Example 4 (Demographic Parity (DP) (Calders et al. 2009)). A model he is fair for Demographic Parity when the probability of predicting a binary label is independent of the sensitive attribute, that is, vl E V, Vr S
(h)(2)) = I|s = r) = F (x) = I).
It means that need to partition the space into K |>/ S groups and, VI E Vr E S, define F(i.)) as
(T, he) = PP (h)(x) ( r)
=
Σ P
((r))))))
(P (
1
Σ
IP PP( (===,,--/) P (ho - f
(T.))
PP (y==l) - F (y = i|s =-)
where the law of total probability was used to obtain the last equation. Thus, Demographic Par- ity satisfies all our assumptions with c(lrr)
(1.r) ( (y== = r) P (y
1| r). (
P = -,s=-r)) with r' # (. = p Pyy= = -P c(l.'')
(lr)
(1,r)
=
P Py= with r' # r, and C
(I.)) (y= = =] - P (u [ (===)).
C PROOF OF LEMMA
Lemma 2 (Negative weights are necessary.). Assume that the fairness notion under consideration
Accuracy Parity. Let hy be the most accurate and fair model. Then using negative weights is necessary long
min max P (ho(x) # T(Th) < P (l)((z) # v)
hoEH 7s
haunfoir
14

---

Under review as a conference paper at ICLR 2023
Proof. To prove this Lemma, one first need notice that, for Accuracy Parity, since Ek=I 1 we have that
P (Tx)
K
CK = (IP (Tx)
1) +
Σ
(Tx.)
0.
k K'gh
This implies that
K
K
Σ P (Tx) + Σ
c
AM
=1
This implies that, whatever choice of A, the weights will always sum to one. In other words, A₂ch 0 by definition, for a given hypothesis he, have that
since also have that 1
K P (ho(z) + y(TA)
K
max A
(T)))
+
ck
Ax
(5)
ER k=1
k'=1
max #1 RE
P (ha (x) + Tx) wn
(6)
e.t. w
where, given W1, wi...., WK, the original values of lambda can be obtained by solving the linear system C² = w where
Cl
Ck
A1
W1 (Tt)
c=
A =
w
C
AK
WK - P (Tk))
which is guaranteed to have infinitely many solutions since the rank of the matrix C' K - and the rank the augmented matrix (Clw) is also K 1. Here are using the fact that P (Ts) # 0, Ak since all the groups have to be represented to be taken into account.
We will now assume that all the weights are positive, that Wi 2 O, vk. Then, the best strategy to solve Problem (6) is to put all the weight on the worst off group k, that is set wh = 1 and
uk. 0, Vk. * It implies that
K Σ
max W1 WI......KE s.f. nx. w0 k=1
P (he(x) # TT))) we = max (he(x) + TTT)).
Furthermore, notice that, for fair models with respect Accuracy Parity, we have that P (1) # v(Ta) P (ho(x) # v). Vk. Thus, if it holds that
min max P (ho # vlTx) < TP (1()(z))))) haER
hounfair
where h2 is the most accurate and fair model, then the optimal solution of Problem (3) in the main paper will be unfair. It implies that, in this case, using positive weights is not sufficient and negative weights are necessary.
D FAIRGRAD FOR E-FAIRNESS
To derive FairGrad for e-fiimess we first consider the following standard optimization problem arg min (ha(x) + v) ho
S.t. VK E Fx ha) Se Vk [K], FT ho) -C.
15

---

In this section, we provide additional details related to the basclines and the procedure. We then provide descriptions of the datasets and finally the results. tuning Under review as a conference paper at ICLR 2023
We, once again, use standard multipliers approach to obtain the following unconstrained formula- tion:
K L (ho, At...., A ,t,.0,,,,,.... IP (1) v) + A (Fe(T, ho) -e) -d (TT hw) + E)
where . Ax and ó 6x are the multipliers that belong to R that is the set of positive reals. Once again, to solve this problem, we will use an alternating approach where the hypothesis and the multipliers are updated one after the other.
Updating the Multipliers. To update the values ....... Ag. we will use standard gradient as- cent procedure. Hence, noting that the gradient of the previous formulation is
FT
V A L (he, Ai...., AK, 81 (k.
(k((Tho)
Fi (T hha -
Sa C (hg, AI,..., ,,K,, 81 0x))
FK((TT (Tc(T,ho) .ho
we have the following update rule vk [K]
X T++1 max (0, AT + (T, - -()
6T+1 max (0, 0% 17 (T, hb) é))
where y) is a learning rate that controls the importance of each weight update.
Updating the Model. To update the parameters 0 RD of the model hg, we proceed as before, using a gradient descent approach. However, first, we notice that given the fairness notions that we consider, Equation is equivalent to
K
C (hg .... Sx.., 5k) =
P (he(a) # Tx)
P(T)) +
Σ ==1
C (A 6x:)
(8)
k=1
K
Σ
(Au + 5x) e
+
(Dx
k=1
1=1
Since the additional terms in the optimization problem do not depend on he, the main difference between exact and e-fairness the nature of the weights. More precisely, at iteration t, the update rule becomes
0- -0- Tje
P (Tx)
Σ ch (Aw 5v.) VoP (ho(a) + Tx) k==l
where is learning rate. Once again, obtain simple reweighting scheme where the weights depend on the current faireess level of the model through Ag..... .......Ax and òg....,ōg, the relative size of each group through IP (Tx). and the faimesss notion through the constants C.
E EXTENDED EXPERIMENTS
16

---

Under review as a conference paper at ICLR 2023
E. BASELINES
Adversarial: One of the common ways of removing sensitive information from the model's representation is via adversarial learning. Broadly, consists of three components, namely an encoder, a task classifier, and an adversary. One the on hand, the objective of the adver- sary to predict sensitive information from the encoder. On the other hand, the encoder aims to create representations that are useful for the downstream task (task classifier) and, at the same time, fool the adversary. The adversary is generally connected to the encoder via gradient reversal layer (Ganin & Lempitsky 2015) which acts like an identity function during the forward pass and scales the loss with a parameter - À during the backward pass. In our setting, the encoder is Multi-Layer Perceptron with two hidden layers of size 64 and 128 respectively, and the task classifier is another Multi-Layer Perceptron with a single hidden layer of size 32. The adversary the same the main task classifier. We use ReLU as the activation function with the dropout set to 0.2 and employ batch normalization tuning, we did grid
with default PyTorch parameters. As part of the hyper-parameet search over A, varying it between 0.1 to 3.0 with an interval of 0.2.
BiFair (Ozdayi et al., 2021): For this baseline, we fix the weight parameter to be of length 8 as suggested in the code released by the authors. In this fixed setting, we perform grid search over the following hyper-paree parameterasssss:
Batch Size: 128,256,512
Weight Decay: 0.0, 0.001
Fairness Loss Weight: 0.5, 2,
Inner Loop Length: 5, 50
Constraints: We use the implementation available in the TensorFlow Constrained Opti- mization library with default hyper-parameters.
FairBatch: We use the implementation publicly released by the authors.
Weighted ERM: We reweigh each example in the dataset based on inverse of the propor- tion of the sensitive group it belongs to.
In our initial experiments, we varied the batch size, and learning rates for both Constraints and FairBatch. However, we found that the default hyper-parameters as specified by the authors result in the best performances. In the spirit of being comparable in terms of hyper-parameter search budget, we also fix all hyper-parameters We experiment with two different batch sizes namely, 64 or 512 for the standard fairness dataset. Similarly, we also experiment with three weight decay values namely, 0.0, 0.001 and 0.01. Note that also vary weight decay and batch sizes for FairBatch, Adversarial, Unconstrained, and BiFair approach.
FairGrad, apart from the batch size and weight decay.
For all our experiments, apart from BiFair, we use Batch Gradient Descent as the optimizer with a learning rate of 0.1 and a gradient clipping of 0.05 to avoid exploding gradients. For BiFair, we employ the Adam optimizer as suggested by the authors with learning rate of 0.001.
selection procedure. As mentioned above, all our baselines come with num- ber of pyerr-parameters (learning rates, batch size, weight decay.... and selecting the best combi- nation is often key to avoid undesirable behaviours such over-fitting. In this paper, we proceed as follows. First, for each method, we consider all the X possible we run the training procedure for 50 epochs for each combination. Then, we retain all the models returned by the last 5 epochs, that is, for a given method, we have 5X models and the goal is to se- lect the best one among them. Since we have access to two measures of performance, we can select either the most accurate model, the most fair model, or trade-off between the two depending on the goal of the practitioner. In this paper, we chose to focus on the third option and we select the model with the lowest fairness score between certain accuracy intervals. More specifically, let a" be the highest validation accuracy among the 5.X models, we choose the model with the lowest validation fairess score amongst all models with a validation accuracy in the interval [a 0.03, a*).
combinations and
For FairGrad, FairBatch and Unconstrained, we considered 6 BiFair, we considered 72 such combinations, while for Adversarial, there were 90 combinations.
combinations. For
17

---

Under review as a conference paper at ICLR 2023
E.2 DATASETS
Here, we provide additional details on the datasets used in our experiments. We begin by describ- ing the standard fairness datasets for which we follow the pre-processing procedure described in Lohaus et al. (2020).
Adul The dataset (Kohavi 1996) is composed of 45222 instances, with 14 features each describing several attributes person. The objective to predict the income of a person (below or above 50k) while remaining fair with respect gender (binary in this case). Following the pre-processing step of Wu et al. (2019), only 9 features were used for training.
CelebA The dataset (Liu al., 2015) consists of 202, 599 images, along with 40 binary attributes associated with each image. We use 38 of these as features while keeping gender as the sensitive attribute and "Smiling" as the class label.
Dutch The dataset (Zliobaite et al., 2011 is composed of 60, 420 instances with cach instance described by 12 features. We predict "Low Income" "High Income" as dictated by the occupation as the main classification task and gender as the sensitive attribute.
Compas] The dataset (Larson et al. 2016) contains 6172 data points, where each data point has 53 features. The goal is to predict if the defendant will be arrested again within two years of the decision. The sensitive attribute is race, which has been merged into "White" and "Non White" categories.
composed of 1994 Communities and Crime The dataset (Redmond & Baveja 2002)
instances with 128 features, of which 29 have been dropped. The objective is to predict the number of violent crimes the community, with race being the sensitive attribute.
German Credi The dataset (Dua al. 2017) consists of 1000 instances, with each having 20 attributes. 'he objective is to predict person's creditworthiness (binary), with gender being the sensitive attribute.
Gaussian 10 duced in Lohaus et al. (2020). It is constructed by drawing points from different Gaussian distributions. We follow the same mechanism as described Lohaus et al. 2020)) and
is a toy dataset with binary task label and binary sensitive attribute, intro-
sample 50000 data points for each class.
Adult Folktables" This dataset (Ding et al. 2021 is an updated version of the original Adult Income dataset. We use California census data with gender as the sensitive attribute. There 195665 instances, with 9 features describing several attributes of person. We use the same preprocessing step as recommended by the authors.
For all the dataset, we use a 20% of the data as test set and 80% as train set. We further divide the train set into two and keep 25% of the training examples as a validation set. For each repetition, we randomly shuffle the data before splitting it, and thus we had unique splits for each random seed. As last pre-processing step, we centered and scaled each feature independently by substracting the mean and dividing by the standard deviation both of which were estimated on the training set.
Twitter Sentiment Analysis 12 The dataset (Blodgett et al., 2016) consists of 200k tweets with binary sensitive attribute (race) and binary sentiment score. We follow the setup proposed by Han ctt al. (202T) and Elazar & Goldberg (2018) and create bias in the dataset by changing the proportion of each subgroup (race-sentiment) in the training set. With two sentiment classes being happy and sad, and two race classes being AAE and SAE, the training data consists of 40% AAE-happy, 10%
'tttps//arrcive.ii ics. www.iicc..ccii...... www...ddw//n//dattstts/uuuulll
http://mwwma.........e. wwt.....d...//rrrrecttCCC....... html
https:/sitess.. www.:lesssoogge..o..... httpg/////c//////tiioiiiiiiaaaaa
https://gthhbb https://iluu.ccom// hwt.llluuu.omcccooameeeeeeeeeeee httpp////wopppp.ccccccaapaaaaaaa
http://rchiie... uci. www.eeu...unncc..
ics.
&ata synthetic data.py
com http:/ihhhh..oo///////l/ottaaaaa
ttp://slanglah.c.. wwwss. wwwww.cwa/uaaa..... edu/Twittr/AAF
18

---

Under review as a conference paper at ICLR 2023
AAE-sad, 10% SAE-happy, and 40% SAE-sad. The test set remains balanced. The tweets encoded using the DeepMoji (Felbo et al. 2017) encoder with no fine-tuning, which has been pre- trained over millions of tweets predict their emoji, thereby predicting the sentiment. Note that the train-test splits are pre-defined and thus do not change based on the random seed of the repetition.
E.3 STANDARD FAIRNESS DATASETS
0.08
0.08
Fai-Grad
FaiGGrod
FairGrad
Accuracy Partyy
0.00
Weigheed
Constraints
Odds
0.055
Fai-Batch
Weighted Erm
Equal Opportunity
0.06
BLAG
Weighted Erm
0.04
0.04
BiFair
Constraints
0.04
BiFair
Constrinns
0.03
0.03
0.03
1,02
11.02
1,02
0.00
0.00
0.00
0.:00
0.00
0.00
0.77
0.81
0.83
0.84
0.86
0.77
0.79
0.83
0.84
0.06
0.77
0.81
0.83
0.84
0.86
Accuracy
(a) Linear- AP
(b) Linear EOdds
(c) Linear EOpp
11.8
0.86
FairGrad
Fai-Grad
Accuracy Partly
0.00
0.06
Unconstrained
Adversarial
Equal Odds
0,00
Uscont
Equal Opportunity
0.00
0.06
Usconst mined
Weigheed Erm
Adveroriall
Advensriaal
Weighted
Weigheed
0.08
0.02
0.00
0.01
0.00
0.01
0.000
0.000
0.77
0.17
Accuracy
Accuracy
Accuracy
(d) Non Linear AP
(e) Non Linear- EOdds
(f) Non Linear
EOpp
Figure 6: Results for the Adult dataset with different fairness measures.
Table 3: Results for the Adult dataset with Linear Models. All the results are averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY f
MEASURE MEAN ABS. 1
MAXIMUM
MINIMUM
Unconstrained
0.8456 + 0.0033 AP
0.0571 + 0.0022 0.077 + 0.0029
-0.0373 + 0.0017
Constant
0.751 + 0.0
AP
0.102 + 0.0
0.138 0.0
0.067 0.0
Weighted ERM
0.8442 +: 0.0016
AP
0.0581 + 0.0021
0.0783 + 0.0028
-0.0379 + 0.0014
Constrained
0.783 + 0.007
AP
0.005 + 0.003
0.007 = 0.005
0.004
FairGrad
0.8124 + 0.005
AP
0.0097 = 0.0029
0.0131 + 0.004
-0.0063 + 0.0019
Unconstrained
0.846 + 0.0028
Eodds
0.0453 = 0.0039
0.048 + 0.0043
-0.0878 + 0.01
Constant
0.748 + 0.0
Eodds
0.0 + 0.0
0.0
0,0
Weighted ERM
0.8475 = 0.0024
Eodds
0.044 + 0.0043
0.0477 0.0031
-0.0837 + 0.0124
Constrained
0.805 + 0.004
Eodds
0.007 + 0.005
0.019 0.017
0,002 0.001
BiFair
0.793 + 0.009
Fodds
0.036 0.008
0.085 + 0.027
-0.03 0.016
FairBatch
0.8437 + 0.0013
Eodds
0.0228 = 0.0071
0.0411 0.0105
-0.0245 + 0.0183
FairGrad
0.8284 + 0.004
Eodds
0.0051 0.0021
0.0078 + 0.0068
-0.0078 + 0.0054
Unconstrained
0.8457 + 0.0028 Eopp
0.0263 + 0.0024
0.0157 + 0.0011
-0.0893 + 0.0083
Constant
0.754 + 0.0
Eopp
0.0 0.0
0.0
0.0
Weighted ERM
0.8475 0.0024
Eopp
0.0246 0.0036
0.0148 0.002
-0.0837 + 0.0124
Constrained
0.846 + 0.002
Eopp
0.011 + 0.004
0.039
0.012
0.0
BiFair
0.8 0.009
Eopp
0.031 + 0.024
0.019
+ 0.014
-0.107 0.083
FairBatch
0.8457 + 0.0016
Eopp
0.0098 + 0.0068
0.0225
0.0174 +
-0.0166 + 0.0241
FairGrad
0.8353 0.0106
Eopp
0.0053 + 0.006
0.0177
0.021
0.021
-0.0037 + 0.0033
19

---

Under review as a conference paper at ICLR 2023
Table 4: Results for the Adult dataset with Non Linear Models. All the results are averaged over 5 runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.8438 + 0.0025 AP
0.0575 = 0.0025 0.0776 0.0033 -0.0375 + 0.0018
Constant
0.751 + 0.0
AP
0.102
0.138 0.0
0,067 0.0
Weighted ERM
0.8469 + 0.0035 AP
0.0564 + 0.003
0.0761 + 0.0038
-0.0368 + 0.0021
Adversarial
0.8364 + 0.0063 AP
0.0526 + 0.0017
0.0709 + 0.0025
-0.0343 + 0.0009
FairGrad
0.8054 + 0.0051 AP
0.0034 0.0033
0.0033 + 0.0031
-0.0036 + 0.0042
Unconstrained
0.8299 + 0.0142 Eodds
0.0448 + 0.0109 0.0404 + 0.0136 -0.0977 + 0.0422
Constant
0.748 + 0.0
Eodds
0.0 + 0.0
0.0
0.0
Weighted ERM
0.8285 0.0085
Eodds
0.0102 = 0.0025
0.0196
-00099 0.0047
Adversarial
0.8202 = 0.0068
Eodds
0.0145 = = 0.0052
0.17
-0,0153 0.0067 0,006
BiFair
0.823 + 0.017
Fodds
0.038 + 0.009
0.09
-0.038 0.015
FairBatch
0.8379 + 0.0009
Eodds
0.02 + 0.0088
0.0327
0
-0.0244 + 0.0218
FairGrad
0.827 0.0071
Fodds
0.0118 0.0024
0.022 0.014
-0.0165 + 0.0135
Unconstrained
0.8382 0.0076 Eopp
0.0242 + 0.0031
0.0145 + 0.0017
-0.0822 + 0,0108
Constant
0.754 + 0.0
Eopp
0.0 + 0.0
0.0 0.0
0.0
Weighted ERM
0.8293 0.0091
Eopp
0.0051 = 0.0033
0.0141 0.0137
-0.0002 0.0038
Adversarial
0.8324 + 0.0058
Eopp
0.007 = 0.0044
0.159
BiFair
0.815 + 0.014
Eopp
0.03 = 0.015
0.019
0.009
-0.103 0.053
FairBatch
0.8415 + 0.0054 Eopp
0.0082 = 0.0073
0.0157
0.0121 -0.017 0.0271
FairGrad
0.8373 0.0043 Eopp
0.0053 = 0.0047 0.0099 + 0.0146
-0.0112 + 0.0127
FaiGGad
FaleGrnd
FairGrad
Accuray Partty
B..4
0.05
Weighted
Usconstraned
Comstraints
Odds
0.09
Fai-Batch
Weighted
Equal Opportunity
0,06
8.04
Weighted
0.00
0.00
Comntrints
0,00
Conatrints
BiFair
BiFair
0.02
0,002
0.00
0.00
0.0L
01.00
0.79
0.80
0.82
0.81
0.86
0.87
0.00
0.79
0.90
0.82
0.84
0.86
0.67
0.00
0.79
0.80
0.82
0.81
0.86
0.87
Accuracy
Accuracy
(a) Linear- AP
(b) Linear EOdds
(c) Linear EOpp
Fai-Grod
Fai-Grad
Fai-Grad
0.06
0.00
FairBatch
Equal Opportunity
ILAG
Accuracy Parity
0.04
Adversarial
Weighted Erm
Odds
0.04
Uncomstranee
Adversarial
0.04
Adversarial
Uncomstranned
0.003
0.003
Weigheed Erm
0.08
Weighted Erm
BiFair
BiFair
0.02
0,00
0.00
0.01
0.01
0.00
0.00
0.79
0.82 Accuracy
0.87
0.00
0.70
0.80
0.84
0.86
0.00
0.79
0.82
0.87
Accuracy
Accuracy
(d) Non Linear AP
(c) Non Linear EOdds
Non Linear EOpp
Figure 7: Results for the CelebA dataset with different fairness measures.
20

---

Under review as a conference paper at ICLR 2023
Table 5: Results for the CelebA dataset with Linear Models. All the results averaged over rums. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY 1
MEASURE MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.8532 0.0009 AP
0.0204 0.0022
0.017 0.0019
-0.0238 + 0.0025
Constant
0.516 + 0.0
AP
0.072 + 0.0
0.084 + 0.0
0.06 0 +.0
Weighted ERM
0.853 0.0008
AP
0.0193 + 0.0021
0.0161 + 0.0018
-0.0225 + 0.0023
Constrained
0.799 + 0.013
AP
0.01 + 0.001
0.012 + 0.002
0.009 + 0.001
FairGrad
0.835 + 0.0028
AP
0.0012 + 0.0009
0.0011 + 0.0007 -0.0014 + 0.0011
Unconstrained
0.8532 0.0009 Fodds
0.0499 0.0019 0.0538 + 0.0024
-0.1011 1 + 0.0033
Constant
0.518 + 0.0
Eodds
0.0
0.0 =
0.0
Weighted ERM
0.853 + 0.0009
Eodds
0.0504 = 0.0019 0.0019
0.0532 + 0.0024
-0.1001 0.0032
Constrained
0.802 + 0.004
Eodds
0.006 0.001
0.01
0.002 =
BiFair
0.845 + 0.007
Eodds
0.021 0.005
-0.336 0.009
FairBatch
0.8518 = 0.0009
Eodds
0.0226 = 0.0017 0.0218
0.0028
-0.0411 0.0053
FairGrad
0.8274 + 0.002
Eodds
0.0025 + 0.0009 0.0038
+ 0.0018
-0.0046 + 0.0026
Unconstrained
0.8532 + 0.0009 Eopp
0.0387 + 0.0014 0.0538 + 0.0024
-0.1011 + 0.0033
Constant
0.518 + 0.0
Eopp
0.0 0.0
0.0
0.0 0.0
Weighted ERM
0.853 + 0.0008
Eopp
0.0383 = 0.0014
0.0531 + 0.0024
-0.0999 + 0.0032
Constrained
0.834 + 0.005
Eopp
0.002 + 0.001
0.005
0.002
0.0
BiFair
0.848 + 0.004
Eopp
0.014 + 0.006
0.02 0.009
-0.037 + 0.017
FairBatch
0.8498 = 0.001
Eopp
0.0102 = 0.0016
0.0142 0.0022
-0.0268 + 0.0042
FairGrad
0.844 + 0.0022
Eopp
0.0013 + 0.0009 0.0025
+ 0.0021
-0.0028 + 0.0018
0.35
0.35
1..5
Accuracy Partyy
Uncomtrained
Weighewd Erm
Equal Odds
0.30
Unconstraine
Equal Opportunity
0:30
Uscond rained
Comstmints
Weighted Erm
Weighted Erm
0.15
0.15
0.15
Conatmints
0.10
0.10
0.00
0.00
0.00
0.000
0.00
0.00
0.72
0.75
0.78
0.81
0.81
0.87
0.72
0.75
0.78
0.81
0.84
0.87
0.72
0.78
0.81
0.81
0.87
Accuracy
Accuracy
Accuracy
(a) Linear AP
(b) Linear EOdds
(c) Linear -EOpp
0.35
0.35
0.25
Accuracy Parity
Adversarial
Uncomatransed
Equal Odds
FaieBatch
Uncontrraned
Equal Oppartunity
FairBatch
Unconstrainel
Adversarial
Adivessriial
0.15
0.15
Weighted
0.15
Weigheed
BiFar
BiFair
0.05
0.06
0.05
0.72
0.75
0.78 Accuracy
0.81
0.84
0.87
0.72
0.75
0.78
0.81
0.84
0.87
0.72
0.75
0.78
0.81
0.84
0.87
Accuracy
Accuracy
(d) Non Linear AP
(c) Non Linear EOdds
(f) Non Linear EOpp
Figure Results for the Crime dataset with different fairness measures.
21

---

Under review as a conference paper at ICLR 2023
Table 6: Results for the CelebA dataset with Non Linear Models. All the results are averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group. respectively.
FAIRNESS
METHOD (L.) ACCURACY
MEASURE MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.8587 0.0015 AP
0.0184 0.0014 0.0012
-0.0215 + 0.0016
Constant
0.516 0.0
AP
0.072 0.0 0.084 + 0.0
0.06
Weighted ERM
0.8593 0.0018 AP
0.018 0.0017
-0.021 0.0019
Adversarial
0.8588 + 0.0012 AP
0.0178 + 0.0014 0.0148 0.0012
-0.0208 + 0.0015
FairGrad
0.8359 = 0.0033 AP
0.0023 = 0.0012 0.0025 = 0.0015
-0.0021 = 0.0009
Unconstrained
0.8583 0.0012
Eodds
0.0432 + 0.003
0.0475 + 0.0028 -0.0893 + 0.0049
Constant
0.518 + 0.0
Eodds
0.0 0.0
0.0
0.0
Weighted ERM
0.8589 0.0009
Eodds
0.0419 = 0.0021
0.0459 + 0.0025
-0.0844 0.0038
Adversarial
0.8567 = 0.0014
Eodds
0.0223 = 0.002
0
-0.051 0.0073
BiFair
0.856 + 0.004
Eodds
0.023 0.002 0.028 + 0.005
-0.052 0.009
FairBatch
0.8533 + 0.0037
Eodds
0.0217 + 0.0014 0.0197 + 0.0026 -0.0321 + 0.005
FairGrad
0.8304 = 0.0031
Eodds
0.0037 = 0.0017
0.0048 + 0.0018 -0.0055 + 0.0023
Unconstrained
0.8585 + 0.0016 Eopp
0.0341 + 0.002
0.0473 :: 0.003
-0.0889 + 0.0052
Constant
0.518 + 0.0
Eopp
0.0 + 0.0
0.0
0.0
Weighted ERM
0.859 + 0.0009
Eopp
0.0331 = 0.0014
0.046 = 0.0023
-0.0866 + 0.0035
Adversarial
0.8557 = 0.0019
Eopp
0.0161 = 0.002
0.0223 0.0029
-0.0419 0.0053
BiFair
0.854 + 0.004
Eopp
0.015 + 0.009
0.021 + 0.012
-0.039 0.022
FairBatch
0.8475 + 0.0043 Eopp
0.0051 0.0024
0.007 0.0033
-0.0131 + 0.0063
FairGrad
0.8439 + 0.0063
Eopp
0.0009 = 0.0008 0.002 + 0.0022
-0,0016 + 0.0011
Table Results for the Crime dataset with Linear Models. All the results are averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.8145 0.0136 AP
0.0329 = 0.0195 0.0199 0,0258 + 0.0162
-0.0399 + 0.0229
Constant
0.734 + 0.0
AP
0.272 0.0
0.377 0.0
0.168 + 0.0
Weighted ERM
0.808 + 0.0246
AP
0.0361 + 0.0108 0.0284 + 0.0091
-0.0438 + 0.0129
Constrained
0.775 + 0.015
AP
0.025 0.019 0.031 + 0.025
0.019 + 0.014
FairGrad
0.814 + 0.0102
AP
0.0403 = 0.0181 0.0316 + 0.0147 -0.049 + 0.0218
Unconstrained
0.8035 + 0.0212
Eodds
0.2152 + 0.0215 0.1038 + 0.0231 -0.396 + 0.0433
Constant
0.677 + 0.0
Eodds
0.0 0.0
0.0
0.0 0.0
Weighted ERM
0.8045 = 0.0271
Eodds
+ 0,2086
0,0974
0.0165
-0.3747 + 0.0679
Constrained
0.751 0.014
Eodds
0.036 0.012
0.088
0.007 0.004
BiFair
0.76 + 0.03
Eodds
+
0.048 + 0.03
-0.163 0.092
FairBatch
0.8306 0.0237
Eodds
0.2015 + 0.035
0.1054 + 0.0333
-0.3704 + 0.067
FairGrad
0.7634 + 0.03
Eodds 0.0938 = 0.0144
0.0491 + 0.016
-0.1927 + 0.0362
Unconstrained
0.804 + 0.0215
Eopp
0.1215 + 0.0183 0.1009
0.0238
-0.3852 + 0.0549
Constant
0.697 0.0
Eopp
0.0 + 0.0
0.0
0.0 0.0
Weighted ERM
0.8171 + 0.0213
Eopp
0.1209 = 0.0154
-0.3851 + 0.0599
Constrained
0.762 + 0.021
Eopp
0.044 0.021
0.138 0.066
0,0 0.0
BiFair
0.806 + 0.01
Eopp
0.085 + 0.038
0.073 + 0.042
-0.268
0.112
FairBatch
0.8225 0.0252
Eopp
0.1126 + 0.0259
0.1002 + 0.0281
0.3501 0.0821
FairGrad
0.7755 0.0233
Eopp
0.0609 + 0.0149
0.0507 + 0.0166
-0.193 0.0456
22

---

Under review as a conference paper at ICLR 2023
Table 8: Results for the Crime dataset with Non Linear Models. All the results averaged 5 runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group. respectively.
FAIRNESS
METHOD (L) ACCURACY 1
MEASURE MEAN ABS. +
MAXIMUM
MINIMUM
Unconstrained
0.8165 + 0.019
AP
0.0535 + 0.0199 0.0423 0.0155
-0.0648 + 0.0251
Constant
0.734 + 0.0
AP
0.272 0.0
0.377 0.0
0.168 0.0
Weighted ERM
0.8271 = 0.0114
AP
0.0483 = 0.0167 0
-0.5844 0.02
Adversarial
0.809 = 0.0175
AP
0.0592 = 0.0173 0.0464 0.0135
-0,0719 0.0223
FairGrad
0.822 + 0.0203 AP
0.0434 + 0.0206 0.0341 + 0.0162
-0.0526 + 0.0252
Unconstrained
0.8115 = 0.014
Eodds
0.1635 = 0.0395 0.0854 0.014
-0.3326 = 0.0649
Constant
0.677 0.0
Eodds
0.0 0.0
0.0
0.0
Weighted ERM
0.8135 + 0.0137
Eodds
0.1739 + 0.0394
0.0861
0.012
-0.339 0.0778
Adversarial
0.791 + 0.007
Fodds
0.1464 + 0.0168
+
-0.3001 0.0296
BiFair
0.793 + 0.022
Eodds
0.161 0.032
0.091
-0.339 0.048
FairBatch
0.8391 + 0.0195
Eodds
0
0 0.0313
-0.3828 + 0.0671
FairGrad
0.7734 + 0.0251
Eodds
0.0982 0.0513
0.0511
+ 0.0179
-0.2016 + 0.0771
Unconstrained
0.817 + 0.0152
Eopp
0.1044 = 0.0133 0.0856
0.0123
-0.3321 = 0.0489
Constant
0.697 + 0.0
Eopp
0.0 + 0.0
0.0
0.0
Weighted ERM
0.8205 0.0184
Eopp
0.1159 + 0.0191 0.0955 0.019
-0.368 0.0642
Adversarial
0.795 + 0.0148
Eopp
0.0959 0.0153
-0.3000 0.042
BiFair
0.807 + 0.025
Eopp
0.11 0.031
0.091
1:351
FairBatch
0.8411 + 0.0177 Eopp
0.1217 = 0.0277 0.1083
0.0311
-0.3784 + 0.0891
FairGrad
0.7799 0.0243
Eopp
0.0675 + 0.0179 0.0556
+ 0.0147
-0.2143 + 0.0592
0.00
FaiGGrad
0.00
Fai-Grrd
0.00
FairGrad
Accuracy Party
Fai-Batch
Equal Opportunity
FakBatch
Weighted Erm
Usconstrateed
Unconstrasted
Weightod Erm
Weighted Erm
0.05
0.05
0.05
0.04
0.04
1.00
0.00
0.00
0.02
0,02
0.02
0.00
0.0
0.00
0.00
0.00
Accuracy
Accuracy
Accuracy
(a) Linear AP
(b) Linear -OOdds
(c) Linear EOpp
(1,08)
1.:0
Accuracy Party
0.08
Uncondtrined
Eougl Opportuniky
0.08
FaieBatch
0.07
Adverariaal
0.07
Uncomitrined
0.07
Uncomstransed
Weighted Erm
Equal Odes
Advenariial
Advensriaal
1L.43
Weighted
11.0
0.08
0.08
0.08
0.00
0.00
0.00
0.76
0.86
0.69
0.72
0.79
0.00
0.76
0.86
Accuracy
Accuracy
Accuracy
(d) Non Linear AP
(e) Non Linear EOdds
(1) Non Linear EOpp
Figure 9: Results for the Adult with multiple groups dataset with different fairness measures.
23

---

Under review as a conference paper at ICLR 2023
Table 9: Results for the Adult with multiple groups dataset with Linear Models. All the results are averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY 1
MEASURE MEAN ABS. +
MAXIMUM
MINIMUM
Unconstrained
0.8451 + 0.0042 AP
0.0559 + 0.0047 0.0985 + 0.0111 -0.042 + 0.003
Constant
0.754 + 0.0
AP
0.097 0.0
0.159 0.0
0,024 + 0.0
Weighted ERM
0.8454 + 0.0032
AP
0.0562 = 0.0042 0.0993 + 0.0117 -0.0426 = 0.0018
FairGrad
0.807 0.0022
AP
0.0148 = 0.0041 0.0256 + 0.0048
-0.0107 = 0.0045
Unconstrained
0.844 + 0.0011
Eodds
0.0558 + 0.0062 0.0578 + 0.0069 -0.1586 + 0.0621
Constant
0.75 = 0.0
Eodds
0.0 = 0.0
0.0
0.0
Weighted ERM
0.8448 0.0038
Eodds
0.0586 = 0.0097 0.0567
0.004 -0.1702 + 0.0776
FairBatch
0.8396 + 0.0034
Eodds
0.0308 + 0.0057 0.0565 0.0116
-0.6441 0.0234
FairGrad
0.8162 0.0052
Eodds
0.0197 + 0.0118 0.0373
+ 0.0233
-0.0493 + 0.0403
Unconstrained
0.8431 0.002
Eopp
0.0391 + 0.0052 0.0297
0.0131
-0.169 0.0565
Constant
0.762 0.0
Eopp
0.0 0.0
0.0
0.0 0.0
Weighted ERM
0.8443 0.0038
Eopp
0.0415 + 0.01 0.0316
0.0145
-0.1767 + 0.0797
FairBatch
0.8392 = 0.004
Eopp
0.0219 = 0.0055
0.05
0.013
-0,0749 = 0.0285
FairGrad
0.834 = 0.0044
Eopp
0.0201 = 0.0099
0.0442 0.0415
-0.0679
+ 0.0808
Table 10: Results for the Adult with multiple groups dataset with Non Linear Models. All the results averaged over 5 runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute faimess value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE
MEAN ABS |
MAXIMUM
MINIMUM
Unconstrained
0.8427 0.0041 AP
0.0546 = 0.0026 0.0966 + 0.0098
-0.0421 +-t 0.0022
Constant
0.754 + 0.0
AP
0.097 + 0.0
0.159 + 0.0
0.024 + 0.0
Weighted ERM
0.8408 0.0031 AP
0.0575 + 0.0035
0.101 0.0106
-0.0443 + 0.0026
Adversarial
0.8358 + 0.0043
AP
0.0527 + 0.0028
0.0889 + 0.0066
-0.0401 + 0.0022
FairGrad
0.7991 = 0.0036 AP
0.013 + 0.0051 0.0257 + 0.0138
-0.0125 = 0.0043
Unconstrained
0.8347 +: 0.0129 Eodds
0.0523 + 0.0126 0.0495 + 0.0166 -0.1772 + 0.0512
Constant
0.75 = 0.0
Eodds
0.0 0.0
0.0
0.0
Weighted ERM
0.8199 0.002
Eodds
0.0223 = 0.0065 000744000017
0.0287 = 0.0076
-0.1013 0.0543
Adversarial
0.8251 = 0.0064
Eodds
-0.0667 1.00667700000 0.0559
FairBatch
0.8212 + 0.0103 Eodds
0.0806 + 0.0137 0.0522 + 0.0076 -0.2545 + 0.0525
FairGrad
0.8128 + 0.0102 Eodds
0.0196 + 0.0061 0.0392 + 0.0176 -0.0433 + 0.0342
Unconstrained
0.8373 + 0.0123 Eopp
0.0331 + 0.008
0.0183 + 0.0045 -0.1587 + 0.0643
Constant
0.762 + 0.0
Eopp
0.0 + 0.0
0.0
0.0 0.0
Weighted ERM 0.8216 + 0.0031 Eopp
0.0245 = 0.008
0.0243
-0.101 0.0543
Adversarial
0,8343 + 0.0036
Eopp
0.0209 = 0.0093
0,0327
-0,0927 + 0.0589
FairBatch
0.821 + 0.0097 Eopp
0.067 + 0.0168 0.047
-0.2484 1 0.0535
FairGrad
0.8341 + 0.0053
Eopp
0.0176 + 0.0059
0.0302
-0.0331 + 0.0543
24

---

Under review as a conference paper at ICLR 2023
D.I
0.1
0.11
0.30
0,10
0,30
Uncandtrined
0,00
Fabfiatch
Loual Opportunity I
Fabfiatch
Weigheed Erm
Usconstakeed
Unconstaaieed
Constrainte
Weighted
Weighted Erm
0.06
0,06
0.00
Accurao
1..0
11:06
0.04
0.M
0.04
1.00
[1.00
1.00
1.40
11/1
1142
0.0
0.01
0.01
0.56
0.58
0.61
0.64
0.67
0.71
0.56
0.58
0.61
0.64
0.67
0.71
0.56
0.58
0.61
0.64
0.67
0.71
Accuracy
Accuracy
Accuracy
(a) Linear AP
(b) Linear EOdds
(c) Linear EOpp
0.1
Fak
Fak
Unconstraned
FaieBatch
Equal Oyportunie munb
FairBatch
Adverariial
Uncomttrined
Uncomttmined
2 0.00
0,00
Adversarial
Acorn 69
11.06
0.00
Weighted
11.0G
Weighted
0.05
0.05
BiFair
0.05
Bifarr
1.04
[104
0.00
0.00
0.00
11002
0.002
11:0
0.01
0.000
0.00
0.54
0.58
0.61
0.64
0.67
0.71
0.54
0.58
0.61
0.64
0.67
0.71
0.54
0.58
0.61
0.64
0.67
0.71
Accuracy
Accuracy
Accuracy
(d) Non Linear- AP
(e) Non Linear EOdds
(f) Non Linear EOpp
Figure 10: Results for the Compas dataset with different faimnsss measures.
Table 11: Results for the Compas dataset with Linear Models. All the results are averaged over 5 runs. Here MEAN ABS.. MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.6644 + 0.0137 AP
0.0091 = 0.0025
0.0076 + 0.0031
-0.0107 + 0.004
Constant
0.545 + 0.0
AP
0.066 + 0.0
0.085 0.0
0,047 0.0
Weighted ERM 0.6671 + 0.0169
AP
0.0088 = 0.004
0.0061 = 0.0028
-0.0115 + 0.0051
Constrained
0.65 + 0.012
AP
0.014 + 0.005
0.018 + 0.006
0.009 0.003
FairGrad
0.6708 + 0.0166 AP
0.0083 + 0.0068
0.0057 + 0.0048
-0.0108 + 0.0088
Unconstrained
0.6636 + 0.0104 Eodds
0.0827 + 0.0165 0.0758 + 0.0133
-0.1553 + 0.0259
Constant
0.527 0.0
Eodds
0.0 + 0.0
0.0 0.0
0,0 + 0.0
Weighted ERM
0.6685 0.0073
Eodds
0.082 + 0.0137
0.0697 + 0.0115 -0.1618 + 0.0222
Constrained
0.564 = 0.014
Eodds
0.007 + 0.004
0.014
0.002
BiFair
0.672 0.021
Eodds
0.076 = 0.023
0.071
-..15
FairBatch
0.6847 + 0.0175
Fodds
0.09 0.0094
0.0854 + 0.0149
-0.1727 + 0.0304
FairGrad
0.6557 0.0075
Fodds
0.0593 0.0128
0.0524 + 0.0102
-0.1241 + 0.0202
Unconstrained
0.6609 0.0106 Eopp
0.052 + 0.0107
0.062 0.0145
-0.1461 + 0.0286
Constant
0.35 + 0.0
Eopp
0.0 0.0
0.0
0.0 + 0.0
Weighted ERM
0.6695 0.0055
Eopp
0.0554 + 0.0074
0.0659 + 0.0107
-0.1557 + 0.0194
Constrained
0.565 + 0.015
Eopp
0.004 0.003
0.009
0.0
BiFair
0.68 = 0.013
Eopp
0.054 0.016
0.064
+
FairBatch
0.6865 + 0.0171
Eopp
0.0618 = 0.0134 0.0715
0.0173
-0.1755
0.0364
FairGrad
0.6565 0.0152
Eopp
0.0467 = 0.0046
0.0554 + 0.0071
-0.1313 + 0.0119
25

---

Under review as a conference paper at ICLR 2023
Table 12: Results for the Compas dataset with Non Linear Models. All the results averaged over runs, Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the well-off group, and the fairness level of the worst-off group. respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS. 4
MAXIMUM
MINIMUM
Unconstrained
0.6593 + 0.0192 AP
0.0119 = 0.0072 0.0095 + 0.004
-0.0144 + 0.0107
Constant
0.545 + 0.0 AP
0.066 + 0.0
0.085
0.0
0.047 0.0
Weighted ERM
0.6687 = 0.0138 AP
0.0127 = 0.0061
0.011 = 0.0034
-0,0145 = 0.0099
Adversarial
0.6583 0.0157 AP
0.0078 = 0.0051
0.0066 = 0.0044
-0.009 0.0069
FairGrad
0.6672 + 0.0099 AP
0.0113 + 0.005
0.0095 + 0.0023
-0.0131 + 0.0082
Unconstrained
0.6562 = 0.0154 Eodds
0.0782 = 0.014
0.0715 0.0136
-0.1521 = 0.0277
Constant
0.527 = 0.0
Eodds
0.0
0.0
0,0 0.0
Weighted ERM
0.6615 + 0.0175
Eodds
0.0789 + 0.0131
0.0726 + 0.0077
-0.1496 + 0.0313
Adversarial
0.6504 + 0.0157
Fodds
0.059 0.0138
0.0549 +
+ 0.0107
-0.1294 + 0.0183
BiFair
0.661 0.009 Eodds
0.013
0.068 0.018
-0.133 0.016
FairBatch
0.6792 + 0.0086
Eodds
0.071 + 0.0083
0.0663 + 0.0091 -0.1508 + 0.0304
FairGrad
0.6457 0.0088
Eodds
0.061 + 0.0075
0.0564 + 0.0065
-0.127 0.0081
Unconstrained
0.6552 0.0137 Eopp
0.0553 = 0.0108 0.0659 0.015
-0.1552 + 0.0281
Constant
0.55 = 0.0
Eopp
0.0 0.0
0,0
0
Weighted ERM
0.6604 0.0163
Eopp
0.0519 + 0.0111
0
-0.1458 0.0299
Adversarial
0.6494 + 0.0148
Eopp
0.0472 = 0.007 +
-0.1327 0.0183
BiFair
0.669 + 0.01 Eopp
4.042::000
0.05 0.02
-0.117 0.055
FairBatch
0.6802 + 0.0114 Eopp
0.0536 = 0.0133 0.062 0.0167
-0.1526 0.0367
FairGrad
0.6586 0.0118
Eopp
0.0476 + 0.0056
0.0563
+ 0.0067
-0.1339 0.0163
FairGrad Weighted Ere
Fai-Grad
FairGred
Accuracy Parky
0.03
Equal Oóds
0,00
Usconstrased Weighted Erm
Equal Opportunity
0.00
Fai-Batch
Unconstrasedd
Constraints
Weighted Erm
1.002
0,00
Constraints
0.002
Constraints
0.00
0.00
0.0L
0.000
0.00
0.00
Accuracy
Accuracy
Accuracy
(a) Linear AP
(b) Linear -OOdds
(c) Linear EOpp
Accuracy Pariyy
11.03
Adversarial
Unconstrinee
Equal Odds
11,03
Unconstrined
Equal Oppartunity
1L0G
Uncomstransed
Weighted Erm
Advenariial Weighted
Advwnsrial
0.00
0.00
0.00
0.80
0.83
1.00
0.79
1100
0.80
0.83
Accuracy
Accuracy
Accuracy
(d) Non Linear AP
(e) Non Linear EOdds
(f) Non Linear EOpp
Figure 11; Results for the Dutch dataset with different fairness measures.
26

---

Under review as a conference paper at ICLR 2023
Table 13: Results for the Dutch dataset with Linear Models. All the results are averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute faimesss value, the fairness level the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L.) ACCURACY
MEASURE
MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.8049 + 0.007
AP
0.0281 + 0.006
0.0281 + 0.006
-0.0282 + 0.0061
Constant
0.524 + 0.0
AP
0.151 + 0.0
0.152 0.0
0.15 + 0.0
Weighted ERM
0.8052 + 0.0073
AP
0.028 0.006
0.028 + 0.006
-0.0281 + 0.006
Constrained
0.799 + 0.009
AP
0.009 0.006
0.009 0.006
0.009 0.006
FairGrad
0.8042 + 0.0046 AP
0.0048 = 0.0033
0.0048 = 0.0033
-0.0048 + 0.0032
Unconstrained
0.8071 + 0.0072 Eodds
0.0212 = 0.0018
0.0322 + 0.009
-0.0256 + 0.0052
Constant
0.522 = 0.0
Eodds
0.0 0.0
0.0
0,0 0.0
Weighted ERM
0.8074 + 0.0074
Eodds
0.0213 = 0.002
0.032 0.0086
-0.0254 + 0.0051
Constrained
0.79 = 0.005
Eodds
0.005 + 0.003
BiFair
0.804 = 0.008
Eodds
0.021 = 0.003
0.025 = 0.004
-0.033 0.01
FairBatch
0.809 + 0.0096
Eodds
0.018 +: 0.0016
0.0262 + 0.0039
-0.0211 + 0.004
FairGrad
0.7978 + 0.0064
Eodds
0.0053 + 0.0019 0.007 + 0.0019
-0.009 + 0.0049
Unconstrained
0.8129 + 0.0021 Eopp
0.0075 + 0.0034 0.0107 = 0.0049
-0.0193 + 0.0086
Constant
0.524 + 0.0
Eopp
0.0 +: 0.0
0.0
0.0 0.0
Weighted ERM
0.8077 + 0.0078
Eopp
0.0076 = 0.0034
0.011 0.0049
-0.0196 + 0.0087
Constrained
0.814 = 0.003
Eopp
0.003 + 0.002
0.007 = 0.006
0.0
BiFair
0.808 = 0.01
Eopp
0.005 = 0.005
0.008 0.007
-0.012 0.012
FairBatch
0.8149 + 0.0117
Eopp
0.0031 + 0.0014
0.0044
0.002
-0.0079 + 0.0036
FairGrad
0.8144 + 0.0021
Eopp
0.004 + 0.0037 0.006 0.0052
-0.0099 + 0.0097
Table 14: Results for the Dutch dataset with Non Linear Models. All the results are averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.7937 0.0052 AP
0.0252 0.0091 0.0252 + 0.009
-0.0252 + 0.0091
Constant
0.524 0.0 AP
0.151 0.0
0.152 + 0.0
0.15 0.0
Weighted ERM
0.7954 + 0.0023 AP
0.0257 + 0.0089
0.0257 + 0.0089
-0.0257 + 0.0089
Adversarial
0.7939 + 0.0043 AP
0.0232 + 0.0071 0.0232 + 0.0071
-0.0232 + 0.007
FairGrad
0.8043 + 0.0071 AP
0.0052 + 0.0026 0.0052 + 0.0026 -0.0052 + 0.0026
Unconstrained
0.7914 + 0.006
Eodds
0.0162 + 0.0062 0.0193 + 0.0071 -0.0263 + 0.0142
Constant
0.522 +: 0.0
Eodds
0.0 0.0
0.0
0.0 0.0
Weighted ERM
0.7958 = 0.0027
Eodds
0.0168 = 0.0053
0.0202 + 0.0048
-0.0261 + 0.0131
Adversarial
0.7928 0.0077
Eodds
0.0148 = 0.0041
0.0000011111 0.006
BiFair
0.819 + 0.003
Eodds
0.021 + 0.004
0.03
+.0000000
+0.088 0.007
FairBatch
0.8091 + 0.012
Eodds
0.018 0.0021
0.0254
0.0058
-0.2448 + 0.0062
FairGrad
0.8013 + 0.0073
Eodds
0.0069 = 0.0031
0.0099 + 0.0038
-0.0095 + 0.0068
Unconstrained
0.8149 +: 0.0034 Eopp
0.0055 + 0.0024 0.0079 + 0.0035
-0.014 0.0061
Constant
0.524 0.0
Eopp
0.0 + 0.0
0.0 0.0
0.0
Weighted ERM
0.8179 + 0.0044 Eopp
0.0066 = 0.0026
0.0095 + 0.0037
-0,01 0.0065
Adversarial
0.8156 0.0038 Eopp
0.004 + 0.0039
- +
-0.0102 0.01
BiFair
0.819 + 0.003
Eopp
0.009 0.002
0.012 0.003
-0.022 0.006
FairBatch
0.8174 0.0031
Eopp
0.002 + 0.0012
0.0029
0.0017
-0.0052 + 0.0031
FairGrad
0.8158 + 0.0051
Eopp
0.0036 + 0.0031
0.0051 + 0.0045
-0.0092 + 0.0079
27

---

Under review as a conference paper at ICLR 2023
Uscandtrined
Soual Dpportunik
Accurayy 2aa
[1/6
Weighte Erm
Unconstrined Weighted
Uncomtrained
0.07
Constraints
0.07
0.07
Weighted Erm
0..0
0..65
0.06
0.00
0.06
1.00
0.00
0.00
1.00
0.00
1.:2
11.0.
11.01
10.01
0.70
0.76
0.79
066
0.67
0.73
0.76
0.70
0.76
0.79
Accuracy
Accuracy
Accuracy
(a) Linear AP
(b) Linear EOdds
(c) Linear EOpp
0.00)
Unconstranced
0,00
Fateatch
Letall Opportuniey 8
FairBatch
ag Pantt
0.08
Adveruriial
0.08
Unconttmined
0..8
Uncomttmined
0.07
0.07
Adversarial
0.07
Adversarial
Accuray
0,00
Weighted
1100
Weighted
BiFair
BiFair
0.04
0.04
0.04
0.08
0.00
0.03
11:02
0.02
0.00
0.00
0.01
0.01
0.64
0.67
0.70
0.73
0.76
0.79
0.64
0.67
0.70
0.73
0.76
0.79
0.64
0.67
0.70
0.73
0.76
0.79
Accuracy
Accuracy
Accuracy
(d) Non Linear- AP
(c) Non Linear FOdds
(f) Non Linear EOpp
Figure 12: Results for the German dataset with different fairness measures.
Table 15: Results for the German dataset with Linear Models. All the results are averaged over 5 runs. Here MEAN ABS.. MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.692 + 0.0232
AP
0.0226 + 0.0181 0.0169 + 0.0111 -0,0284 + 0.0256
Constant
0.73 + 0.0
AP
0.05 0.0
0.00
0.031 + 0.0
Weighted ERM
0.707 + 0.0344
AP
0.0243 0.0191 0.0186
-0.0299 0.027
Constrained
0.733 = 0.033
AP
0.024 0.025
0.033
0.015 = 0.017
FairGrad
0.744 + 0.0357
AP
0.0274 + 0.0212 0.0215
0.0123
-0.0334 + 0.0306
Unconstrained
0.69 + 0.0266
Eodds
0.0316 + 0.0207 0.0499 + 0.0341
-0.0618 + 0.0471
Constant
0.7 + 0.0
Eodds
0.0 0.0
0.0
0.0 = 0.0
Weighted ERM
0.709 + 0.0296
Eodds
0.0324 +: 0.0338
0.0461
0.046
-0.055 + 0.0626
Constrained
0.739 + 0.027
Eodds
0.037 + 0.012
0.072
0.01 0.004
BiFair
0.698 + 0.039
Eodds
0.033 0.01
0.023
0.059
FairBatch
0.7 = 0,0247
Eodds
0.0706 0.0184
0.1102
-0.1134 0.0518
FairGrad
0.734 0.0358
Fodds
0.0464 0.0201
0.0784
0.0232
-0.0721 + 0.0496
Unconstrained
0.704 0.0193 Eopp
0.0053 + 0.0035 0.0096
0.004
-0.0116 + 0.0117
Constant
0.7 + 0.0
Eopp
0.0 0.0
0.0
0.0
Weighted ERM
0.706 + 0.0328
Eopp
0.0048 0.0039
0.0097
0.0091
-0.0096
0.0092
Constrained
0.741 + 0.019
Eopp
0.005 + 0.002
0.155
0.0 0.0
BiFair
0.703 + 0.037
Eopp
0.007 = 0.006
0.014
0.011
FairBatch
0.718 + 0.0229
Eopp
0.0172 - 0.0124
0.0272
-.00
FairGrad
0.723 + 0.0425
Eopp
0.0125 + 0.0043
0.0212 + 0.0087
-0.288 0.0162
28

---

Under review as a conference paper at ICLR 2023
Table 16: Results for the German dataset with Non Linear Models. All the results averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the well-off group, and the fairness level of the worst-off group. respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS. +
MAXIMUM
MINIMUM
Unconstrained
0.695 + 0.0122 AP
0.0426 + 0.0241
0.0144
0.0144 -0.0537 + 0.0345
Constant
0.73 + 0.0
0.05 + 0.0
0.069 0.0
0.031 0.0
Weighted ERM
0.703 = 0.0183
AP
0.035 = 0.0237
0.0265
-0,0436 0 0.0338
Adversarial
0.681 = 0.0156 AP
0.041 0.0254
1 2.0327=00000
-0,0492 + 0.0368
FairGrad
0.714 + 0.026 AP
0.037 + 0.0222
0.0291 + 0.0119
-0.0448 + 0.0331
Unconstrained
0.689 + 0.0213 Eodds
0.0089 + 0.0052
0.0117 = 0.0045
-0.0144 = 0.0116
Constant
0.7 = 0.0
Eodds
0.0 0.0
0.0
0.0
Weighted ERM
0.703 + 0.034
Eodds
0.0211 0.0106
0.0305
0.0186 -0.0372 0.0158
Adversarial
0.684 + 0.0097
Eodds
+
0.0263
0.0201 -0.0339 + 0.0237
BiFair
0.725 + 0.031
Eodds
0.016 0.015
0.021
00.027 -0,0822 0.0983
FairBatch
0.692 = 0.026
Eodds
0.0489 0.0382
0,0607
FairGrad
0.695 0.0237
Eodds
0.0095 0.004
0.0121
0.0046
-0.0155 + 0.0076
Unconstrained
0.686 + 0.0215
Eopp
0.0124 0.0075
0.0227
0.0128
-0.0269 + 0.0227
Constant
0.7 + 0.0
Eopp
0.0 = 0.0
0.0
0.0
Weighted ERM
0.7 0.0261
Eopp
0.0066 0.0057
0.0131
0.0071
-0,0133 + 0.0173
Adversarial
0.687 + 0.0129
Eopp
0.0085 + 0.0051
-0.0137 0.0099
BiFair
0.727 + 0.023
Eopp
0.015 = 0.013 0.023
-0.036
FairBatch
0.697 + 0.025
Eopp
0.0084 + 0.0079 0.0235
-0,0102 0.0226
0.0226
0.0094
FairGrad
0.696 + 0.0166
Eopp
0.0052 + 0.0038 0.0093
+ 0.0064
-0.0115 0.0108
B.12
IL12
I1.12
[.]
FaiGrad
0.1.
Fai-Grad
[.]]
0.10
Unconstrained
0,10
FaieBatch
Quet Reportunk UnR R
00.10
Fak-Batch
(L.t)
Weighted Erm
(1,0)
Usconstraied
(Lit)
Unconstrasted
Constraints
11005
Weightod Erm
TLIS
Weighted Erm
0.00
Constraints
Constraints
11002
1100
1.00
01.00
0.00
0.00
Accuracy
Accuracy
Accuracy
(a) Linear AP
(b) Linear EOdds
(c) Linear EOpp
0.12
0.12
0.12
0.1
0.1
0.1.
3 11:02
11.10
Uncondrained
11.10
11.10
FaieBatch
Advenariaal
(1,0)
Uncomitmined
(1/0)
Uncomatranee
0.08
Weighted Erm
0.06
Advenariial
0.08
Advwnsriaal
0.07
0.07
Weighted
0.07
0.06
0100
0.06
0.05
0.05
0.05
0.0
0.65
0.90
1.00)
0.57
0.00
0.65
0.90
Accuracy
Accuracy
Accuracy
(d) Non Linear AP
(e) Non Linear EOdds
(1) Non Linear EOpp
Figure 13: Results for the Gaussian dataset with different fairness measures.
29

---

Under review as a conference paper at ICLR 2023
Table 17: Results for the Gaussian dataset with Linear Models. All the results are averaged over 5 runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS. +
MAXIMUM
MINIMUM
Unconstrained
0.8689 0.0037 AP
0.0966 + 0.0029 0.0957 + 0.0028
-0.0974 +: 0.0036
Constant
0.497 = 0.0
AP
0.001 0.0
0.00::00
0.001 = 0.0
Weighted ERM
0.869 = 0.0039
AP
0.0966 = 0.0026
0.0957
0.0023
-0.0974 0.0034
Constrained
0.799 + 0.004
AP
0.003 + 0.002
0.003 + 0.002
0.003 + 0.002
FairGrad
0.8516 + 0.0064
AP
0.0558 + 0.0094 0.0553 + 0.0093
-0.0562 + 0.0096
Unconstrained
0.869 = 0.0037
Eodds
0.0971 = 0.0026 0.1872 + 0.0067 -0.1896 = 0.0056
Constant
0.499 + 0.0
Eodds
0.0 + 0.0
0.0 + 0.0
0.0 + 0.0
Weighted ERM
0.869 + 0.0039
Fodds
0.0971 + 0.0023 0.1869 + 0.0063 -0.1894 + 0.0051
Constrained
0.497 + 0.003
Eodds
0.0 0.0
0.0
0,0 0.0
BiFair
0.873 = 0.004
Eodds
0.013
0.21
-0,213 0.004
FairBatch
0.8649 0.0025
Eodds
0.0902 + 0.0035
0.1717 + 0.0046
-0.1719 0.0079
FairGrad
0.8459 + 0.01
Eodds
0.0786 = 0.0051
0.1504 + 0.0102
-0.1527 + 0.0142
Unconstrained
0.8598 + 0.0121 Eopp
0.0928 = 0.0012 0.1845 + 0.0041
-0.1869 + 0.0041
Constant
0.498 + 0.0
Eopp
0.0 + 0.0
0.0 0.0
0.0 + 0.0
Weighted ERM
0.8599 + 0.0121
Eopp
0.0931 = 0.0011
0.1849 + 0.004
-0.1874 + 0.004
Constrained
0.698 + 0.005
Eopp
0.004 0.002
0.008 0.005
0,0 0.0
BiFair
0.863 + 0.009
Eopp
0.2
-0,202 0.006
FairBatch
0.8635 0.0024
Eopp
0.085 + 0.0023
0.17 0.0032
-0.1702 + 0.0065
FairGrad
0.8431 + 0.0065
Eopp
0.0752 = 0.0043 0.1494 + 0.0087
-0.1514 0.0094
Table 18: Results for the Gaussian dataset with Non Linear Models. All the results are averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY 1
MEASURE MEAN ABS. |
MAXIMUM
MINIMUM
Unconstrained
0.88 + 0.0038
AP
0.0897 = 0.0045 0.0888 + 0.0035 -0.0905 + 0.0055
Constant
0.497 + 0.0
AP
0.001 0.0
0.001 + 0.0
0.001
0.0
Weighted ERM
0.8809 0.0048 AP
0.0903 = 0.0045 0.0894 0.0033
-0.0911 0.0057
Adversarial
0.8725 = 0.0115 AP
0.0858 = 0.0077
0.0851 = 0.0076
-0.0866 1000866=000 0.0081
FairGrad
0.8542 = 0.0047 AP
0.0352 = 0.0047 0.0349 + 0.0048 -0.0355 = 0.0046
Unconstrained
0.8814 + 0.0024 Eodds
0.093 + 0.0032
0.1807 + 0.0066 -0.183 0.005
Constant
0.499 + 0.0
Eodds
0.0 + 0.0
0.0 1.0 0.0
0.0 + 0.0
Weighted ERM
0.8821 = 0.0031
Eodds
0.0939 = 0.0013
0.1826 + 0.0042
-0.185 0.0033
Adversarial
0.8775 + 0.0091
Eodds
0.0852 = 0.007
0.1643
-0.1666 + 0.0146
BiFair
0.868 + 0.013 Eodds
0.092 0.011
0.167
4.035
-0.168 + 0.031
FairBatch
0.8735 0.0032
Eodds
0.0749 = 0.0041
0.1455
0.005
-0.1456 + 0.0056
FairGrad
0.8539 + 0.0056
Eodds
0.0596 + 0.0068
0.1013 + 0.0147
-0.1025 + 0.0144
Unconstrained
0.8801 + 0.004 Eopp
0,0902 + 0.0017 0.1792 + 0.0041
-0.1816 + 0.0053
Constant
0.498 + 0.0
Eopp
0.0 + 0.0
0.0
0.0 + 0.0
Weighted ERM 0.8805 + 0.0046
Eopp
0.0912 = 0.0008
0,1812 0.0024
-0.1837 + 0.0045
Adversarial
0.8754 0.0086
Eopp
0.0808 + 0.0066
0.1605 + 0.0128
-0.1628 + 0.0143
BiFair
0.88 + 0.003 Eopp
0.086 + 0.005 0.17 + 0.013
-0.172 + 0.009
FairBatch
0.874 = 0.0035
Eopp
0.0733 + 0.0029 0.65 + 0.0054
-0.1467 + 0.0066
FairGrad
0.8543 + 0.0082
Eopp
0.0517 = 0.0095
0.1028 0.0191
-0.1041
+ 0.0192
30

---

Under review as a conference paper at ICLR 2023
11.20
11.30
11.20
Unconstranie Weigheed Erm
FatBatch
Equal oportunitty
FaiBatch Unconstaareed
Accuracy Parity
0.15
Constraints
Equal Odds
0.15
Weighted
Usconstakeed
0.15
Weigheed Erm
0.10
0.30
BiFair
0.30
BiFair
0.69
0.71
0./2
0.74
0.75
0.77
0.09
0.71
0.72
0.74
0.75
0.77
0.69
0.71
0.72
0.74
0.75
0.77
Accuracy
Accuracy
Accuracy
(a) Linear-
AP
(b) Linear- EOdds
(c) Linear- EOpp
FaiGGrad
Grad
FairGrad
Unconstrined
FairBatch
Equal Opportunity
FairBatch
Accuraty Parity
0.15
Adversarial
0.15
0.15
Woighted
Equal Odds
Adversarial
Wrighted Erm
Weighted Erm
0.10
0.10
BiFair
0.10
BiFair
0.05
0.05
0.05
0.000
0.00
0.00
69
0.71
0.77
0.74
0.75
0.77
0.69
0.71
0.72
0.74
0.75
0.77
0.69
0.71
0.77
0.74
0.75
0.77
Accuracy
Accuracy
Accuracy
(d) Non Linear AP
(c) Non Linear EOdds
(f) Non Linear EOpp
Figure 14: Results for the Twitter Sentiment dataset with different fairness measures.
Table 19: Results for the Twitter Sentiment dataset with Linear Models. All the results are averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS. J
MAXIMUM
MINIMUM
Unconstrained
0.7211 + 0.004
AP
0.0426 + 0.001 0.0426 0.0011
-0.0426 + 0.0011
Constant
0.5 : 0.0
AP
0.0
0.0
0,0 0.0
Weighted ERM
0.7212 + 0.0044
AP
0.0426 + 0.0011
0.0426 + 0.0011
-0.0426 + 0.0011
Constrained
0.72 + 0.002
AP
-.00000003
0.04 1 0.003
0.04 # 0.003
FairGrad
0.7219 0.0027 AP
0.0462 0.0021
0.0462 + 0.0021
-0.0462 + 0.0021
Unconstrained
0.7237 + 0.0054
Eodds
0.1867 + 0.0052
0.2287 + 0.0078
-0.2288 + 0.0078
Constant
0.5 : 0.0
Eodds
0.0 + 0.0
0.0
0,0 0.0
Weighted ERM
0.7234 0.0054
Eodds
0.188 0.0033
0.2314 0.0056
-0.2315 + 0.0056
Constrained
0.72 = 0.004
Eodds
0.019 0.005
0.006 0.005
BiFair
0.736 + 0.009
Fodds
0.041 0.012
0.056
0.022
-0.056 0.022
FairBatch
0.7413 + 0.0014
Eodds
0.1391 + 0.0043
0.1755 + 0.0084
-0.1756 + 0.0084
FairGrad
0.7193 0.0062
Eodds
0.0154 = 0.0051
0.0204 + 0.0098
-0.0204 + 0.0098
Unconstrained
0.7244 +: 0.0051 Eopp
0.0719 + 0.0012 0.1439 + 0.0023
-0.1438 + 0.0023
Constant
0.5 :: 0.0
Eopp
0.0 0.0
0.0
0.0 + 0.0
Weighted ERM
0.72 = 0.0054
Eopp
0.0718 = 0.0013
0.1437 - 0.0026
-0.1436 = 0.0026
Constrained
0.752 + 0.004
Eopp
0.002 0.001
0.005 + 0.001
0.0
BiFair
0.746 + 0.009
Eopp
0.009 + 0.004
0.017 0.009
-0.017
0.009
FairBatch
0.7426 + 0.001
Eopp
0.0429 + 0.0005
0.0858
+ 0.0011
-0.0858 + 0.0011
FairGrad
0.7518 + 0.0069
Eopp
0.0024 + 0.002
0.0049 + 0.004
-0.0049 + 0.004
31

---

Under review as a conference paper at ICLR 2023
Table 20: Results for the Twitter Sentiment dataset with Non Linear Models. All the results averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.715 + 0.0043
AP
0.0392 = 0.0055 0.0392 :: 0.0055 -0.0392 + 0.0055
Constant
0.5
AP
0.0
0.0
0.0
Weighted ERM
0.7183 + 0.0042 AP
0.0427 = 0.0019 0.0427 = 0.0019
-0,0427 = 0.0019
Adversarial
0.7385 0.0075 AP
0.0367 = 0.0027 0.0367 = 0.0027
-0.0368 = 0.0027
FairGrad
0.7154 + 0.0047 AP
0.0368 + 0.0079 0.0367 + 0.0078
-0.0368 + 0.0079
Unconstrained
0.7167 = 0.0126
Eodds
0.1854 = 0.0061 0.2349 = 0.0091
-0.235 = 0.0091
Constant
0.5 0.0
Eodds
0.0 0.0
0.0
0,0 0.0
Weighted ERM
0.718 + 0.0137
Eodds
0.1882 + 0.0062 0.2379 + 0.0073
-0.2381 + 0.0073
Adversarial
0.7393 + 0.0024
Eodds
0.0382 + 0.0056 0.06 +
+ 0.0151
BiFair
0.74 = 0.01
Eodds
0.039 0.016
1.058
-0.058 0.017
FairBatch
0.7318 + 0.004
Eodds
0.1313 = 0.0057 0.1724
+ 0.0055
-0.1725 0,0055
FairGrad
0.717 0.0082
Eodds
0.0109 0.0027 0.0165
+ 0.0053
-0.0165 + 0.0053
Unconstrained
0.7147 0.0118 Eopp
0.0653 = 0.0062 0.1306
0.0124
-0.1306 + 0.0124
Constant
0.5 = 0.0
Eopp
0.0 0.0
0.0 0.0
0.0
Weighted ERM
0.7074 0.0158
Eopp
0.0672 + 0.0062
0.1346 0.0125
-0.1345 0.0125
Adversarial
0.7471 + 0.0042 Eopp
0.005 + 0.0035
+.0099
-0.0099 +.0009900000 0.007
BiFair
0.747 + 0.009 Eopp
0.007 0.005 0.013
-0.013 0.01
FairBatch
0.7359 + 0.0011
Eopp
0.0368 = 0.0012 0.0736
+ 0.0025 -0,0736 0.0025
FairGrad
0.7401 0.0059
Eopp
0.0049 + 0.0041
0.0099
+ 0.0083
-0.0099 + 0.0083
BL06
FainGrad
Fai-Grad
BLGG
Unconstrained
Equal Opportunity
Fai-Batch
Accuracy Partyy
0.04
Weighted Ere
Equal Oéds
0.04
Usconstrased
0.04
Usconstrateed
Constraints
Weightod Erm
Weighted Erm
0.00
0.00
Constraints
8,00
Constraints
0.00
0,00
0,00
0.00
0.00
0.0L
0.000
0.00
0.00
Accuracy
Accuracy
Accuracy
(a) Linear AP
(b) Linear -OOdds
(c) Linear EOpp
11.45
11.03
Accuracy Party
Advenariaal
Unconstrined
Equal Odés
Unconttrined
Equal Opportunity
Uncomatransed
FaieBatch
Weighted Erm
Advenariial
Advwnsrial
0.08
0.08
Weighted
0.08
0.00
0.00
0.00
1..0
0.58
0.83
1.00
0.35
0.47
0.70
0.81
0.35
0.58
0.83
Accuracy
Accuracy
Accuracy
(d) Non Linear AP
(e) Non Linear EOdds
(1) Non Linear EOpp
Figure 15: Results for the Folktables Adult dataset with different fairness measures.
32

---

Under review as a conference paper at ICLR 2023
Table 21: Results for the Folktables Adult dataset with Linear Models. All the results are averaged runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (L) ACCURACY
MEASURE MEAN ABS.
MAXIMUM
MINIMUM
Unconstrained
0.7905 0.0033 AP
0.0131 + 0.0021
0.0123 + 0.0021
-0.0138 + 0.0022
Constant
0.666 = 0.0
AP
0.053 0.0
0.056 0.0
0.051 0.0
Weighted ERM
0.7906 + 0.0032 AP
0.0127 0.0023
0.0119 0.0022
-0.0134 0.0024
Constrained
0.467 + 0.115 AP
0.036 + 0.003
0.039 + 0.003
0.034 0.003
FairGrad
0.7837 + 0.0049 AP
0.0023 + 0.0009
0.0023 + 0.001
-0.0022 + 0.0008
Unconstrained
0.789 + 0.0026 Eodds
0.0301 = 0.011
0.0377 = 0.0153
-0.0458 + 0.0184
Constant
0.667 + 0.0
Eodds
0.0 + 0.0
0.0 0.0
0.0 + 0.0
Weighted ERM
0.7886 + 0.0032
Eodds
0.0294 + 0.012
0.0364 + 0.0169 -0.0443 + 0.0206
Constrained
0.663 + 0.032
Eodds
0.008 0.003
1013
0,004 0.002
BiFair
0.768 + 0.007
Eodds
0.008 0.005
0.011
-0,011 0.008
FairBatch
0.788 0.0027
Eodds
0.0045 = 0.0033
0.0069
0.0065 -0.0063 + 0.0049
FairGrad
0.7885 + 0.0027
Eodds
0.0043 = 0.0019
0.0073
+ 0.0037
-0.0068 + 0.0045
Unconstrained
0.7902 + 0.0038 Eopp
0.0094 = 0.0031
0.0162 + 0.0053 -0.0215 + 0.0071
Constant
0.667 + 0.0
Eopp
0.0 + 0.0
0.0
0.0 0.0
Weighted ERM 0.7893 + 0.0031
Eopp
0.009 + 0.003
0.0155 + 0.0051
-0.0206 + 0.0069
Constrained
0.706 + 0.002
Eopp
0.004 0.0
0.01
0,0 0.0
BiFair
0.77 = 0.002
Eopp
0.019 0.01
0.033
0.017
-0.044 0.023
FairBatch
0.79 = 0.0031
Eopp
0.0012 = 0.0015
0.0022
0.0026
-0.0026 0.0034
FairGrad
0.7893 + 0.0026
Eopp
0.0011 = 0.0009 0.0024 0.002
-0.0021 = 0.0016
Table 22: Results for the Folktables Adult dataset with Non Linear Models. All the results are averaged over runs. Here MEAN ABS., MAXIMUM, and MINIMUM represent the mean absolute fairness value, the fairness level of the most well-off group, and the fairness level of the worst-off group, respectively.
FAIRNESS
METHOD (NL) ACCURACY 1
MEASURE MEAN ABS
MAXIMUM
MINIMUM
Unconstrained
0.8037 0.0037 AP
10031 0.0123 + 0.0016 -0.0139 + 0.0017
Constant
0.666 + 0.0
AP
0.053 0.0
0.056
0051:00
Weighted ERM
0.8046 0.0049 AP
0 0.0014 0.0123 0.0014
-0.0138 0.0015
Adversarial
0.8016 = 0.0053 AP
0.0122 = 0.0016 0.0115 0.0015
0.0000111 0.0016
FairGrad
0.7917 = 0.0025 AP
0.0016 + 0.0011 0.0016 + 0.0011 -0.0016 0.001
Unconstrained
0.7947 +: 0.0078 Eodds
0.0314 + 0.0059 0.0373 + 0.0058
-0.0454 + 0.0066
Constant
0.667 + 0.0
Eodds
0.0 - 0.0
0.0 0.0
0.0 + 0.0
Weighted ERM
0.7902 + 0.0049
Eodds
0.0327 + 0.0061
-0.0488 + 0.0077
Adversarial
0.806 + 0.0047
Eodds
0.0035 + 0.0018
0.0051 + 0.0021
-0.0053 + 0.0028
BiFair
0.793 + 0.006
Eodds
0.006 = 0.003 0.007 = 0.003
-0.007 + 0.004
FairBatch
0.8061 = 0.0044
Eodds
0.0051 + 0.0015 0.0087 = 0.0048
-0.0084 + 0.0029
FairGrad
0.7997 = 0.0087
Eodds
0.0045 + 0.0029 0.0067 + 0.0045
-0.0071 + 0.0058
Unconstrained
0.7902 + 0.0044 Eopp
0.0097 + 0.0026 0,0168 + 0.0045 -0.0222 + 0.006
Constant
0.667 0.0
Eopp
0.0 = 0.0
0.0 0.0
0.0 0.0
Weighted ERM
0.7947 = 0.0022
Eopp
0.0105 + 0.0027
0.0181 0.0047
-0.024 0.0062
Adversarial
0.8108 0.0161
Eopp
0.0034 +: 0.0057
0.0041
0.005 -0.0095 0.017
BiFair
0.793 + 0.008
Eopp
0.028 + 0.017 0.048 + 0.029
-0.064
0.039
FairBatch
0.8038 0.0063 Eopp
0.0008 + 0.0005
0.0014 1..001440.0000 0.0009
-0.0018 0.0012
FairGrad
0.8058 + 0.0035
Eopp
0.0014 + 0.0014
0.003 + 0.0031
-0,0026 + 0.0024
33


Reviews:
[{"review_id": "qJ0wCw5fMx", "reviewer": "Reviewer_chVG", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a python package and an approach called fairgrad. The goal of fairgrad is to learn model parameters that satisfy accuracy parity or its approximate equivalent across groups of a dataset. Fairgrad does this by formulating a constrained optimization problem, which is to minimize a model perfomance metric subject to accuracy parity constraints across demographic groups that partition the training dataset. This formulation can then be relaxed and put in a langrangian form where the lagrange multipliers are applied to the accuracy parity constraint for each group. This entire formulation can then be 'solved' via an alternating gradient descent formulation. The first component of the alternating gradient descent formulation applied gradient descent to the langrange multipliers. Given the current setting of the langrange multipliers, gradient descent is then applied again to the model parameters. This scheme is iterated until convergence or a pre-specified number of iterations. The Fairgrad package implements this scheme automatically. The empirical performance of the proposed scheme is demonstrated on a variety of benchmarks.", "summary_of_the_review": "The paper proposed a formulation, FairGrad, that is presented in a python package that is then tested across a wide variety of datasets. Overall, the proposed approach can be easily applied by others in the community.", "strength_and_weaknesses": "### Strength\n\nIn my opinion, the primary strength of this work is the python package that is being released as part of this paper. Given the package, the proposed formulation can then be applied across a variety of different settings more easily. In my opinion, this alone, is an important enough contribution of this paper.\n\n\n### Weaknesses\n\n- Novelty in formulation: As a reviewer, I actually don't place any emphasis on the 'novelty' of a work, so I don't see this as a weakness. My only point here is that the formulation here is not different that those in [Jiang & Nachum, Cotter et al.] and others that all employ the Langrange multplier formulation. The key difference seems to be that this formulation allows for negative weights/multipliers which other formulations don't. I think the paper is written to introduce fairgrad as a new method, but in my opinion it is actually a generalization of existing methods. Perhaps a better pitch for this paper would be that these existing methods are generalized and implemented as part of clean python/pytorch package. In my opinion, that alone, is a worthy ICLR submission. The push to make Fairgrad a unique method  seems a stretch to me. I am willing to change my opinion if the authors believe that this is not the case.\n\n- The empirical results are helpful, but I would recommend that the authors jettison the traditional Adult dataset and use the newly proposed Adult dataset from the Neurips 2021 paper by Ding et. al. titled retiring adult. Results on that dataset would be more convincing. \n\n- In the experiments, reporting the average fairness level for all groups is slightly misleading since all the improvement might be in the dominant group. I think these should be broken into groups.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "aVwNqghxYN", "reviewer": "Reviewer_BapZ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work, the authors concentrate on variants of the fair classification problem. They unify some of the most popular fairness notions in a general representation framework by defining a quantity whose magnitude reveals (dis-)advantage to specific groups. By using the aforementioned quantity, they represent the fair classification optimization problem, derive a gradient descent (GD) algorithm to solve this problem and provide intuition on the \"weights\" that are used to update the GD iterations by bridging them with the protected groups.", "summary_of_the_review": "In general, the paper covers an interesting topic, provides a useful Python package, and has extensive numerical experiments; however, I think the work lacks novelty as the problems solved and the solution techniques are all known.", "strength_and_weaknesses": "**Strength:** I believe the literature is summarized very well, and the paper is written well in general. The motivation is clear and the topic covered is relevant to the research our community is conducting. I really like equation (1), and the related proofs in Example 1 and Appendix B.\n\n**Weaknesses:** \n*I list my major and minor concerns that made me recommend a rejection. However, I still like some aspects of the paper, hence I hope that my comments would be found useful by the authors to improve the paper's quality. I am also going to stay active during the discussion period in case the authors have updates or questions regarding my review.*\n\n***Major Weaknesses/Questions***\n- I think this framework is just providing us with a higher-level notation for the fair classifier optimization problem, but unless I am not seeing something, we already know these gradient descent updates in each of the covered settings. For example, take the algorithm explained in 3.1, and replace $F_k(\\mathcal{T}, h_\\theta)$ with the specific definition of Accuracy Parity (AP). Then the resulting gradient descent algorithm is already known. Same for any other fairness notion covered in this work. In general, I do not think there is much advantage that comes with the new expression $F_k(\\mathcal{T}, h_\\theta)$ in addition to providing some intuition.\n- The optimization/duality/GD discussions are lacking assumptions, discussions, and citations. The convexity of the functions is never discussed, but duality is being used. Convergence to local solutions instead of global solutions is not 'warned'. The paper concentrates on the computational tools and numerical experiments that the theoretical arguments look weaker. \n- The proposed 'weighting' is obtained immediately in the alternating GD setting. There is no selection of these weights, rather everything is a corollary of using GD.\n- Lemma 1 looks redundant, as negative weights come automatically unless we rule them out.\n- Further questions and minor weaknesses that are enumerated next.\n\n***Minor Weaknesses/Questions***\n- Introduction: Maybe the authors can mention that the designing phase of the algorithms has access to the so-called sensitive attributes, as there is a new line of research where the optimizer is oblivious. See, e.g., Mozannar, H., Ohannessian, M., & Srebro, N. ICML (2020)\n- In the introduction, the 'reweighting scheme' is not clear. This is understood later.\n- Page 2: \"They are also often limited in the range of ...\" the word 'often' sounds a little subjective. Could the authors please consider adding citations to or deleting such sentences?\n- Page 2: \"gradient descent based methods\" -> do the authors mean gradient descent based solutions instead?\n- Page 2: \"update the weights of the examples\" is not clear yet.\n- python -> I think 'p' should be capitalized\n- Page 2: while discussing that the library will be very similar to PyTorch functions, could the authors please also discuss that the user needs to provide the sensitive attributes?\n- I personally think the notation abuse of $\\mathbb{P}(E)$ referring to both empirical and true probabilities is more than a slight notational abuse. For example, in equation (1), the term $\\mathbb{P}(h_\\theta(x) \\neq y | \\mathcal{T}_{k'})$ is hard to grasp.\n- Section 2.1: When the authors state $F_k$ as a representation of whether or not a group is advantaged, maybe they can highlight this with respect to a classifier. Sometimes in fairness literature \"advantaged\" groups mean those who have historically been associated with better/higher rewards.\n- Domain of $F_k$ includes the set of distributions, but $F_k(\\mathcal{T}, h_\\theta)$ takes a dataset. Probably the notation should be formalized with $\\mathcal{T}$ denoting the **empirical distribution** of the training set instead.\n- Maybe the authors can define $F_{(r)}$ notation uses $(r)$ as the group number $r$ refers to?\n- \"$0.01$ .. is a good rule of thumb\" -> why? This sounds subjective.\n- Section 3.4, \"$80\\%$ rule in the US\" -> similarly, there are no references here.\n-  Minor comment: In the appendix, the proof is re-sated as Lemma 2, maybe this can be fixed to Lemma 1 again?\n- The notation $[1,K]$ is confusing as $k \\in [1,K]$ is not continuous, hence we probably need the $k \\in [K]$ shorthand instead.\n- In the optimization problems `arg min` and `st.` are not aligned.\n- The gradient on top of page 4 is missing its function.\n\n***Optimization Related Issues/Questions***\n- Firstly, problem (2) is very general and there are lots of very strong techniques proposed to solve special cases of this problem. I think the use of GDs should be motivated further. \n- Shouldn't the Lagrangian dual problem flip the max and min operators (e.g., in (3))? Because the Lagrangian dual function first minimizes the Lagrangian function over the primal optimization parameters, and the Lagrangian dual problem maximizes the Lagrangian function over the dual variables $\\lambda$?\n- To use duality we need a lot of assumptions. Here nothing is mentioned. The compactness of the feasible sets, convexity, whether or not strong duality holds, whether the proposed algorithm converges to a global solution or not and whether it means the fairness is not satisfied due to the relaxations, are not discussed. **Update: The authors replied to this concern, hence I am marginally increasing my score.**\n- The optimization problems use $\\arg\\min$ instead of $\\min$. However, $\\arg\\min$ returns sets and the duality arguments are for optimal values instead. It is not very usual (even correct) to use $\\arg\\min$ in my view. Unless we try to address a specific optimal solution that belongs to the $\\arg\\min$ set, if we are referring to an optimization problem, the standard approach is to use $\\inf$ (or $\\min$ if existence is discussed).\n- Does the use of surrogates result in the promised fairness level not holding anymore? In general, the proposed method might give *a* solution, but is it feasible? Overall, I am not very convinced with the current discussions. \n- The values $\\omega_k$ seem to be coming immediately from the gradients, am I wrong? I think they are an immediate result of splitting the updates of the primal and dual variables.\n- When it comes to the proposed decomposition of alternatingly updating the primal and dual variables, there are many existing works already, hence I believe this approach is not new and looks like a less sophisticated version of the primal-dual methods that the literature uses.\n- Does \"Algorithm 1\" add anything to the paper? Maybe it should be clear from Section 3.1.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "QBZIBkMzgZT", "reviewer": "Reviewer_93K5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose group reweighting algorithms for achieving group fairness.", "summary_of_the_review": "see my comments above.", "strength_and_weaknesses": "+: \n\nThe performance of the algorithm seems to be better than the state of the arts.\n\nThe paper writing is clear.\n\n-:\n\nThe technical novelty seems limited. The difference between this work and the previous work [Agarwal et al., 2018] seems very marginal. The only difference here is that this work uses a gradient descent ascent algorithm while Agarwal et al. used the online convex optimization algorithm (exponentiated gradient) with the best response for one of the two players. \n\nFurthermore, the difference between this work and the previous work [Roh et al., 2020] also seems marginal -- (1) min max optimization vs bilevel optimization and (2) negative group weights are allowed. \n\nThe breadth of baseline algorithms is somewhat limited. Many new open-source frameworks implement a large number of fairness algorithms, and I believe that they will be helpful in running more extensive evaluations. \n\nAlso, many of the algorithms have tuning knobs, so it's more helpful if one compares the entire trade-offs achieved by each method.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "7dUje60R7g", "reviewer": "Reviewer_xqvP", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose a modification to Stochastic gradient descent where they alternatively learn the lagrange multipliers for the fairness constraint along with learning the parameters of the model during training. Fairness here is modelled as a constraint, and different data-points are associated with different weights (one weight per class). They empirically show the power of this method, and analyze the accuracy/fairness tradeoff obtained by this method and compare it against full constrained optimization.", "summary_of_the_review": "Overall, the paper is well-written and the direction is important and promising. However, I would like the authors to address the above weakness in a meaningful form so that the submission can be made even more stronger. given a reasonable response to the above, I would be able to move my score based on that.", "strength_and_weaknesses": "Strength\n\n+ Although the paper positions this as a problem of addressing fairness, this method can be applied more broadly to any domain where the dataset contains multiple distributions, and we would like the hypothesis to not reflect the same underlying distribution from which the dataset is generated and learn to generalize across the various classes equally well. Fairness is one such example, but there are many practical applications to this such as data coming from different distributions (e.g., dataset collected over many days from a slowly changing environment).\n\n+ The procedure is very simple to implement, and can be readily accommodated in a typical SGD training loop. Moreover, the additional overhead is very minimal since the aditional information the algorithm requires (i.e., average accuracy per group) is anyway computed in a typical SGD update step.\n\nWeakness\n\n+ The first weakness of the paper I find is that the paper does not compare to more baselines that are natural. First, since we know the groups already, one could in-principle first compute the ratios of prevalence of the various group in a random sample of the dataset and use an IPS estimate to correct for this bias (i.e., weight_i = 1/Pr[random sample belongs to class i]). The advantage with this method is that, this is direct and one does not need to worry about learning additional parameters. Second, one could try to learn the weights lambda outside of the SGD update using some form of non-differentiable method (such as BO) which are much more sample efficient. \n\n+ The second weakness of this paper is that the paper does not try this method on more modern datasets/architectures (e.g., GLUE/superGLUE and transformer). This may be needed, because even minor additional overhead during training will be more visible when training on large models on big datasets. I would recommend the authors to perform ablations on overhead more thoroughly and/or add the caveat that the overhead could be minimal but more experiments are needed.\n\n\n--- EDIT\n\nThe authors have run additional experiments to resolve both my main worries: scalability as a function of architecture complexity and natural baselines. So I am increasing my score.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject