FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods
Authors: Xiaotian Han
Jianfeng Chi
Yu Chen
Qifan Wang
Han Zhao
Na Zou
Xia Hu
Venue: ICLR 2024 poster (2024)
Keywords: Fairness Benchmark
Bias Mitigation
Algorithmic Fairness
Primary area: datasets and benchmarks
TL;DR: The Fair Fairness Benchmark (FFB) is introduced as a comprehensive framework for assessing in-processing group fairness methods in machine learning.

Abstract: This paper introduces the Fair Fairness Benchmark (FFB), a benchmarking framework for in-processing group fairness methods. Ensuring fairness in machine learning is important for ethical compliance. However, there exist challenges in comparing and developing fairness methods due to inconsistencies in experimental settings, lack of accessible algorithmic implementations, and limited extensibility of current fairness packages and tools. To address these issues, we introduce an open-source standardized benchmark for evaluating in-processing group fairness methods and provide a comprehensive analysis of state-of-the-art methods to ensure different notions of group fairness. This work offers the following key contributions: the provision of flexible, extensible, minimalistic, and research-oriented open-source code; the establishment of unified fairness method benchmarking pipelines; and extensive benchmarking, which yields key insights from 45,079 experiments, 14,428 GPU hours. We believe that our work will significantly facilitate the growth and development of the fairness research community. The benchmark is available at https://github.com/ahxt/fair_fairness_benchmark.

Published as a conference paper at ICLR 2024
FFB: A FAIR FAIRNESS BENCHMARK FOR IN- PROCESSING GROUP FAIRNESS METHODS
Xiaotian Han Jianfeng Chi Yu Chen² Qifan Wang² Han Zhao" Na Zou Xia Hu
'Texas A&M University Meta AI AAnytime Al 'UICC SUniversity of Houston 'Rice University
han@tamu. edu (jianfengchi, wafcr)emeta com ycheneanytine-ai.. com hanzhao@illinois. edu nzou2@uh. edu xia.hu@ries edu
ABSTRACT
This paper introduces the Fair Fairness Benchmark (FFB), a benchmarking frame- work for in-processing group fairnsss methods. Ensuring fairness in machine learning is important for ethical compliance. However, there exist challenges comparing and developing fairness methods due to inconsistencies in experimental settings, lack of accessible algorithmic implementations, and limited extensibility current fairness packages and tools. To address these issues, introduce open- source standardized benchmark for evaluating in in-processing group fairness methods and provide comprehensive analysis of state-of-the-ee methods to ensure differ- ent notions of group fairness. This work offers the following key contributions: the provision of flexible, extensible, minimalistic, and research-oriented open-source code; the establishment of unified fairness method benchmarking pipelines; and extensive benchmarking, which yields key insights from 45, 079 experiments, 14, 428 GPU hours. We believe that our work will significantly facilitate the growth and development of the fairesss research community. The benchmark is available https: /gg comp http///tttaaaaaaaaa/////a fainneesbbbbbbbbbccanr
INTRODUCTION
Machine learning models trained on biased data have been found to perpetuate and even exacer- bate the bias against historically underrepresented and disadvantaged demographic groups when deployed (Mehrabi et al., 2021; Pessach & Shmueli, 2022; Caton Haas, 2020; Wan et al., 2023). As result, concerns about fairness have gained significant attention, especially applications of these models expand to high-stakes domains such as criminal justice, hiring process, and credit scoring. To mitigate such algorithmic bias, variety of fairness criteria and algorithms have been proposed, which impose statistical constraints on the model to ensure equitable treatment under the respective fairness notions (Hsu al., 2022; Chai et al., 2022; Reddy al., 2021b). However, fair and objective comparison between the proposed and existing algorithms to enforce fairness can be difficult due to the following reasons:
Hard to compare the performance of two objectives: utility' and fairness. Often, there is a trade-off' between these two objectives (McNamara et al., 2019: Zhao & Gordon, 2022: Xian et al., 2023). Besides, the instability of the fairness performance of those methods during the training process can complicate the pursuit of an optimal balance (Xian et al., 2023).
Inconsistent experimental settings (Madras et al., 2018; Lowizos al., 2016): Faimess methods can also be hindered by variations in dataset preprocessing and the use of different backbones. These discrepancies can lead to inconsistent (Table 1) and unfair comparison, further complicating the comparison of methods.
Stable and customizable implementation of commonly used fairness methods are not accessible. The fairness methods are often implemented in different programming languages and frameworks, complicating the reproduction and the comparative analysis of various fairness approaches.
*This work was done while the author was an intern at Meta.
WW use utility represent the performance of the downstream tasks, which is commonly used in faimesss community (Dwork al., 2012; Li & Liu, 2022; Baumann al., 2022).

---

Published as a conference paper at ICLR 2024
Current fairness packages (Bellamy et al., 2018; Bird al., 2020) and tools often suffer from a lack of extensibility. This can make it difficult for researchers and practitioners to build upon existing methods and develop new ones.
This work aims to facilitate the growth and development of the fairness research community by addressing existing challenges, promoting more accessible and reproducible methods for fairness implementation, and establishing efficient benchmarking techniques. To achieve these goals, we develop a standardized benchmark for evaluating and comparing in-process group fairness methods, which we make open-source. We also conduct comprehensive experiments and analysis of fairness methods. The major contributions of our benchmark are summarized follows:
We offer open-source implementation for all preprocessing, methods, metrics, and training results, thus facilitating other researchers to utilize and build upon this work. The Fair Fairness Benchmark is publicly available, making easy for researchers and practitioners to use and contribute to.
Extensible, Minimalistic, and Research-orientee Open-Source Code:
Unified Fairness Method Benchmarking Pipelines: Our benchmark includes unified fairness method development and evaluation pipeline, with three key components: First, we provide a thorough statistical and experimental analysis of widely-used fairness datasets and identity some widely-used datasets unsuitable for studying faimess issues; second, we standardize preprocessing for these datasets, ensuring consistent, comparable evaluations; lastly, we present range of bias mitigation algorithms and comprehensive evaluation metrics for group faimess.
Comprehensive Benchmarking and Detailed Obsevations: We conduct comprehensive experi- ments on 14 datasets (each with two sensitive attributes), 6 utiliity metrics and 9 fairness metrics. We run a total of 45, 079 experiments. Our experiments yield the following key insights: ① Not all widely used fairness datasets stably exhibit fairness issues. (Important for evaluation). @ The HSIC achieves the best utility-faan ffirness trade-off among the tested methods. (Not popular baseline before) @ Utility- fairness trade-offs are generally controllable by the hyperparameter. (Despite adversarial debiasing method) @ Stopping training while learning rate is lower enough effective. (Practical strategy).
Scope of this Work. We focus on the problem of in-processing group fairness, which is defined as discrimination against demographic groups. We consider the fairness methods in the context of binary classification and binary sensitive attributes.
Notation. The dataset is denoted as {(x,, $t, 1) where N is the number of samples. For the sample i in the dataset, x; E Rd is non-sensitive attributes, E {0, }} represents the binary sensitive attribute, and y/ E {0. 1} is the label of the downstream task. We use ú E {0, 1} to denote the predicted label of the downstream task, which is obtained by thresholding the output of machine
learning model fx) Idd + [0, 1] with trainable parameter 0. Accordingly, we use X, Y, S and Y to denote the random variables.
WHY IS THIS BENCHMARK NEEDED?
Ensuring fairness in algorithmic predictions is crucial in the field of machine learning. However, fairly comparing the current fairness method challenging due to inconsistencies in data pre- pre-processing and a lack of flexibility in existing fairness packages. This section aims to analyze the critical issue of current fairness methods and discuss the urgent need for a benchmark to address these challenges.
Current fairness packages lack flexibility for researchers. AIF360 (Bellamy et al., 2018) and Fair- Learn (Bird al., 2020) are two well-known fairness packages that have successfully mitigated bias in machine learning algorithms. As popular open-source Python packages, they provide practitioners with toolkits for detecting and mitigating bias their models and evaluating model fairness. AIF360 comprehensive fairness toolkit offering variety of algorithms and metrics for addressing bias in machine learning models. FairLearn also provides multiple algorithms and fairness metrics for assessing model performance. While both packages are highly recognized within the fair machine learning community, they may not give researchers the desired flexibility for research purposes. We provide more in-depth comparison between AIF360, FairLearn, and FFB in Appendix A.
Inconsistent experimental setting leads to unfair comparison. Prior research has often experienced inconsistencies in data pre-processing and train test split, which has led to divergent performance
2

---

Published as a conference paper at ICLR 2024
Table Fairness definitions and metrics used the experiments. Appendix lists a more compre- hensive list of fairness metries and their details. Their simple code implementations are at this url.
Abbreviation
Definition
dp (Dwork et al., 2012)
Demographic//aatisticlll Parity
P(P s = r(1
prule (Zalar al., 2017)
p- Rule
P(P =1 S= 1)/P((( =1:0 p/100
ppv (Chouldechova, 2017) 7)
Predictive Parity Value Parity
P(Y =1 -
bnego (Kleinberg
2016)
Balance for Negative Class
B[((X) = ==0.8=0=3(((0) 0,
bposc (Kleinberg et al., 2016)
Balance for Positive Class
B[((X) ==1.5=0]=3//[x]
eopp (Hardt 2016)
Equality Opportunity
P( =1] = |
eodd (Hardh et al., 2016)
Equalized Odds
P( = = P( 1= 1011==10 g) {0
(Han al., 2023)
CDF
So (Fota) (x)x)
results that hinder comparison and reproducibility. Minor variations in data preparation and dataset split can significantly impact the performance of machine learning algorithms. Due to these issues, the reported accuracy of tabular data aadult,German varies in different papers (Madras et al., 2018; Zemel et al., 2013; Edwards & Storkey, 2015; Feldman et al., 2015; 5; Louizos al., 2016) shown in Table 1. To tackle these issues, we propose a standardized and consistent data pre-processing and split approach, including data normalization, outlier removal, and the implementation of uniform train-test split ratio.
Sufficient and in-depth analysis of fairness meth- ods is urgently needed. The current fairness method lacks comprehensive comparison, such the training curves and stability, the influence of the millity-fairness, trade-off parameters, and how to select the best utility- fairness trade-offs during the training process. Our benchmark addresses these shortcomings by offering more in in-depth and multifaceted analysis. To present more thorough understanding of fairness methods, we investigate the training stability, model performance under various fairness constraints, and the selection of best-performing models.
Table 1:
The reported accuracy of tabular data varies in different papers.
Paper
Adult German
Madras et al. (2018)
Ro 0.85
Zemel al. (2013)
0.70 0.69
Edwards Storkey (2015)
Ro 0.83
Feldman et al. (2015)
Rs 0.68
Rs 0.69
Louizos et al. (2016)
RJ 0.82
A 0.78
Amax
0.17
0.09
PPercentage
20%
13%
3 FFB: FAIR FAIRNESS BENCHMARK
To overcome the above limitations of the previous methods, introduce the Fair Fairness Benchmark (FFB), a extensible, minimalistic, and research-oriented fairness benchmark package. Compare to other fairess packages, FFB codebase has the following main characteristies:
We provide the source code for fairness methods implementation, allowing researchers modify, extend, and tailor these methods to suit their specific requirements and implement new ideas upon our code. This fosters more customizable approach to developing fairness methods.
Extensible:
Minimalistic: We focus on delivering core fairness methods and allowing users understand the fundamental techniques comprehensively without being overwhelmed by unnecessary complexity. This approach ensures that users can easily implement and integrate our methods into their existing workflows while maintaining a solid grasp of the underlying concepts.
Research-oriente We include benchmark datasets and evaluation metrics that facilitate assessing fairness methods. This simplifies the research process, allowing researchers to quickly compare and analyze the effectivvness of different methods various scenarios.
3.1 GROUP FAIRNESS METRICS
To provide comprehensive comparison for bias mitigating methods, consider multiple faireess metrics, including demographic parity, p-rule, equality of opportunity, equalized odds, the area between CDF curves, etc. Table 2 presents the faimess metrics used in Section 4 and Section 5.
3

---

Published as a conference paper at ICLR 2024
Table 3: The summary of the benchmarking datasets. The #nFeat/#cFeat is the number of the numeri- cal/categorical features and the FallFeat is the total number of the features after our preprocessing. The you /1/80 $1 is the ratio of two classes of the target label and the sensitive attributes. More details about datasets presented in Appendix D. The dataset loading codes this url.
Dataset
Task
SensAur #Instances #nFeat #cFeat #allFeat
(Ist) % (2nd)
Adult (Kohavi Becker, 1996)
income gender, race credit gender, age
2.088
9:20
German (Dua & Graff, 2017)
1,000
13
6
58
1 2.33
1: 1:2:3
4:26
KDDCenSus (Dua & Graff, 2017)
income gender, race
292, 550
32
8
509
1: 14.76
1: 0.99
8.14
COMPAS (Larson al., 2016)
age credit gender, race income gender, race employment gender, race
recidivism
6, 172
400
5
405
1: 0.83
1 1:2.25
Bank Graff, 2017)
41,
62
0.13
1
:7.58
ACS-1 (Ding al., 2021)
195, 663
8
908
0.70
1.8.8
1 1.62
ACS-E (Ding et al., 2021)
378, 817
15
0
187
1 0.84
1:003
1: 1.39
ACS-P al., 2021)
public gender, ruce mobility gender, race
138,
18
0
1696
1 0.58
1:277
1:31
ACS-M 2021)
80,
20
2678
1 3.26
:.95
1 1.32
ACS-T (Ding al., 2021)
traveltime gender, race
172, 508
1567
1 0.94
0.89
CelebA-A (liu et al., 2015)
attractive gender, age wavy hair gender, age
202, 539
48 48
1 0.95
1 1.44
1:2.29
CelebA-W (Liu al., 2015)
599
48
1:1.3
1:1.4
0:2.2
CelebA-S (Liu al., 2015)
smiling gender, age
1.07
1:440
UTKFace (Zhang et ul., 2017)
age gender, race
23. 705
48 x 48
1: 1.15
1:101
1: 1.35
3.2 BENCHMARKING DATASETS
To provide a comprehensive comparison of fairness methods, we adopted multiple commonly-uses fairness datasets (Le Quy et al., 2022) for our experiments, including tabular and image datasets. Table 3 summarizes the datasets used. they are publicly available and can be downloaded from the corresponding websites. We also present the number of features and the ratio of the target label and sensitive attributes. For example, the target label ratio of KDDCensus is 14.76, which is an extremely unbalanced dataset.
3.3
DATA PREPROCESSING
To ensure a fair comparison and maintain the reproducibility of the fairness approach, we adhere to a conventional data preprocessing strategy. We apply standard normalization for numerical features while employing one-hot encoding to process the categorical features. We also split the data into training and test sets with random seeds. We use the training set to train the model and the test set to evaluate the model's performance. We use the same data preprocessing strategy for all the datasets.
3.4 BENCHMARKING FAIRNESS METHODS
In this section, we introduce the benchmarking methodology employed in our experiments. The benchmarking methods be classified into three categories: surrogate loss, independence con- straints, and adversarial debiasing techniques. In this paper, we focus on in-processing methods for fairness primarily due to the following: 1) They intervene directly in the learning algorithm to ensure fairness, which provides more nuanced and effective approach to mitigating bias; 2) The emergence of more in-processing techniques designed in deep neural networks calls for systematic comparison; 3) In- pr-processing methods are susceptible to information leakage since they do not require sensitive attributes during inference. In particular, we consider the following three types of in-processing methods. Gap Regularization (Chuang & Mroueh, 2020) simplifies optimization by offering a smooth approximation to real loss functions, which are often non-convex or difficult to optimize directly. This approach includes DiffDP, Di ffoodd, and Di ffEopp. Independence intro- duces fairness constraint into the optimization process to minimize the impact of protected attributes on model predictions while maintaining performance. This approach includes PRemover (Kamishima et al., 2012) and HSIC (Li et al., 2019). Adversarial Learning² minimizes the utility loss while preventing the adversary from accurately predicting the protected attributes. This approach includes
Forr adversarial learning methods, we gradient reversal (Gamin al., 2016) for better training stability.
4

---

Published as a conference paper at ICLR 2024
Table 4: Bias examination for all datasets. We identify the biased dataset specified with sensitive attributes with the reported utility and fairness performance of ERM. Numbers (c.g., 1.35 mean that the bias is too small, indicating is not suitable for faimess assessment. This only reflects our own recommendation and fairness metrics should be appropriately chosen for different applications. The biased datasets are marked with v, while the unbiased datasets are marked with x. The K indicates that the bias exists but with large standard deviation. The results are based on 10 trials with varying
data splits and training seeds, to ensure reliable outcomes. The table is generated from 910 runs.
Uniiy
Falmess
Dotaset
SenAtrr
acc
auc
ap
dp
abcc
prule
eodd
copp
Recommend?
Bank-
Age
97.17
94,05-0.. 62.4 54.31aia
10.88
10.64
44.48aa3e 10.7
6.18
25.42.2.0a
78.55
80.21
63.02.144
7.361438
4.89
90.47
14.45
2.74
German-
Age
75, 19.3.
77.21-210
28
62.90au
12.20
10.01
84.44
17.974307
6,20
Gender
86.30an
78-20gmm
66.78 78
16.67
18.36
32.54gm
14.16gs
7.90
Race
85.21anar
91.10
78.65440
06.85
12.20
12.59
41.56.2
13.12
8.812m
Gender
72.00 67.07.m
67. m7
59. 30.77
13.43gm
6.80g
65. 63.1224m
19.67 in
11.54ga m
COMPAS
Race
67. 13
72.98
39
74
60
16.83pa.e ce
8.16
0333 os
20,05
KD0Census
Rave
14.492m7s
94,40.a
94.03-004
99 55.
97.13
17.32:
3.61:1c0
1-3623-97
3.31.815
5.20
14.97a2.1 16.35
98-66
6.50aa
0.28cmmn
V
x
Gender
62. 32.30.
90.28....
88.02
77.91inm
9.10
8.27
79.01
3.38
4.75
ACS-T
Race
22.40
90.40
174174
78. 18n1
7.71
77.24
9.72
6.21
ACE-E.
Gonder
81 13
11.99
90.00
on
81.3120 13
81.38:031
4:00
4.56
07-2
96-872me
10.77am
3/882030
x
X
Gonder
71. 12.922n
75.25
67 33
52:93
91. 01.262a9
4-50
x
AES-P-
Race
71. 20..0
75.00..0.1
67.01.n0s
52.061074
0 0-181030
1.98
97.87
1.031
4.03
ACS- M
Gender
76.08amxs
70.81
m3
72.85
a
88.53gmm
85.40
80,54ima1
80.70am1
0.11aumr
0:dBerest
0.84
1.15ga
90-00t
90
0.82g1 ox
0.18
X
X
Gender
06.36
13.54
71.39uu
06.51ams1
8.00am
5.02
12.90 84.65
5.72
ACS-T
Race
66.45enos
13
71.67
66 27
9.62am1
6.07
83:09am
15.19e:
6.50
Gender
78 44
86.67
86.60iane
62.30)
37.67
30.42a
70.84a
33.58e1
CelebA-A
Agu
78, 19
86.07.0.3
86,061
79. 17:077
41 11.9001000
11.15
33
37
02.222200
18.83.1
Gender
82 10.30.ar
88.38-0..
80.38
70.14
33.92.1.0
29.52
16.89.2m
52.71
39.02
Celeba-M
Age
#2.50 42.50
88.18-01
80.38
14.14
10.27
10.61
64.30
10.63 is
6.48gr os
CelebA-S
Gender
$0.95.s.
89.08
14.09
13.02
72.76.a
6.90
6.5larm
Age
80.00ss
96.51
96.67
80.08
5:91
5.50
88.21g5
6.46
0.30
Gender
83.34am
91.78ua.
81 61.6820es
25 68....
20.57a1a
54.6122
17.16:2
UTKFace
Race
83.311mm
91.78..0.7
91.661
$1.66 11.60
23 25
18.99
59.87
23.07
16.08.2m
Gender
68, 10ines
74,81
74.02
65.82
8.17
4.41-1.11
83.11is.
13.35
a.
Jigsom
Race
68 10
74.81:0.0
71.02100
65 82:327
4 4.561a2e
90.25
7.92.a.0
3.18 26
AdvDebias (Zhang et al., 2018: Louppe et al., 2017; Beutel al., 2017: Edwards & Storkey, 2015; Adel et al., 2019) and LAFTR (Madras et al., 2018). The fairness methods are present as follows:
ERM is standard machine learning method that minimizes the empirical risk of the training data. It is common baseline for fairness methods.
Di DiffDP, DiffEopp, DiffEodd are the gap regularization methods for demographic parity, equalized opportunity, and equalized odds (Chuang & Mroueh, 2020). As these faimess definitions cannot be optimized directly, gap regularization is alternative loss and differentiable and can be optimized using gradient descent.
PRemover (Kamishima et al., 2012) (PrejudiceRemover) minimizes the mutual information be- tween the prediction accuracy and the sensitive attributes.
HSIC (Gretton et al., 2005; Baharlouei et al., 2020; Li et al., 2019) minimizes the Independence Criterion between the prediction accuracy and the sensitive attributes.
AdvDebias (Zhang et al., 2018; Louppe et al., 2017: Beutel et al., 2017; Edwards & Storkey, 2015; Adel et al., 2019) (adversarial debiasing) maximize classifier for prediction ability and simultaneously minimizes an adversary to predict sensitive attributes from the predictions.
LAFTR (Madras et al., 2018) a fair representation learning method that aims to learn an interme- diate representation that minimizes the classification loss, reconstruction error, and the ad adversary's s ability to predict the sensitive attributes from the representation.
5

---

Published as a conference paper at ICLR 2024
DECEOP
F3eEove
HBDO
LAFIR
AdvDebiaa
ERR
Utility (acc)
Utility [acc)
Unilnn (anc)
Utility (anc)
0.33
Fuirness (abcc)
Fairuss (dp)
Pairuss (abcc)
Faimesss (dp)
(a) Tabular Data, across Adult, t, German, Bank, KDDCensus, ACS-I/E/P/M/T datasets.
PRanover
E3K
(acc)
Unitty (acc)
Utllity (auc)
Unility (ascc)
05
10
0.3
05
Flimass (aboc)
Faimmess (dp)
Paimes (abcc)
Phimess (dp)
(b) Image Data, across CelebA-A/W/S, UTKFace datasets with multiple targets.
OirEEopp
DiffBood
ERN
DiffEopp DiffBood
1.00
1.00
Utillty (anc)
Unilit (auc]
Unility (axc)
Utiliy (aac)
00
0.50
Fairness (eodd)
Fairness
(eopp)
Fairness (eodd)
Fairnss (eopp)
(c) Tabular Data on eodd and eodd.
(d) Image Data on eodd and eodd.
The utility-faimess trade-offs of current faimess methods DiffDP, PRemover, HSIC, LAFTR, and AdvDebias. To plot the faimess and utility performance in one figure, for each dataset, we normalize the utility (acc,auc) and fairness (abcc, dp) based on the performance of ERM, which denoted as the point (1.0, 1.0). The figures clearly show that utility-fairness exhibits trader-offs. These figures are generated from total of 27568 runs.
Figure 1:
4 BIAS EXAMINATION FOR WIDELY USED FAIRNESS BENCHMARK DATASETS
The presence of bias in the current widely used dataset is not well examined and investigated it should be, even though such bias can significantly influence the assessment of fairness methods. As such, our work endeavors to delve deeper into this issue by exploring the inherent bias in the widely used dataset. We aim to assess the suitability of these datasets for fairness evaluation critically.
4 Not all widely used fairness datasets stably exhibit fairness issues. We systematically identify datasets that not only have demonstrated bias but are also prevalently used in fairness research. We found that in some cases, the bias in these datasets is either not consistently present its manifestation varies significantly. This finding indicates that relying on these datasets for fairness analysis might not always provide stable or reliable results, suggesting the need for more rigorous dataset selection or bias evaluation methodologies in future fairness studies. The biased datasets are marked with / while unbiased ones are with x. The X indicates that the bias exists but with a large standard deviation.
BENCHMARKING CURRENT FAIRNESS METHODS
This section presents comprehensive experiments to benchmark the performance of existing in- processing group fairness methods.
5.1 How THE BIAS MITIGATING METHODS PERFORM ON UTILITY-FAINNES TRADE-OFFS?
In this section, we present the results of experiments conducted to assess the performance of existing in processing faimess methods in terms of the utility-faimess trade-offs. We analyze how well these methods balance optimizing utility and ensuring fairness in decision-making. To accurately reflect the performance of the different methods, aggregate the performance across different datasets in

---

Published as a conference paper at ICLR 2024
DiFIOF
PRemovex
HSTC
AdvDebiaa
LAFTR
Nonnalloed dp
Nomuaaioe dp
Normalioet dp
Normalzzel dp
6.50
Normalleed
0.5
0..5
0.25
0.0
50
100
160
0.00
0.00
60
T0U
150
50
T0U
120
50
100
150
Triining Stops
Training Stops
Treining Steps
Training Steps
Thiining Stopp
DiffOP
PRemover
Nomalized aboc
Notralieed dize abeo
Noet slee aboo
Nomrallied aliel aboo
1.0
0.50
0.4
0.500
0.5-
0.25
110
150
1.00
100
Thiiing Steps
Tratiin Stegus
Training
Triinigg Sterss
Trining
Figure 2: The fairness performance with varying fairness control hyperparameters. The intensity of the color represents the size of the control parameters. In most cases, the larger value of control parameters yields better fairness performance, while small ones have worse fairness performance. These figures are generated from 13110 runs of experiments.
Trido
vil
Leal
FRM
Difrop
Plenover
RSTC
AdvDebiat
1AFTTR
70
70
40
150
10
5.1
8
20
164
150
50
100
150
50
300
150
100
150
100
150
10
4b05
abcc
sboo
11:
223
11
300
1041
150
100
120
100 Tranting Stape
100
1000
170
Trancnng
Str
Trotzing Stope
Tewiting Strpe
Troning Stope
Trining Stogs
Figure 3: The training curves on tabular dataset. The training curves for fairness metrics typically have larger standard deviation than utility performance, showing the instability of fairness performance.
one figure. To do so, we normalize the utility (acc,auc) and fairness (abcc, dp) performance based on the performance of the ERM. From the results, make the following major observations:
@ The utility-fairness performance of the current fairness method exhibits trade-offs. We first present the fairmes trade-offs of the existing in- in-processing fairness methods in Figure 1. We conduct experiments using various in-processing fairness methods and analyze the ability to adjust the trade-offs cater to specific needs while maintaining balance between accuracy and fairness.
The HSIC method achieves the best -ailety-fairness trade-off overall. The HSIC method consistently excels in balancing utility and fairness, outperforming other approaches across our tests. This method, depicted in green in Figure 1, shows particular effectiveness when applied tabular data. It exhibits significant ability improve fairness measures without compromising the precision of utility, maintaining high accuracy in predictions. This quality affirms the robustness of the HSIC method in preserving utility-fairness equilibrium under various conditions. However, when this method is applied to image data, it exhibits relative performance decline, showing lower fairness and utility scores.
① Adversarial debiasing methods exhibit instability, As illustrated in Figure 1, the utility-fairness points representing the AdvDebias method are scattered randomly across the figures, failing to depict consistent trade-off pattern. This randomness suggests inherent instability in adversarial debiasing methods. This inconsistency further demonstrated in subsequent experiments, where the training curves reveal that these methods difficult control effectively.
7

---

Published as a conference paper at ICLR 2024
5.2 CAN THE UTILITY-FAIRNESS TRADE-OFFS BE CONTROLLED?
Hereby we investigate the extent to which the utility-fairmess trade-oft's can be controlled. We conduct experiments using various in-processing fairness methods and analyze the ability of trade-offs.
The -airetys trade-offs are generally controllable. The intensity of color in Figure corresponds the size of the control parameters. With the exception of adversarial debiasing, we find that the performance of most methods can be modulated effectively through the adjustment of hyperparameters. Specifically, larger control hyperparameters tend yield lower dp and abcc values, indicating enhanced fairness. This implies that the trade-offs between utility and fairness can be actively managed in most cases, providing a crucial degree of flexibility in -rriene data processing tasks. In comparison, the adversarial debiasing method (AdvDebias) is hard to control.
5.3 How DO UTILITY AND FAIRNESS PERFORMANCE CHINNE DURING TRAINING PROCESS?
In this section, we thoroughly exam- ine the training curves of existing in- processing fairness methods, which are not sufficiently explored in previ- ous studies. We conduct a series of ex- periments to track the evolution of util- ity and fairness performance through- out the training process and evaluate the impact of these dynamics on the fi- nal results. We presented the training curves in Figures 3 and 4 for tabular data and image data, respectively.
38M/resnet18
DiffDP 1resmet
ISIC/resnet18
80
80-
600
60-
40%
100
150
731
20
20
10
100
150
501
20-
The training curves of utility are stable, while those of fairness are unstable. Results on both tabular and image data show that the standard de- viation for fairness metrics is signifi- cantly larger than that for utility met- rics. Among the faimess methods, LAFTR shows the most stable fairness performance. Even though the value of fairness metrics is small, the large standard deviation still suggests unstable fairness performance. The results indicate future direction focused on enhancing fairness training stability.
aboc
abou
10
10-
50
1100 Steps
100
60
100
150
101
100
150
Training
Thaining
Steps
Teaizing Steps
Figure 4: The training curves on image dataset. The results similar to tabular dataset that training curves for fairness metrics typically have larger standard deviation than utility performance.
/ Stopping training while the learning rate is lower enough proves effective. In our work, we halt the model training when the learning rate diminishes to value less than - which is decayed by multiplying by 0.1 every 50 training steps. The utilization of learning rate decay to halt training results in stable faimness metrics, thereby affirming its efficacy and reasonableness.
5.4 EXPERIMENT ON TEXT DATA
In this section, we conducted experiments on text data, specifically on comment toxicity classification using the Jigsaw toxic comment dataset (Jigsaw, 2018). The task is to predict whether a comment is toxic or not. Part of this dataset is labeled with identity attributes, such as gender and race. In our study, we regard race and gender and race as sensitive attributes. We follow the setting in (Chuang & Mroueh, 2020) to transform each comment into vector using BERT (Devlin et al., 2019). Subsequently, we employ an ML.P to predict based on the encoded vector. We present the result on the hillit--firnnsss trade-offs in Figure 5, training process in Figure 6, and controllability of trade-offss in Appendix H.
From these results, we observe similar trend of utility-fairness trade-offss in text data in tabular and image data in most cases, but there are some slight differences. We made the following observations: 1) Utility and fairness typically exhibit trade-offs. 2) The training stability has an opposite trend compared to tabular/image data. The training curves for utility are unstable, while those for fairness relatively unstable for most bias mitigation methods.
8

---

Published as a conference paper at ICLR 2024
Diccce
Piexovez
EBIC
Usiliy (acc)
Usilit (anc)
Usiliy (acc)
Uullyy
65
Fairness (abcc)
Faimeess
(dp)
Fairnss (aboc)
Fairnes (dp)
Figure 5: The utility--airress trade-offss of current fairness methods - Di ffDP, PRemover, and HSIC on text data. To plot the fairness and utility performance one figure, for each dataset, we normalize the utility (acc,auc) and fairness (abcc, dp) based on the performance of ERM, which is denoted as the point (1.0, 1.0). The figures show that
exhibits trader-offs.
train
val
ERM
DiffDP
HSIC
PRemover
70
60
60-
Bcc
acc
acc
acc
60-
50
40
40
0
30
100
150
0
30
100
150
0
50
100
150
0
50
100
150
Training Steps
Training Steps
Training Steps
Training Steps
ERM
Dif DiffDP
HSIC
PRemover
20
6U
60-
50
25
25
25
0
30
100
150
0
30
100
150
0
30
100
150
0
50
150
Training Steps
Training Steps
Training Steps
Training g Steps
ERM
DiffDP
HSIC
PRemover
40
40
40
10
abcc
abcc
abcc
abcc
5
20
20
20
50
100
150
0
50
100
150
0
50
100
150
0
50
100
150
Training Steps
Training Steps
Training Steps
Training Steps
Figure 6: The training curves text dataset. The training curves for fairness metrics typically have larger standard deviation than utility performance, showing the instability of fairness performance.
6 DISCUSSIONS
This paper introduces Fair Fairness Benchmark (FFB) to benchmark the in-processing group fair- ness models, offering extensible. minimalistic, and research- open- code, as well as comprehensive experiments.
Social Impact. Our benchmark, with its extensible, minimalistic, and research-oriented open- source code, is designed facilitate researchers and practitioners to explore and implement fairness methods. Standardized dataset preprocessing and reference baseline implementation will help reduce inconsistencies and make fairness more accessible, especially for beginners in the field. Ultimately, our work aims to stimulate future research on fairness and foster the development of fairness models.
I Limitations. When the ground truth biased, relying solely on traditional accuracy can be problem- atic, as mentioned in (Wick et al.. 2019; Zajko, 2022; Ruggieri et al., 2023). We employ multiple metrics (accuracy, average precision, AUC) for evaluating "utility" instead of just relying on accuracy, aiming alleviate this problem some extent.
Future work, Our plan for subsequent phases of this work involves extending the scope of the FFB to include wider range of in- in-processing group faimess methods. Moreover, intend to incorporate additional definitions of fairness into our evaluations. Exploring methods to measure unbiased accuracy is promising direction for future research in faimess.
9

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENT
We thank the anonymous reviewers for their constructive suggestions and fruitful discussion. Han Zhao supported in part by a research grant from the IBM-IL Discovery Accelerator Institute (IIDAI). This work is also supported in part by NSF IIS 1900990, 1939716, 2239257, and 1750074.
REFERENCES
Tameem Adel, Isabel Valera, Zoubin Ghahramani, and Adrian Weller. One-network adversarial fairness. In Proceedings of the AAAI, pp. 2412-2420, 2019.
Wael Alghamdi, Hsiang Hsu, Haewon Jeong, Hao Wang, Peter Michalak, Shahab Asoodeh, and Flavio Calmon. Beyond adult and compas: Fair multi-class prediction via information projection. Advances in Neural Information Processing Systems, 35:38747-38760, 2022.
Sina Baharlouei, Maher Nouiched, Ahmad Beirami, and Meisam Razaviyayn. Rényi fair inference. In International Conference Learning Representations, 2020. URL https:// net/ Forum?id=HkgsUJrthn
Joachim Baumann, Anikó Hannák, and Christoph Heitz. Enforcing group fairesss in algorithmic decision making: Utility maximization under sufficiency. In Proceedings of the 2022 ACM Conference Fairness, Accountability, and Transparency, pp. 2315-2326, 2022.
Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanic Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, and Yunfeng Zhang. AI Fairesss 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias, October 2018. URL https:
00000/111000000 01943.
Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint 100.:0707.000755 2017.
Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Mi- lan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. Fairlearn: A toolkit for assessing and improving fairness in ai. Technical Report MSR-TR-2020-32. Mi- crosoft, May 2020. URL https://www www.msoootttommmmmmmmmmmmo.m...t http:://www.microooff..cm fairlearn- a
lassessinn--nd--iae and - inneess--i-ii....
Maarten Buyl and Tijl De Bie. Optimal transport of classifiers faimess. In Advances in Neural
Information Processing Systems, 2022.
Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. Optimized pre-processing for discrimination prevention. In I. Guyon, U. Von Luxburg, S. Bengio, Н. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL https: httpttt:::///ceedtthht /ptp://inoooeeee..///eeee/ neurips httpscccppppreeeeee fitpee/fiile//ppee///////eee 45492255dB45a44faaebbbbr/ 13ee:3333366666P8e
Simon Caton and Christian Haas. Fairness in machine learning: A survey. arXiv preprint arXiv:2010.04053 2020.
Junyi Chai, Taeuk Jang, and Xiaoqian Wang. Fairesss without demographics through knowledge distillation. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances Neural Information Processing Systems, 2022. URL https://openreviewwnee net/ Forum?id-8gjiwWNNSoff
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153-163, 2017.
Ching- Yao Yao
Chuang and Youssef Mroueh. Fair mixup: Fairness via interpolation. In ICLR, 2020.
10

---

Published as a conference paper at ICLR 2024
Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic deci- sion making and the cost fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 17, pp. 797-800 New York. NY. USA, 2017. Association for Computing Machinery. ISBN 97814 9781450348874, doi: 10.1 145/34999999 3098095. URL https::/doi..rrg/// 110e00..1114380 1145/3097983 3098095.
Jacob Devlin, Ming- Weig-Wii&e Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1), 2019.
Jimmy Ba Diederik P. Kingma. Adam: a method for stochastic optimization. CoRR, abs/ 412.6990, 2014.
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt, Retiring adult: New datasets for fair machine learning. NeurIPS, 2021.
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http:htt://archive..t uci.edu/ml.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Conference Innovations in Theoretical Computer Science (ITCS), 2012.
Harrison Edwards and Amos Storkey. Censoring representations with an adversary. arviv preprint ar..v:1511.05897, 2015.
Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubra- manian. Certifying and removing disparate impact. In proceedings the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 259-268, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Nomain-adversarri training of neural networks. Journal of Machine Learning Research, 17(59):1-355 2016. URL htttp://jmle org/ hrrgpppprs/ v17/15-239, html.
Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. Measuring statistical dependence with hilbert-schmidt norms. In International conference on algorithmic learning theory, pp. 63-77. Springer, 2005.
Xiaotian Han, Zhimeng Jiang, Hongye Jin, Zirui Liu, Na Zou, Qifan Wang, and Xia Hu. Retiring Sidelta Mext/DP)S: New distribution-lewe metrics for demographic parity. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview. seappenrrevee..eet/oorrrr IdWLLjDFWWWWa.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning, Advances in neural information processing systems, 29, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Brian Hsu, Rahul Mazumder, Preetam Nandy, and Kinjal Basu. Pushing the limits of fairness impossibility: Who's the fairest of them all? In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances Neural Information Processing Systems, 2022. URL https://openrevvie. net//forum?/---ott5550uu
Ray Jiang, Aldo Pacchiano, Tom Stepleton. Heinrich Jiang, and Silvia Chiappa. Wasserstein fair classification. In Uncertainty in Artificial Intelligence, pp. 862-872. PMLR, 2020.
Jigsaw. Jigsaw unintended bias in toxicity classification, 2018. URL https://ww..aggle.coom wwwptaneeeeeebb-----oo-ooiiieae-
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimi- nation. Knowledge and information systems, 33(1):1-33, 2012.
11

---

Published as a conference paper at ICLR 2024
Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-awar classifier with prejudice remover regularizer. In Joint European conference on machine learning and knowledge discovery in databases. Springer, 2012.
Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determi- nation of risk scores. arXiv preprint arXiv: 1609000879 2016.
Ronny Kohavi and Barry Becker. Uci adult data set. UCI Meachine Learning Repository, 5, 1996.
Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. Propublica compas analysis data and analysis for 'machine bias'. https://github.com wttp:/tppppopppppcco iaa/ anaaaaiss, 2016. Accessed: 2023-03-13.
Tai Le Quy, Arjun Roy, Vasileios Iosifidis, Wenbin Zhang, and Eirini Ntoutsi. A survey on datasets for faimess-awware machine learning. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, (2(3)ee1552 2022.
Peizhao Li and Hongfu Liu. Achieving faireess utility cost via data reweighing with influence. In International Conference on Machine Learning, pp. 12917-12930. PMLR, 2022.
Zhu Li, Adrian Verez-Suay, Gustau Camps- Valls, and Dino Sejdinovic. Kernel dependence
regularizers and gaussian processes with applications to algorithmic fairness. arXiv preprint 191iv:1911. 11t::1911.04322, 2019.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In Proceedings of International Conference Computer Vision (ICCV), December 2015.
Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard S Zemel. The variational fair autoencoder. International Conference on Learning Representations (ICLR), 2016.
Gilles Louppe, Michael Kagan, and Kyle Cranmer. Learning to pivot with adversarial networks. NeurlPS, 30, 2017.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. International Conference on Machine Learning, 2018.
Daniel McNamara, Cheng Soon Ong, and Robert Williamson. Costs and benefits of fair representa- tion learning. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 263-270, 2019.
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6): 2021.
Anay Mehrotra and Nisheeth Vishnoi. Fair ranking with noisy protected attributes. Advances in Neural Information Processing Systems, 35:31711-31777 2022.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer. James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in neural information processing systems, pp. 8026-8037, 2019.
F. Pedregosa. G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825 12:2825-2830. 2011.
Dana Pessach and Erez Shmueli. A review on faimess in machine learning. ACM Computing Surveys (CSUR), 35(3):1-44, 2022.
Charan Reddy, Deepak Sharma, Soroush Mehri, Adriana Romero Soriano, Samira Shaba- nian, and Sina Honari. Benchmarking bias mitigation algorithms in representation learn- ing through fairness metrics. In J. Vanschoren and S. Yeung (eds.), Proceedings of the Neural Information Processing Systems Track Datasets and Benchmarks, volume 1, 2021a. URL https: / perprsseeeeeccrrss-srrrreeennnie prw.eediiooceced.n..oe.. wwweeeiinnnnnuuiiiiss.ee.. httppp..ccppaae cc/ppppr//22////// /fille/ 22220000888850002288000776686-00 round1 pdf..
12

---

Published as a conference paper at ICLR 2024
Charan Reddy, Deepak Sharma, Soroush Mehri, Adriana Nomero-Soriano, Samira Shaba- nian, and Sina Honari. Benchmarking bias mitigation algorithms in representation learning through fairness metrics. In Thirty-fifth Conference Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021b. URL wttttts:///pppreiiieee wtw./ewwwne/ffuummiieeee OTnqQUEwPKu.
Salvatore Ruggieri, Jose M Alvarez, Andrea Pugnana, Franco Turini, al. Can we trust fair-ai? In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 15421-15430, 2023.
Saeed Sharif--Malvaeedii Michael Kearns, and Aaron Roth. Average individual fairesss: Algorithms, generalization and experiments. Advances neural information processing systems, 32, 2019.
Changjian Shui, Gezheng Xu, Qi Chen, Jiaqi Li, Charles X Ling, Tal Arbel, Boyu Wang, and Christian Gagné. On learning fairness and accuracy multiple subgroups. Advances in Neural Information Processing Systems, 35:34121-34135, 2022.
Mingyang Wan, Daochen Zha, Ninghao Liu, and Na Zou. In-processing modeling techniques for machine learning fairness: survey. ACM Transactions on Knowledge Discovery from Data, 17 (3):1-27, 2023.
Michael Wick, Jean- -aptiste Tristan, et al. Unlocking fairness: trade-off revisited. Advances in neural information processing systems, 32, 2019.
Ruicheng Xian, Lang Yin, and Han Zhao. Fair and optimal classification via post-processing. In International Conference Machine Learning. 2023.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna Gummadi. Fair- ness constraints; Mechanisms for fair classification. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
Mike Zajko. Artificial intelligence, algorithms, and social inequality: Sociological contributions to contemporary debates. Sociology Compass, 16(3):e12962, 2022.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning, pp. 325-333. PMLR, 2013.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2018
Guanhua Zhang, Yihua Zhang, Yang Zhang, Wenqi Fan, Qing Li, Sijia Liu, and Shiyu Chang. Fairness reprogramming. arXiv preprint 2rXiv:2209.10222 2022.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/reeression by conditional adversarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017.
Han Zhao and Geoffrey Gordon. Inherent tradeoffs in learning fair representations. Journal of Machine Learning Research, 23(57):1 2022. URL http://jmll org. httppaperss/23/2111444
html.
13

---

Published as a conference paper at ICLR 2024
Appendix
Table of Contents
A Related Work
15
B Details of the Group Fairness
16
C Experimental Setting
16
D Details of the Benchmarking Datasets
17
E Detailed Experimental Settings
F More Experiment Results Adult
19
F.I
Utility-Faireess Trade-offs.
19
F2 Training Curves and Hyperparameters for Controlling Faimesss
19
G More Experiment Results on CelebA-A
21
Utility-Faireess Trade-offs.
21
G.2 Training Curves and Hyperparameters for Controlling Fairness
21
H More Experiment Results Jigsaw
22
H.l How does Model Size Influence Fairness Performance?
23
Implementation Comparison with AIF360 and FairLearn
24
14

---

Published as a conference paper at ICLR 2024
A RELATED WORK
Algorithmic Fairness Fairness in machine learning has garnered considerable attention in recent years. The goal of fairness in machine learning is to ensure that the machine learning models are fair and unbiased towards an individual or group. Thus, faimess in machine learning can be divided into categories: group fairness (Dwork et al., 2012: Hardt et al., 2016; Corbett- Davies al., 2017; Zafar et al., 2017; Madras et al., 2018) and individual fairness (Dwork et al., 2012; Sharifi Malriiieel et al., 2019). Group fairness aims to ensure that the machine learning models are fair to different groups of people, while individual faimess aims to "similar individuals should be treated similarly." To mitigate fairness and bias problems in machine learning models, bias mitigation methods can be divided into three categories: pre-processing (Kamiran & Calders, 2012: Calmon et al., 2017), in-processing (Kamishima et al., 2012; Zhang et al., 2018; Madras et al., 2018; Zhang et al., 2022; Buyl & De Bie, 2022; Alghamdi et al., 2022: Shui et al., 2022; Mehrotra & Vishnoi, 2022), and post-processing (Hardt et al., 2016; Jiang et al., 2020). Given that the group fairness metrics are widely adopted in real-world applications and the emergence of more in-processing techniques designed in deep neural network models, we focus on benchmarking in-processing methods for group fairness for neural network models for tabular and image data.
Fairness Packages and Benchmarks There are many faimess packages in the literature. Among them, AIF360 (Bellamy et al., 2018) and FairLearn (Bird et al., 2020) are the most widely used Python packages that provide a set of metrics and algorithms to measure and mitigate bias in machine learning models. They provide a set of metrics to measure the bias of machine learning models, including disparate impact, statistical parity difference, and equal opportunity difference, and set of algorithms to mitigate the bias of machine learning models. However, both AIF360 and FairLearn implement the bias mitigation algorithms using Scikit-learn (Pedregosa et al., 2011) API design (c.g., the use of fit() function) with complicated class inheritance, making the understanding and direct modification of implementation difficult. In comparison, our benchmark decouples the implementation of different bias mitigation algorithms using Pytorch-style (Paszke et al., 2019) training scripts and provides unified fairness evaluation interface for comprehensive list of group fairness metrics. One recently proposed benchmark (Reddy et al., 202 10) also aims to benchmark bias mitigation algorithms. However, their benchmark only includes adversarial learning methods and datasets (e.g.. a synthetic dataset and CI-MNIST) without fairness implications and uses dp and eodd as the fairness metrics. In contrast, our benchmark is more comprehensive in terms of algorithms. datasets, and fairness evaluation metrics.
15

---

Published as a conference paper at ICLR 2024
B DETAILS OF THE GROUP FAIRNESS
In this section, we provide the details of the group fairness. We first introduce the definition of group fairness. Then, introduce the existing group fairness metrics and algorithms.
dp (Demographic Parity or Statistical Parity) (Zemel et al., 2013). A classifier satisfies de- mographic parity if the predicted outcome Y is independent of the sensitive attribute S, i.e., P(Y S = 0) p(Ý S 1).
prule (Zafar et al., 2017). A classifier satisfies p%-rule if the ratio between the probability of subjects having a certain sensitive attribute value assigned the positive decision outcome and the probability of subjects not having that value also assigned the positive outcome should be no less than p/100, i.c., PP(Y 1 1 1)/P(Ý = 1 S 0) S p/100.
eopp (Equality of Opportunity) (Hardt et al., 2016). A classifier satisfies equalized opportunity the predicted outcome Y is independent of the sensitive attribute S when the label Y 1, i.e.. P(Y S Y, 1) P( S 1,Y
eodd (Equalized Odds) (Hardt et al., 2016). A classifier satisfies equalized odds if the predicted outcome y is independent of the sensitive attribute S conditioned on the label Y, i.e., P(Ý) S = 0, Y = y) p(Ý S = 1,Y = y). y E 10.
acc (Aceuracy Parity). A classifier satisfies accuracy parity the enror rates of different sensitive attribute values are the same, i.e., P( * Y S = 0) = P( +Y 1),y (0, 1}.
aucp (ROC AUC Parity). A classifier satisfies ROC AUC parity if its area under the receiver operating characteristic curve of w.r.t. different sensitive attribute values are the same.
ppv (Predictive Parity Value Parity) A classifier satisfies predictive parity value parity if the probability of a subject with a positive predictive value belonging to the positive class w.r.t. different sensitive attribute values are the same, i.c., P(Y = Y, = 0) = P(Y = 1 y,5 = 1).
bnegc (Balance for Negative Class). A classifier satisfies balance for the negative class if the average predicted probability of a subject belonging to the negative class is the same w.r.t. different sensitive attribute values, i.e., [x)
0, = 1]. bposc (Balance for Positive Class). A classifier satisfies balance for the negative class the average predicted probability of a subject belonging to the positive class is the same w.r.t. different sensitive attribute values, i.c., (X) Y S, = 0J [[f(X) Y 1.$= 1].
0] (x
abcc (Area Between Cumulative density function Curves) (Han al., 2023) proposed to precisely measure the violation of demographic parity at the distribution level. The new fairness metrics directly measure the difference between the distributions of the prediction probability for different demographic groups
C EXPERIMENTAL SETTING
For tabular datasets, we use two-layer Multi-layer Perceptron with 256 neurons each for all datasets. We use different bath sizes for different datasets based on the total number of instances of each dataset. For image datasets, we use various neural networks (such as ResNet-15 18 (He et al., 2016) and ResNet-152) as the backbone. We don't 't use weight decay for all datasets and all fairness methods. We use linear learning rate decay for all datasets and all fairness methods. We use Adam (Diederik Kingma, 2014) as the optimizer with learning rate of 0.001 for both tabular and image data. As these objectives, utility and faimess, often present trade-offs, it can be challenging to determine when to stop model training, and this issue is rarely discussed in the previous literature. In this work, we experience. We employ linear decay
adopted straightforward stopping strategy based strategy for the learning rate, halving it every 50 training steps. The model training is stopped when the learning rate decreases value below 16
16

---

Published as a conference paper at ICLR 2024
D DETAILS OF THE BENCHMARKING DATASETS
In this section, we provide the details of the benchmarking datasets. We first introduce the bench- marking datasets. Then, we introduce the data preprocessing and data splitting.
Tabular Datasets
Adult (Kohavi & Becker, 1996). The Adult dataset widely used in machine learning and data mining research. It contains 1994 U.S. census data. The task of the dataset is to predict whether a person makes over $50K a year, given an individual's s demographic and financial information. The dataset includes sensitive information such age and gender. In the literature, gender is mostly used as the (binary) sensitive attribute to evaluate group fairness.
COMPAS* (Larson al., 2016). The COMPAS dataset contains records of criminal defendants, and it is used to predict whether the defendant will recidivate within two years. The dataset includes attributes related to the defendant, such as their criminal history, and demographic information, such gender and race.
German' (Dua & Graff, 2017). The German Credit dataset contains information individuals who applied for credit at a German bank, including their financial status, credit history, and demographic information (e.g., gender and age). It is used to predict whether an individual should receive a positive or negative credit risk rating.
Bank (Dua & Graff, 2017). The bank marketing dataset is used to analyze the effectiveness of marketing strategies of Portuguese banking institution by predicting if the client will subscribe term deposit. The input variables of the dataset include the bank client's 's personal information and other bank marketing activities related to the client. Age was studied as the sensitive attribute in (Zafar et al., 2017).
KDDCenSus' (Dua & Graff, 2017). Similar the Adult dataset, the task of the KDD Census dataset is to predict whether the individual's 's income is above S50k with more instances. The sensitive attributes are gender and race.
ACS-I/E/P/M/T* (Ding et al., 2021). The ACS dataset provides several prediction tasks (e.g., predict whether an individual's income is above $50K or whether individual is employed). constructed from the American Community Survey (ACS) Public Use Microdata Sample (PUMS). All tasks contain features for race, gender, and other task-related features.
Image Datasets
CelebA-A/W/S* (Liu et al., 2015). The CelebFaces Attributes dataset comprises 20k face images from 10k celebrities, Each image is annotated with 40 binary labels indicating specific facial attributes such as gender, hair color, and age.
UTKFaae 10 (Zhang et al., 2017). The UTKKaace dataset is a large-scale face dataset that contains over 20k face images of people from different ethnicities and ages. The images annotated with age, gender, and ethnicity information.
Text Datasets
Jigsaw"! (Jigsaw, 2018). The CelebFaces Attributes dataset comprises 20k face images from 10k celebrities. Each image is annotated with binary labels indicating specific facial attributes such gender, hair color, and age.
"https://archiiveiiss http://www........eeed//////eeee /ccc.edd//m//aaaaeett/aauut htta:///aaaieee...s....ce////aas httprs/s//httta.cccoooooooooolos
"https://susangp:
17

---

Published as a conference paper at ICLR 2024
E DETAILED EXPERIMENTAL SETTINGS
We present the details of the experimental setting in Tables to 7. Table presents the common hyperparameters used by both Tabular and Image datasets, including an initial learning rate of 0.01, Adam as the optimizer, zero weight decay, StepLR as the scheduler with step of 50, a gamma value of 0.1, and 150 training steps total. Table presents the range of control hyperparameters used for various fairness methods. Each method has a unique range of these parameters. Table 7 indicates the batch sizes chosen for various datasets during training, ranging from 32 for the German and COMPAS datasets to a large 4096 for the KDDCensus and UTKFace datasets using a batch size 128, which are determined by the number of instances of the datasets.
datasets, with CelebA-A/W/S and
Table 5: Common paraeerparameters.
Dataset Initial LR Optimizer Weight Decay Scheduler StepLR step StepLR StepLR_gamma gamma Training steps
Tabular Image
0.01
Adam Adam
0.0
StepLR
50
0.1
150
0.01
0.0
StepLR
50
0.1
150
Table 6: The fairness control hyperparameter selections.
Dataset
Control hyperparameter
Di FFDP
0.2, 0,00,,40000 0.8, 1.0, 1.2, 1.6, 1,8, 2,0,
Di ffoddd
024, 0.8, 1.0, 1.2, 1,,, 3.5 0.2, 0.4, 0.8,
Di ffEopp
1,,,,,,,,,
PRemover
0.05, 0.1, 0.25, 0.3, 0.35, 0.40, 0.45, 0.50, 0.8, 1.0
HSIC
100, 150, 200, 300, 350, 4000, 500, 600, 700, 900, 1000 1.8, 2,0,2,2,2,,3333333,,
AdvDebias
0.2,0.4,066, 02,00,,,.66000,,,0.,1,,, 1.0,
LAFTR
0.1, 0.3 0.5
Table 7: The batch size for different datasets during the training.
Dataset
Bank German Adult COMPAS KDDCenSus ACS-I/E/P/M/T CelebA-A/W/S UTKFace
Batch Size
1024
32
1024
32
4096
4096
128
128
18

---

Published as a conference paper at ICLR 2024
F MORE EXPERIMENT RESULTS ON Adult
In this appendix, we present the experimental results on Adult datasets. The same experiment results are presented http://TODO.
F.1 UTILITY-FAIRNES TRADE-OFFS
We plot the utiliit--farress trade-offs for the Adult it dataset with gender as the sensitive attribute and present the results in Figures 7 and 8.
sdulit/ineoeee /sex
edule Tincomnn
aduilt /income foex
adult /inessse/ses /see
adul /isccoene faes
prate
addl1 fo/onoon
adult t/f Lacorme frees
sdult /iacoene
adult /intonueey sirs
edid
pp7
bit
htpp
Figure 7: The Cuility-Fairness Trade-off's with acc as utility metric.
ado1t 5, /ir /iroome Trane
adult fincome (sex
addsli /inecmm//aax
adu't t. /nneemo//we
adull www.coooeeeeeeeeeeooo faces
abes
acep
aucp
prote
Fnnsste fsere
adull /incoone
adu't /incomoo//rr
adul Alncame
00.3
6000
ppv
Figure 8: The Utility-Fairness Trade-off's with acc as utility metric.
F.2 TRAINING CURVES AND HYPERPARAMETER FOR CONTROLLING FAIRNESS
We plot the utility and fairness training curves for varying fairness control hyperparameters on the Adult dataset, and present the results in Figure 9, The intensity of the color represents the size of the control parameters. In most cases, the larger value of control parameters yields better faimeess performance, while small ones have worse fairness performance.
19

---

Published as a conference paper at ICLR 2024
4/v
Dhieegn Strce
Thaace Shipe
-Yendag Strpe
Srandaa Strpe
Traad22 Strpe
D40
330
51
150
50
100
110
301
300
50
391
150
HELC
HRIC
BeIe
BBIC
BBIC
52.
13
Thining Stage
Thilknng Stzge
Theaking Strne
Thankig Stege
Thanking Strge
Thaiking Strge
68
Stote
Stew
Truacea
Cheading Stape
Ciaadios Strpp
Vrundia Stope
CAPTR
LAFTR
LAF 7TL
LAPTI
1AP
1 7TL
100
130
120
100
300
10
Strge
Srine
Strge
Strge
DLEEDP
Basekag Stope
Thakea Shope
Canakie Str
Thaadaa Stape
Figure 9: Hyperparameters for Controlling Fairness on Adult t dataset.
20

---

Published as a conference paper at ICLR 2024
G MORE EXPERIMENT RESULTS ON CelebA-A
In this appendix, we present the experimental results on the CelebA-A dataset.
G.1
TRADE-OFFS
We plot the utility--fain fairness trade-offs for the CelebA-A dataset with gender as the sensitive attribute and present the results Figure The results show the utility-fairness trade-offs in CelebA-A dataset.
celeba eelleba/ee /attractiv//gendee
caleba, caabba/attractive/ggenne genderr celeba/attrace /ttpaaaavvttteenneeeeeee /gender
seleba/attractive /ender
abc
abi
20
26 ap
abec
btp
bfn
celeba/atte www.bbaaaaatiieeeeeeooooeeoe gender
celeba celleba/aee cel lttaaaaatiieegeeeeelee
celeba /attrantiit gender
eeleba/fe malleba/atrinn /tttraattiveeeeeeee gender
acc
aGc
acc
aboc
dp
btp
bfn
Figure 10: The Utility-Faimnent Trade-offs
G.2 TRAINING CURVES AND HYPERPARAMETERS FOR CONTROLLING FAIRNESS
We plot the utility and fairness training curves for varying faimness control hyperparameters on the CelebA-A dataset, and present the results Figure The intensity of the color represents the size of the control parameters. In most cases, the larger value of control parameters yields better fairness performance, while small ones have worse fairness performance.
ERW
ERN
EBM
ERN
ERM
ERM
Treasna, Steae FRonover
Thening Sugo PRenover
Trinso, Stepa PRemscer
Thuskig Siio Phemover
Trassia S4c.0 FReeover
Treadie Stop FRenover
700
0:00
50-
Tianiing 2arte
Traarieg Strpe D14
Thilwing Steps D1CTOP
Thinking Stope DLFE3P
Thainin Strrc D11fOP
Traniing Stapee DurfDP
D1EEOP
50-
100
31
Tiacusa 2trra
Liendiu Stree
Trsdning Stepa
Uheadin Shipe RSTC
Trasung Metae
Vinaeee Shace
TC
HSIC
HSTe
HETC
RE1C
50-
Stogee
Figure 11: Training Curves and Hyperparameters for Controlling Fairness one CelebA-A dataset.
21

---

Published as a conference paper at ICLR 2024
H MORE EXPERIMENT RESULTS ON Jigsaw
We present the results on fairness performance using various fairness control hyperparameters.
We observed that the faimess metrics abcc and dp cannot be influenced by the fairness control hyperparameters, whereas prule can.
DiffDP
HSIC
PRemover
Normalized abcc
Normslized abcc
10
Normalizod abco
10
10-
51
5
5
0
50
150
50
100
150
150
Training Steps
Training Steps
Training Steps
DiffDP
HSIC
PRemover
Normalized dp
Normalized dp
Normalized dp
50
100
150
50
100
150
50
100
150
Training Steps
Training Steps
Training Steps
DiffDP
HSIC
PRemover
Normalized mladd prule
Normalized prule
Norunlized prule
1.0
1.0
0.5
0.5-
0.5
100 Training Steps
50
150
100 Training Steps
50
150
50
100
150
Training Steps
Figure 12: The fairness performance with varying faimness control hyperparameters on text data. The intensity of the color represents the size of the control parameters. In most cases, the larger value of control parameters yields better fairness performance, while small ones have worse faimeess performance.
22

---

Published as a conference paper at ICLR 2024
H.1 How DOES MODEL SIZE INFLUENCE FAIRNESS PERFORMANCE?
We conducted experiments to explore the influence of model size on fairness performance. We use various neural networks with the number of neural network trainable parameters spanning from 11.6M to 126.9M. The results are presented in Figure 13.
ERM
Planover
EBIC
XII
XII
o. pesmices
XUI
XIP
XII
Figure 13: The performance with varying sizes of neural networks. The x-axis the number of model parameters. There is no clear relationship between model size and performance.
The architecture does not significantly influence fairness performance. An increase in neural network parameters does yield significant changes in utility fairness performance. This suggests larger models do not naturally mitigate that dataset bias. The results observed from the Di FFDP method indicate potential correlation between utility performance and bias: fairness performance may deteriorate as utility increases, exhibiting trade-offs between them across different models.
1PResNet-18 (11.6M), ResNet-34 (21.8M), ResNet-50 (25.6M), ResNet-101 (44.5M), ResNet-152 (60.2M).
ResNext-50 (25.0M), ResNext101 (88.8M), wide_ ResNet-0 (68.9M), and wide ResNet101 (126.9M).
23

---

Published as a conference paper at ICLR 2024
I IMPLEMENTATION COMPARISON WITH AIF360 AND FAIRLEARN
In this section, we provide the implementations of adversarial debiasing AIF360, Fairlearn, and FFB demonstrate FFB extensible, minimalistic, and research-oriented compared to existing fairness packages. Algorithm 1, Algorithm 2, and Algorithm show the implementation of adversarial debiasing in AIF360, FairLearn, and FFB, respectively.
We can see that both AIF360 and FairLearn use Scikit-learn API design (e.g., the use of fit() function), whereas FFB use Pytorch-style implementation, which provides a unified data preprocessing pipeline and fairness evaluation interface in single script. Thus, researchers using FFB can use the bias mitigation method and reproduce the experimental results us- ing one line of command. Additionally, AIF360 and FairLearn use complicated class inheri- tance (e.g., wwwerssrriallaiinssssssssseeeeee in FairLearn inherents ClassifierMixin) AdversariialDebiasttee in AIF360 inherents Transformer), and other exter- nal dependencies (e.g., Adivrrsarialaairness FairLearn uses backendEngine. to implement the training step), making the implementation hard to read. This makes researchers hard to understand and re-implement the bias mitigation methods.
and
Algorithm Adveebias in AIF360
class diveraariaalebaassnggTTTanarrrs (self:
((s features, features_dim, keep_prob): deine classifier
def addversary_nodeleee pred prgi_sogits, true_labels): deine adversary model
def predict(self, dataset):
fit(self, dataset):
with tf. wf.variable_scopee wcw.ssscepppeesooossse sccpee_neeoe_
e tf graph construction
self. sess. (tf. wuntf.ggobbl.....mmmmoomooo variables_initiallae self.sess..ru
for epoch in rsa.f nun_epohhh):: training
for (onge(nuuttaiie stap:rralsallle/e/sseleer batch_size):
batch_feee feed_dict = (self. features_ohe batch_features,
true_labels_phh: batch_labels.
self. detected_attributes..o aatch_protecte__attrbbttee
self. keep_prob: 0.8)
selff.dbbiae lalis_ooss__lllll
pred_ labels
pre.oroteceeaLatribteess.sss
self.sess.rus
blassifier_minimzzer
adversary prrrsarr_ninimizer,
pred_Labels__es, www.prrreceed_atriiuttess loss], rrro.ddictbbttch_eeeeddd logging
if 280
else:
ppw.eeeddeee.. labe__ labels_loss_veeeee self, runs
filassifier minimizer. sllasifier_minimzzer.
pred_ pred_labels_lose
ftee.ddicttattch.eeed.it)))
if x 200
logging
24

---

Published as a conference paper at ICLR 2024
Algorithm/ 2 AdvDebias in FairLearn
class Alveraarallaimmmsssseesssnnno (self:
setupp(eel. x, Y, A):
def fit(self, x, * sensitive_ features::None): x, Y, A self._ vnnuucate_inputtX. y, fensurs,e_feeture, reinitialize=Frue)
for epoch in range(epochs): baich slice(
batch batch_size,
min((batch 1) batch_size, s.shape[))),
(LP, LA) = self. www..raeeeeennnnee.eeer... train_sstet x[batch_slice), slicel, A[batch_slice]
predictor ceatttor_ossssssees append)))))))))))
adversary adversary_leases. appeess)))
fatsee, y, sensitive__fatuurssNone)
cunisiio_uunction(sefr x):
def predict(self, x):
def _vaaaiddte_ipputceal X, Y, reinitialiie=Falsse:
set oee_.preiccoor_funttiionoe111
class Adversarial (attrrarrresssssssrseessssssraan
ClassifierMixin).
def init__ (self):
कम Inittilliee model by setting the predictor loss and function. *** = "classifier"
self
super aooerdddeersaralaaatnesssaasss
self).__nitt_O)
25

---

Published as a conference paper at ICLR 2024
Algorithm 3 AdvDebias in FFB
class Adversary(tn.Mooti
class ModuleM)
(model, test__loaerr, criterion, device, args=None):
def train(clf, adv, lotd_, elf_criterion, adv_criterion, clf_optimizer, adv_optimizer):
if __nae__.. == 'ainnin'
parser argaarse. Arguenntraguetrrrsserr
parser.add_ed aed.rrddd_drrrree argunent((( ("meeentt(---taaa d'--dataset' type=str, default="adultt)
parser adderr.dddarrgeeeee mt-nmeell typesstr, default= "HLP")
args parser parse_args()
Dataset selection
if args, dataset "adult":
x, y, $ load_adult wtt.:aatttttttt..atrrrrreeere elif args. dataset args.dataset
"compas";
X, y, load_compas_datae sensitive_atriiuutearg...m
Unified Dataset preprocessing (e.g., train/test split, )
define network architecture, etc. optimizer
features,,, mum_classe=1,, mlp_Layers=[512, 256, 643) to(vvice)) aff_criterion in. BCELoss() elf_optinizer optin. Adan( alf.parameters(), Ireargs.1r)
adv Adversary( n_sensitive (devvce)
adv_criterion nn.BCELossrreduction "nenn""""") adv_optimizer optin. par () 1r=args.
for epoch in range(1, nu__scchsts)) begin training train(cIf, adv, train_loader, clf clf_criterion, adv_criterien, elf_optimizer, adv_optimizer)
epoch args. logging loggllog steiss or epoch args. nun_epochs:
test_metrics test (nodel=clf,, lost__ldretttet__ong Leadeett__tdr, criterionscl f_rritrriotttiioon devicesdevice) logging metrics
26


Reviews:
[{"review_id": "bwDzLd3K7b", "reviewer": "Reviewer_5ZmP", "summary": "This paper proposes a new fairness benchmark called FFB (Fair Fairness Benchmark). FFB targets to support group fairness metrics and in-processing fairness algorithms, and the system contains several well-known fairness algorithms with metrics. The paper also describes various observations, which are gathered by using the proposed benchmark. For example, the paper observes that the model architecture usually does not significantly affect the fairness performances, which shows that the biases are mainly from the training data. These observations are aggregated based on more than 45000 experiments.", "questions": "My main concerns are described in the above weakness section. I hope to hear the authors’ response to them.\n\n--------------------\nThe score is updated after rebuttal.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "- The paper aims to solve a very important problem in the fairness literature, the lack of great benchmarks.\n- The paper well states the challenges in making fairness benchmarks and proposes a new one called FFB to help the fairness literature.\n- The paper performs extensive experiments and summarizes their observations in several aspects, including model performance and stability.", "weaknesses": "Although I appreciate the paper’s contribution on proposing a new benchmark, I have several concerns on the manuscript as a research paper.\n- The paper needs to explain more clearly how to use FFB and the strengths of the system itself.\n  - The paper does not clearly explain how to use FFB for testing new algorithms or new datasets. The paper currently focuses on the predefined algorithms and models in FFB, but as a paper that proposes a new benchmark, demonstrating how to utilize their system can be more important.\n  - Similarly, it would be better if the paper could provide more explanations on the strengths of FFB itself. It seems some explained characteristics (like minimalistic aspect) are not supported by enough convincing explanations.\n- Currently, the paper explains their observations on several algorithms and datasets by using FFB, but many of the observations are not very surprising and already discussed in the literature. Thus, it is a bit unclear to me whether such observations themselves can be a strong contribution of this paper. Although such observations are still noteworthy to summarize in the paper, it may be better to not oversell them in the paper. It would be better if the paper could clearly connect these observations to the previous knowledge in the fairness literature.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bRM5xxtWbm", "reviewer": "Reviewer_dSax", "summary": "The paper introduces the Fair Fairness Benchmark (FFB), a framework for evaluating group fairness methods in machine learning. It aims to address the challenges of inconsistent experimental settings, limited algorithmic implementations, and extensibility issues in fairness tools. FFB offers an open-source benchmark, standardized code, and extensive analysis from 45,079 experiments, making it a valuable resource for the fairness research community.", "questions": "Figure 1: Are the results different because of different random seeds or did you changed the gamma values of the loss functions to obtain different acc-fairness trade-offs?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper address a prominent problem in the Fairness community which is the inconsistency of different results from various papers. A lot of experiments are conducted in a standardized manner using different datasets, methods and evaluation metrics. The writing is clear and linking the contribution points with 1,2,3.. in the text makes it quite convenient to read. The open source code looks good and is in a state which can be easily adopted for other researchers.", "weaknesses": "In Table 4 you give some recommendations if one should use the dataset or not. In the text you explain the different discussion. It can be that I missed it but are there any quantifiable measurements to check if a dataset is good for fairness or not - like a metric? And if so can this value be included in the study? Also you said you had 10 trials to produce this table. Did you do any HP optimization with some hold-out splits or how exactly was this done? More detailed information about this would be appreciated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hZ69R2cBKe", "reviewer": "Reviewer_Tj8b", "summary": "This paper proposes a group fairness benchmark for in-processing methods. A wide range of fairness definitions are used along with multiple datasets. Three types of in-processing methods are compared: gap regularization, independence, and adversarial learning. Several observations are made on fairness-utility tradeoff, stability, and others.", "questions": "Please address the weak points above.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Since the fairness literature is vast, it is a good time to make a comprehensive comparison.\n* The comparison of in-processing methods using group fairness measures looks reasonable.", "weaknesses": "* While proposing a fairness benchmark is a worthy effort, the scope of this study seems a bit limited because it only considers in-processing methods. Even AIF360 proposed in 2018 compares pre-processing, in-processing, and post-processing methods, so it is expected that a new benchmark should at least subsume this scope. Pre-processing methods should not be ignored because some of them are designed to complement in-processing methods for the best results. Even for in-processing methods only, there are reweighing and sampling methods [1,2,3] that should be compared.\n\nNowadays, performing in-processing training without sensitive attributes is also actively studied, and a comparison with these methods (e.g., [4,5]) would be interesting.\n\n[1] Jiang et al., Identifying and Correcting Label Bias in Machine Learning, AISTATS 2020\n[2] Roh et al., FairBatch: Batch Selection for Model Fairness, ICLR 2021\n[3] Iosifidis and Ntoutsi, AdaFair: Cumulative Fairness Adaptive Boosting, CIKM, 2019\n[4] Lahoti et al., Fairness without Demographics through Adversarially Reweighted Learning, NeurIPS 2020\n[5] Hashimoto et al., Fairness Without Demographics in Repeated Loss Minimization, ICML 2018\n\n* Concluding that HSIC is the best approach seems misleading because not all in-processing methods were compared as explained above.\n\n* Some of the key observations are already known in the fairness community. Observation 4 (adversarial debiasing has instability) is not surprising and is mentioned in the papers that use this approach. Observation 5 (utility-fairness trade-off is controllable) does not seem revealing either. What's actively studied nowadays is whether there has to be a trade-off or not, and there is a line of research that discusses when utility and fairness align instead of conflict. It would be interesting to empirically verify if the claims made here are actually true. In observation 6 (training curve stability), a future direction is suggested to \"focus on enhancing fairness training stability\". However, it is not clear why enhancing the fairness stability is the most important research direction among other challenges. What does it mean to be not stable enough?\n\n* There is an emphasis on making the benchmark research-oriented, but this term is rather vague. Instead, benchmarks should target practical applications as they actually show which fairness method works.\n\n* One of the future works is to include a wider range of in-processing group fairness methods. This direction should not be a future work as the current paper claims to be a complete benchmark for such methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AvQXCNmNxe", "reviewer": "Reviewer_8vvg", "summary": "Paper introduces the Fair Fairness Benchmark, a benchmarking framework for in-processing group fairness methods.\nContributions are: the provision of flexible, extensible, minimalistic, and research-oriented open-source code;\nthe establishment of unified fairness method benchmarking pipelines.", "questions": "I think that authors should elaborate on the limited amount of of state of the art (un)fairness mitigation methods.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Good amount of fairness metrics.\nGood amount of datasets.", "weaknesses": "I think that the main missing point of the paper is a larger series of state of the art (un)fairness mitigation methods to use as baseline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)