Retiring $\Delta \text{DP}$: New Distribution-Level Metrics for Demographic Parity
Authors: Xiaotian Han
Zhimeng Jiang
Hongye Jin
Zirui Liu
Na Zou
Qifan Wang
Xia Hu
Venue: Accepted by TMLR ()

Abstract: Demographic parity is the most widely recognized measure of group fairness in machine learning, which ensures equal treatment of different demographic groups. Numerous works aim to achieve demographic parity by pursuing the commonly used metric $\Delta DP$. Unfortunately, in this paper, we reveal that the fairness metric $\Delta DP$ can not precisely measure the violation of demographic parity, because it inherently has the following drawbacks: i) zero-value $\Delta DP$ does not guarantee zero violation of demographic parity, ii)  $\Delta DP$ values can vary with different classification thresholds. To this end, we propose two new fairness metrics, Area Between Probability density function Curves (ABPC) and Area Between Cumulative density function Curves (ABCC), to precisely measure the violation of demographic parity at the distribution level. The new fairness metrics directly measure the difference between the distributions of the prediction probability for different demographic groups. Thus our proposed new metrics enjoy: i) zero-value ABCC/ABPC guarantees zero violation of demographic parity; ii) ABCC/ABPC guarantees demographic parity while the classification thresholds are adjusted. We further re-evaluate the existing fair models with our proposed fairness metrics and observe different fairness behaviors of those models under the new metrics.

Published in Transactions on Machine Learning Research (05/2023)
Retiring ADP: New Distribution-Leve Metrics for Demographic Parity
Xiaotian Han' Zhimeng Jiang' Hongye Jin' Zirui Liu?, Na Zou', Qifan Wang', Xia Hu? (han, zhômengj, jhy0410, ] {zirui.liu, liu, zua Brice.edu, wafer@meta.com
Texos A&M University, 2 RRice University, 3 Meta Al
Reviewed on OpenReview: https: openreview. net/ forum? Ld-LIDFIAWVV
Abstract
Demographic parity is the most widely recognized measure of group fairness in machine learn- ing, which ensures equal treatment of different demographic groups. Numerous works aim to achieve demographic parity by pursuing the commonly used metric ADP 1. Unfortunately, in this paper, we reveal that the fairness metric ADP can not precisely measure the violation of demographic parity, because it inherently has the following drawbacks: i) zero-value ADP does not guarantee zero violation of demographic parity, ùij ADP values can vary with different classification thresholds. To this end, we propose two new fairness metrics, Area Between Probability density function Curves (ABPC) and Area Between Cumulative density function Curves (ABCC), to precisely measure the violation of demographic parity al the distribution level. The new fairness metrics directly measure the difference between the dis- tributions of the prediction probability for different demographic groups. Thus our proposed new metrics enjoy: i) zero-value ABCC/ABPC guarantees zero violation of demographic parity; ti ABCC/ABPC guarantees demographic parity while the classification thresholds are adjusted. We further re-evaluate the existing fair models with our proposed fairness metries and observe different fairness behaviors of those models under the new metrics. The code is available at https:/ /ttpp::/githhe http://ggtttt.ttte thub. com/ homm/hxt/new metric for www.deggrppic_ppriiiiee
1 Introduction
Machine learning has been extensively adopted in various high-stake decision-making process, including criminal justice (Heidensohn, 1986; Berk et al., 2021; Tolan et al., 2019), healthcare (Ahmad et al., 2020; Cappelen & Norhcim, 2006), college admission (Friedler et al., 2016), loan approval (Mukerjee et al., 2002; Kozodoi et al., 2022), and job marketing (Johnson et al., 2009; Raghavan et al., 2020). Since such high-stake dacision-making could have a life-long effect on individuals, the fairness issue in these machine learning systems has raised increasing concerns (Chai et al., 2022; Zhang et al., 2022). Lots of fairness definitions (Garg al., 2020; Mehrabi al., 2021; Verma & Rubin, 2018; Saxena al., 2019; Zafar et al., 2017: Mchrabi et al., 2020; Dwork et al., 2012; Hardt al., 2016; Kleinberg et al., 2016) (e.g., demographic parity, equalized opportunity) has been proposed to solve different types of fairness issues. In this paper, focus the measurement of demographic parity, ADP, which requires the predictions of machine learning model should be independent on sensitive attributes (Menon & Williamson, 2018; Ustun et al., 2019; Kamishima et al., 2012).
To develop effective fair models, a faithful metric is critical to guide the development of a fair machine learning system since an "irrational' fairness metric may mislead the development of fair models and even lead to opposite conelusions. Despite the extensive efforts to develop various fairness metrics, one critical and fundamental question is still unclear: Are the existing metrics really reasonable to quantify demographic parily violation?
*Equal contribution.
1APP includes ADP, ADP, this paper, the details presented Section 2.1.

---

Published in Transactions on Machine Learning Research (05/2023)
Model (unfair)
Model 2 unfair) )
Model (fuir)
2.5
Male
2.0
2.0
Female
Density
2
1.5
1.0
0.5
0.5
0.0
0.2
0.6
1.0
0.0
0.4
0.8
0.0
0.2
0.8
9
ADP,= 0.0451 =DP.= 0.0608
ADP,= 0.0233 A 0.0041 X
ADP,= 0.0186 ADP 0.0085 0.0055
ABPC= 0.1067 ABCC= 0.0659
ABPC = 0.2239 ABCC= 0.0349
ABPC = 0.0418 ABCC= 0.0091
Figure 1: The distribution of the predictive probability of male and female groups on the ACS-Income dataset. ADP measures Model and Model 3 correctly. However, it fails to assess (x) Model since it obtains a "fair" assessment on an "unfair" model. Our proposed metries, ABPC and ABCC, can assoss all the models precisely, Experimental results on more datasets are presented in Section 6.2.
In this paper, we rethink the rationale of ADP and investigate its limitations measuring the violation of demographic parity. There are two commonly used implementations of ADP 2, including ADP: (i.e., the difference of the mean of the predictive probabilities between different groups, used in papers (Chuang & Mroueh, 2020; Zemel et al., 2013)) and ADP, (i.e., the difference of the proportion of positive prediction between different groups, used in papers (Dai & Wang, 2021; Creager et al., 2019; Edwards & Storkey, 2015; Kamishima ct al., 2012)). We argue that ADP, as a metric, has the following drawbacks: First, zero-valuc ADP does not guarantee zero violation of demographic parity, One fundamental requirement for the demographic parity metric is that the zero-value metric must be equivalent to the achievement. of demographic parity, and vice versa. However, zero-value ADP does not indicate the establishment of demographic parity since ADP is a necessary but insufficient condition for demographic parity. An illustration of ACc-mn data is shown in Figure/ to demonstrate that ADP fails to assess the violation of demographic parity since it reaches (ncarly) zero on an unfair model (the middle subfigure in Figure 1). Sccond, the value of ADP does not accurately quantify the violation of demographic parity and the level of fairness. Different values of the same metric should represent different levels of unfairness, which still true even in a monotonously transformed space. ADP does not satisfy this property, resulting in it being unable to compare the level of fairness based solely on its value. Third, ADP, value is highly correlated to the selection of the threshold for the classification task. To make decision based on predictive probability, one predefined threshold is needed. If the threshold for downstream tasks changes, the proportion of positive predictions of different groups will change accordingly, resulting in a change in ADP, (Corbett-Davits et al., 2017; Menon & Williamson, 2017; Pleiss et al., 2017; Canetti et al., 2019). The selection of the threshold greatly affects the value of ADP, (validated by Figures 2 and () Wu, 2020; Barata al.). However, threshold tuning is needed in practice. Adjusting the threshold is a common practice for decision-making but can violate demographic parity if the model is evaluated using ADP metric. There are several examples to illustrate dynamic threshold in the downstream task. For instance, in college admissions, the number of admissions and applicants can vary from year to year, necessitating adjustments to the decision-makinn threshold. Similarly, in Al-assisted medical diagnosis, doctors may modily the threshold for diagnosing a disease based on a patient's family history. However, if the model is evaluated using the ADP metric, demographic parity cannot be guarantcod if the threshold is changing. One specific ADP, = 0 can not guarantee demographic parity under the on-the-fly threshold change.
In view of the drawbacks of fairness metrics ADP for demographic parity, we first propose two criteria to thcoretically guide the development of the metric on demographic parity: 1) Sufficiency: zero-value fairness metric must be a necessary and sufficient condition to achieve demographic parity. 2) Fidelity: The metric should accurately reflect the degree of unfairness, and the difference of such a metric indicates the fairness gap in terms of demographic parity. To bridge the gap between the criteria and the current demographic parity metric, we propose two distribution-level metrics, namely Area Between Probability density function Curves (ABPC) and Area Between Cumulative density function Curves (ABCC), retire ADP, and ADP,
We would like to claim that these two kinds of ADP are widely used to evaluate demographic parity in the current literature.
2

---

Published in Transactions on Machine Learning Research (05/2023)
respectively. The advantage is that such independence can be guaranteed over any threshold, while ADP can only guarantee independence over a specifie threshold.
The proposed metrics satisfy all (or partial) two criteria to guarantee the correctness of measuring demographic parity and address the limitations of the existing metrics, as well as estimation tractability from limited data samples. Moreover, we also re-evaluate the mainstream fair models with our proposed metrics. Our main contributions are as follows:
We theoretically and experimentally reveal that the existing metric ADP for demographic parity can not precisely measure the violation of demographic parity, because it inherently has fundamental drawbacks: i) zero-value ADP does not guarantee zero bias, ti) ADP value does not accurately quantify the violation of demographic parily and mi) ADP value is different for varying thresholds.
Motivated by the above observations, we formally established two criteria that a desirable metric on demographic parity should satisfy. This provides a guideline to assess other fairness metrics.
We further propose two distribution-level metrics, Area Between Probability density function Curves (ABPC) and Area Between Cumulative density function Curves (ABCC), to resolve the limitations of ADP, which are theoretically and empirically capable of measuring the violation of demographic parity.
We re-evaluate the mainstream fair models with our proposed metrics, ABPC and ABCC, and re-assess their fairness performance on real-world datasets. Experimental results show that the inherent tension between fairness and accuracy is stronger, which has been underestimated proviously.
It is worth noting that instead of invalidating the fairness definition of demographic parily, claim that the current widely used metrics for demographic parity (i.e., ADP, and ADP.) cannot precisely reflect the violation of demographic parity. In our paper, to precisely mcasure the violation of demographic parity, we propose two new and reasonable metrics, ABPC and ABCC, to measure the violation of demographic parily.
2 Preliminaries
Notation. Without loss of generality, we consider the binary classification task with binary sensitive attributes in this paper. We denote the dataset D= [(x, ya, s,] f=1= where N represents the number of samples, x; Rd is the features excluding sensitive attribute, 34 E {0,1) is the label of the downstream task, and the E {0. }} is the sensitive attributes of i-th sample. The index set of groups with sensitive attributes is defined as So = {i= 8, = 0} with No samples, and S = {i= : 8 1} with N₁ samples. The predictive probability is denoted as ÿ E [o, 1] by the machine learning model f Rdd + [0, The binary prediction is than threshold f0. 1}, where 0 = 1,, and (-)(() represents the indicator function of whether larger denotes the random variable that takes values x, Y denotes the random variable that takes values y- 5 denotes the random variable that takes values Y denotes the random variable that takes values We use fo/1(-) and (-)) to denote the probability and cumulative distribution of the predictive value ÿ over the group with sensitive attribute s = 0/1, respectively.
denoted as I E
2.1 Demographic Parity and Violation Measurement
The main idea behind demographic parity is that the prediction of the machine learning model should not be correlated the sensitive attributes. Demographic parity can be achieved if and only if the predictive probabilities are independent of the sensitive attributes.
To measure the violation of demographic parity, several metrics has been proposed to evaluate the demographic parity. The widely used metrics are A.Poontinuous (short for ADP.), which is calculated by the predictive probabilities (comlinuous), and Pt binary (short for ADP;), which is calculated by the binary prediction (binary) with respect to a threshold t. Here we present their formal definitions.
ADP (Dai & Wang, 2021: Creager et al., 2019; Edwards & Storkey, 2015; Kamishima et al., 2012) is the difference between the proportion of the positive prediction between different groups, which uses the difference
3

---

Published in Transactions on Machine Learning Research (05/2023)
of the average of the binary prediction between different groups as follows:
ADP = Enes, Dh No
Enes, Dh
(1)
N₁
where the 1hh _ [Dn] is the binary prodiction of the downstream task based on pre-defined threshold t, .N is the number of the instances, and No/1 is the number of the samples in the group with sensitive attribute 0/1.
ADP, (Chuang & Mroueh, 2020; Zemel et al., 2013) measures the difference of the average of the predictive probability between different demographic groups as follows:
Enes, In No
Enes, In N1
ADP-
(2)
where the ÿ is the prediction probability of the downstream lask, is the lotal number of the instances, Ng=0/1 is the total number of the samples in the group with sensitive attribute 0/1. The key condition for ADP. is that the average predictive probability ğ among the same sensitive attribute group is good approximation of the true conditional probability P(Y 1|$ 0) or P( 1|$ 1).
Discussion Relying solely on ADP, and ADP. as measurements for machine learning models can lead to trivial and unfair solutions, potentially misguiding the development of fair models. The current fairness methods usually result in trivial solutions if we use ADP; and ADP,, as shown in the results of Model in Figures and 5. The "unfair" solution may obtaín a. lower ADP and ADPs, but with unfair prediction. However, ADP, and ADP. have become the de facto measurements for the current fairness metric, as many previous works use them as fairness metrics (e.g., [2] use ADP, and [4][]] use ADP.). This could mislead the development of fairness methods in terms of demographic parity, ADP, = 0 (asymptotically) if and only if 9 is independent of the sensitive attributes since the conditional probability distribution given sensitive altribule S = 0 and S = are the same for the same ratio of positive samples across different
demographic groups, i.e., ADP: Enesa Ensei 2h. O. However, we argue that our proposed metrics N1
No
are a stronger version of ADP, especially for dynamic thresholds. In other words, when proposed metric ABCC or ABPC is zero, ÿ is independent of the sensitive attributes for any threshold. Conversely, ADP; 0 implies that ÿ is independent of the sensitive attributes for specific thresholds. In nutshell, our proposed fairness metrics are more general and stronger denographic parity metrics, which can be adopted in scenarios with dynamic thresholds.
The relation between ADP. and ADP. In the existing literature, ADP. and ADP are both commonly used, where the former mainly focuses on the fairness over original predictive probability and the latter fairness over the final decision making. Although these two metrics are adopted for different objectives, there is an intrinsic relation between them as shown in Theorem 2.1 (Please see Appendix A for more details.).
Theorem 2.1 The fairness measurement over original predictive probability ADP. is upper bounded by the mean fairness measurement over binary prediction ADP; with uniform-distributin dintributttttionnn threshold t, i.e., ADP. S In ADP Additionally, there must erist certain threshold t* so that the two fairness mensurements are equivalent, i.e., ADP. = ADP!..
3 Why does ADP Fail?
In this sention, we demonstrate that ADP is problematic in measuring the violation of demographic parity. We provide theoretical evidence of measurement failure of ADP. In addition, we also empirically demonstrate that ADP cannot measure demographic parity properly.
3.1 Theoretical Analysis
Argument 1: ADP 0 is only a necessary but insufficient condition for demographic parity to hold. ADP, relying the difference between the average prediction probability, is not sufficiently
4

---

Published in Transactions on Machine Learning Research (05/2023)
PDF
CDF
Threshold
1.0
ADP = 0.0608
Probabiitt
0:10
1.5
0.6
4DPpet
Biased
0.05
0.9
4DP
0.0451
0.0
0.4
0.6
1.0
Debiaved MLP
Dersiy
1.5
ADP.= 0.0051
Probebiitty
0.6
0.5
ADP 0.0233
0.05
0.0
0.00
Figure 2: The estimated PDFs and empirical CDFs of the predictive probability over different groups (i.e., male, female) the ACS-Enn CCS-Employyent dataset. Top: Biased MLP model. Bottom: Debiased MLP model with ADP: as regularization. The PDFs and CDFs of biased MT.P are obviously different, illustrating that the machine learning model is biased to gender. Debiased MLP achieves much lower ADPo than the biased MLP, however, the PDFs of different groups are different even though the "mean" of them are almost the same. The bottom figures show that ADP, is the difference between the CDF lines while the threshold 0.5. We also provide the same set of results on Adult data in Appendix G.
reliable to quantify model prediction bias. According to characteristie function in probability theory (Sasvári, 2000), the samne probability function is equivalent to the same r-th origin moment for any However, ADP only adopts average prediction (1-th origin moment) to define the metric. In other words, machine learning modols can still be biased even if such metrics are zero. The insufficiency to guarantee demographic parity damages the authority of fairness measurement. In summary, ADP. 0 and ADP; = are necessary but insufficient conditions for demographic parity, which requires that the prediction should be independent of the sensitive attributes. We conclude that ADP. and AD are necessary but insufficient conditions for demographic parity, which requires that the prediction should be independent of the sensitive attributes We present loy example to illustrate this argument. Let's consider we have the model's prediction g [0.4, 0.4, 0.4, 0.4, 0.5, 0.5, 0.5, 0.5, 0.5,0.9), 0.9), and the sensitive attribute s = [0, 0, 0, 0, 1,1,,, OJ. We can see that the model is unfair since tends to predict low values for samples with sensitive attribute O, while predicting high values for samples with sensitive attribute 1, despite an outlier (the last sample). However, ADP. 0, indicating a fair prediction. Thus, ADP. may fail to measure fairness.
Argument Threshold Rules harm the measurement accuracy of ADP; In practice the decision-making systems typically first predict a probability for the positive class and then obtain the binary prediction with a predefined threshold t, denoted prediction is 1 else the prediction will be O. This decision-making process is named Threshold Rules (Mitchell et al., 2021; Corbett-Davies Goel, 2018). The establishment of ADP; (Equation (1)) depends on the predefined threshold since the binary predictions ÿ is determined by the threshold t. However, the so-called threshold rules could harm the measurement accuracy of ADP, Changing the threshold for dccision-making may lead to the changing demographie parity violation. The changing ADP highlights the fundamental drawbacks of the metric of demographic parity. We present toy example to illustrate this argument. Let's consider the we have the model'ss prediction ğ = [0.35, 0.45, 0.55, 0.65) and the sensitive attribute = [0, 1, 4], if we set the threshold to 0.5 for classification, ADP = O, indicating a fair prediction. However, if we set the threshold to 0.6, ADP = 0.67, indication unfair prediction. Thus, with different. thresholds, ADP may fail to measure fairness when the threshold changes.
1 10] For example, let t = 0.5, if ÿ 2 0.5 then the
3.2 Empirical Investigation
In this section, we empirically explore why ADP fails to measure the violation demographic parity. We train one multilayer perceptron (Biased MLP) and the other MLP with fairness constraint (Debiased MLP)
3 3please see more details in Appendix R.
5

---

Published in Transactions on Machine Learning Research (05/2023)
the UCI Adult and ACS-Emmloyyment dataset. Then adopt Kernel Density Estimation (KDE) to estimate the PDFs of the predictive probability ÿ on the test dataset and also present the empirical CDFs. The results for ACS-Em SSSS--mplyymentt are presented in Figure 2, and the results on the Adult dataset are in Appendix G. From the experimental results, we have the following findings:
Finding The predictive probability distribution violates demographic parity. On both the Adult and ACS- EoS--mmloyyent datasets, the PDFs (Top-Left subfigure in Figure 2) are different among different groups, indicating the prediction is related to sensitive attributes. The difference between the PDFs reflects the degree of the violation of demographic parity. Similarly, the difference between the CDFs (Bottom-Left subfigure in Figure 2) also indicates the violation of demographic parity, and the area between the CDFs reflects the degree of fairness as well.
Finding 2: ADP, only measures the difference of the probability with a specific threshold t. The CDFs (Middle-Left subfigure in Figure 2) vary among different groups, indicating the disparity of predictive probability distribution. Even with the debiased MLP, the CDFs (Middle-Right figures in Figure 2) still vary. Typically, the practitioners make decisions using 0.5 threshold for binary classification. Thus, the ADP is the difference of CDF' when t = 0.5, making the ADP unreliable measurement for violation of demographic parity.
Finding 3: ADP =0 with specific threshold can not guarantee the demographic parity. On both the Adult and mestemployment datasets, the PDFs (Top-Right subfigure in Figure 2) are different among different groups but with the (nearly) same mean prediction, making ADP, O. This finding indicates that the ADPg ? is a thresholdddepeeddent metric, the value of which will be different if we choose different. thresholds in downstream tasks.
4 Criteria for Fairness Measurement
Tell me how you measure me and I will tell you how I will behave. If you measure me in an illogical way... do not complain about illogical behavior...
Eliyahu Goldratt
In this section, we provide a systematic understanding of fairness measurement from scratch, i.c., the criteria of fairness measurement design and int rinsie rationale. Given a dataset D = ((xi, Vi, 8i) and well-trained model ÿ = f(x), the fairness measurement can be determined as Bias(D,
Criterion (Sufficiency): One metric should be necessary and sufficient to measure the demographic parity. The fairness measurement. is adopted to characterize the derivation of the model from the "ideal" unbiased one, therefore, the prediction value is independent of sensitive attributes if and only if the fairness metric is zero.
Criterion 2 (Fidelity): One metric should be invariant with respect to monotone transformations of the distributions. Different values of the fairness metric should represent different levels of fairness. As mentioned in the introduction, for example, we change the scale on the predictive probability of one model, the fairness metric should be consistent in the original and scaled space. This property is referred as invarionce over invertible transformation. Thus we can compare the degree of fairness by comparing the values of ADP. Inspired by this, we formally provide the following definition of invariance:
Definition 4.1 (Invariance) For any invertible transformation T [0, 1] + [0, 1], the fairness measurement DDass f) salisfies mensurement invariance condition if for any dolaset D and machine learning model f((), (Da T o f) = Bias(D, f) always holds.
5 The Proposed Metrics
Following these criteria, in this section, we propose two metrics to mcasure the violation of demographic parity, namely Area Between PDF Curves (ABPC) and Area Between CDF Curves (ABCC), and prove that both ABPC and ABCC satisfy (or partially satisfy) the desired criteria.
6

---

Published in Transactions on Machine Learning Research (05/2023)
5.1 ABPC and ABCC Metrics
In this section, we formally define two distribution-level metrics to measure the violation of demographie parity. ABPC is defined as the total variation distance (TV) between probability density functions with different sensitive attributc groups as follows:
ABPC SV() - L. [f(()) fi)))
(3)
where fo( and S1 (1) are the PDFs of the predictive probability of different demographic groups. Similarly, ABCC is defined as the total variation between prediction cumulative density functions with different sensitive altribute groups as follows:
ABCC (()) = -f
- f( [Fo(z) Fi dx,
(4)
where Fo(r) and (z) are the CDF of the predictive probability of demographic groups.
Note that the proposed metrics can be easily extended to the multi-value sensitive attribute setting. Suppose the sensitive attribute has m values, then we compute the ABPC of each pair of groups with different sensitive attributes and then average them. ABCC for multi-value sensitive attributes with m values can also be computed in similar way. Since the is small in practice, the computational complexity is acceptable.
5.2 Theoretical Properties of ABPC and ABCC
In this section, we analyze the properties of ABPC and ABCC (Please see proofs in Appendix C), as well their relation.
Theorem 5.1 (Properties of ABPC) The proposed ABPC has the following desired properties: 1) ABPC 0 holds if and only if demographic parity is established. 2) ABPC is invariant to invertible transformation. 3) ABPC has a range of [0.2].
Relation to ADP.: The proposed fairness metric ABPC upper bounds ADP i.c., ABPC 2 ADP which overcomes the drawback of ADP, regarding the insufficient condition of demographic parity. Please see Appendix D for more details.
Theorem 5.2 (Properties of ABCC) The proposed ABCC has the following desired properties: 1) ABCC if and only if demographic parity establishes. 2)ABCC is continuous over model prediction. 3) ABCC has range [0, 1].
Relation to ADP': The ABCC generalizes ADP ADP, is the difference of positive prediction proportion
over different groups with respect to threshold rule t, thus ADP = (Fa(t) (i (t)). Considering the definition of ABCC = Jo [F(()) - (1() dr, we have ABCC = fo ADP dt. This indicates that ABCC is the average of the ADPE over all possible threshold t.
Relation between ABPC and ABCC. Hereby show the relation between ABPC and ABCC that they are both Wasserstein distance between the prediction PDFs of two groups, but with different transport cost functions (See Appendix F. for more details). The proposed fairness metric ABPC is Wasserstein distance with cost function co(2, 3) = 2- 1(x # y), defined as W., (fox)) f1()). The proposed fairness metric ABCC is Wasserstein distance with 1 cost. c(x, y) = Ir - yl between the prediction PDFs with different sensitive attributes. other words, ABPC and ABCC can be interpreted as Wasserstein distance between the PDFs for different sensitive attribute groups with different transport cost functions.
5.3 Estimating ABPC and ABCC
In real-world scenarios, the fairness mcasurement basod on the PDFs and CDFs of predictive probability, which can be estimated from finite data samples. Hereby we present the estimation methods for both ABPC and ABCC and present the theoretical and empirical analysis for the estimation.
7

---

Published in Transactions on Machine Learning Research (05/2023)
For the proposed ABPC, the PDFs can be estimated via kernel density estimation (KDE) given predictive probability (Tn. n E } for each sensitive attribute group with 8 = i. The basic idea for PDF estimation is that the prediction value for each sample represents a local PDF component (kernel function) and the overall mixed local PDF is the estimated PDF. Given the smoothing kernel function ((x) (e.g., Gaussian kernel) salisfying normalization condition f K (x))dz = 1, and bandwidth h, then the estimated PDF is f((z)= - B. Enes, k(*T/=), for i = 1, Subsequently, ABPC can be estimated via Equation (3).
For the proposed ABCC, directly adopt the empirical distribution function (Shorack & Wellner, 2009) estimated CDF, which mcasures the fraction of samples' predictions that are less or equal to the specified threshold. Formally, given predictive probability {ūn: n E Sc,,, the empirical distribution function F3 for sensitive attribute 8 = i is given by (x) = N; Enes, g= where i = 0, 1. Hence, based on the definition of proposed metrics, we have ABPC = So 1fo()) - (x)))dd and ABCC = fo Fo(x)) - R (x)]dx. Firstly, we provide the following definition to measure estimation tractability with finite data samples as follows:
Definition 5.3 ((N, ((tratttaaa)))l)))t)) Given the dalasel 'D with N data samples, underlying yet unknown data distribution PD, and well-trained machine learning model f(),, the fairness measurement satisfies (N,e)- tractability if for any dataset D and machine learning model f(), the meon square estimation error condition Ep Bi - Bias( Pv. f) < holds.
Subsequently, provide theoretical analysis the estimation error for estimated metrics, ABPC and ABCC, as follows (more details in Appendix F):
Lemma 5.4 (Estimation Tractability) Suppose we adopt KDE to estimate the prediction probability density function and directly calculate the estimated fairness metrics ABPC and ABCC, such fairness metrics estimation satisfies (N, ()N--)))-trrctiiie of data samples. Furthermore, for the number of samples N = O = and N = -0(8-1e3 for ABPC and ABCC, with probability at least 1 1-8, 8, [Bias(D. f) Bias(PP. ff) <E <..
and (N, ((N--))-trattaabllii where N represents the number
Lemma 5.4 demonstrates the reliability of our proposed metric with the finite data samples using the proposed estimation method. Recall demographic parity requires independent prediction with respect to sensitive altribules, such independence should be guaranteed at the distribution level. However, we only observe limited data samples, instead of the prediction distribution. Hence, the fairness measurement must be reliably estimated from limited observed data samples. The estimation error convergence provides a dispensable foundation for fairness measurement and model comparison in terms of fairness in practice.
Computation Complexity. The computation of the proposed ABPC and ABCC metrics, defined Equations (3) and (4), are based on estimated probability density function f((x) and cumulative function F(()), and numerical integration over [o, 1] with M probing points. For ABPC, the estimation of PDFs is given 1 Enes, K( K(==(=). The computation complexity of f((1)) with M probing points is O(MN), by f((x) (. where N is the number of samples, and the computation complexity of numerical integration is ((M). Therefore, the computation complexity of ABPC is O(MN). For ABCC, the computation complexity for the estimation of CDF with M probing points and numerical integration is O(MN) and O(M), respectively. Thus, the computation complexity of ABCC is also ((MN).
Discussion about Direct Optimization of ABPC and ABCC. The proposed ABPC and ABCC metrics are both diffcrentiable w.r.t. model parameters and thus can be directly optimized. The key reason is that the (conditional) prediction probability density can be estimated based on KDE, i.e., f((x) = श Enes, K( (= ), where K(x) is smoothing Gaussian kernel function, In is the model prediction for n-th sample, and h is pre-defined bandwidth. Note that ABPC is differentiable w.r.t. (conditional) prediction probability density, and (conditional) prediction probability density is differentiable w.r.t. model prediction and model parameters, ABPC can be directly optimized. Similarly, ABCC also can be directly optimized. We also clarify that this paper mainly focuses on the fairness metric side, the bias mitigation method development for ABPC and ABCC can be left. for future work.
K(=)(=)
8

---

Published in Transactions on Machine Learning Research (05/2023)
ABPC
ABCC
0.3
Relalive
MI
120
SourCH
0.2
0.1
64
0.0
0.2
0.4
0.6
0.8
1.0
Sample Numbers
Prediction
target
0.14
3.5
Relative Error
0.10
0.12
ABPC
ABCC
3.0
Daget
0.08
MI
0.08
2.0
0.04
1.5
0.02
1.0
0.00
10a
10"
0.0
0.2
0.4
0.6
0.8
1.0
target
Sample Numbers
Prediction
Figure 3: Relative estimation error for ABPC, ABCC, and mutual informa- tion. Top: The synthotic data is from (0.33, 0.1) and (0.4, 0.1). Boltom: The synthetic data from (V..2, and (0..400.1)
The source and target distributions and bias density visualization for vanilla MLP model. Bottom: The source and target distributions and bias density visualization for adversarial debiasing. The fair model represents the diagonal optimal transport plan, i.c., the source and target distribution is aligned everywhere.
Figure 4: Bias density visualization. Top:
5.4 Comparison with Related Work
We comprehensively analyze the existing demographic parity metrics and provide their inherent relations and differences. In other words, our work is metric-centric and provides insights to identify the promising properties, namely sufficiency, and fidelity, of fairness metrics. We highlight the differences with works (Jiang et al., 2020; 2022b) in terms of fairness metric and estimation tractability.
Difference with Jiang et al. (2020): For fairness metric, the motivation and justification differ significantly. In this work, we start with the proposed two critoria (i.c., sufficiency, fidclity), and then design fairness metries as the distance of probability density function (PDF) and cumulative distribution analyze that the proposed fairness metrics are actually some version
function (CDF). Furthermore, of Wasserstein distance. Instead, (Jiang et al., 2020) directly starts with Wasserstein distance to enforce strong demographic parity. For metric justifications, we mainly focus on the analysis of whether the proposed two criteria hold or not. Instead, (Jiang et al., 2020) provides an in-depth analysis of Wasserstein distance and demonstrates several equivalent formats in Section 3.1. On top the analysis, Wasserstein penalized logistic regression method and post-proccsse proepprocssing method are proposed to achieve fair prediction.
Difference with Jiang et al. (2022b): The results on eslimation tractability is similar to (Jiang et al., 2022b). However, we clarify that the target metrics are differ significantly. Jiang et al. (2022b) propose GDP to measure the bias for continuous sensitive attributes via the difference of average prediction across different sensitive attributes. Instead, we only focus on the proposed two fairness metrics for binary sensitive attributes by measuring the distribution distance. Our result shows that the proposed metrics have the same estimation tractability that in GDP. The main reason is that the estimation method of our paper and Jiang el al. (2022b) are both derived from non-parametric estimation theory (Sampson & Guttorp, 1992). These results demonstrate the proposed metrics can be reliable estimations in practice.
5.5 Experimental Evaluation
We empirically evaluate the estimation tractability of our proposed fairness metrics and visualize the bias density. First, we show the relative estimation error of our proposed metrics lower than that of mutual information (MI) in the experiments with synthetic data. Subsequently, we visualize the bias density for vanilla. MT.P and adversarial debiasing method in Ancome dataset.
9

---

Published in Transactions on Machine Learning Research (05/2023)
5.5.1 Estimation Tractability on Synthetic Data
We test the relative estimation error of the proposed ABPC and ABCC compared with mutual information, which precisely measures the dependence between two random variables. For data generation, we first. generate Gaussian mixture distribution with different means and variances for different groups. Then we use the sigmoid function to map all data into [0, 1]. For metric estimation, we adopt KDE and empirical CDF to estimate the ground-truth PDF and CDF, and then directly calculate estimated metrics via numerical integration. The relative estimation error is defined as the deviation of the estimated metric by ground-truth metrics.
Figure 3 shows the relative estimation error of different fairness metrics with respect to sample numbers for the synthetic data with different parameters. We observe that the relative estimation error for ABCC and mutual information are the lowest and the largest for different numbers of data samples, respoctively. In other words, our proposed metrics embrace higher estimation tractability with finite data samples.
In this experiment, mutual information is calculated based on prediction probability and conditional prediction probability, i.e., MI(Ý; S) = H - -H(YIS) = H(Y) - P(S = 0) (Y15 0) - P(S = HY|S = 1), where the entropy function H(Y) Sg fy (ū)dū, and H(YS= = = ( for 0, Mutual information between model prediction Y E [0. 1] and sensitive attribute S E 0, can be adopted to measure the independence. Compared with the definitions of ABPC and ABCC, the calculation of MI is also based on the estimated (conditional) prediction probability, as shown in Scction 5.3. The advantage of our proposed ABPC and ABCC is the lower metric relative estimation error.
5.5.2 Bias Density Visualization
Section 5.2 shows that ABCC is the 1a-sssssrrteein distance of two PDFs of different groups (Please see Appendix T for more details). In other words, ABCC can be decompased into the integration of bias density over prediction domains via calculating the optimal transport plan. Suppose 7" (z, y) is the optimal transportation plan between these two PDFs For ABCC, we can define the bias density as p(x, y) _ |x ul>"(1, v) since ABCC satisfies ABCC = f p(x, y)dzdy. In other words, the bias can be decomposed across the prediction value domain for these two sensitive attribute groups. We visualize the bias density of the vanilla method and adversarial debiasing on ACS-Ineome dataset. Figure 4 shows the estimated PDFss for two groups (source and target distribution) and visualizes the bias density. For the top subfigure, the source distribution is higher than the target distribution at around 0.5 and 0.9 prediction values. The bias density shows that this excess part at the source distribution should be moved toward the target distribution at around 0.2 and 0.6 prediction values. For the boltom subfigure, these two distributions are extremely close and thus lead to a low fairness metric. The optimal transport plan is close to the diagonal line, demonstrating that most parts of the source distribution keep the original value.
6 Re-evaluating Existing Fair Models
In this section, we conduct experiments on various datasets to re-evaluate the commonly-used fair models.
6.1 Experimental Setting
We consider the vanilla MLP model (MLP) and widely used debiasing methods, including regularization (REG), and adversarial debiasing (ADV). MLP (Multilayer Perceptron) is the vanilla multilayer perceptron to minimize the empirical risk of downstream tasks. Thus MLP tends to be biased the underprivileged group, making it biased model. In our experiments, network and utilize ReLU (Nair & Hinton, 2010) as the activation function. The same network architecture is also adopted as the classification network for other baselines. ADV (Adversarial Debiasing) (Louppe et al., 2017) jointly trains a classification network and an adversarial network. The adversarial network takes the output of the classifier as its input and is trained to distinguish which sensitive attribute group the output comes from. We train the classifier to provide correct predictions for the input data while training the adversarial network not to distinguish groups from the output of this classifier aimultaneously.
Debiasing Methods.
adopt 4-layer fully-connected
10

---

Published in Transactions on Machine Learning Research (05/2023)
Table 1: The fairness performance on the tabular dataset for existing fair models and we consider race and gender sensitive attributes. represents the accuracy improvement compared to MLP. A higher accuracy metric indicates better performance. & represents the improvement of fairness compared to MLP. A lower fairness metric indicates better fairness. The results are based on 10 runs for all methods.
Accuracy
Fairness
Methods
Acc(%)
AP(%%
ADP(
(DP.
(APP((%)
ABCC(%)
MLP 85.54=0.19
77.33=+0.2
19.27
20.03=0.44
64.65=1007
20.03=0044
Race
REG 85.31+0.12
-0.27%
75.55+00115
-2.0%
14.18+0.67
26.41%
0.81=0.62
95.96%
62.21-2.68
3.77%
10.79+0.53
46. 46.13%
Adult
ADV 80.03
-6.44%
4..84444844
-20.06%
3.8941194
79.81%
1.6211.41
91.91%
21.2611375
67.12%
2.60
87.02%
MLP 85.52=0.12
77.33
21.84+0.59
25.9 25.93=0.60
98. 98.62=00.53
25. 25.93#0.50
Gender
REG 85.03+0.22
-0.57%
73.55+00.5
-4.89%
15.58+0..9
28.66%
1.30=1.05
94.99%
80.62+4.62
18.25%
12.86+0.30
50.40%
ADV 76,3120.01 -10.73%
69. 38..8.82.89
-10.28%
0.31.00.9
58.58%
0.5610.72
97.84%
87.92::33.2
10.85%
0.56.0.72
97.841%
RDD Censuns
MLP 94,9420.01
99. 50:00.00
2.64::0.11
3.73=0000
18.91
3.73:10.08
Race
REG 94.81+0.06
-0.14%
99.42++0.0
-0.08%
1.61=0.21
39.02%
0.78=0.27
79.09%
10. 19.43
46.11%
0.83+0.25 26
77.75%
ADV 93.7220.29
-1.29%
99. 18.13.00.00 16
-0.37% %
0.14
94. 70%
0.32
91.12%%
11.11
41.25%
0.3410027
90.88%
MLP 94.84=0.06
99.45=#0.00
4.75=0.36
5,99=0.30
40.96=00.0
5.99+0.30
Gender REG 94.45+0.06
-0.41%
19..1++0.0
-.14%
1. 1.38=0.20
70.95%
0.86=0.23
85.64%
8.57+0.19
79.08%
1.04+0.14
82.64%
ADV 93.6620.17
-1.21%
98.62:10.4
-0.83%
0.35.10.7
92.63%
0.36110.6
93. 93.99%
5.111:2.09
87,52%
0.455:0:22
92. 99%19%
MLP 10:00.0.0
84. 83:10.1
9.57
7.47=00.22
16.82:00.44
7.47:0.12
Race
REG 81.15+0.18
-0.79%
83.92++0.12 12
-1.07%
3.11+0.43
07.50%
1.66=0.37
77.78%
9.19 0.1400.52
45.36%
2,48+0.18
66.80%
ACS-T
ADV 77.7220.28 -4.99%
79.06:10.39
-6. -6.80%
0.45.10.50
95.30%
0.26110.26
96.52%
2.7844074
83.47%
0.38:10.19
91.91%
MLP 81.7820.09
84. 11.665100.10 16
9. 9.14::0.16
7.97=0.11
14.74:0.59
7.97
Gender REG 80.93+0.06
-1.04%
83.61 +0.17
-1. 1..23%
2.10=0.22
77.02%
1.60==.20
79.92%
3. 6.6444
74.97%
1. 1.60+0.20 20
79.92%
ADV 78.4220.85
-4.11%
79.62:10.42
-5.94%
0.32:10.0
96.50%
0.241.0.14
96.99%
2.5410772
82.77%
0.48-10.10
93.98%
ACS.E
MLP 81.7820.04
85, 26:10.05
5.49910.79 79
6.73
42.18:20.70
7.1210.44
Gender REG 81.57+0.06
-0.26%
84.51+0.13
-0.88%
1.02#+.65
81.42%
0.39+0.33
94.21% %
22. 22.40=1.00
46.89%
3.30+0.12
53.65%
ADV 77.71 -4.98%
81.39:10.36
-4.51%
1.15220.74
79.05%
0.82=0.77
87.82%
3.67
91.30%
0.97:10.62
86.38%
REG (Regularization) is a kind of in-process method that adds a fairness-related regularization term to the objective function (Chuang& & Mroueh, 2020; Kamishima et al., 2012). This kind of method improves the fairness of the model with the regularization term simultaneously optimized during training. In our experiments, REG takes ADP, as the regularization term. The objective function is defined as Lee + ACdpp where Lee is the cross-cntropy loss for downstream task and Lip is fairness constraint (Equation (2)).
Dataset. We consider the following datasets in our experiment. including tabular dataset and image dataset. (See more experimental results in Appendix H). UCI Adult (Dua & Graff, 2017) contains clean information about 45, 222 individuals from the 1994 US Consus. One instance is described with 15 attributes. The task is to predict whether the income of a person is higher than $50k given attributes about the person. We considered gender and race as sensitive attributes. ACS-Incone (Ding al., 2021) derives from the American Community Survey (ACS) Public Use Microdata Sample (PUMS). Like UCI Adult, the task on this dataset is to predict whether individual's income is above $50k. The dataset contains 1, 664, 500 data points. We choose gender and race as sensitive attributes. ACp-Employment (Ding et al., 2021) also derives from the ACS PUMS. This task is to predict whether an individual is employed. The number of data in this dataset is 3, 236 107. In this dataset, gender is used as the sensitive attribule. KDD Census (Dua & Graff, 2017) contains 284, 556 clean instances with 41 attributes. The task of this dataset is also to predict whether the individual's income is above $50k. The sensitive attributes are gender and race.
Evaluation Metric. The fairness metrics are ADP ADP, ABPC and ABCC. The evaluation metrics for model accuracy are: 1) Acc: the accuracy is the fraction of correct predictions; 2) AP: the average precision (AP) is defined as AP = E,, (R, - Pn where P, and Rn-1 are the precision and recall at the n-th threshold. The value is between 0 and 1, and higher is better.
6.2 Will ABPC and ABCC Measure the Violation of Demographic Precisely?
We empirically validate the effectiveness of the proposed metries in this section. We train three different models and plot the distribution of the predictive probability for different groups, and we report the fairness
11

---

Published in Transactions on Machine Learning Research (05/2023)
Model I (unfair)
Model (unfair)
Model 3 (fair)
Male Female
Density
Densiy
Density
4
2
0
0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
9
9
ADP,= 0.1887
ADP:= 0.1998
ADP, = 0.1554
ADP:= 0.0174X
4DP;= 0.0315
ADP:= 0.0264
ABPC = 0.6961 ABCC= 0,1998
ABPC= 0.6387
ABCC= 0.1137
ABPC= 0.1331 ABCC= 0.0271
Figure 5: The distribution of the predictive probability of male and female groups on the Adult dataset.
metries ADP and &DP;, ABPC and ABCC for them in Figures 1 and 5. Figures 1 and 5 show the predictive probability distribution of three machine models. The Models 1, 2, and 3 are unfair, unfair, and fair, respectively, since the distributions of Model 2 for different groups are the same while others are not. One can see that ADP can measure the unfair model (Model 1) and fair model (Model 3) correctly since it has a larger ADP value for Model 1 than Model 3. However, it fails to assess Model 2 (unfair) since it obtains relatively low value on unfair model. Our proposed metrics ABPC and ABCC can assess all the models correctly. The wrong fairness assessment on Model 2 demonstrates the ADP is problematic in measuring the violation of demographic parity, ADP, was able to measure the violation of Model 2 in Figure 5 on the Adult dataset, where ADP, = 0.1554 indicates that the model is unfair. However, for Model 2 in Figure 1 on the InS-me dataset, both ADP, and ADP, failed to evaluate the demographic parity violation. Therefore, ADR, cannot consistently evaluate models across different datasets, which is another experimental evidence of its failure to measure the violation of demographic parity. Regarding the model side, this may be due to the fact that current fairness methods often result in trivial solutions when using ADP, and ADP. measurements.
6.3 How the Existing Fair Models Performs with ABPC and ABCC?
We conduct experiments on the aforementioned four datasets and baselines. Since there is no universal best model that optimizes both fairness and accuracy objectives, trained all the models with fixed number of epochs and reported the performance on the test dalaset. We train MLP and REG for 10 epochs and train ADV for 40 epochs. We use ten different dataset splits and report the mean and standard deviation of Acc and AP. We report the prediction performance and the fairness performance of the downstream task in Table 1. We have the following Observations:
Obs. 1: REG method always achieves a lower ADP. but a relatively high ADP. Since the REG directly optimizes the ADP., REG always obtains lower ADP.. However, REG achieves much higher ABPC and ABCC than other methods.
Obs. 2: ADV is a better fair model to achieve demographic parity. In terms of the proposed metrics, the ADV gains 6 better ABPC among 8 cases and 8 better ABCC among 8 cases, showing that ADV achieves the best fairness performance. The reason why ADV performs better using our proposed metrics twofold. First,, adversarial learning can intuitively and theoretically ensure that the predicted probabilities (a continuous value) are independent of the sensitive attributes. This is supported by both theoretical and empirical evidence. Second, our two proposed metrics are designed to be 0 only when the predicted probabilities are fully independent of the sensitive attributes. This provides more strict and precise mcasure of demographic parity, which may make it casier for the ADV method to achieve better performance on this metric. On the other hand, the REG method in our paper uses ADP. as a fairness regularization term (as well as a surrogate loss function for ADP.). By using the difference of the average predicted probabilitics for demographic groups, it can ensure that the prodicted probabilitics are fully independent of the sensitive attributes. Thus, may achieve secondary performance on our proposed metric. The experimental observation that ADV performs better with respect to their metrics somehow shows that.
12

---

Published in Transactions on Machine Learning Research (05/2023)
UCI Adult
KDD Census
ACSIncome
95.5
82.0
81.5
Gender
Accuracy
84.0
Accuracy
95.0
Accuracy
22.0
94.5
80.5
80.0
80.0
78.0
94.0
79.5
0
5
10
15
20
25
Fairness (ABCC)
Fairness (ABCC)
Fairness (ABCC)
95.5
81.5
95.0
Race
Accuracy
02.0
Accuracy
Accura
81.0
80.0
78.0
94.0
79.5
REG
76.0
93.5
79.0
10
15
20
2
Fairesss (ABCC)
Fairness (ABCC)
Fairness (ABCC)
Figure 6: The accuracy and fairness tradc-off on the tabular datasct. The Parcto fronticrs show the accuracy- fairness trade-off of different fair models.
measuring demographic parity from the distribution of predicted probabilities can provide more insight into the behavior of existing fairness methods.
Obs. 3: The inherent tension between fairness and accuracy is underestimated. Tt the prevailing wisdom that model's fairness and its accuracy are in tension with one another. Although the REG does not harm the model accuracy dramatically, it can not achieve ideal fairness. In contrast, the ADV achieves better fairness but harms the model accuracy dramatically. This observation demonstrates that if we tend to achieve fair decision-making, we have to sacrifice more accuracy.
6.4 How Accuracy and Fairness Trade-off with the Proposed Metrics?
In this section, we provide the "Parcto front" to investigate the tension between the model accuracy (Acc) and fairness (ABCC) shown in Figure 6. For REG, we set different values of the trade-off hyperparameter À [o, 1] to control the accuracy-fairness trade-off and À E [10, 180] for ADV. We have the following Observations:
Obs. 1: REG tends to gain a better accuracy performance while ADV tends to gain better fairness performance. The Pareto front of REG is higher than that of ADV, indicating that REG obtains a better accuracy performance. In contrast, the Pareto front of ADV is lower than that of REG, but it can reach a lower ABCC area, indicating that REG obtains a better fairness performance. The results show that our proposed metrics can better evaluate the performance of fair models and provide new standards analyze model debiasing.
The inherent tension between fairness and accuracy is underestimated. The REG method cannot achieve lower ABCC, which may limit its application to real-world tasks. Although the ADV can achieve much lower ABCC, the accuracy drop is too high. In other words, ADV sacrifices too much performance in pursuit of fairness. The area (indicated as F) points out the potential dircction of the lair model development.
Obs. 2:
7 Related Work
In this section, we present related works about fairness and metrics for demographic parity.
Fairness. Algorithmic fairness is legally mandatory in various high-stake real-world applications. The various fairness definitions have been proposed and categorised into individual fairness (Yurochkin et al., 2019; Mukherjce al., 2020; Yurochkin Sun, 2020; Kang et al., 2020; Mukherjee et al., 2022), group fairness (Hardt al., 2016: Verma & Rubin, 2018; Li et al., 2020; Ling al., 2023; Jiang et al., 2022a), and counterfactual fairness (Kusner et al., 2017; Agarwal et al., 2021; Zuo et al., 2022). In this paper, we focus
13

---

Published in Transactions on Machine Learning Research (05/2023)
demographic parity, widely studied group fairness. The metric for group fairness measures the disparity between subgroups defined by sensitive attributes (e.g., gender and race). For example, demographic parity (Dwork et al., 2012) aims to render the independence of the prediction and sensitive attribute, and ADP mcasuring the average prediction disparity among different groups is widely adopted as the fairness metrics for demographic metrics. Several works leverage other measurements (i.e., Wasserstein distance, ACC) for bias mitigation or measurement (Chzhen et al., 2020; Miroshníkov et al., 2020; 2021; Kallus Zhou, 2019; Barata et al.). Robust fairness is also well studied (Mehrotra & Vishnoi, 2022; Ma et al., 2022; Chai & Wang, 2022; An et al., 2022; Giguere et al., 2022; Jiang et al., 2023), such under distribution shift and with limited sensitive attributes.
Metrics for Demographic Parity. Metrics for the violation of demographic parity are proposed to evaluate the demographic parity. The widely used metrics are contiinnuuus (short for ADP;), which is calculated binary (short for ADP)), which is calculated by the binary prediction (binary) with respect. to a threshold L. Here we present. their formal definitions. ADP; (Dai & Wang, 2021; Creager et al., 2019; Edwards & Storkey, 2015; Kamishima et al., 2012) is the difference between the proportion of the positive prodiction between different groups, which uses the difference of the average of the binary prediction between different groups. ADP₂ (Chuang & Mroueh, 2020; Zemel et al., 2013) measures the difference in the mean of the predictive probability, which uses the difference in the average of the predictive probability between different groups. In addition, the metric based on Wasserstein distance for demographic parity also is proposed in (Jiang et al., 2020). The authors (Jiang et al., 2020) directly propose strong demographic parity on predictive probability using Wasserstein distance. Instead, in this paper, we start with the proposed two criteria (i.e., sufficiency, and fidelity), and then design fairness metrics as the distance of probability density function (PDF) and cumulative distribution function (CDF). Furthermore, we analyze that the proposed fairness metries are actually some version of Wasserstein distance. The fairness metric for contimuous sensitive attributes with Kernel Density Estimation (KDE) has also been proposed in works (Mary et al., 2019; Grari et al., 2020; Jiang et al., 2022b).
by the predictive probabilities (continuous), and A Pinarry
8 Conclusion
In this paper, we rethink the rationale of the widcly adopted fairness metric ADP and propose two metrics for demographic parity with sufficiency and fidelity. Specifically, we first point out that the existing fairness metric is not the necessary and sufficient condition for demographic parity. We propose two fundamental criteria for fairness measurement design. Further, we propose two fairness metrics, namely ABPC and ABCC, which satisfy the proposed criteria with theoretical and experimental justification. Finally, we re-evaluate the three standard baselines on tabular and image datasets considering all fairness metrics of demographic parity. We believe the proposed metrics would contribute to the fairness of academic and industrial communities by properly evaluating the performance of fair models and suggesting way to model development.
Acknowledgements
We are grateful for the detailed reviews from the reviewers. Their thought.ul comments and suggestions were invaluable in strengthening this paper. We also would like to sincerely thank everyone who has provided their generous feedback for this work. This work was supported in part by National Science Foundation (NSF) IIS-1900990, IIS-1939716, IIS- 2239222, and NSF IIS-2224843.
References
Chirag Agarwal, Himabindu Lakkaraju, and Marinka Zitnik. Towards a unified framework for fair and stable graph representation learning. In UAI 2021: Uncertainty in Artificial Intelligence, 2021.
Muhammad Aurangzeb Ahmad, Arpit Patel, Carly Eckert, Vikas Kumar, and Ankur Teredesai. Fairness in machine learning for healthcare. In Proceedings of the 26th ACM SIGKDD, pp. 3529-3350003 00--3,30, 2020.
Bang An, Zora Che, Mucong Ding, and Furong Huang. Transferring fairness under distribution shifts via fair consistency regularization. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.),
14

---

Published in Transactions on Machine Learning Research (05/2023)
Advances in Neural Information Processing Systems, 2022. URL httgs://opeamrviis...ee/ooooooi Jp_
C Ballantine and J Roberts. A simple proof of rolle's theorem for finite fields. The Americon mathematical monthly, 109(1):72 74, 2002.
António Pereira Barata, Frank W Takes, H Jaap van den Herik, and Cor J Veenman. Fair tree classifier using strong demographie parity.
Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50(1):3-44, 2021.
Ran Canetti, Aloni Cohen, Nishanth Dikkala, Govind Ramnarayan, Sarah Scheffler, and Adam Smith. From soft classifiers to hard decisions: How fair can we be? In Proceedings of the conference on fairness, accounlabilily, and transparency, pp. 309 318, 2019.
Alexander W Cappelen and Ole Frithjof Norheim. Responsibility, fairness and rationing in health care. Health policy, 76(3):312-33 319, 2006.
Junyi Chai and Xiaoqian Wang. Self-supervised fair representation learning without demographics. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreviee http://eeeeeeoom////////o..o....
Junyi Chai, Taeuk Jang, and Xiaoqian Wang. Fairness without demographics through knowledge distillation. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Newral Information Processing Systems, 2022. URL https: attps://openreviee htt.:/r.......ooooomoo//gg/ageae
Mingliang Chen and Min Wu. Towards threshold invariant fair classification. In Conference on Uncertainty in Artificial Intelligence, pp. 560-569. PMLR, 2020.
Ching-Yao Chuang and Youssef Mrouch. Fair mixup: Fairness via interpolation. In ICLR, 2020.
Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimíliano Pontil. Fair regression with wasserstein barycenters. Advances in Newrol Information Processing Systems, 33:7321 7331, 2020.
Sam Corbett-Davier and Sharad Goel. The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint ar Kiw:1808.0009 2018
Sam Corbett- Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. In Proceedings of the 29rd acm sigkdd international conference on knowledge discovery and dala mining, pp.
2017.
Elliot Creager, David Madras, Jörn-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and PMLR,
Richard Zemel. Flexibly fair representation learning by disentanglement. In ICML, pp. 2019.
Enyan Dai and Suhang Wang. Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. In Proceedings of the 14th ACM WSDM, pp. 680-688 2021.
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine lcarning. NewrIPS, 2021.
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http:://archiie..ccs.uu uci. ml.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214-226, 2012.
Harrison Edwards and Amos Storkey. Censoring representations with an adversary. arXiv preprint arxiv:1511 151.00097, 2015.
15

---

Published in Transactions on Machine Learning Research (05/2023)
Sorelle A Friedler, Carlos Scheidegger, and Suresh Venkatasuuramannian. On the (im) possibility of fairness. Xrviv preprint Xiv: 1511:600.000 1609.07266, 2016.
Pratyush Garg, John Villasenor, and Virginia Foggo. Fairness metries: A comparative analysis. In 2020 IEEE International Conference on Big Data (Big Data), pp. 3662 3666. IEEE, 2020.
Stephen Giguere, Blossom Metevier, Yuriy Brun, Philip S. Thomas, Scott Niekum, and Bruno Castro da Silva. Fairness guarantees under demographic shift. In International Conference on Learning Representations, 2022. URL. http////rrrreeeeeeeeetoormmeeeea
Vincent Grari, Sylvain Lamprier, and Marcin Detyniecki. Fairness-aware neural rényi minimization for continuous features. In IJCAI, 2020.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning, NewrIPS, 29, 2016.
Frances Heidensohn. Models of justice: Portia or persephone? some thoughts on equality, fairness and gender in the field of criminal justice. International journal of the sociology of law, 14(3/4)2287-90 1986.
Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, and Silvia Chiappa. Wasserstein fair classification. In Uncertainty in Artificial Intelligence, pp. 862 872. PMLR, 2020.
Zhimeng Jiang, Xiaotian Han, Chao Fan, Zirui Liu, Na Zou, Ali Mostafavi, and Xia Hu. Fmm: Toward fair graph message passing against topology bias. arXiv preprinl arXim:220.044 041877 2022a.
Zhimeng Jiang, Xiaotian Han, Chao Fan, Fan Yang, Mostafavi Ali, and Hu Xia. Generalized demographic parity for group fairness. In ICLR, 2022b.
Zhimeng Jiang, Xiaotian Han, Hongye Jin, Guanchu Wang, Na Zou, and Xia Hu. Weight perturbation can help fairness under distríbution shift. arXiv preprint ar Xiv:29993 03300, 2023.
Jeff Johnson, Donald M Truxillo, Berrin Erdogan, Talya N Bauer, and Leslic Hammer. Perceptions of overall fairness: are effects on job performance moderated by leader-member exchange? Human Performance, 22 (5), 2009.
Nathan Kallus and Angela Zhou. The fairness of risk scores beyond classification: Bipartite ranking and the xauc metric. Advances in neural information processing systems, 32, 2019.
Toshihiro
Kamishima, Shotaro Akaho, Hidcki Asoh, and Jun Sakuma. Fairness-aware classifier with prejudice remover regularizer. In Joint European conference on machine learning and knowledge discovery in databases. Springer, 2012.
Jian Kang, Jingrui He, Ross Maciejewski, and Hanghang Tong. Inform: Individual fairness on graph mining. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pp. 379-389, 2020.
Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. arXiv preprint 1xXi::1609005007 2016.
Nikita Kozodoi, Johannes Jacob, and Stefan Lessmann. Fairness credit scoring: Assessment, implementation and profit implications. European Journal of Operational Research, 297(3):1083 1094, 2022.
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. NewrIPS, 30, 2017.
Peizhao Li, Yifei Wang, Han Zhao, Pengyu Hong, and Hongfu Liu. On dyadic fairness: Exploring and mitigating bias in graph connections. In International Conference on Learning Representations, 2020.
Hongyi Ling, Zhimeng Jiang, Youzhi Luo, Shuiwang Ji, and Na Zou. Learning fair graph representations via automated data augmentations. In International Conference Learning Representations, 2023.
16

---

Published in Transactions on Machine Learning Research (05/2023)
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of ICCV, December 2015.
Gilles Louppe, Michael Kagan, and Kyle Cranmer. Learning to pivot with adversarial networks. NewrIPS, 30, 2017.
Xinsong Ma, Zekai Wang, and Weiwei Liu. On the tradeoff between robust and fairness. Tn Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL attps://openrevieen htwp://nwwreeeeeeeoeooooooeeeeao wttt://orrurrt.11...........
Jérémic Mary, Clément Calauzenes, and Noureddine El Karoui. Fairness-aware learning for continuous allribules and Treatments. In ICML. PMLR, 2019.
Ninarch Mchrabi, Yuzhong Huang, and Fred Morstatter. Statistical equity: A fairness classification objective. arXiv preprinl ar Xxx:2005.07293, 2020.
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6):1-33, 35, 2021.
Anay Mehrotra and Nisheeth K Vishnoi. Fair ranking with noisy protected attributes. In Alice H. Oh, Alckh Agarwal, Danielle Belgrave, and Kyunghyun Cho (ods.), Advances in Neural Information Processing Systems, 2022. URL https / /pwpreeveeeeeeeeeeeeeeeeeeeeeeee wwwpwviiweeeeeefooeeeeeeeeeeeeee orruriidan
Aditya Krishna Menon and Robert C Williamson. The cost of fairness in classification. ar preprint Stfi:1705.00000 2017.
Aditya Krishna Menon and Robert C Williamson. The cost of fairness in binary classification. In Conference Fairness, accountability and transparency, pp. 107-118. PMLR, 2018.
Alexey Miroshnikov, Konstandinos Kotsiopoulos, Ryan Franks, and Arjun Ravi Kannan. Wasserstein-based fairness interpretability framework for machine learning models. arxie preprint arXiv:2011.03152 2020.
Alexey Miroshnikov, Konstandinos Kotsiopoulos, Ryan Franks, and Arjun Ravi Kannan. Model-agnostic bias mitigation methods with regressor distribution control for wasserstein-based fairness metrics. arXiv preprint (rXic:2111.1190 2021.
Shira Mitchell, Eric Potash, Solon Barocas, Alexander D' D'Amour, and Kristian Lum. Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8:141-163, 2021.
Amitabha Mukerjee, Rita Biswas, Kalyanmoy Deb, and Amrit Mathur. Multi-objective evolutionary algorithms for the risk-return trade-off off in bank loan management. International Transactions in operational research, 9(5):583 597, 2002.
Debarghya Mukherjee, Mikhail Yurochkin, Moulinath Banerjee, and Yuekai Sun. Two simple ways to learn individual fairness metries from data. In JCML, 2020.
Debarghya Mukherjee, Felix Petersen, Mikhail Yurochkin, and Yuekai Sun. Domain adaptation meets individual fairness. and they get along. In Alice H. Oh, Alckh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL attps://opps /ttp://///////eee/////e .et.eeereeeeeeeeeeeeeeeee forum? ? XXG.XAXA..A.....
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On fairness and calibration. Advances in neural information processing systems, 30, 2017.
Manish Raghavan, Solon Barocas, Jon Kleinberg, and Karen Levy. Mitigating bias in algorithmic hiring: Evaluating claims and practices. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pp. 469 481, 2020.
17

---

Published in Transactions on Machine Learning Research (05/2023)
Ludger Rtischendorf. The wasserstein distance and approximation theorems. Probability Theory and Related Fields, 10(1):117-129, 1985.
Paul Sampson and Peter Guttorp. Nonparametric estimation of nonstationary spatial covariance structure. Journal of the Americon Statistical Association, 1992.
Zoltán Sasvári. Characleristic functions and moment sequences: posilive definiteness in probability. Nova Publishers, 2000.
Nripsuta Ani Saxena, Karen Huang, Evan DeFilippis, Goran Radanovie, David C Parkes, and Yang Liu. How do fairness definitions fare? examining public attitudes towards algorithmic definitions of fairness. In Proceedings of the 2019 AAAT/ACM Conference on Al, Ethics, and Sociely, pp. 99 106, 2019.
Galen R. Shorack and Jon A Wellner. Empirical processes with applications to statistics. SIAM, 2009.
Songül Tolan, Marius Miron, Emilia Gómez, and Carlos Castillo. Why machine learning may lead to unfairness: Evidence from risk assessment for juvenile justice in catalonia. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Low, pp. 83 92, 2019.
Berk Ustun, Yang Liu, and David Parkes. Fairness without harm: Decoupled classifiers with preference guarantees. In International Conference on Machine Learning, pp. 6373 6382. PMLR, 2019.
Sahil Verma and Julia Rubin. Fairness definitions explained. In 2018 ieee/acm international workshop software fairness (fairware), pp. 1-7. IEEE, 2018.
Mikhail Yurochkin and Yuekai Sun. Sensei: Sensitive set invariance for enforcing individual fairness. arXiv preprint ar Xiv:20006. 14168, 2020.
Mikhail Yurochkin, Amanda Bower, and Yuckai Sun. Training individually fair ml models with sensitive subspace robustness. orXiv preprinl (xXix:1977000020 2019.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics, pp. 962-970. PMLR, 2017.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In ICML, pp. 325-333. PMLR, 2013.
Guanhua Zhang, Yihua Zhang, Yang Zhang, Wenqi Fan, Qing Li, Sijia Liu, and Shiyu Chang. Fairness reprogramming. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openrevii/
net net/forumiid-laas BateaayrrrBdZyyi
Aoqi Zuo, Susan Wei, Tongliang Liu, Bo Han, Kun Zhang, and Mingming Gong, Counterfactual fairness with partially known causal graph. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Newral Information Processing Systems, 2022. URL cttps://openrevil forum? i 9aLbntH2lUq.
18

---

Published in Transactions on Machine Learning Research (05/2023)
A Proof of Theorem 2.1
Firstly, we provide the relation between binary prediction ÿ and the original predictive probability g. i.c., the original predictive probability ÿ is the mean of binary prediction O with wilform-distribution threshold t:
[. ah f
0n dt = In:
12 =
(5)
Furthermore, can derive the relation between these two fairness measurements based the definition of ADPo and ADP; in Equation (1) and Equation (2):
!. ADppdt= S( Enes, In
Enes, Ih
dt
No
N1
So Enes In
Enes,
In
dt
No
N₁
So dn
So In
Σ néSp
Σ
No
Ni
neS,
Enes,
In
Enes,
In
= ADP
(6)
No
N1
Eneo, 12 No
Enesi 22 = 0 ADP.
Note that if we set threshold as = 0, it is casy to obtain that ADP?
N1
Additionally, according to Rolle's theorem (Ballantine & Roberts, 2002) and Equation (6), there exist certain threshold to so that the following equation holds
ADP = L' ADP ADP.
(7)
Considering the continuity of fairness measurement ADP over threshold t, there exists threshold t* [0, tol that ADP5 ADP. holds, which completes the proof.
B More Discussion on Insufficiency of ADP
Proposition B.1 ADP! 0 and ADP, are necessary but insufficient conditions for demogrophic parily, which requires that the prediction should be tndependent of the sensitive attributes.
If the predictive values satisfy demographic parity, the predictive probability should be independent of the sensitive attributes. Obviously, the distributions of the predictive probability of different groups follow the identical distribution, thus, the mean of the predictive probability of different groups should be the same. In practice, if the number of instances is large enough, ADP, = 0 and A 0 for any threshold hold. On the contrary, ADP- = 0 or ADPA different groups follows the ident ical distribution. Thus we obtain that ADP, = 0 or ADP' = 0 is necessary but not sufficient conditions for demographic parity. This proposition illustrates that zero-value fairness measurements ADP, and ADP. = can not guarantce that the model prediction and sensitive attributes are independent.
for a certain threshold will not guarantee that the predictive probability of
C Proof of Properties of ABPC and of ABCC
Firstly, it is easy to check the continuity condition since the PDF and CDF estimation are continuous over model prediction, and proposed bias metrics ABPC and ABCC also continuous w.r.t. the PDF and CDF estimation. As for the invariance over invertible transformation T, supposed the PDF fo(x) and fi()) represent the PDF of model prediction for different groups, and the transformed prediction PDF is fo(2)
19

---

Published in Transactions on Machine Learning Research (05/2023)
and (z) with Tx)), note that the transformation T is invertible, according to probability theory, the relation between fo( and fz(z) satisfies fo(z)dr fo(z)z. Therefore, we have
ABPC, - - -f: f(e)ddd S. [fo(z) - fi(x) dr ABPC,i
Lastly, consider the necessary and sufficient conditions demographic parity. It is easy to obtain that demographic parity represents the independent prediction w.r.t. sensitive attributes. Hence, ABPC and ABCC are zero, and vice versa.
D The Relation between ABPC and ADP.
Based on the definition of ABPC, we have
ABPC
[fo(z) fx()ddd
(8)
2 1 (xfo(t) - xfi(x))dx
2 xfo(z)dx L xf((z)dr)
(9)
ADPc
i.e., the bias metric ABPC is rigorously larger than ADPs, which conquers the drawback of the insufficient condition of demographic parity.
E The Relation between ABPC and ABCC
Based on the definition of Wasscrstein distance with cost function co(x, y) 2 1(x + v), we have
Wer (fo(x),
2
inf I 1(x # y)>(2, ddddy (er(f(=)).1(())
inf (ET((f(=)). Jo (1 (1- (x- (,) y)drdy
min (i)
[a((z) - fi
(10)
2
ABPC.
2
According to work (Shorack & Wellner, 2009), when the PDF of prediction has a limited expectation, we take the following equation:
/ [Fo(x)) Fi (x))drr
ABCC
[' (F) -F
(11)
((
where Fo F (t) represents the inverse function of the original CDF Fo(z). Therefore, the proposed two metrics ABPC and ABCC are both the distance for these two PDFs for different. groups except the distance metrics.
20

---

Published in Transactions on Machine Learning Research (05/2023)
F Proof of Estimation Tractability on ABPC and ABCC
According to the non-parametric estimation theory (Sampson & Guttorp, 1992; Jiang et al., 2022b), the estimation error for PDF satisfies Error Ex [1][((())) f(x)) O(N 4) for kernel density estimator.
Hence, for bias metric ABPC estimation error, we have
EBTOCABPC = Ep [ABCC- - ABPC]]]
Ex || I [fo(z) f1(x)/dx - [ [fo(x) - (x)))dx]²]
(a)
Ex || ! [o(()) fo(x)lde + L [f(()) ((c))dd
= 2 EErl [fo(z) - fo(x))dx)) 2 + EEyl //" [f(()) R (x)))a)))
@ 2 F.Il.. / (/o(c)) -f((t)/dd)) Eu Eall. (f(()) - [(()))
ON #).
where inequality (a) holds due to absolute inequality, and inequality (b) holds based on Cauchy-Schwarz inequality for the integration version. For the number of samples N = 0 ie --)) ), we have
P(IABPC ABPC| <))
P(IABPC - 2 2 2?)
(c)
Ep ||BPC ABPC
(12)
2
-.
where inequality (c) holds due to Markov chain inequality. In other words, for the sufficient number of samples = 06 - JABPC - ABPCI < holds with at least probability 1-
Similarly, for CDF estimation, based on the central limit theorem, vn(F(z) (((()))())))) has asymptotically normal distribution, i.c., Error E F(()) F=))111 O(N-1). Therefore, similar to the derivation of
PDF, we can obtain ErroTABPC O(N For the number of samples N = 0 -2), we can also find that |ABCC ABCC| holds with at least probability 1-
G Experimental Examination on Adult Dataset
In this appendix, provide an experimental examination of the Adult dataset in Figure 7, which is the same as Figure 2. We can observe that the PDF and CDF of biased MLP are obviously different, illustrating that the machine lcarning model is biased to gender. Debiased MLP achieves a much lower ADP. than the biased MLP, however, the PDFs of differrent groups are different even though the "mean" of them are almost. the same. The bottom figures show that ADP, is the difference between the CDF lines while the threshold 0.5. The results are also in line with the finding in Section 3.2.
H Re-evaluate the Fairness on Image Data
In this appendix, conduct experiment the CelebA image dataset to re-evaluate the performance of the fair model in Table 2. The CclebA face attributes dataset (Liu et al., 2015) contains over 200,000 face images, where each image has 40 human-labeled attributes. Among the altributes, we select. Attractive as a binary classification task and consider gender as the sensitive attribute. The results are presented in Table 2. The results show a similar finding with the tabular datasct, demonstrating that 1): REG method always achieves lower ADP. but a relatively high ADP. 2): ADV is a more promising fair model to achieve demographic parity. 3): The inherent. tension between fairness and accuracy underestimated.
21

---

Published in Transactions on Machine Learning Research (05/2023)
PDF
CDF
Threshold
1.0
Biased MLP
0.3
Density
ADP: 0:1998
0.6
ADPpred
ADPt=05
0.1554
0.2
pred
0.2
0.1
0.0
0.2
0.4
0.6
0.0
0.2
0.4
0.6
9
9
Debiased MLP
1.0
0.20
Probabiltyy
Density
ADP.= 0.0174
0.6
0.15
ADP = 0.1887
0.10
0.2
0.05
0.00
0.2
0.4
0.6
y
9
Figure 7: The empirical PDF and CDF of the predictive probability over different groups (i.e., male, female) on Adult dataset. Left: Biased MLP model. Right: Debiased MLP model with ADP, as regularization.
Table 2: The fairness performance on image dataset. t represents the accuracy improvement compared to MLP. A higher accuracy metric indicates better performance. 4 represents the improvement of fairness compared to MLP. A lower fairness metric indicates better fairness.
Accuracy
Fairness
Acc(%)
AP(%)
t
(%)
ADP((%)
ABPC(%)
J
ABCC(%)
MLP
79.12
88.06
44.17
44.11
46.70
44.11
Age
REG
66.28
-16.22
86.08
-2.24
2.10
99.95
14.02
68.21
65.69
-40.66
17.21
60.98
ADV
59.48
24.82
63.93
27.40.
12.31
72.13
12.15
72.46
16.73
64.18
12.15
72.46
Gender
MLP
79.12
88.06
42.21
41.87
43.61
41.87
REG
77.82
-1.64
72.63
-17.52
30.03
28.86
02.00
95.22
19.36
55.61
22.42
46.45
ADV
65.25
-17.53
71.20
-19.15
01.91
95.48
01.91
95.44
04.00
90.83
01.91
95.44
Wasserstein Distance Introduction
The Wasserstein distance (Rtischendorf, 1985) has already been adopted in machine learning due to the power of measuring the difference between two distributions. In the fairness community, the transport problem measures the difference in predictive probability for different groups. Formally, suppose the predictive probability distributions for different groups are fo(x) and fi(),, and the cost function moving from æ y is e( the transportation problem is given by 7 (x.z) arg inf S c(x, y> y)dddy, where f1)(Jo.f1)
distribution set [[fo, fi) (r > 0, y) v)dy = fo(x), (x y)dz = fi (v)) is the collection of all possible transportation plan, i.., joint distribution with margin distribution fo and f1. The optimal transportation plan always exists and is defined y"(x,)) v).
The tdh Wasserstein distance is special case of the optimal transport problem with cost function c(x, y) l y/. The formal definition is given by
(, inf (er(Ooo
/ Iec- yi (1 -o. u))ddd))
Wz(Jo,f)
(13)
22

---

Published in Transactions on Machine Learning Research (05/2023)
Algorithm Python code of ABPC
Algorithm 2 Python code ABCC
def ABCO( y_pred, y_gt, 2_values, (continuous).
10000 ):
def ABPCC y_pred, y_85; z_values, bx_method "scott",
sample_n 5000
y-pred: predicted
y y-pred: predicted values (contimuous).
# y-gt: ground truth label (binary).
z_values: senaitive attributea (binary).
8 =_values: sensitive attributes (binary)
y_gt: ground
(binary)..
sample_u: The number of sample points to use for integration of the area betäeen the CDFs.
bu nethet: the method for estimator bandwidth
sample_n: the number of sample points to use for integration the area between the PDFs.
y_pred = V_.pred () y-gt y-gt.ravel()
y-pzed - y-ppred.ravel))
y-gt y-gt.ravel()
z_values r_values.ravel1.
2_values z_values.ravel(
y-pre_1 y_pred[[z_valee values 1] [2_ values == 0]
y-pre_1 y-pred
y_pre_0 y_pred
y-pre_0 [_pree[2_valuee == 0]
empirical
kde0 goussion_kdely kel_p_nreeddyy__________ bu bu_method bu we_nhhhed) kdet (yyesien_kdeey_pre.... bu_method bw_nethod)
# XDE PDF
ecdf0 ECDF (y_pre___))
ECDP ECDP (y_pre_1)
integration
x np 1i..linsacee(O, 1, sample_n)
integration
up. np.linspace 1, sample_n)
ecdf0_x - ecdf0(x)
kdel_? kd)l(x)
kde0_x kx)0(x))
ecdfi_x ecdfl
abec ap. pep.trapzeeo abb - ecdfi_x), x)
abpc tr.trapz
kdei_x), x)
return abpc
return abcc
Python Code for the Proposed Metrics
In this appendix, we provide the python code for our proposed two metrics, ABPC and ABCC. The provided codes show that our proposed metrics are practical and easy to compute.
The numbers 5, 000 and 10, 000 mentioned for ABPC and ABCC are not the number of data samples. Instead, they refer to the number of points used in the integration process for both metrics. Both ABPC and ABCC are based on estimated functions with range of [0, 1]. The "sample n" parameter represents the number of points within the range [0, 1] used for integration, and the integration is computed using these points. Thus, we can set "sample n" to a relatively large nunber. The default values of 5, 000 and 10, 000 are not strict requirements. In our experiments, we found that a "sample n larger than 100 should be sufficient for accurate results.
23


Reviews:
[{"review_id": "IKhm2uVAvB", "reviewer": "Reviewer_tEuV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe fairness issue of machine learning methods is an important topic. The authors point out the drawback of the metric $\\Delta DP$ and verify their claims both theoretically and empirically. The paper proposes two fundamental criteria for fairness measurement design and propose two fairness metrics which satisfy the proposed criteria.\n\nWeakness:\n\n1. The writing can be improved, the detail suggestions are presented in the \"requested changes\"\n2. ABCC and ABPC cannot be computed directly. The paper may discuss the computational cost of the estimation of ABCC and ABPC.\n3. According to lemma 5.2, the estimation of ABPC requires more samples than ABCC to guarantee the $\\epsilon$ estimation error. But in the experiments, ABPC use 5k samples and ABCC use 10k samples. Why ABCC needs more samples for estimation in the experiments?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the metric of the fairness. The authors first show that the commonly used fairness metric $\\Delta DP$ can not precisely measure the violation of demographic parity, and then propose two novel fairness metrics called ABCC and ABPC. The proposed metrics are: 1) necessary and sufficient to measure the demographic parity; 2) invariant with respect to monotone transformations of the distributions. The authors also re-evaluate existing fair models with the proposed metrics.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The abstract is repeated twice. \n2. In section 3.1, the paper may provide a toy example where the \\Delta DP fails.\n3. In section 5.2, the theoretical properies of ABPC and ABCC should be written as theorems with proofs.\n4. In the last paragraph of section 5.3, \"Similarly, ABCC is also can be directly optimized\" ->  \"Similarly, ABCC can also be directly optimized\" \n5. In Apendix B, $\\Delta_b^t$ seems to be undefined.\n6. The paper may provide some discussion on the issues presented in the weakness part.", "broader_impact_concerns": "none", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pIdbSpVvcb", "reviewer": "Reviewer_n71L", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The code of the experiment is published in a form that is easy to reproduce.\n- The paper is well organized and comfortable to read.\n- The claims of the paper are supported by convincing theoretical and experimental evidence.\n\nWeaknesses:\n\nThere is a lack of explanation as to when and in what situations the existing metrics ($\\Delta DP_b$ and $\\Delta DP_c$) are problematic,\nespecially in the introduction.\nThis paper focuses on models that output predictive probability $\\tilde{y} \\in [0,1]$ once and determines the label $\\hat{y}$ using an appropriate threshold value $t$, i.e., $\\hat{y} = \\mathbf{1} [\\tilde{y} \\geq t]$.\nIn my understanding,\nif $t$ is fixed and if the predicted label $\\hat{y}$ alone is used in the downstream task,\n$\\Delta DP_b^t$ is a \"rational\" fairness metric,\ni.e.,\n$\\Delta DP_b^t = 0$ (asymptotically) iff $\\hat{y}$ is independent of the sensitive attributes.\nIn other words, the issues pointed out by the authors arise only when different threshold values $t$ are used depending on the downstream tasks; otherwise, the conventional metrics are sufficient.\nMy point is not to deny the intrinsic value of this study, but the way the paper is now written, it seems to me that this is overstating the magnitude of the problem of existing methods and the scope of application of the proposed method.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper considers metrics for demographic parity in the classification task.\nThe contribution of this paper includes the following points:\n- This paper points out issues with the standard metrics, $\\Delta DP_b$ and $\\Delta DP_c$.\n- To resolve the issues, this paper proposes new metrics referred to as ABPC and ABCC.\n- Using new metrics ABPC adn ABCC, this paper re-evaluated the fairness performance of existing methods with real-worlds datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would like you to check if there is any misunderstanding about the points I mentioned as weakness.\nIf there is no misunderstanding, please address it.", "broader_impact_concerns": "I have no concerns about ethical aspects.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BUDY3fkZKP", "reviewer": "Reviewer_YqvS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- The DP metric is a very common metric used in fairness literature and building on top of that and highlighting its shortcomings can be interesting for the community.\n\n- The paper is overall well written.\n\n- This notion of distance between distributions as a stronger form of DP has already been introduced in Jiang et. al. And, they also consider the same distribution distance that this work considers. But, as the authors mention, they derive these metrics starting from desired conditions. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Demographic Parity (DP) is a commonly used metric for evaluating the fairness of the models. This work argues that DP has two shortcomings - 1) 0 DP does not imply 0 violation of demographic parity 2) DP can vary based on the threshold chosen for classification. This paper proposes two new metrics Area Between Probability density function Curves (ABPC) and Area Between Cumulative density function Curves (ABCC) to deal with these shortcomings. These metric measure the distance between the distributions instead of the average values. Then, they evaluate a few machine learning models trained with different fairness regularizations and compare the values of different fairness metrics.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- This paper studies DP and points out concerns with this measure. This has already been done by previous works. For example, [1] already pointed out how DP is not a sufficient metric and introduced the Equalized odds metric. Can the authors please comment on what different metrics have been introduced to improve over DP’s shortcomings and how their method is different from existing ones?\n\n- On Page 1 in the introduction, the first line containing DP mentions “In this paper, we rethink the rationale of ∆DP”. It would be useful for the reader to get a 2 line introduction to the metric first and then mention that this paper rethinks that.\n\n- DP and the new proposed metrics don’t not look at the feature distribution at all. Can the authors include a discussion on metrics which also depend on feature distribution and what are the tradeoffs between two two types of metrics?\n\n- In [2], the authors argue that some natural fairness conditions can not be all satisfied simultaneously. Can the authors comment on their metrics in this regard that what conditions are not satisfied by the proposed metrics?\n\n- In section 5.5.1, it is not clear mutual information is calculated between which variables.\n\n- In figure 5, the authors claim that DP_c is not a good metric since it is lower for plot b compared to c. How about DP_b? Is that a good metric for this case?\n\n- Do the authors have some intuition on why ADV performs better with respect to their metrics?\n\n- The authors comment on how different fairness regularization methods work w.r.t the proposed metrics in section 6.3. Is there some way to verify that they are actually better? \n\n- Is there a way to directly optimize the proposed metrics?\n\nTypos:\n\n- Preliminaries section: where n represents the -> where N represents the\n- Figure 2 description: not clear if talking about the top or bottom figure in the first two lines. The figure mentions Left and Right instead of Top and Bottom. The y label of the rightmost plot is cut.\n\n[1] Equality of Opportunity in Supervised Learning\n\n[2] Inherent Trade-Offs in the Fair Determination of Risk Scores", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper highlights shortcomings in existing widely used metrics and also proposes some new ones, derived from desirable criteria for such metrics. It also develops some theory around these and provides some empirical results. 
Decision: Accept (poster)