G ENERALIZED D EMOGRAPHIC P ARITY FOR G ROUP
F AIRNESS
Anonymous authors
Paper under double-blind review
A BSTRACT
This work aims to generalize demographic parity to continuous sensitive attributes while preserving tractable computation.
Current fairness metrics for continuous sensitive attributes largely rely on intractable statistical independence between variables, such as Hirschfeld-Gebelein-Renyi (HGR) and mutual information.
Statistical fairness metrics estimation relying on either tractable bounds or neural network approximation, however, are not sufﬁciently trustful to rank algorithms prediction bias due to lack of estimation accuracy guarantee.
To make fairness metrics trustable, we propose Generalized Demographic Parity (GDP), a group fairness metric for continuous and discrete attributes.
We show the understanding of GDP from the probability perspective and theoretically reveal the connection between GDP regularizer and adversarial debiasing.
To estimate GDP, we adopt hard and soft group strategies via the one-hot or the soft group indicator, representing the membership of each sample in different groups of the sensitive attribute.
We provably and numerically show that the soft group strategy achieves a faster estimation error convergence rate.
Experiments show the better bias mitigation performance of GDP regularizer, compared with adversarial debiasing, for regression and classiﬁcation tasks in tabular and graph benchmarks.
I NTRODUCTION
Fairness problem has attracted increasing attention in many high-stakes applications, such as credit rating, insurance pricing and college admission (Mehrabi et al., 2021; Du et al., 2020; Bellamy et al., 2018), the adopted machine learning models encode and even amplify societal biases toward the group with different sensitive attributes.
The majority of existing fairness metrics, such as demographic parity (DP) (Feldman et al., 2015), equal odds (EO) (Hardt et al., 2016), presumably consider discrete sensitive variables such as gender and race.
In many real-world applications including urban studies and mobility predictions (Tessum et al., 2021), however, individuals’ sensitive attributes are unavailable due to privacy constraints.
Instead, only aggregated attributes presenting in continuous distributions are available, and thus fairness requires unbiased prediction over neighborhood or region -level objects.
Additionally, the sensitive attributes, such as age and weight, are inherently continuous (Mary et al., 2019;
Grari et al., IJCAI’20).
The widely existing continuous sensitive attributes stimulate further fairness metrics deﬁnition and bias mitigation methods.
Existing fairness metrics on continuous sensitive attributes rely on the statistical measurement of independence, such as Hirschfeld-Gebelein-Renyi (HGR) maximal correlation coefﬁcient (Mary et al., 2019) and mutual information (Jha et al., 2021; Creager et al., 2019), which are computationintractable due to the involved functional optimization.
Note that the mutual information involves the ratio of probability density function, it is intractable to directly estimate mutual information via probability density function estimation due to the sensitivity over probability density function, especially for the low probability density value.
Previous works (Roh et al., 2020; Lowy et al., 2021; Cho et al., 2020) adopting mutual information or variations as regularizer, however, rely on tractable bound, or computationally complex singular value decomposition operation (Mary et al., 2019), or training-needed neural network approximation (Belghazi et al., 2018), such as Donsker-Varadhan representation (Belghazi et al., 2018), variational bounds (Poole et al., 2019).
Nevertheless, it is unreliable to adopt the mathematical bound of fairness metric to evaluate different algorithms since lower metrics bound does not necessarily imply lower prediction bias.
A question is raised:
∆𝐺𝐷𝑃 =𝑇𝑉( , )
∆𝐷𝑃 =𝑇𝑉( , ) ∆𝐷𝑃 =𝑇𝑉 , = 𝑇𝑉( , )
Can we extend DP for continuous attributes while preserving tractable computation?
In this work, we provide positive answers via proposing generalized demographic parity (GDP) from regression perspective.
Figure 1 provides an illustrative example for DP and GDP.
The local prediction average (blue solid curve ) and the global prediction average (red dashed line ) represent the average prediction value given sensitive attributes and the whole data samples, respectively.
The local and global prediction average should be consistent at any speciﬁc continuous sensitive attributes.
Therefore, we deﬁne GDP, via the weighted total variation distance, to measure the distance between the local and global prediction average, where the weight is the probability density of continuous sensitive attributes.
We also theoretically demonstrate the equivalence of GDP and DP for binary sensitive attributes, provide an understanding of GDP from probability perspective, and reveal the bias mitigation methods connection between GDP regularizer and adversarial debiasing.
Although GDP is clearly deﬁned on the unknown underlying joint distribution of prediction and sensitive attributes, only data samples, in practice, are available.
To this end, we propose two GDP estimation methods, named histogram estimation (hard group strategy) and kernel estimation (soft group strategy) methods, where kernel estimation is provable with faster estimation error convergence rate w.r.t.
data sample size.
Speciﬁcally, histogram estimation manually divides continuous sensitive attributes into several disjointed and complementary sensitive attributes bins, and then each sample only belongs to one speciﬁc bin containing the sensitive attribute of the sample.
In other words, the group indicator of the sample is one-hot.
As for kernel estimation, instead of quantizing continuous sensitive attributes as several groups, the group indicator of the sample is treated as a kernel function.
In other words, to calculate the mean prediction value given speciﬁc sensitive attributes, group smoothing strategy is adopted via a soft indicator determined by the sensitive attribute distance between the sample sensitive attribute with target-speciﬁc sensitive attribute.
In short, the contributions of this paper are:
• We develop a tractable group fairness metric GDP for continuous sensitive attributes.
We theoretically justify GDP via demonstrating the equivalence with DP for binary sensitive attributes, providing GDP understanding from probability perspective, and revealing the connection between GDP regularizer and adversarial debiasing.
• We propose histogram and kernel GDP estimation and provably demonstrate the superiority of kernel GDP estimation method with faster estimation error convergence rate w.r
t. sample size.
• We experimentally evaluate the effectiveness and expansibility of GDP on different domain benchmarks (e g., tabular, graph, and temporal graph data), tasks (e.g, classiﬁcation and regression tasks), and compositional sensitive attributes.
R ELATED W ORK
Machine Learning Fairness Fair machine learning targets bias mitigation for automated decisionmaking systems.
Various fairness deﬁnitions, such as group fairness and individual fairness, have been proposed (Zemel et al., 2013).
Group metrics, such as DP and EO, measure prediction difference between the groups with different sensitive attributes such as gender, age (Louizos et al., 2016;
Hardt et al., 2016).
While pre- and post-processing methods have been proposed for fairness boosting, these methods can still lead to higher prediction bias (Barocas et al., 2017) compared with in-processing methods, such as adding regularizer, adversarial debiasing and data augmentation.
For example, the covariance between the predictions and sensitive attributes regularization are imposed to boost the independence in (Woodworth et al., 2017).
(Zafar et al., 2017) constrains the decision boundaries of classiﬁer to minimize prediction disparity between different groups.
Adversarial training has been originally proposed for deep generative modeling (Goodfellow et al.
, 2014)and has been introduced for prediction debias in representation learning (Zhao et al., 2020; Beutel et al., 2017; Louppe et al., 2017) and transfer learning (Madras et al., 2018).
Data augmentation, such as fair mixup (Chuang & Mroueh, 2020), can improve the generalization ability for fairness.
Representation neutralization is proposed to boost fairness without sensitive attribute (Du et al., 2021).
Kernel Density Estimation and Kernel Regression Kernel density estimation (KDE) is a nonparametric method to estimate the continuous probability density function of a random variable (Davis et al., 2011; Parzen, 1962).
Given ﬁnite data samples, KDE smoothly estimate the probability function via weighted summation, where the weight is determined via kernel function (Epanechnikov, 1969).
Kernel regression is a non-parametric technique to estimate the conditional expectation of a random variable (Nadaraya, 1964).
Nadaraya-Watson Kernel regression function estimator is proposed for regression via locally normalized weighted average in (Bierens, 1988), where the sample weight is determined by kernel function.
G ENERALIZED D EMOGRAPHIC P ARITY
Without loss of generality, we consider a binary classiﬁcation task to predict the output variable
Y given the input variable X , while avoiding prediction bias for sensitive attribute S .
Deﬁne the input X ∈ X ⊂ R d , labels Y ∈ { 0 , 1 } , and machine learning model f : R d → [0 , 1] provides prediction score ˆ Y =
f ( X ) .
Fairness requires predictor ˆ Y to be independent of sensitive attribute
S , regardless of continuous or discrete, i.e., P ( ˆ Y = ˆ y ) =
P ( ˆ Y =
ˆ y | S = s ) for any support value y and s (Beutel et al., 2017).
Since the independent constraint is difﬁcult to optimize, the relaxed demographic parity (DP) (Madras et al., 2018) metrics are proposed to quantitatively measure the predictor bias for binary sensitive attribute S ∈ { 0 , 1 } .
Formally, the demographic parity is deﬁned as ∆ DP = | E ˆ Y
[ ˆ Y | S = 0] − E ˆ Y
[ ˆ Y | S
= 1] | , where E [ · ] represents variable expectation.
For categorical sensitive attribute S ∈ S , work (Cho et al., 2020) introduces a fairness metric, named difference w.r.t.
demographic parity (DDP) ∆ DDP = (cid:80) s ∈S [Equation] E
[Equation] Y [
[Equation] S [Equation] s ]
[Equation]
ˆ Y [
[Equation] Y ] | .
Although DP has been widely used to evaluate the prediction bias, it is still inapplicable for continuous sensitive attributes since the data samples cannot be directly divided into several distinctive groups based on the sensitive attributes.
Without loss of generality, we assume continuous sensitive attributes S ∈ [0 , 1] and propose GDP to extend tractable DP for continuous sensitive attributes.
Assume the joint distribution of tuple ( S, ˆ Y ) is P S, ˆ Y ( s, ˆ y ) , the local prediction average and global prediction average are deﬁned as the prediction expectation given sensitive attribute S
= s and without any sensitive attribute condition, i.e., local prediction average
m ( s ) (cid:52) =
E [ ˆ Y | S
= s ] and global prediction average m avg (cid:52) =
E S [ m ( S )]
=
E [ ˆ Y ] , respectively.
Then, we adopt weighted total variation distance on local prediction average and global prediction average, where the weight is speciﬁed by the probability density function of the sensitive attribute.
The formal deﬁnition of the discrepancy demographic parity for continuous sensitive attributes is as follows:
[Equation]
We also provide the connection of GDP and DP for binary sensitive attributes, which implies that
GDP is equivalent to DP for binary sensitive attributes, as follows:
Theorem 1 (Connection between DP and GDP) .
For binary sensitive attribute S ∈ { 0 , 1 } , GPD and DP are equivalent except the coefﬁcient only dependent on datasets.
Speciﬁcally, the relation of ∆ GDP and ∆ DP satisﬁes ∆ GDP = 2 P S ( S = 1) · P S ( S = 0) · ∆ DP .
The proof of Theorem 1 is presented in Appendix A. For categorical sensitive attributes, it is easy to obtain that ∆ GDP =
(cid:80) s ∈S P S ( S = s )
| E ˆ Y
[ ˆ Y | S
= s ] − E
ˆ Y
[ ˆ Y ] | , i.e., GDP is weighted DDP for categorical sensitive attributes.
In a nutshell, GDP is a natural fairness metric extension for binary and categorical sensitive attributes.
Since the independence between prediction ˆ Y and sensitive attribute S implies that the joint distribution P S, ˆ Y ( s, ˆ y ) and product marginal distribution P S ( s ) P ˆ Y (ˆ y ) are the same, the bias can be measured by the distance of the joint distribution and product marginal distribution.
Subsequently, we show the connection of GDP and prediction-weighted total variation distance between these two distributions as follows:
Theorem 2 (Probability View of GDP) .
Assume the joint distribution of ( ˆ Y , S ) with support [0 , 1] 2 is P S, ˆ Y ( s, ˆ y ) .
Deﬁne the prediction-weighted total variation distance as TV pred ( P 1 , P 2 ) (cid:52) =
(cid:82) 1 0 (cid:82) 1
ˆ y | P 1 (ˆ y, s ) − P 2 (ˆ y, s ) | dˆ y d s .
Then the proposed fairness for continuous senstive attribute is upper bounded by prediction- weighted total variation distance between the joint distribution and product marginal distribution:
[Equation]
The proof of Theorem 2 is presented in Appendix B. Theorem 2 demonstrates that GDP is actually a lower bound for prediction-weighted total variation distance between these two distributions and implies the necessity of GDP for bias measurement.
GDP E STIMATION
GDP is deﬁned based on the underlying joint distribution P S, ˆ Y ( s, ˆ y ) of tuple ( S, ˆ Y ) , where S, ˆ Y ∈ [0 , 1] .
The underlying joint distribution, however, is unknown.
Thus, we aim to estimate GDP given samples { ( s n , ˆ y n ) , n ∈ [ N ] } , where [ N ] (cid:52) = { 1 , · · · , N } .
To bridge this gap, we propose histogram GDP estimation and kernel GDP estimation methods based on different group strategies.
Speciﬁcally, histogram GDP estimation hardly groups data samples via creating consecutive, nonoverlapping intervals bins, and the local prediction average is estimated by the average prediction among the data samples in the bin.
As for kernel GDP estimation, a soft group indicator, determined by the kernel function and sensitive attribute distance, is adopted to provide group smoothing strategy.
Speciﬁcally, the local prediction average for target sensitive attribute is calculated via weighted prediction, where the sample with sensitive attribute close to the target sensitive attribute possesses large weight.
Histogram GDP Estimation
A histogram is originally an approximate representation of the underlying data probability distribution via creating several consecutive, non-overlapping intervals or bins with, usually but not required, equal bandwidth.
In this paper, we assume all bins with equal bandwidth h and the number of bins N h (cid:52)
=
1 h is an integer.
In other words, the data tuples can be divided into N h groups, and the bin intervals are given by B
= [0 , h ) , B 2 =
[ h, 2 h ) , · · · ,
B N h
=
[( N h − 1) h, 1] .
Deﬁne indicator function I ( A ) as 1 if event
A happens, otherwise is 0 .
Thereby, the group indicator w h ( n, i ) of sample ( s n , ˆ y n ) for i − th bin is given by I ( s n ∈ B i ) .
Note that all bin intervals are complementary; each sample belongs one and only one bin, i.e., the indicator vectors w h ( n ) =
[ w h ( n, 1) , · · · , w h ( n, N h )] for sample n is one-hot, which formally deﬁne the hard group strategy.
The local prediction average and probability of sensitive attribute can be point-wisely estimated based on the empirical expectation and distribution.
Speciﬁcally, for s ∈ B i , the local and global prediction average are given by
[Equation]
Similarly, the probability of sensitive attribute is given by ˆ P hS ( s ∈ B i ) =
{+(cid:80) Nn =
 I ( s n ∈ B i ) N .+}
Finally, we combine all estimations to calculate prediction bias as follows:
[Equation]
Estimation Histogram GDP estimation provides a hard group strategy via creating several bins.
However, a tiny sensitive attribute permutation can lead to different group indicators and thus histogram estimation is not robust on sensitive attribute.
On the other hand, a data tuple not necessarily belongs to one group for continuous attributes.
For example, the data sample with s = 0 .
5 may have the same probability belonging to target sensitive attributes s = 0 .
4 and s = 0 .
Based on these observations, we propose kernel GDP estimation via group smoothing.
Intuitively, when calculating local prediction average or probability density function for target sensitive attribute, the tuple with more close attribute is entrusted higher weight.
Speciﬁcally, we introduce a symmetric one-dimensional smoothing kernel function K ( s ) ≥ 0 satisfying normalized condition (cid:82) K ( s )
d s = 1 , symmetry condition (cid:82) sK ( s )
d s = 0 and ﬁnite variance σ 2 K (cid:52) =
(cid:82) s 2 K ( s )
d s > 0 .
Deﬁne h > 0 as the kernel bandwidth.
For target sensitive attribute s , the tuple weight for sample with sensitive attribute s n is given by w ( s n , s ) (cid:52) = 1 h K ( | s n − s | h ) .
In short, kernel function provides the group smoothing strategy based on sensitive attribute distance of tuple pair.
Given the smoothing kernel function K ( s ) , the local and global prediction average can be obtained via normalized weighted average (Nadaraya–Watson kernel estimator) as follows:
[Equation]
Similarly, the probability of sensitive attribute is given by ˜ p hS ( s ) =
Nh (cid:80) Nn
=1 K ( s n − s h ) .
Finally, we combine all estimations to calculate kernel GDP estimation as follows:
[Equation]
Estimation Error Analysis We provide the theoretical analysis on GDP estimation error and prove the superiority of kernel GDP estimation.
Assume that each data sample is independent and identically distributed random variables.
Therefore, the estimated GDP is still a random variable, and we adopt the expectation of the mean squared error (MSE) to quantify the accuracy of the estimation method.
Formally, the error of histogram estimation and kernel estimation are given by
[Equation]
where the expectation is taken across N tuples.
Here, we provide an asymptotic analysis on estimation error and show the superiority of kernel GDP estimation in the following:
Theorem 3 (Estimation Error Convergence Rate) .
Assume that the mean prediction function m ( s ) , given sensitive attribute s , is smooth and satisﬁes L - Lipschitz condition 1 on local average | m ( s ) − m ( s (cid:48) )
| ≤ L | s − s (cid:48) | for any s, s (cid:48) .
the optimal bandwidth choice for histogram and kernel estimation methods are h ∗ hist =
O ( N − 13 ) and h ∗ kernel =
O ( N − 15 ) .
Additionally, the estimation error satisfy
Err hist =
O ( N − 23 ) and Err kernel
=
O ( N − 45 ) , where O ( · ) is big O notation.
The proof of Theorem 3 is presented in Appendix C.
The proof sketch is to separately provide upper bounds for the estimation error of local average and sensitive attribute probability density estimation.
Finally, we combine these two estimations to obtain the estimation error for GDP.
Computation Complexity Analysis Since GDP calculation involves in integral operations, the approximated numerical integration is usually adopted with M probing sensitive attribute.
The complexity to calculate local prediction average and probability density at M probing sensitive attributes are O ( MN ) and thus, the complexity for histogram and kernel estimation both are O ( MN ) .
In (Mary et al., 2019), intractable HGR coefﬁcient equals second large singular value of distribution ratio matrix or can be upper bounded by tractable chi-square distance between the joint distribution and marginal product distribution.
Assume that there are M probing prediction, the complexity for two-dimensional probability density function estimation and SVD is O ( M 2 N ) and O ( M 3 ) , and that of the chi-square distance is O ( M 2 N ) .
Therefore, the computation complexity for HGR is O
( M 2 ( M + N )) .
As for the neural based approximation for HGR or mutual information, the complexity for training is quite large compared with directly computation.
A NALYSIS ON GDP R EGULARIZER AND A DVERSARIAL D EBIASING
With GDP bias measurement for continuous sensitive attributes, it is natural to add GDP regularizer to enforce fairness.
Another bias mitigation is adversarial debiasing, a two-player framework for predictor and adversary with regression task.
We establish the connection between GDP regularizer and adversarial debiasing, and demonstrate that adversarial debiasing with speciﬁc adversary regression objective is actually minimizing GDP implicitly.
GDP Regularizer:
As a reminder, our goal is to learn a predictor ˆ Y = f
( X ) that approximates the label Y for each input while reducing the prediction bias ∆ GDP w.r.t.
continuous sensitive attributes S .
It is natural to add GDP regularization to enforce fairness.
Given the prediction loss
L pred , the fairness-enforcing objective function is min f L pred (cid:16) f
( X ) , Y (cid:17) + λ ∆ GDP , where L could be regression or classiﬁcation task loss and λ is the hyperparameter to control the trade-off between the prediction performance and prediction bias reduction.
Adversarial Debiasing: Adversarial debiasing is another natural method to ensure fair prediction via a two-player game between predictor and adversary.
Speciﬁcally, the predictor f yields prediction ˆ Y , and given prediction ˆ Y , the adversary g tries to predict continuous sensitive attributes ˆ S
= g ( ˆ Y ) in regression task.
Similar to adversarial debiasing (Louppe et al., 2017), we adopt the same classiﬁer and adversary structure, where the input of adversary is the output of the classiﬁer.
For objective function of adversary, we adopted the utility function proposed in (Madras et al., 2018) 2 to represent sensitive attribute prediction accuracy.
In this case, the predictor aims to generate accurate prediction and fool the adversarial simultaneously, while adversary targets high utility for accurate sensitive attribute prediction.
Let L pred denote the prediction loss and L adv represent the adversarial utility.
Then adversarial debiasing is trained following the min-max procedure min f max g L pred (cid:16) f
( X ) , Y (cid:17) + λ L adv (cid:16) g (cid:0) f
( X ) (cid:1) , S (cid:17) .
Deﬁne g ∗ =
arg min g
L adv (cid:16) g (cid:0) f
( X ) (cid:1) , S (cid:17) as the optimal adversarial given predictor f , the min-max procedure is simpliﬁed to the objective function min f L pred (cid:16) f ( X ) , Y (cid:17) + λ L adv (cid:16) g ∗ (cid:0) f
( X ) (cid:1) , S (cid:17) .
Theoretical Connection:
We provide an inherent connection between GDP regularizer and adversarial debiasing.
Speciﬁcally, we demonstrate that the optimal adversarial utility
L adv (cid:16) g ∗ (cid:0) f
( X ) (cid:1) , S (cid:17) is actually the upper bound of GDP ∆ GDP as long as the utility function in adversary is
L adv ( ˆ S, S )
= 1 − | ˆ S − S | in the following theorem:
Theorem 4 (GDP and Adversarial Debiasing Connection) .
Considering a predictor ˆ Y = f
( X ) and adversary ˆ S = g
( ˆ Y ) , given adversary utility L adv ( ˆ S, S )
= 1 − | ˆ S − S | and optimal adversary g ∗ , then GDP ∆ GDP is bounded by the utility function L adv (cid:16) g ∗ (cid:0) f
( X ) (cid:1) , S (cid:17) with optimal adversary, i.e., L adv (cid:16) g ∗ (cid:0) f ( X ) (cid:1) , S (cid:17) ≥ ∆ GDP .
The proof of Theorem 4 is presented in Appendix E. Intuitively, a fair classiﬁer aims to minimize the adversary utility to make prediction fair, and thus induces lower GDP metric.
Theorem 4 reveals the inherent connection between GDP regularizer and adversarial debiasing: adversarial debiasing behaviors like predictor loss optimization with GDP regularization, as long as the adversary is optimal.
Additionally, such connection not only holds for underlying data distribution, but also for empirical data distribution.
The proof sketch is similar via replacing underlying data distribution as empirical data distribution in the expectation operation.
In practice, the alternative optimization is usually adopted in adversarial debiasing via alternatively updating either predictor or adversary at each training step while keeping the other one ﬁxed.
E XPERIMENTS
We evaluate the effectiveness and expansibility of GDP .
First, we show the lower GDP estimation error for kernel GDP estimation 3 with group smoothing, compared to that of histogram, via two synthetic experiments.
We empirically evaluate the effectiveness and expansibility of kernel estimation for multiple prediction tasks, including classiﬁcation and regression tasks, and multiple domain real-world datasets, including tabular, graph and temporal graph data (See Appendix H.1).
Moreover, the kernel estimation is also applicable for compositional continuous sensitive attributes.
For a fair comparison, we compare our method kernel , adding kernel GDP estimation as regularization, with (a) vanilla : training with empirical risk minimization (ERM) without any regularization; (b) histogram : histogram estimation with continuous sensitive attribute; (c) adv : adversarial debiasing
(Louppe et al., 2017); (d) adv-bn : adversarial debiasing with binary-quantized sensitive attribute; (e) hgr : the upper bound of HGR as regularizer (Mary et al., 2019); and (f) hgr-bn : the upper bound of HGR as regularizer with binary-quantized sensitive attribute.
Speciﬁcally, we demonstrate the trade-off between prediction performance and GDP by varying the hyper-parameter λ .
In particular, we adopt accuracy (Acc) for classiﬁcation task and mean absolute error (MAE) for regression task.
Details about the baselines and experimental setups for each dataset can be found in Appendix H.2.
S YNTHETIC E XPERIMENTS
We test GDP estimation error and investigate the robustness for histogram and kernel estimation via two synthetic experiments.
The goal is to investigate the effect of bandwidth choice and number of samples on GDP estimation error.
For data generation , we ﬁrst generate bivariate Gaussian distribution N ( µ, Σ ) for samples ( S, ˆ Y ) , where expectation µ = [0 . 5 , 1 . 0] and covariance matrix Σ = (cid:20)0 .0 .
5 2 .(cid:21) .
Additionally, we generate the samples based on p S, ˆ Y ( s, ˆ y )
=
s
+ ˆ y if 0 ≤ s, ˆ y ≤ 1 , via acceptance-rejection method (Wells et al., 2004).
For estimation , we select two typical kernel functions (tricube and Aitchison-Aitken kernel (Mussa, 2013)) for local prediction average, and two kernel functions (linear and cosine kernel functions) for probability density function estimation.
Figure 2 shows the GDP estimation error w.r.t.
bandwidth h and number of samples N for bivariate Gaussian distribution and second synthetic bivariate distribution, respectively.
We observe that histogram GDP estimation error is highly sensitive to bandwidth choice, while kernel estimation is robust to bandwidth and kernel function choice due to the ﬂexibility of group smoothing.
As for error rate with number of samples, our experiments show the error rate curve for histogram and kernel estimation.
It is seen that kernel estimation can achieve the fast error convergence rate if searching the optimal bandwidth for each method.
In addition, the estimation error result is almost the same for different kernel functions, which supports the theoretical results in Theorem 3.
6. E XPERIMENTS ON T ABULAR DATA
Dataset: We consider two benchmark tabular datasets, UCI Adult and Crimes, to evaluate the effectiveness of kernel estimation for classiﬁcation and regression tasks.
UCI Adult dataset 4 contains more than 40 , 000 individual information from 1994 US Census.
The classiﬁcation task is to predict whether a person’s income exceeds $50 k/yr (KOHAVI, 1996).
We consider normalized age 5 sensitive attribute to measure the fairness of algorithms.
The Crime dataset 6 includes 128 attributes for 1 , 994 samples from communities in the US.
The regression task is to predict the number of violent crimes per population in US communities.
We adopt the black group ratio as continuous sensitive attribute.
Model: We adopt two-layer selu networks model (Klambauer et al., 2017) with hidden size 50 and report the mean prediction performance and GDP with 5 running times.
[Figure]
Results: We compare the bias mitigation performance, i.e., the tradeoff between prediction performance and GDP, of kernel estimation with other baselines for the two tabular datasets in Figure 3.
The hyper-parameter λ in Eq.
(5) controls the tradeoff between prediction performance and GDP.
Speciﬁcally, we choose accuracy metric for classiﬁcation task in Adult dataset, and MAE metric for regression task in Crimes datset.
For adversarial debiasing, we vary the regularization weights to obtain the tradeoff curve.
Overall, we make the following observations : (a) kernel estimation outperforms all other baseline methods in terms of performance-fairness tradeoff curve for Adult and Crimes datasets.
Speciﬁcally, kernel estimation can achieve more than 1% accuracy improvement if GDP is lower than 0 .
02 for adult dataset, and more than 2% MAE reduction for Crimes dataset; (b) kernel estimation has lower computation complexity compared with adversarial debiasing.
Kernel estimation and histogram decrease more than 50% running time on an average across all tabular dataset, which makes kernel and histogram estimation readily usable for large scale realworld datasets; (c)
The histogram estimation and binary-quatized sensitive attribute would lead to mitigation performance drop for kernel estimation, adversarial debiasing and HGR.
This fact implies the importance of order information in continuous sensitive attributes for bias mitigation.
In addition, we observe adversarial debiasing training and HGR are not stable and large hyper-parameter unnecessarily leads to bias mitigation.
6. E XPERIMENTS ON G RAPH DATA
Dataset : We consider two real-world graph datasets, Pokec-z and Pokec-n , sampled from a larger one Facebook-like social network Pokec in Slovakia.
User proﬁles contain gender, age, interest, education, working ﬁeld and etc.
We treat the normalized age as the continuous sensitive attributes and the node classiﬁcation task is to predict the working ﬁeld of the users.
Model : We use three graph neural network backbones, graph convolutional networks (GCN) (Kipf & Welling, 2017), graph attention networks (GAT) (Veliˇckovi´c et al., 2018) and Simplifying graph convolutional networks (SGC) (Wu et al., 2019) with 64 feature dimensions.
We train GNN with 200 epochs with 5 running times and report the average accuracy and GDP.
In each trial, the dataset is randomly split into a training, validation, and test set with 50% , 25% , and 25% partition, respectively.
Results: We compare the mitigation performance of kernel estimation with other baselines for two datasets with three backbones in Figure 4.
Similarly, kernel estimation consistently outperforms the other baselines by a large margin and binary-quantized sensitive attributes inevitably deteriorate the mitigation performance.
Another observation is that GDP can be reduced at least 80% at the cost of 2% accuracy for kernel estimation in two datasets and three backbones, while results in larger accuracy drop for other baselines.
Additionally, adversarial debiasing and HGR are also unstable during training and higher hyper-parameter may lead to larger GDP.
[Figure]
6. E XPERIMENTS ON C OMPOSITIONAL C ONTINUOUS S ENSITIVE A TTRIBUTES
Dataset : Similarly, the same two benchmark tabular datasets are adopted to evaluate the effectiveness of kernel estimation for compositional continuous sensitive attributes.
Speciﬁcally, we treat the normalized age and education number for UCI dataset, and black group ratio and normalized age for Crimes dataset as compositional continuous sensitive attributes.
to evaluate bias mitigation performance.
Model : We also adopt two-layer selu networks model with hidden size 50 with 5 running times and report the average mean prediction performance and GDP.
Results : Figure 5 shows bias mitigation performance between prediction performance and GDP.
Again, kernel estimation consistently achieves a better tradeoff compared with all other baselines, and binary-quantized compositional sensitive attribute leads to mitigation performance drop.
[Figure]
C ONCLUSION
We generalize demographic parity fairness metric, named GDP, to continuous sensitive attributes while preserving tractable computation.
We theoretically justify the uniﬁcation of proposed GDP for continuous and discrete sensitive attributes, and show the necessity of GDP via demonstrating the connection with joint and product margin distributions distance.
We also propose two GDP estimation methods, named histogram and kernel, with linear computation complexity via hard and soft group strategies, and provide corresponding estimation error analysis.
For the superiority of kernel estimation, we provably demonstrate the faster estimation error convergence rate compared with histogram estimation, and experimentally show better bias mitigation performances in multiple domains, multiple tasks and compositional sensitive attributes.
A P ROOF OF T HEOREM
For binary sensitive attribute S ∈ { 0 , 1 } , the probability of sensitive attribute follows Bernoulli distribution (cid:16) P S ( S = 0) , P S ( S = 1) (cid:17) .
Therefore, the global prediction average is given by m avg =
P S ( S = 0) m (0) + P S ( S = 1) m (1) and GDP is
[Equation]
GDP
[Equation]
(cid:12)(cid:12)(cid:12)
[Equation]
∆ DP.
where the coefﬁcient 2 P S ( S = 0) P S ( S = 1) only depends on data.
This fact demonstrates the applicability of GDP for categorical sensitive attribute with discrete measure choice for sensitive attributes, and it is equivalent to demographic parity for binary sensitive attribute.
B P ROOF OF T HEOREM
Notice that the fairness constraint ideally requires the independence of prediction ˆ Y and sensitive attribute S , i.e., the joint distribution and product margin distribution equals: P ˆ Y ,S (ˆ y, s ) =
P S ( s ) P ˆ Y (ˆ y ) .
Aiming to measure independence deviation, a natural intuition is to quantify the probability deviation via the distance between the joint distribution and product margin distribution.
We show the connection between the proposed GDP and prediction-weighted total variation distance of these two distributions.
Recall the deﬁnition of GDP, it is easy to obtain
[Equation]
which completes the proof
[Equation]
C P ROOF OF T HEOREM
Notice that histogram and kernel estimation require local prediction average and probability density function estimation for sensitive attributes, we start with the analysis on local prediction average and probability density function estimation, and then provide the proof of GDP error analysis.
. P ROOF FOR H ISTOGRAM E STIMATION
Recall that the number of bins is N h with same bandwidth h , and bins interval are given by B 1 = [0 , h ) , B 2 =
[ h, 2 h ) , · · · , B N h =
[( N h − 1) h, 1] .
Firstly, we will separately analyze the error for local prediction average and sensitive attribute probability.
Next, these two estimation error results can be combined for GDP estimation error.
Since the continuous sensitive attributes is considered, we deﬁned the probability density function of sensitive attribute S as p S ( s ) .
The estimated probability density function is given by:
[Equation]
Subsequently, we deﬁne the pointwise MSE error MSE pdfhist ( s ) to measure probability density function estimation error given sensitive attribute s as follows:
[Equation]
where the expectation is took across N samples.
Then we have following Lemma 1 on optimal pointwise MSE error for probability density function:
Lemma 1.
Assume the mean prediction function m ( s ) , given sensitive attribute s , is smooth and satisﬁes L -Lipschitz condition | m ( s ) − m ( s (cid:48) )
| ≤ L | s − s (cid:48) | for any s, s (cid:48) .
Given N i.i.d.
samples { (ˆ y n , s n ) , n ∈ [ N ] } , then the optimal MSE error min h MSE pdfhist ( s ) is O
N − 23 ) , where the optimal bandwidth satisﬁes h ∗
=
O ( N − 13 ) for any sensitive attribute s .
As for the local prediction average, similarly, we have the estimated local prediction average as follows:
[Equation]
Subsequently, we also deﬁne the pointwise MSE error MSE reghist ( s ) to measure local prediction average estimation error given sensitive attribute s as follows:
[Equation]
where the expectation is took across N samples.
Then we have following Lemma 2 on optimal pointwise MSE error for local prediction average:
Lemma 2.
Assume the mean prediction function m ( s ) , given sensitive attribute s , is smooth and satisﬁes L -Lipschitz condition | m ( s ) − m ( s (cid:48) )
| ≤ L | s − s (cid:48) | for any s, s (cid:48) .
Deﬁne the bounded prediction variance σ 2 ( s ) ≤ σ 2 , given sensitive attribute s , as σ 2 ( s ) (cid:52) =
E ˆ Y | S
[( ˆ Y − m ( s )) 2 | S = s ] .
Given N i.i.d.
samples { (ˆ y n , s n ) , n ∈ [ N ] } , then the optimal MSE error min h MSE reghist ( s ) is
O ( N − 23 ) , where the optimal bandwidth satisﬁes h ∗ =
O ( N − 13 ) for any sensitive attribute s .
Proof for histogram estimation : Based on the deﬁnition of MSE error in Eq.
(6), we have
Err hist
[Equation]
where inequality ( a ) holds due to absolute value inequality, inequality ( b ) holds due to | a 1 b 1 − a 2 b 2 | ≤ | a 1 || b 1 −
b 2 | + | a 1 − a 2 || b 2 | , inequality ( c ) holds based on ( a + b ) 2 ≤ 2( a 2 + b 2 ) and
Cauchy–Schwarz inequality, inequality ( d ) holds based on Lemmas (1) and (2).
Note that the order of the optimal bandwidth are the same in Lemmas (1) and (2), the optimal bandwidth for minimizing
GDP MSE is O ( N − 1 3 ) .
C. K ERNEL E STIMATION
Recall that we assume that kernel function satisﬁes normalized condition (cid:82) K ( s )
d s = 1 , symmetry (cid:82) sK ( s )d s = 1 and ﬁnite variance σ 2 K (cid:52) =
(cid:82) s 2 K ( s )
d s > 0 .
We also deﬁne ˜ σ 2 K = (cid:82) K 2 ( y )d y .
Similarly, the probability density function of sensitive attribute is given by p S ( s ) .
The estimated probability density function is given by:
[Equation] [Equation]
Subsequently, we deﬁne the pointwise MSE error MSE pdfkernel ( s ) to measure probability density function estimation error given sensitive attribute s as follows:
[Equation]
where the expectation is took across N samples.
Then we have following Lemma 3 on optimal pointwise MSE error for probability density function:
Lemma 3.
Given N i.i.d.
samples { (ˆ y n , s n ) , n ∈ [ N ] } , then the optimal MSE error min h MSE pdf kernel ( s ) is O
( N − 4 5 ) , where the optimal bandwidth satisﬁes h ∗ =
O ( N − 1 5 ) for any sensitive attribute s .
As for the local prediction average, similarly, we have local prediction average estimation as follows:
[Equation]
Subsequently, we also deﬁne pointwise MSE error MSE regkernel ( s ) to measure local prediction average estimation error given sensitive attribute s as follows:
[Equation]
(14) where the expectation is took across N samples.
Then we have following Lemma 4 on optimal pointwise MSE error for local prediction average:
Lemma 4.
Deﬁne the bounded prediction variance σ 2 ( s ) ≤ σ 2 , given sensitive attribute s , as σ 2 ( s ) (cid:52) =
E ˆ Y | S
[( ˆ Y − m ( s )) 2 | S = s ] .
Given N i.i
[Equation]
d. samples { (ˆ y n , s n ) , n ∈ [ N ] } , then the optimal
MSE error min h MSE regkernel ( s ) is O
( N − 45 ) , where the optimal bandwidth satisﬁes h ∗
=
O ( N − 15 ) for any sensitive attribute s .
Proof for kernel estimation : Based on the deﬁnition of MSE error in Eq.
[Equation]
we have
[Equation]
where inequality ( e ) holds due to absolute value inequality, inequality ( f ) holds due to | a 1 b 1 − a 2 b 2 | ≤ | a 1 || b 1 −
b 2 | + | a 1 − a 2 || b 2 | , inequality ( g ) holds based on ( a + b ) 2 ≤ 2( a 2 + b 2 ) and
Cauchy–Schwarz inequality, inequality ( h ) holds based on Lemmas (3) and (4).
Note that the order of the optimal bandwidth are the same in Lemmas (3) and (4), the optimal bandwidth for minimizing
GDP MSE is O ( N − 1 5 ) .
D P ROOF OF L EMMAS D. P ROOF OF L EMMA
Proof.
Note that there exists bias-variance tradeoff for MSE pdfhist ( s ) , i.e.,
[Equation]
Next we analyze the bias and variance for probability density function MSE.
For the bias part, note that, for s ∈ B i , the expectation of estimated probability density function satisﬁes:
[Equation]
where the last equality holds by the mean value theorem.
Therefore, the bias satisﬁes
[Equation]
As for variance part, note that the variance of Bernoulli distribution with parameter p is p (1 − p ) , for s ∈ B i , we have the variance as follows,
[Equation]
Combining the bias and variance part, we have
[Equation]
It is easy to obtain the optimal bandwidth h ∗ =
[ p S ( s
[Equation]
) 2 L 2 N ] 13 =
O ( N − 13 ) , and the minimized MSE is lower than [ Lp s ( s ∗ ) 2 N ] {+
+}
= O ( N − 23 ) .
D. P ROOF OF L EMMA
Proof.
Note that the local prediction average by histogram, for s ∈ B i , is given by ˆ m ( s )
= (cid:80) N
[Equation]
n ∈ B
i
[Equation]
Y
[Equation]
(cid:80) Nn =
I ( s n ∈ B i ) .
Deﬁne the normalized weight as w n ( s n ) =
I ( s n ∈ B i )
[Equation]
Nn =1 I ( s n ∈ B i ) , then local prediction average is given by ˆ m ( s )
= (cid:80) Nn =1 w n ( s n ) m ( s n ) .
Similarly, we can obtain the bias and variance tradeoff for local prediction average error MSE reghist ( s ) as follows,
[Equation]
For the bias part, based on Lipschitz condition on the mean prediction function, note that (cid:80)
Nn =1 w n ( s n ) = 1 , we have
[Equation]
For the variance part, we have variance for local prediction average as follows,
[Equation]
Combining the bias and variance part, we have
[Equation]
It is easy to obtain the optimal bandwidth h ∗ =
[ σ 2 2 L 2 N ] 13 = O ( N − 13 ) , and the minimized MSE is lower than [ Lσ 2 2 N ]
=
O ( N − 23 ) .
D. P ROOF OF L EMMA
Proof.
Note that there exists bias-variance tradeoff for MSE pdf kernel ( s ) , i.e.,
[Equation]
Next we analyze the bias and variance for probability density function MSE.
For the bias part, the expectation of estimated probability density function satisﬁes:
[Equation]
where the last equality holds by adopting transformation
y =
s n − s h .
By Taylor expansion, when h is small, we have
[Equation]
Based ob Eqs.
(19) and (20)
[Equation]
we have
Bias
[Equation]
As for the variance part
[Equation]
we have
[Equation]
Combining the bias and variance part, we have
[Equation]
It is easy to obtain that the optimal bandwidth h ∗
=
O ( N − 15 ) , and thus, the minimized MSE is lower than O ( N − 45 ) .
D. P ROOF OF L EMMA
Proof.
Recall that the mean prediction conditioned on sensitive attribute s is given by m ( s ) =
E [ ˆ Y | S = s ] , we rewrite prediction as ˆ Y = m ( S ) + e , where e is the regression noise and satisﬁes E [ e ] = 0 and E [ e 2 | S
= s ]
= σ 2 ( s ) .
Note that the prediction Y = m ( s ) + [ m ( S ) − m ( s )]
+ e , we have:
[Equation]
Based on Eq.
(13), we have
[Equation] [Equation]
Since E [ e | S = s ] = 0 , we have the expectation of E [ ˜ m 2 ( s )]
= 0 since E [ K ( s n − s h ) e ] =
E [ K ( s n − s h ) E
[ e | S = s n ]] = 0 .
As for the variance of ˜ m 2 ( s ) , we have
[Equation]
Subsequently, we consider the expectation and variance of ˜ m 1 ( s ) .
Speciﬁcally, for expectation, we have
[Equation]
As for the variance of ˜ m 1 ( s ) , we have
[Equation]
Combining the bias and variance part, we have
[Equation]
[Equation]
Based on the inequality of arithmetic and geometric means, it is easy to obtain the optimal bandwidth h ∗
=
O ( N − 15 ) , and the minimized MSE is lower than O ( N − 45 ) .
E P ROOF OF T HEOREM
Given predictor ˆ Y = f
( X ) , adversary ˆ S
=
g
( ˆ Y ) , and adversary utility L adv ( ˆ S, S )
= 1 − | ˆ S − S | ,
GDP and adversary utility are given by
[Equation]
Intuitively, higher model prediction implies larger sensitive attribute if mean prediction function m ( s ) is more close to s compared with 1 − s and vice versa.
Therefore, we construct adversary g as follows:
[Equation]
Suppose without loss of generality (WLOG)
that E S [ | S − m ( S ) | ] ≤ E S [ (cid:12)(cid:12) S − (cid:0) 1 − m ( S ) (cid:1)(cid:12)(cid:12) ] , i.e., higher model prediction implies larger sensitive attribute.
Then adversary utility is given by
[Equation]
where the inequality holds due to Jensen’s inequality and convex function | x − t | for any constant t .
Next we show, under the constructive adversary g , the adversarial utility L adv ≥ 12 ≥ ∆ GDP .
Firstly, notice that, for any function m ( S ) ∈ [0 , 1] and S ∈ [0 , 1] , we have
[Equation]
which implies that
[Equation]
As for the analysis on GDP, we consider the worst case of mean prediction function m ( s ) since
GDP satisﬁes ∆ GDP =
E S
[ (cid:12)(cid:12) m ( S ) − E S
[ m ( S )] (cid:12)(cid:12) ] .
Note that function | x | is strictly convex and the solution to maximize a strictly convex function over all ﬁnite support given ﬁrst moment is achieved by a distribution of two mass extreme points, GDP can achieve maximal value when m ( S ) is 0 or 1 .
Deﬁne p pos =
E S (cid:104) P (cid:16) m ( S ) = 1 (cid:17)(cid:105) , then E S [ m ( S )]
= p pos and ∆ GDP = p pos (cid:16)
1 − p pos (cid:17) + (1 − p pos ) p pos ≤ 12 .
Therefore, for the optimal adversary g ∗ , we have
[Equation]
F D ATA S TATISTICS
For fair comparision with previous work, we perform the classiﬁcation and regression task on ﬁve datasets, including Crimes, Adult, Pokec-n, Pokec-z and Harris dataset.
The ﬁrst four dataset have been widely adopted to study the fairness problem in tabular data and graph data, while Harris dataset is collected by ourself for temporal graph data.
Table 1 presents additional information on the real-world tabular, graph and temporal graph datasets.
For task type column, “Reg” and “Clf” represents regression task and classiﬁcation task, respectively.
[Table]
G M ORE D ETAILS ON S YNTHETIC E XPERIMENTS G. GDP C ALCULATION
We ﬁrstly provide ground truth analysis in synthetic experiments so that we can evaluate propsoed two GDP estimation methods error.
Considering bivariate Gaussian distribution with mean µ = [ µ 1 , µ 2 ] and covariance matrix Σ = (cid:20) σσ 12 σσ(cid:21) and note that covariance matrix is positive deﬁnite matrix, it is easy to obtain inverse covariance matrix Σ − 1 =
(cid:20) λλ 12 λλ(cid:21) , where λ
22 =
σ 11 | Σ | and λ 12 = − σ 12 | Σ | .
The joint distribution of ( S, ˆ Y ) follows p S, ˆ Y ( s, ˆ y )
= 1 √ 2 π | Σ | exp (cid:16) − 12 λ 11 ( s − µ 1 ) 2 − 12 λ 22 ( s − µ 2 ) 2 + λ
[Equation]
s − µ 1 )(ˆ y − µ 2 ) (cid:17) .
Based on probability theory, we can have the condition function p ˆ Y | S (ˆ y | s ) as follows:
[Equation]
which means the mean prediction function m ( s )
=
σ 11 µ 2 + σ 12 ( s − µ 1 ) σ.
Notice that the probability density function of sensitive attribute is also Guassian with N ( µ 1 , σ 11 ) , therefore, the GDP is
[Equation]
Next, we calculate GDP for the second synthetic probability density function p S, ˆ Y ( s, ˆ y )
=
s
+ ˆ y if 0 ≤ s, ˆ y ≤ 1 .
It is easy to obtain the conditional probability p ˆ Y | S (ˆ y | s )
= s
+ˆ y s +
if 0 ≤ s, ˆ y ≤and thus the mean prediction function m ( s )
=
E [ ˆ Y | S
= s ]
= 12 s + 13 s + 1 2 .
Similarly, the probability of sensitive attribute is p S ( s ) =
s +
if 0 ≤ s, ˆ y ≤ 1 .
Thus, the GDP satisﬁes ∆ GDP = (cid:90) (cid:12)(cid:12)(cid:12) m ( s ) − E [ m ( S )] (cid:12)(cid:12)(cid:12) p S ( s )d s = (cid:90) (cid:12)(cid:12)(cid:12) 12 s +s + 12 − 12 (cid:12)(cid:12)(cid:12) ( s + 1 2)d s = 1 48 .
G. M ORE S YNTHETIC E XPERIMENT R ESULTS ON E STIMATION E RROR
Figure 6 shows local prediction average and sensitive attribute probability density function estimation results for different kernel function choice.
The top two subﬁgures show the local prediction average estimation error w.r.t.
bandwidth choice.
It is seen that the tricube and aitchison aitken kernel function achieve better and robust local prediction average estimation compared with Gaussian kernel function.
The mid subﬁgures show the local prediction average result for different sensitive attribute and bottom two subﬁgures shows probability density function estimation results with different kernel and histogram choice.
It is seen that kernel estimation possesses more smooth and accurate probability density function estimation.
[Figure]
G. M ORE S YNTHETIC E XPERIMENT R ESULTS ON R OBUSTNESS
In this subsection, we aim to investigate the robustness of GDP estimation error over different mean and covariance parameters.
Note that only the parameters expectation µ = [ µ 1 , µ 2 ] =
[0 . 5 , 1 . 0] and covariance matrix Σ = (cid:20) σσ 12 σσ(cid:21)
=
(cid:20)0 .0 .
5 2 .(cid:21) are adopted on synthetic experiments, we adopt these parameters as default and only modify the one parameter to inspect GDP estimation error.
Figures 7 and 8 show the GDP estimation error with mean parameters µ 1 and µ 2 , and covariance parameters σ 11 , σ 12 , and σ 22 given the same data size 1000 .
We have two observations: (1) kernel-based GDP estimation methods consistently achieves lower estimation errors than histogrambased counterpart for any Gaussian distribution parameter, which further validates the superiority of kernel-based method.
(2) kernel-based GDP estimation error is almost the same for any kernel function pair choice, which validates our theoretical analysis in Theorem.
GD
P e
[Figure]
o r
[Figure]
H M ORE D ETAILS ON R EAL -W ORLD E XPERIMENTAL R ESULTS H. M ITIGATION P ERFORMANCE FOR T EMPORAL G RAPH D ATA
The temporal graph data, provided by data intelligence company Cuebiq (Cuebiq, 2021), is collected from anonymous human movement activities, including the coordinates and time of mobile devices at stop points, during August 2017 in Harris County (Houston) Texas, USA.
To generate the temporal graph, we ﬁrst divide the Harris County into several grid cells with equal size approximately 1 km × 1 km .
Each grid cell is treated as a graph node, and temporal link between two nodes represents at least one user movement in hourly basis duration.
The node features are generated from socio-demographic data of the American Community Survey (ACS) 2014–2018 (5-year) data by the U.S. Census Bureau (Bureau, 2021).
In the experiment, the white race ratio is treated as continuous sensitive attributes and our task is to predict whether the income of each node is high or low.
We adopt temporal graph attention (TGAT)
(da Xu et al., 2020)
7 with map and product attention mechanism to efﬁciently aggregate temporal-topological neighborhood features and report the mean prediction performance and GDP with 5 running times.
We compare the mitigation performance of kernel estimation and other baselines for private Harris dataset with two backbones in Figure 9.
Simialrly, the hyper-parameter λ control the tradeoff between accuracy and GDP.
Again, kernel estimation consistently outperforms the other baselines by a large margin and binary-quantized sensitive attributes inevitably deteriorate the mitigation performance.
[Figure]
H. P REDICTION P ERFORMANCE AND GDP TRADEOFF C URVE
DURING T
RAINING
Training curve on tabular data : Aiming to inspect the dynamic prediction performance and GDP during model training, we provide prediction performance and GDP tradeoff curve for Adult and
Crimes dataset in Figure 10.
The left and right y-axis represent the prediction performance and GDP metric, respectively.
It is seen that, for kernel or histogram as regularization, the hyperparameter can control the prediction performance and GDP tradeoff, while the training is highly unstable with large variance for adversarial debiasing.
Additionally, kernel estimation as regularization possesses better bias mitigation performance for Adult and Crimes dataset.
[Figure]
M A E ( s o li d ) kernel-5.
histogram-5.
adv-500.
adv_bn-500.
G D P ( d o tt e d )
Training curve on graph data : Figures 11 and 12 demonstrates prediction performance and GDP tradeoff curve for Pokec-n and Pokec-z datasets using GCN and GAT model.
It is seen that, for kernel or histogram as regularization, the hyperparameter can control the prediction performance and GDP tradeoff, while the training is highly unstable with large variance for adversarial debiasing.
Additionally, kernel estimation as regularization possesses better bias mitigation performance for
Pokec-n and Pokec-z datasets in GCN and GAT model.
[Figure]
[Figure]
Training curve on temporal graph data : Figure 13 demonstrates prediction performance and GDP tradeoff curve for Harris datasets using TGAT with map and product attention mechanism.
It is seen that, for kernel or histogram as regularization, the hyperparameter can control the prediction performance and GDP tradeoff, while the training is highly unstable with large variance for adversarial debiasing.
Additionally, kernel estimation as regularization possesses better bias mitigation performance for TGAT with map and product attention mechanism.
Training curve on compositional sensitive attribute : Figure 14 demonstrates prediction performance and GDP tradeoff curve for Adult and Crimes dataset.
It is seen that, for kernel or histogram as regularization, the hyperparameter can control the prediction performance and GDP tradeoff, while the training is highly unstable with large variance for adversarial debiasing.
Additionally, kernel estimation as regularization possesses better bias mitigation performance for Adult and Crimes dataset. I F UTURE W ORKS [Figure] [Figure]
li
[Figure]
There are several interesting future work related to proposed GDP:(1)Extend other fairness metrics, such as Equal Odds (EO), for continuous and discrete sensitive attributes. The connection between extended fairness metric and adversarial debiasing is also interesting. (2) Provide a more ﬁnegrained analysis on estimation error, including the exact optimal bandwidth and estimation error expression, the lower bound of the estimation error for histogram and kernel GDP estimation over all data distribution and model prediction, the estimation error analysis over compositional sensitive attributes. (3)Investigate the better GDP estimation method with faster convergence guarantee.