Original: Fair Synthetic Data Does not Necessarily Lead to Fair Models
Revised: Fair Synthetic Data Does not Necessarily Lead to Fair Models

Original: Abstract
Revised: Abstract

Original: The Wasserstein GAN (WGAN) is a well-established model allowing for the generation of high-quality synthetic data approximating a given real dataset.
Revised: The Wasserstein GAN (WGAN) is a well-established model allowing for the generation of high-quality synthetic data approximating a given real dataset.

Original: We study
Revised: We study

Original: TabFairGAN, a known tabular variation of WGAN in which a custom penalty termis added to the generator’s loss, forcing it to produce fair data.
Revised: TabFairGAN, a known tabular variation of WGAN in which a custom penalty termis added to the generator’s loss, forcing it to produce fair data.

Original: Here we measurethe fairness of synthetic data using demographic parity, i.e., the gap in the proportions of positive outcome between different sensitive groups.
Revised: Here we measurethe fairness of synthetic data using demographic parity, i.e., the gap in the proportions of positive outcome between different sensitive groups.

Original: We reproducesome results from the paper and highlight empirically the fact that although thesynthetic data achieves low demographic parity, a classification model trained onsaid data and evaluated on real data may still output predictions that achieve highdemographic parity – hence is unfair.
Revised: We reproducesome results from the paper and highlight empirically the fact that although thesynthetic data achieves low demographic parity, a classification model trained onsaid data and evaluated on real data may still output predictions that achieve highdemographic parity – hence is unfair.

Original: In particular, we show empirically this gapholds for most parts spectrum of the fairness-accuracy tradeoff, besides the largepenalty case where the model mode collapses to the most frequent target outcome,and the low-penalty case where the data is not constrained to be fair.
Revised: In particular, we show empirically this gapholds for most parts spectrum of the fairness-accuracy tradeoff, besides the largepenalty case where the model mode collapses to the most frequent target outcome,and the low-penalty case where the data is not constrained to be fair.

Original:   Introduction
Revised:   Introduction

Original: The problem of constructing fair machine learning models has been well studied, with a range ofpossible approaches, such as using different thresholds for different protected classes (Jang et al.[2022]), adding an optimal transport penalty to a neural network’s loss function (Gordaliza et al.[2019]) and more.
Revised: The problem of constructing fair machine learning models has been well studied, with a range ofpossible approaches, such as using different thresholds for different protected classes (Jang et al.[2022]), adding an optimal transport penalty to a neural network’s loss function (Gordaliza et al.[2019]) and more.

Original: Among these approaches, generating synthetic data under fairness constraints isa prevalent one (see Xu et al.
Revised: Among these approaches, generating synthetic data under fairness constraints isa prevalent one (see Xu et al.

Original: [2018, 2019], van Breugel et al. 
Revised: [2018, 2019], van Breugel et al. [2021], Jang et al.

Original: A common way to evaluate thefairness of synthetic data is using demographic parity.
Revised: A commonway to evaluate the fairness of synthetic data is using demographic parity.

Original: Given a data set with sensitive binary attribute S and binary target Y , the demographic parity of the data measures the dependence between thedistributions of S and Y .
Revised: Given a data set withsensitive binary attribute S and binary target Y , the demographic parity of the data measures thedependence between the distributions of S and Y .

Original: Similarly, the demographic parity of a classifier f trained on said syntheticdata measures the dependence between S and predictions f 
Revised: Similarly, the demographic parity of a classifierf trained on said synthetic data measures the dependence between S and predictions f ( X, S ) where

Original: ( X, S ) where X consists of the set ofunsensitive variables.
Revised: X consists of the set of unsensitive variables.

Original: In this latter case, the evaluation of the classifier’s fairness is done on the realdata.
Revised: In this latter case, the evaluation of the classifier’sfairness is done on the real data.

Original: Thus, training a classifier on the synthetic fair data and evaluating its demographic parity overa holdout of the real data gives us another viable way to measure the fairness of the data.
Revised: Thus, training a classifier on the synthetic fair data and evaluatingits demographic parity over a holdout of the real data gives us another viable way to measure thefairness of the data.

Original: In fact, ifone’s goal is to use synthetic fair data to train models which are fair in production, the latter metricis intuitively more relevant.
Revised: In fact, if one’s goal is to use synthetic fair data to train models which are fairin production, the latter metric is intuitively more relevant.

Original: In this paper we focus on TabFairGAN (Rajabi and Garibay [2022]), a known variant of WGAN(Arjovsky et al.
Revised: In this paper we focus on TabFairGAN (Rajabi and Garibay [2022]), a known variant of WGAN(Arjovsky et al.

Original: [2017]) in which a demographic parity penalty term is added to the generators lossfunction, forcing it to produce data with low dependence between S and Y .
Revised: [2017]) in which a demographic parity penalty term is added to the generators lossfunction, forcing it to produce data with low dependence between S and Y .

Original: We measure the fairnessof the data generated by the aforesaid model in both ways, on two common benchmark datasets,and show that besides in trivial cases, the demographic parity of a classifier tends to be significantlyhigher than the demographic parity of the synthetic data it has been trained on for most of thefairness-accuracy spectrum.
Revised: We measure the fairnessof the data generated by the aforesaid model in both ways, on two common benchmark datasets,and show that besides in trivial cases, the demographic parity of a classifier tends to be significantlyhigher than the demographic parity of the synthetic data it has been trained on for most of thefairness-accuracy spectrum.

Original: This implies that although the synthetic data generated by our modelis fair, using this data for training may not be a good way to create models which produce similaroutput distributions when conditioned on different protected classes.
Revised: This implies that although the synthetic data generated by our modelis fair, using this data for training may not be a good way to create models which produce similaroutput distributions when conditioned on different protected classes.

Original:  Background2.1 Demographic Parity
Revised:  Background2.1 Demographic Parity

Original: Given a dataset D =
Revised: Given a dataset D =

Original: { X, s, y } where the vector X ∈ R n represents the unprotected attributes,s ∈ { 0 , 1 } , is the protected attribute and y ∈ { 0 , 1 } is the target feature, the demographic parity of
Revised: { X, s, y } where the vector X ∈ R n represents the unprotected attributes,s ∈ { 0 , 1 } , is the protected attribute and y ∈ { 0 , 1 } is the target feature, the demographic parity of

Original: D is defined by:
Revised: D is defined by:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Given a classifier f : R n → { 0 , 1 } trained on the dataset D train , and a test dataset D test , the demographic parity of f is defined by:
Revised: Given a classifier f : R n → { 0 , 1 } trained on the dataset D train , and a test dataset D test , the demographic parity of f is defined by:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: It is worth noting that the notion of demographic parity can be generalized to the case of multiplesensitive classes and multiple targets and that there exist many more notions of fairness (see Walbyand Armstrong [2011]).
Revised: It is worth noting that the notion of demographic parity can be generalized to the case of multiplesensitive classes and multiple targets and that there exist many more notions of fairness (see Walbyand Armstrong [2011]).

Original: 2.2 WGAN
Revised: 2.2 WGAN

Original: The architecture of WGAN is based on the Kantorovich-Rubinstein duality (Edwards [2011]), atheorem in the field of optimal transport (Villani [2009]), and involves a generator G , and a critic
Revised: The architecture of WGAN is based on the Kantorovich-Rubinstein duality (Edwards [2011]), atheorem in the field of optimal transport (Villani [2009]), and involves a generator G , and a critic

Original: C .
Revised: C .

Original: The generator’s role is to transform the distribution of a random noise z ∼ N (0 , 1) into adistribution P g which approximates some real data distribution P r .
Revised: The generator’s role is to transform the distribution of a random noise z ∼ N (0 , 1) into adistribution P g which approximates some real data distribution P r .

Original: The critic’s role is to evaluatethe quality of a given data point.
Revised: The critic’s role is to evaluatethe quality of a given data point.

Original: The kantorovich Rubinstein duality suggests the critic should beapproximately 1-Lipschitz
Revised: The kantorovich Rubinstein duality suggests the critic should beapproximately 1-Lipschitz

Original: and so we add a gradient penalty term to the discriminator’s loss.
Revised: and so we add a gradient penalty term to the discriminator’s loss.

Original: Thus,the loss functions that the two models try to minimize are:
Revised: Thus,the loss functions that the two models try to minimize are:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: where P ˆ x is a distribution of points, uniformly sampled along straight lines between pairs of pointssampled from P r and P g , ( ∥ ▽ ˆ x C (ˆ x ) ∥ 2 is the gradient norm of the critic’s scores on ˆ x , and γ isthe gradient penalty coefficient.
Revised: where P ˆ x is a distribution of points, uniformly sampled along straight lines between pairs of pointssampled from P r and P g , ( ∥ ▽ ˆ x C (ˆ x ) ∥ 2 is the gradient norm of the critic’s scores on ˆ x , and γ isthe gradient penalty coefficient.

Original: The latter has been kept constant at the value of 10 throughout allof WGAN training trials, as recommended by the authors of the WGAN-GP paper (Gulrajani et al.[2017]).
Revised: The latter has been kept constant at the value of 10 throughout allof WGAN training trials, as recommended by the authors of the WGAN-GP paper (Gulrajani et al.[2017]).

Original: The main reason we use WGAN instead of the original GAN (Goodfellow et al.
Revised: The main reason we use WGAN instead of the original GAN (Goodfellow et al.

Original: [2014]) isthat it is less prone to mode collapse.
Revised: [2014]) isthat it is less prone to mode collapse.

Original:  Methodology3.1 Fair Data Generation
Revised:  Methodology3.1 Fair Data Generation

Original: As previously mentioned, to produce fair synthetic data we use tabFairGAN, a common variant of
Revised: As previously mentioned, to produce fair synthetic data we use tabFairGAN, a common variant of

Original: WGAN which involves adding a fairness penalty term to the generator loss of a standard WGANmodel.
Revised: WGAN which involves adding a fairness penalty term to the generator loss of a standard WGANmodel.

Original: Thus the generator’s loss becomes:
Revised: Thus the generator’s loss becomes:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where D = { x i , s i , y i }
Revised: where D = { x i , s i , y i }

Original: Ni =1 and ( x i , s i , y i ) ∼ P g .
Revised: Ni =1 and ( x i , s i , y i ) ∼ P g .

Original: The second term is equal to the demographic parityof the generated data squared, and λ is a hyper-parameter used to control the synthetic data’s utilityfairness trade-off.
Revised: The second term is equal to the demographic parityof the generated data squared, and λ is a hyper-parameter used to control the synthetic data’s utilityfairness trade-off.

Original: As the generator is trained to minimize its loss, introducing the new penalty termforces the generator to produce data with lower demographic parity.
Revised: As the generator is trained to minimize its loss, introducing the new penalty termforces the generator to produce data with lower demographic parity.

Original: Following tabFairGAN, the generator’s network consists has Gumbel-Softmax heads for categoricalvariables, however, we do not use variational Gaussian normalization for continuous variables.
Revised: Following tabFairGAN, the generator’s network consists has Gumbel-Softmax heads for categoricalvariables, however, we do not use variational Gaussian normalization for continuous variables.

Original: 3.2 Data
Revised: 3.2 Data

Original: In order to test our hypothesis, we use the Census Adult data-set (Blake [1998]), and Compas Recidivism data-set (Angwin et al.
Revised: In order to test our hypothesis, we use the Census Adult data-set (Blake [1998]), and Compas Recidivism data-set (Angwin et al.

Original: [2016]), two common benchmarking data-sets in the fairness field –both considered in Rajabi and Garibay [2022].
Revised: [2016]), two common benchmarking data-sets in the fairness field –both considered in Rajabi and Garibay [2022].

Original: The first is a classification problem of 48842 records,in which we try to predict whether a person earns over 50K dollars a year, using features like education, family status, etc.
Revised: The first is a classification problem of 48842 records,in which we try to predict whether a person earns over 50K dollars a year, using features like education, family status, etc.

Original: The sensitive attribute here is the gender of a person.
Revised: The sensitive attribute here is the gender of a person.

Original: The second is aclassification problem of 16267 records where we try to predict whether ex-criminal offenders havea high or low chance to commit another crime, based on features like previous offense type, language, etc.
Revised: The second is aclassification problem of 16267 records where we try to predict whether ex-criminal offenders havea high or low chance to commit another crime, based on features like previous offense type, language, etc.

Original: Studies have shown that this dataset is biased against African Americans (Chouldechova[2017]).
Revised: Studies have shown that this dataset is biased against African Americans (Chouldechova[2017]).

Original: Therefore ethnicity is chosen to be the protected attribute for this study.
Revised: Therefore ethnicity is chosen to be the protected attribute for this study.

Original: We shall use analtered version of the original dataset which was used in the original TabFairGAN paper.
Revised: We shall use analtered version of the original dataset which was used in the original TabFairGAN paper.

Original: In both cases, the target and the protected attribute are binary.
Revised: In both cases, the target and the protected attribute are binary.

Original: We pre-process the data using one-hotencoding on all categorical features and standard scaling on all numeric features.
Revised: We pre-process the data using one-hotencoding on all categorical features and standard scaling on all numeric features.

Original: 3.3 Metrics
Revised: 3.3 Metrics

Original: We concern ourselves with three properties of synthetically generated data, its utility, its population fairness, and the fairness of a model trained using it.
Revised: We concern ourselves with three properties of synthetically generated data, its utility, its population fairness, and the fairness of a model trained using it.

Original: Thus, the three metrics which we willconcentrate on are the following: we will measure the data’s utility using the accuracy of a logisticregression model trained on the synthetic data and assessed on a test set of real data.
Revised: Thus, the three metrics which we willconcentrate on are the following: we will measure the data’s utility using the accuracy of a logisticregression model trained on the synthetic data and assessed on a test set of real data.

Original: we will measurethe data’s population fairness using The demographic parity of the synthetic data itself, and finally,we will measure the fairness of a model trained on the data using the demographic parity of theoutput of a logistic regressions model trained on the synthetic data and evaluated on the test data.
Revised: we will measurethe data’s population fairness using The demographic parity of the synthetic data itself, and finally,we will measure the fairness of a model trained on the data using the demographic parity of theoutput of a logistic regressions model trained on the synthetic data and evaluated on the test data.

Original:  Experiments
Revised:  Experiments

Original: In the first experiment, we report accuracy and demographic parity scores on the Census dataset (see
Revised: In the first experiment, we report accuracy and demographic parity scores on the Census dataset (see

Original: Figure 1).
Revised: Figure 1).

Original: As the fairness weight λ grows, the accuracy of the classifier trained on the synthetic data decreases,starting with a value as high as that of a classifier trained on the real data, and ending with anaccuracy close to that of a classifier which assigns only the outcome with the highest probability(this achieves an accuracy of 0.76), and has a perfect data demographic parity as a result.
Revised: As the fairness weight λ grows, the accuracy of the classifier trained on the synthetic data decreases,starting with a value as high as that of a classifier trained on the real data, and ending with anaccuracy close to that of a classifier which assigns only the outcome with the highest probability(this achieves an accuracy of 0.76), and has a perfect data demographic parity as a result.

Original: We alsosee that both the demographic parity of the synthetic data, and that of the logistic regression classifiertrained on it, start at around 0 .
Revised: We alsosee that both the demographic parity of the synthetic data, and that of the logistic regression classifiertrained on it, start at around 0 .

Original: 19 , which is the demographic parity of the real data, and end at aroundzero.
Revised: 19 , which is the demographic parity of the real data, and end at aroundzero.

Original: However, more importantly, we observe a gap between data and model demographic parity.
Revised: However, more importantly, we observe a gap between data and model demographic parity.

Original: It isindeed required to pay a significantly higher price in accuracy to reach the same level of parity forsynthetic data and model.
Revised: It isindeed required to pay a significantly higher price in accuracy to reach the same level of parity forsynthetic data and model.

Original: We observe similar conclusions in the Compas experiment (see Figure 2).
Revised: We observe similar conclusions in the Compas experiment (see Figure 2).

Original: In this case, the gap is significantly higher – for an accuracy of 85% , the data demographic parityis at 0 .
Revised: In this case, the gap is significantly higher – for an accuracy of 85% , the data demographic parityis at 0 .

Original: 02 , while the model demographic parity is of 0 .
Revised: 02 , while the model demographic parity is of 0 .

Original: This suggests that care need to be takenwhen leveraging synthetic data achieving fairness constraints as this does not necessarily means amodel trained on said data, and tested on a real test holdout will lead to fair predictions.
Revised: This suggests that care need to be takenwhen leveraging synthetic data achieving fairness constraints as this does not necessarily means amodel trained on said data, and tested on a real test holdout will lead to fair predictions.

Original: Also, the only places where the gap is minor is when the fairness weight is close to 0 , in which casethe data is not constrained to be fair, and the case where the fairness weight becomes very large,in which case the classification model simply mode-collapses to the most frequent target outcome,(and hence achieves perfect model demographic parity as can
Revised: Also, the only places where the gap is minor is when the fairness weight is close to 0 , in which casethe data is not constrained to be fair, and the case where the fairness weight becomes very large,in which case the classification model simply mode-collapses to the most frequent target outcome,(and hence achieves perfect model demographic parity as can

Original: Figure 1)
Revised: Figure 1)

Original: [Figure]
Revised: [Figure]

Original: Census Adult Dataset Experiment Results
Revised: Census Adult Dataset Experiment Results

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Compas Recidivism Dataset Experiment Results
Revised: Compas Recidivism Dataset Experiment Results

Original: [Figure]
Revised: [Figure]

Revised:  Discussion

Revised: Demographic parity is solely dictated by the correlation between the target variable Y and the sensitive variable S .

Revised: However, it does not account for proxy variables, and can be low despite the data having other dependencies, which may bias the results.

Revised: This implies that a synthetic data generation model which is trained with demographic parity constraints, can achieve these constraints bychanging P ( X, s ) rather then changing P ( y | X, s ) (i.e. changing the population itself rather thenthe targets dependency on the population).

Revised: This drift leads to a fairness gap, i e., the training datamay look artificially fair, but a model trained on it and deployed (tested) on a holdout dataset fromthe true population can still lead to unfair predictions.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we reproduce and study a minor variation of TabFairGAN, a recently proposed approach to train GANs under demographic parity constraints.
Revised: In this paper we reproduce and study a minor variation of TabFairGAN, a recently proposed approach to train GANs under demographic parity constraints.

Original: We observed that while this class of model allows to achieve low data demographic parity with negligible accuracy drop, achievingdownstream model demographic parity is more challenging.
Revised: We observed that while this classof model allows to achieve low data demographic parity with negligible accuracy drop, achievingdownstream model demographic parity is more challenging.

Original: This suggests that care should be takenwhen leveraging synthetic data with demographic parity constraints as models trained on said datamay not be downstream-fair.
Revised: This suggests that care should be takenwhen leveraging synthetic data with demographic parity constraints as models trained on said datamay not be downstream-fair.