Generative Fairness Teaching
Authors: Rongmei Lin
Hanjun Dai
Li Xiong
Wei Wei
Venue:  (2021)
Keywords: fairness
student teacher model
counterfactual generative model

Abstract: Increasing evidences has shown that data biases towards sensitive features such as gender or race are often inherited or even amplified by machine learning models. Recent advancements in fairness mitigate such biases by adjusting the predictions across sensitive groups during the training. Such a correction, however, can only take advantage of samples in a fixed dataset, which usually has limited amount of samples for the minority groups. We propose a generative fairness teaching framework that provides a model with not only real samples but also synthesized samples to compensate the data biases during training. We employ such a teaching strategy by implementing a Generative Fairness Teacher (GFT) that dynamically adjust the proportion of training data for a biased student model. Experimental results indicated that our teacher model is capable of guiding a wide range of biased models by improving  the fairness and performance trade-offs significantly.

Under review as a conference paper at ICLR 2021
GENERATIVE FAIRNESS TEACHING
Anonymous authors Paper under double-blind review
ABSTRACT
Increasing evidences has shown that data biases towards sensitive features such gender or race are often inherited or even amplified by machine learning models. Recent advancements in fairness mitigate such biases by adjusting the predictions across sensitive groups during the training. Such a correction, however, can only take advantage of samples in a fixed dataset, which usually has limited amount of samples for the minority groups. We propose generative fairness teaching framework that provides a model with not only real samples but also synthesized samples to compensate the data biases during training. We employ such teaching strategy by implementing a Generative Fairnsss Teacher (GFT) that dynamically adjust the proportion of training data for biased student model. Experimental results indicated that our teacher model is capable of guiding a wide range of biased models by improving the faimess and performance trade-offs significantly.
INTRODUCTION
Automated learning systems are ubiquitous across a wide variety of sectors. Such systems can be used in many sensitive environments to make important and even life-changing decisions. Tra- ditionally, decisions are made primary by human and the basis are usually highly regulated. For example in the Equal Credit Opportunity ACts (ECOA), incorporating attributes such as race, color, or sex into credit lending decisions are illegal in United States (Mchrabi et al. 2019). As more and more of this process nowadays is implemented by automated learning systems instead, algorithmic fairness becomes topic of paramount importance. Lending (Hardt et al.) 2016), hiring (Alder & Gilbert 22006), and educational rights (Kusner et al.) (017) are examples where gender or race bi- ased decisions from automatic systems can have serious consequences. Even for more mechanical tasks such as image classification (Buolamwini & Gebru 2018), image captioning (Hendricks al.) 2018), word embedding learning (Garg et al. [018 Bolukbasi et al. 2016), and named co-reference resolution (Zhao et al. 2018). algorithmic discrimination can be a major concern. As the society relies more and more such automated systems, algorithmic fairness becomes a pressing issue. Al- though much of the focus of developing automated learning systems has been on the performance, it is important take fairness into consideration while designing and deploying the systems.
Unfortunately, state-of-the-ar automated systems are usually data driven, which makes it more likely to inherit even amplify the biases rooted in dataset. This is an especially serious is- sue for deep learning and gradient based models, which can easily fit itself into the biased patterns of the dataset. For example, in a dataset with very few female candidates being labeled as hired in a job candidate prediction task, models might choose to give unfavorable predictions to qualified female candidates due to their under-repreesnntaions in the training data. If deployed, such biased predictor will deprive minority groups from acquiring the same opportunities as the others.
Much of the work in the domain of machine learning fairness has been focusing exclusively on leveraging knowledge from samples in a dataset. One straightforward way is to adjust the distribu- tions of the training data through proppecssssng. In the job candidate prediction example above, this means that we can either down-sample the majority class or up-sample the minority ones (Kamiran & Calders 2012). Another family of fairness methods aims at matching the model performance on the majority class to that of the minority ones during training by using one of the fairness criteria (Gajane & Pechenizkiy 2017). Some examples of such methods includes adding regularizations Kamishima et al.J 2012) applying adversarial learning (Madras et al. 2018a). One issue with these approaches is that in many cases minority groups might be heavily under-represented in the
dataset. Model training with fairnsss constraints will typically give up much of the performance ad-

---

Under review as a conference paper at ICLR 2021
vantages (e.g., prediction accuracies) in favor of the fairness metries. Methods concentrate solely on a dataset will often find themselves difficult to maintain good performance fairness trade-off.
One way to make models learn beyond the dataset is to take advantage of causal reasoning (Pearl ctt al. 2009), which borrows knowledge from external structures often formulated as a causal graph. Counterfactual Faimesss (Kusner et al. 2017) and Causal Fairness (Kilbertus et al. 2017) ples of such approaches. One unique characteristic of causal fairness is the fact that they need to be built based on a causal graph. And because those metries are usually optimized and evaluated their own objective, which involves a causal graph, it's not clear how that added knowledge can be used benefit other more commonly used fairesss criteria such as Demographic Parity and Equalized Odds. Although it is possible to create causal structures that subsume conditional independencies in order to benefit DP or EO, we will need those structure information be known in advance and we will have to derive one such structure for each metric we find. This is, what we believed, significant limitation of the current causal methods which we aim to improve.
exam-
In this paper, we propose generative approach for fairness training that is capable of leveraging both real data and "counterfactual data" generated from a causal graph. The counterfactual data is generated in a way that alters the sensitive attribute while keeping other latent factors unchanged. We formulate such generative model using a novel combination of adversarial training with mutual information regularization. Next, the two types of data are organized by an architecture called the teacher, which dynamically determines the proportion of real and counterfactual samples to train particular model. Our model Generative Fairness Teacher (GFT) can be used to improve an arbitrary fairness criteria based on need. Our experimental results indicate that we are able to take advantage of the counterfactual generative model and make it able to achieve significantly better model fairness on a wide range of datasets across models. we are able to improve upon models with different levels of biases.
BACKGROUND
We provides a basic overview for the foundations of our method. Here we assume X to be the input features, while A being the set of sensitive features. We define Y the be favorable outcome and Ý to be the models' prediction of the favorable outcome given the features. The core idea of Faimesss in machine learning is to distribute those favorable outcomes evenly across each of the sensitive group A.
2.1 FORMAL FAIRNESS CRITERIA
There has been many existing work on fairness focusing studying criteria to achieve algorithmic fairness. A straightforward way to define fairness is Demographic Parity Madras al. (201 In Demographic Parity, the chances of allocating the favorable outcomes Y is the same across sensitive groups A. Under that definition, the predictive variable Ý is independent with A, making predictions free from discrimination against sensitive groups. Note that even though A takes the form of binary variable, we can easily extend the definition into the case of multiple values.
Definition Demographic Parity
Other fairness criteria that are built based on input features includes include Fairness Through Un- awareness Gajane & Pechenizkiy (2017). and Individual Faimess Kusner al. (2017). More re- cently, Hardt et al. argued that criteria that only takes into account sample features making it difficult for the algorithms allocate favorable outcomes to the actual qualified samples in both the minority and the majority groups. Such an observation leading to new fairness criteria called Equalized Odds (and its special case Equal Opportunity) Hardt et al. (2016), where the faimess statement includes a condition on target variable Y.
Definition 2 Equalized Odds
P( = =1=
2

---

Under review as a conference paper at ICLR 2021
3 CAUSAL MODELS AND COUNTERFACTUAL EXAMPLES
causal model Pearl et al. (2000) is defined over a triple (U, V, F) where is a set of observed variables and U/ being set of latent background variables. F is defined to be a set of equations for each variable in V. Vi = fi Upai). Here pa, refers to the parent of a causal graph. One importance concept in causal reasoning is intervention, in which case substitute the variable of certain equation 24 v.
We define counterfactual example to be synthesized sample generated from existing data X by manipulating its sensitive feature from a to a'. Here we assume that both the real sample and the counterfactual sample XA-a' are generated from latent code U.
Definition 3 Counterfactual Example
X (U)]xe
(3)
3.1 COMMON TECHNIQUES FOR FAIRNESS
Depending on when the fairness criteria are applied, methods for achieving faimesss can be catego- rized as pre-processing, in-processing and post procepsres Mchrabi et al. (2019).
In-processing Techniques. In-processing techniques apply fairness criteria during the training. Common techniques including fairness regularizer Kamishima et al. (2012) and adversarial training Madras et al. (2)18)). Other methods fall into this category including the reduction based method
Agarwal et al. (2018) and the more traditional discrimination approach in data mining[Hajian & Domingo-Ferrer (2012).
Our implementation applies in-processing techniques although our framework does not deal with in-processing methods directly.
Pre-processing Techniques. Pre-processing methods applies to the models before the actual train- ing happens. Methods fall into this category are almost exclusively data processing techniques that aims at making the dataset free from biases. Re-sampling and re-weighting are two common tech- niques of pre- pre-processing techniques for fairness. Calmon et al, 2017); Kamiran & Calders (012);;
Agarwal al. (2018). Other techniques include that repairs biases in database Salimi et al. 2019
Our method is closely related to the pre-processing techniques because from the perspective of the student model teacher model can be viewed as data pre-processor.
Post-processing Techniques. When fairness adjustments are applied after the training is finished, techniques are called post-processing methods. Post-processing methods can used adapt mod- els with all kinds of biases levels into a fair model model Madras et al. (2018) 18). Other recently proposed in- cluding the method model fairness as a score transtormation problem Wei et al. (2019) and meth- ods enforces Independence between sensitive features and model outcomes through Wasserstein- 1 distances Diang et al.) (2020) Our approach is closely related the post-processing technique as our teacher model can work with an arbitrarily biased student model.
4 GENERATIVE FAIRNESS TEACHING
In this section, we propose a teaching framework for training a student model that is able to work with a wide range of fairness criteria. We first present the overview of our approach in section 4.1 Then in section 4.2 we claborate a novel generative model that can create "counterfactual exam-
ples". In section 4.4 we will show how train such a teacher policy with the given student model and counterfactual generative model.
4.1 FAIRNESS TEACHING FRAMEWORK
Given a training dataset Derain {(X Y = PPrainl where X and Y are ob- served features and label, respectively, and A is some sensitive attribute, we are interested in learn- ing a predictive model (Y X) that is parameterized by 0, such that it maximizes the reward on the validation set:
R(0) E( log pog ( ACcP (Y)))) D Dwatid)
3

---

Under review as a conference paper at ICLR 2021
Algorithm 1 Generative Teaching Procedure
1: Input: initial student model pe, (X)x), counterfactual generator p(X)
2: Input: dataset D, Teacher policy Te
3: for f- 1 to episode length T do
4: Sample minibatch D' {d. [s Mr. a; a M ~ D.
= 0i), 5: Get counterfactual data D' = { d: = [ f1 = 3i, A= 0z): Mi, [t]} for each di E D'.
6: Get current student's state = SDD Po (Y using Eq 11
7: Obtain decision d; (s), and get D = d; E D' 0
uf(d D'uu = =1].
8: Update student: 0, 0:-1 V= [D] Σ (a.y)eD (ogl)
9: end for
10: Return: updated student model poy (YIX)
where FC(:,) stands for the evaluation metric under certain fairness requirement, such as equalized odds. This objective tries balance between the generalization error and the faimess constraint, which is controlled by the hyperparameter Afc-
In our teaching framework, we will teach student predictive model pa XX to minimize Eq 4 Our teacher model TH. is responsible for providing proper data samples for the student at each step of optimization achieve this goal. In most teaching frameworks, the teacher is only responsible for selecting proper samples from existing dataset. However, due to the potential bias in the dataset, such assumption is too limited to achieve the fairness requirement.
Recall the definition of counterfactual example in Eq 3 Given tuple of (U,X= = I, a), changing A by < a' while keeping U fixed will also change X. Thus the change to the predictive distribution p(YAra' (U))ar a) depends on two aspects: 1) the predictive model (Y(())), and 2) a counterfactual generative model p(X) == p(AAaa' (U)) = x, A = a). Suppose we have the model p(X) ready, then it would be possible to regulate (X by generate counterfactual samples during training.
Given the teacher model Te and the counterfactual generative model p(X), , we are ready to present At each teaching stage, the teacher will make binary decisions on using 1) the sample selection from the given dataset Divains or 2) the counterfactual data (X, YY, A = a') coming from data sample (X, A, = a) E Derain and altered by ş( )). The student will then use the selected samples to perform gradient update of O.
our iterative teaching approach for learning (Y) in algorithm [
In the following sections, we will present how such counterfactual generative model p(X) are learned through teaching policy T..
4.2 LEARNING COUNTERFACTUAL GENERATOR ((})
8~ ((
Share weights dx
X ~ p(X = a)
go, (UIX)
Mutual-Info
8 Male
Attr LOSS
Lete
Attr Loss
L-Penta Female
Leta
MGAN 1oss
VAR Loss
Lugan
Lib
Figure 1: Overview of the counterfactual generative model.
4

---

Under review as a conference paper at ICLR 2021
To learn the counterfactual data distribution, first need understanding of the empirical data distribution. In next subsection, we first present our latent variable modeling of the data distribution:
4.2.1 EMPIRICAL DATA MODELING
A)p(U)dU. The design of this graphical model follows Kusner et al. (2017), where we have the dependency U X +- A. Here assumed to be independent from sensitive attribute A, and and A will become dependent when X is observed. The generative process of counterfactual example X depends on both U and an altered A. To learn such latent variable model, we optimize the following lower bound:
We model the empirical data distribution as p( =)= Ju
log p( (x,) 2 Lib = Bys, (U1x) [logp( ( A P)x (XUU, A)]
- DKL (U) ((U)), s.t. T(A:UIX) =0
(5)
The mutual information constraint indicate that the posterior {() should not be informative at predicting data distribution p( ())) and thus disentangles the sensitive and insensitive latent factors. We rewrite the mutual information in the following way:
p(U, A(X)
Cma( == I(AUU] = E log p(U()))((A(x) Egs, Bx(Ajx) ((( +C
(6)
where the constant C' is the entropy (x))) as ((A is the empirical data distribution. Thus suppose have predictive model P&a (A(U)) that trained by minimizing:
(o)( -Ex,AA M.KKA~ (U|x) Poo Poa (AlU))
then we can address the constraint Eq using penalty method, by minimizing Lyn. w.r.t dg-
4.2.2 COUNTERFACTUAN DATA GENERATIVE MODELING
As introduced in section the counterfactual examples are generated by altering the sensitive attribute while keeping the latent factor U unchanged. However we want to make sure these samples are also realistic, the sense that it is close the original data distribution p(X). To match these two distributions, we leverage the technique of WGAN (Arjovsky et al.) 2017) by optimizing the following objective:
Lugaan (),)), D) min max EX-D [(X)] -E (XX.A=a)--DD000000XXx)
[(X))
#(8) D
where D is the discriminator in GAN. We also adopt the gradient penalty (Gulrajani et al. 2017) Cgp with the discriminator to stabilize the training. Note that the counterfactual model shares the same decoder parameters as Pax (X U, A) in 5
We also leverage the attribute labels as auxiliary tasks for D. This auxiliary helps D better distin- guish between the realistic images and the generated counterfactual images. Here create another linear layer on top of D's last hidden layer (denoted as AA) and try to minimize:
Cate = min E(x,Ay<D [log (A[X)) x ( [logPo, (A d'(x)]
(9)
DA
4.3 TRAINING GENERATIVE MODELS
As our generative models have multiple constraints with entangled dependencies (see Table for summary). we design the following learning paradigm that can effectively satisfy these objectives:
Train the encoder 4e, (UIX), decoder p(ox). and discriminator D, DA in alternating way, D, AA) where the last term the L2 with C, := LD Lwgan + Lets + Eop + LI2(0g, 0x,
regularization of neural network parameters;
b) Train the attribute classifier P&* with Catt + LoZ(DA):
Finetune the encoder, decoder and discriminator with L. == L. Lmn-
Intuitively, the first step gets the generators working reasonably well in generating realistic images. The second step learns the attribute classifier from the latent code U, which is also tractable task. In
5

---

Under review as a conference paper at ICLR 2021
Table I: Notation of loss terms.
Notation
Ohjective
Lib
generative modeling of (X, A) with latent U separating the information of A from U learning the attribute classifier from U lcarning the counterfactual generator auxiliary task for attribute prediction gradient penalty L2 regularization
Lyu
Cutt
Lugan
Las
Cop. L12
the last step, we address the mutual information constraint using penalty method, by minimizing the Eq 6 together with other generative model losses. The counterfactual generator would be expected learn adapt to the refined U in the last step. See Figure [] for the visual demonstration of this process. In practice, we can also tune the coefficients of each loss term. See the experiment section for more information.
4.4 LEARNING TEACHER POLICY T
Since the student objective defined in Eq 4 is complicated, which involves with arbitrary fairness metrics, we leverage Policy Gradient to learn the teacher model Tg. Specifically, our objective is
max E ST
Σ (SLIOEET
r(s,, as)
(10)
where r the state-action trajectory sampled from the behavior Tg-- Next we define the state, action and reward in detail. The RL-based teacher act as a data loader in the iterative learning process between teacher and student. will feed the real or fake images to student according to current state. Following the teacher's instruction, the student model will have final terminal reward on the held-out validation set.
state: It contains the information of student's model and current training batch. We denote it as
S(D', po (Y|x)) = (1 E D'}. {at E DDD (po [m))];
FC(po
(11)
fairness current training batch reward: The reward function defined in Eq which is evaluated on held-out validation set. For example, if the Equalized Odds is used as fairness metric, then FC(.,)) = Plog( = 1|A =
labels
sensitive attributes
cross entropy
1,Y=y) - = 1|A 0, Y y))). One can also define the reward for Demographic Parity or other fairness criteria.
action: The teacher needs to make binary decision {am J.M m==1 am {0, 1 on the minibatch of instances, where is the batch size. Here represent using real data, 0 represent using the corresponding counterfactual data sample generated from p(X).
The episode length typically equals to several epochs of the training data. We use the moving average of the final reward as the baseline to reduce the variance. Note that other advanced RI. algorithms techniques that handle delayed reward (Arjona-Medina et al. 2019) also be adapted here to further boost the performance.
5 EXPERIMENTS
5.1 TABULAR DATA
We perform binary classification and faimess metric analysis tabular data to show the improvement of performance using Generative Fairness Teacher (GFT), Prediction per- formance is measured by Testing Error. We choose Equalized Odds defined in Eq be our fairness metrics to illustrate the performance of our model. In practice one can choose to optimize an arbi- trary metrics based on needs. In each of the experiments we evaluate the gap of Equalized Odds, defined as
Experiment Details.
=O= = 1|X = P( 1|X x, A d,
(12)

---

Under review as a conference paper at ICLR 2021
Table 2: Adult Dataset
Table 3: COMPAS Dataset
Table 4: CelebA Dataset
Method
Frron(%)
FO
Method
Er)or(5)
EO
Method
Error %
BO
In-proesssig
16.8
0.048
in-processing
32,9
0.034
Basel: real
21.0
0.426
Post-processing
16.9
0.049
Post-processing
32.8
0.039
Basc2: all fake
48.1
0.610
Basel: all real
15,9
0.179
Basel: all real
32.1
0.224
Base3; random
23.5
0.417
Base2: all fake
15.8
0.141
Base2: all lake
32.3
0.231
Base4; balance
19.6
0.362
Buse3: random
16.0
0.180
Base3; random
32.7
0.247
Base5: lix ratio
20.7
0.242
Base4; balance
15.8
0.157
Base4: balance
32.0
0.229
Base6; reverse
19.2
0.171
GFT
16.1
0.044
GFT
32.8
0.028
GFF
19.1
0.098
We report the maximum among the false positive difference and true positive difference between protected and unprotected groups. We compared our method with anoonentittedggrddeen reduction based in- processsiing algorithm (Agarwal et al.) 2018) and score-based post-processing algorithm (Hardt et al. 2016). In addition these two methods, we also compared the GFT with four different baselines. Basel denotes the model that trains with all original examples, which is also an uncon- strained classifiers. Base2 denotes the model that trains using all counterfactual examples. Base3 is the model that trains with a random combination of original and counterfactual examples. Base4 is the model trained with a balance combination of original and counterfactual examples, which guarantees the proportion of protected and unprotected group in the training set to be the same. We evaluate our method on two well-known tabular datasets, the ProPublica's s COMPAS recidivism dataset and the UCI Adult income dataset. The model are trained on randomly selected 75% samples and evaluated on the rest of 25% testing examples. We follow the same setting as in (Agarwal et al, 2018), which uses logistic regression in scikit-learn as the classifier.
Adult Income Dataset. The Adult Income dataset contains information about individuals from the 1994 U.S. census. There are 48,842 instances and 14 attributes, including sensitive attributes race and sex. From Adult dataset, we select age, education number of years, relationship, race, sex, capital-gain and hours-per-weet peek to be the decision variables. The binary classification task here is to predict whether an individual makes more or less than $50k per year. The results in Table 2 show that our GFT can achieve the lowest EO score with the minimum sacrifice on the testing error compared with other faimess algorithms, achieving the best performance fairness trade-offs. Additionally, GFT outperforms the four baselines, indicating that our generative fairness teaching is indeed more effective than combing the original and the counterfactual data in mechanical way.
COMPAS Dataset. The ProPublica COMPAS dataset has a total of 7,918 instances, each with 53 features. From COMPAS dataset, select age, race, sex, count of prior offences, charge for which the person was arrested and COMPAS risk score to be the decision variables. The binary target outcome is defined whether or not the defendant recidivated within two years. The ProPublica COMPAS dataset has total of 7,918 8 data instances, each with 53 features. From COMPAS dataset, select age, race, sex, count of prior offences, charge for which the person was arrested and COMPAS risk score to be the decision variables. Experimental results are illustrated in Table 3 where one can observe the GFT is consistently better than other methods in terms of Equalized Odds. Similar to the results in Adult income, we see that GFT achieved the best performance fairness trade-offs among all of the methods tested.
5.2 IMAGE DATA
Experiment Details. We also evaluate our GFT on visual recognition task. In order to generate high quality counterfactual examples, we leverage an U-Net like connections between encoder and decoder. The adversarial classifier implemented on the latent representation of the image a stack of 9 convolution layers followed by fully connected layers. We follow the same baseline settings as in section 55T to combine original and counterfactual images, the other two fairness algorithms are not applicable here due to the format of image data. In addition to the four baselines, we add two different settings. Base5 denotes the baseline model that trains on 90% counterfactual examples and 10% original examples. Training data in Base6 is obtained by maintaining a part of the original examples and adding counterfactual examples to reverse the original biased distribution among pro- tected and unprotected group. The EO score we use here is the sum of the false positive difference
7

---

Under review as a conference paper at ICLR 2021
and true positive difference between protected and unprotected groups. In the visual recognition task, the student model is a VGG-16 network trained using momentum SGD optimizer.
CelebA Dataset. CelebA a commonly used large-scale face attribute dataset. There are 202,599 images, each with 40 binary attributes that reflect appearance (hair color and style, face shape, makeup, for example), emotional state (smiling), gender, attractiveness and age. For this dataset, we use 'Gender' as the binary sensitive attribute. Among the other 39 attributes, We choose one of the most correlated attributes to 'Gender', the 'Arched Eycebrows as our classification target to make this task more challenging. As shown in Table 4 GFT reduced the EO score significantly comparing to the baselines. We also achieved 0.9% improvements on testing error, outperforming the balanced baseline.
Improving Fairness across Models with Different Bias Level. We perform post-processing teaching experiment on four different biased student models. A is the sensitive attribute and Y the classification target. We manually select four pre-training image sets according to the certain ratio shown Figure After training on these dataset respectively, we obtained four student base models with different unfair level. The gap of Equalized odds trend in Figure3 shows that the GFT capable to alleviate the unfairness of various student base models a post-processing manner.
A-D Y-0
A-O Y-1
stdeet
studen2
A=1 Y=0
A=1 Y=1
=bident
akidemn
A=D Y=D
A=D Y=1
Cop Squaaiiddd
Suden: 2
A=1 Y=0
A=1 Y=1
A-O Y-0
A-O Y+1
dndert 3
A-1 Y-O
A-1 Y-1
A-O Y-O
A-O Y-1
A-1 Y-D
A-1 Y-1
20
90
100
Number
Figure 2: Different Unfair Ratio
Figure 3: Post-Processing Result
5.3 ANALYSIS
Counterfactual Examples. We include a qualitative evaluation of our counterfactual generator in Fig These visualizations demonstrate the difference of original images and the counterfactual images by manipulating the binary attributes. We choose the male, young and blonde hair as the sensitive attributes to show the effects of manipulating specific property. One can observe that the target attribute 'Arched Eyebrows' in our recognition task is not visually altered between the original example and the counterfactual one. Powered by generative backbone, our counterfactual examples are of high quality.
Teacher Model. We analyze the training dynamics and the teaching behavior of our GFT model in this subsection for the CelebA dataset. We implement a policy gradient based teacher agent as the data loader to student agent. We shoe negative log reward in Figure the training reward here is the final Equalized Odds score on the CelebA held-out validation set. After 50 episodes, the corresponding EO score will be smaller than 0.20, the unfairness is alleviated compared with the unconstrained baseline 0.67. Figure 6 demonstrates the action adopted by the teacher model. We also demonstrated the percentage (moving average with sample size 7) original image in training the student model. As we defined in section 4.4, the teacher will make binary decision of whether to feed original or counterfactual image to student on each iteration. Since the teacher model interacts with student model during the training process, image is also changing. As training progresses, there has been a gradual decline the use of original images (and thus increase in the counterfactual ones).
observe that the percentage of using original
8

---

Under review as a conference paper at ICLR 2021
oriitoll
male
yourg
blondo
Figure 4: Examples of the counterfactual images on CelebA from the male, young and blonde hair attribute. These result are obtained by our counterfactual generator.
M Now M
Peroentage of Original Image (%)
2.5
Triiting Rowands
2.0
MM
1.5
1.0
120
80
Number of Episode
Number of lleration
Figure 5: Training dynamic
Figure 6: Teacher action
CONCLUSIONS
In this paper, we propose the Generative Fairness Teaching (GFT) framework to achieve algorithmic fairness for machine learning models. Our method can generate high quality counterfactual exam- ples, which is novel approach to compensate the biases in dataset. Together with a student teacher architecture, we dynamically adjust the proportion of counterfactual examples and mix with the original ones in order train a fair model. Experimental results indicated that our method strongly out-perform baseline methods both tabular and real image datasets.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reduc- tions approach fair classification, arXiv preprint arXiv: 0003002453, 2018.
G Stoney Alder and Joseph Gilbert. Achieving ethics and fairness in hiring: Going beyond the law. Journal of Business Ethics, 68(4):449-464, 2006.
Jose A Arjona-Medina, Michael Gillhofer, Michael Widrich, Thomas Unterthiner, Johannes Brand- stetter, and Sepp Hochreiter. Rudder: Return decomposition for delayed rewards. In Advances in Neural Information Processing Systems, pp. 13566-13577, 2019.
Martin Arjovsky, Soumith Chintala, and Léon Bottou.
Wasserstein
gan.
arXiv preprint
arXiv: 07875,, 2017.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems, pp. 4349-43577 2016.
Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commer- cial gender classification. In Conference on fairess, accountability and transparency, pp. 77-91, 2018.
Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush Varshney. Optimized pre-proeessing for discrimination prevention. In Advances in Neural Infor- mation Processing Systems, pp. 3992-4001, 2017.
Pratik Gajane and Mykola Pechenizkiy, On formalizing fairness prediction with machine learning. 2017.
arXiv preprint arXiv:
Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115 (16):E3635-E3644 2018
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of wasserstein gans. Advances neural information processing systems, pp- 5767-5777, 2017.
Sara Hajian and Josep Domingo-Ferrer. A methodology for direct and indirect discrimination pre- vention in data mining. IEEE transactions on knowledge and data engineering, 25(7): 445-1499, 2012.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pp. 3315-3323, 2016.
Lisa Anne Hendricks, Kaylee Burns, Kate Sacnko, Trevor Darrell, and Anna Rohrbach. Women also snowboard: Overcoming bias in captioning models. In European Conference on Computer Vision, pp. 793-811. Springer, 2018
Ray Jiang, Aldo Pacchiano, Tom Stepleton. Heinrich Jiang, and Silvia Chiappa. Wasserstein fair classification. In Uncertainty in Artificial Intelligence, pp. 862-872. PMLR, 2020.
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrim- ination. Knowledge and Information Systems, 33(1):1-33 2012.
Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-aware classifier with prejudice remover regularizer. In Joint European Conference Machine Learning and Knowledge Discovery in Databases, pp. 35-50. Springer, 2012.
Niki Kilbertus, Matco Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. Avoiding discrimination through causal reasoning. In Advances in Neural Information Processing Systems, pp. 656-666, 2017.
Matt Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In Advances in neural information processing systems, pp. 4066-4076, 2017.
10

---

Under review as a conference paper at ICLR 2021
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, pp. 3384-3393, 2018a.
David Madras, Toni Pitassi, and Richard Zemel. Predict responsibly: improving fairness and accu- racy by learning to defer. In Advances in Neural Information Processing Systems, pp. 6147-6157, 2018b.
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. arXiv preprint arXiv: 1908.09635, 2019.
Judea Pearl al. Models, reasoning and inference. Cambridge, UK: CambridgeeUniversttyyreess 2000.
Judea Pearl et al. Causal inference in statistics: An overview. Statistics surveys, 3:96-146, 2009.
Babak Salimi, Luke Rodriguez, Bill Howe, and Dan Suciu. Interventional fairesss: Causal database repair for algorithmic fairness. Proceedings of the 2019 International Conference on Manage- ment of Data, pp. 793-810, 2019.
Dennis Wei, Karthikeyan Natesan Ramamurthy, and Flavio du Pin Calmon. Optimized score trans- formation for fair classification, arXiv preprint arXiv:1906.0000 2019.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai- Wei Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. arXiv preprint 2018.

---

Under review as a conference paper at ICLR 2021
Appendix
A EXPERIMENTAL DETAILS
Encoder
Decoder
Discriminator
Adv. Classifier
Conv [4x4, 64, 2]
DeConv [4x4, 1024, 2]
Conv [4x4, 64, 2]
Convl.x [3x3, 512, x]x3
Conv2 [4x4, 128, 2]
DeConv 4x4, 512, 2]
Conv2 [4x4. 128, 2]
Conv2.x [3x3, 256, 1]x3
Conv3 [4x4, 256, 2]
DeConvi [4x4, 256, 2]
Conv3 [4x4, 256,
Conv3 [4x4, 256,
Conv4 [4x4, 512, 2]
DeConvl [4x4, 128, 2]
Conv4 [4x4, 512, 2]
Conv4 128, 1]
Conv5 [4x4, 1024, 2]
DeConv [4x4, 3,
Conv5 [4x4, 1024, 2]
Conv5 [1x1, [Ix1, n class, 1]
Table 5: Our counterfactual generative model architecture. Conv Conv2.x and Conv3.x denote convolution units that may contain multiple convolution layers. E.g., [4x4, 64, 2]x3 denotes cascaded convolution layers with 64 filters of size 4x4 and stride 2.
Generative model settings. The network architectures used in the paper are elaborated in Table 5 There is an U-Net like connections between Encoder Conv4 layer and Decoder DeConvl layer. Ex- cept the Adversarial Classifier, which uses Stochastic Gradient Descent Optimizer with momentum 0.9. The other module use the Adam Optimizer. We use batch size 64 and start with learning rate le - 4. We will first pre-train the Generator and Discriminator for 60 epochs and fix, then train the Adversarial Classifier solely for 20 epochs, finally we fine-tune Generator and Discriminator again for another 60 epochs.
Teacher and student model settings. The network architecture used in the teacher model is three- layer neural network with layer size d x 15 x n, where d and represent the dimension of state and number of action. VGG-16 is used in the face attribute classification task as the student model. We train the teacher model for 500 episodes, within each episode, the student model is re- re-initialized and trained for 20 epochs. Teacher model and student model are optimized by Adam and Stochastic Gradient Descent Optimizer with momentum 0.9 respectively. We start with learning rate 1e 2 3 for Adam and 0.1 for momentum SGD, divide by 10 when the performance is saturated. We use batch size 64. The terminal reward is measured on the held-out validation set after 20 epochs of student training, the final result is measured on the testing set.
Tabular dataset settings. For the tabular data experiments, we follow the same settings in the original paper (Agarwal et al., 2018) and the official repositories | These settings also include the standard data pre-processing steps, which convert the data into suitable format for the ML algo- rithms. Then the data is randomly split into training and testing set in a ratio of 75% and 25%. The only different setting is the choices of the decision variables. We use age, education number of years, relationship, race, sex, capital-gain and bours-per-week to be the decision variables in the Adult dataset. We use age, race, sex, count of prior offences, charge for which the person was arrested and COMPAS risk score to be the decision variables in the COMPAS dataset.
https://gthib..oommaaiirrrtttenn
12


Reviews:
[{"review_id": "yjbq-5hpTZ", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper describes a pre-processing method to reduce certain statistical disparities in the classifier obtained from the training data. The proposed approach involves learning a latent probability model that simulates the training data. The authors then manipulate the learned model to generate \"counterfactual\" samples that belong to the membership of underrepresented demography. A more \"fair\" classifier is trained on the manipulated data mixing with the \"counterfactual\" samples.\n\nThe proposed approach seems sensible. The experiment results support the claims that it reduces the statistical disparities (e.g., equalized odds) of the trained classifier. I like the simplicity of the proposed method while future work is needed to explicate the theoretical condition under which it is sufficient.\n\nAlso, I believe that the clarity of this paper could be improved. For instance, the definition of counterfactual fairness seems to be orthogonal to the \"counterfactual\" samples described in later sections. More specifically, the counterfactual fairness concerns with the potential outcome of prediction $\\hat Y_{a'}$ has the sensitive attribute $A$ been $a'$. On the other hand, the \"counterfactual\" samples is the potential outcomes of the feature $X_{a'}$ had $A = a'$. It does not seem that the concept of counterfactual fairness has be used in later sections. Therefore, I would suggest that the authors could remove the definition of counterfactual fairness and provide a formal introduction to structural causal models and interventions, e.g., see (Causality, Pearl, 2009). \n\n\n------------ Post Rebuttal ------------------\nI read other reviewers' comments and the authors' responses. I like the idea of applying the GAN framework to compute counterfactual distributions. However, I could also see why other reviewers are not particularly excited about it. The authors managed to apply the GAN approach to obtain counterfactual samples in some specific datasets. However, many questions regarding the proposed methods are left unanswered, e.g., under which condition the proposed GAN approach is ensured to obtain unbiased counterfactual samples. With this being said, I think this paper could be most improved by further elaborating how it contributes to the existing causal inference literature, especially in computing counterfactual probabilities. Due to these reasons, I intend to keep my score but won't strongly champion for it.", "extra_scores": {}, "extra_text": {}}, {"review_id": "cnKmzUPEXan", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper combines counterfactual modeling with adversarial training for fair machine learning tasks. For a given fairness metric chosen from a variety of canonical examples, the method ensures fairness by augmenting the data with counterfactual examples during training. The approach has potential, which is best demonstrated on examples where the counterfactual data generation is interesting, like the CelebA data.\n\nI believe the main weakness of the paper is a low degree of novelty. For example, the idea of training with counterfactual data is present already in the Kusner et al. (2017) reference. The current paper expands the uses of that technique and combines it with an adversarial training architecture. So the strength of this work depends on the suitability of the counterfactual model and training architecture.\n\nThe other data examples, Adult and COMPAS, do not contribute much additional value, particularly in light of my previous point. The paper could be improved by replacing these with one or more examples that better leverage the strength of adversarial training. For ICLR it might be best if these examples use types of data where representations can be useful, like text for example. \n\nThe Section 2.2 comments on in/pre/post-processing were confusing. The method is \"related\" to all of them? Does it not fall into any of the categories?", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZLwRbypCUzT", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a teacher-student framework to ensure fairness by letting the teacher choose examples for the student from either the training data or from a counterfactual distribution. The main contributions are a counterfactual generative model and an algorithm for learning the teacher policy.\n\nStrengths:\n========\n1. The idea seems interesting and the proposed teacher-student framework is novel in the area of fair learning.\n\n2. The authors have done a good job of modeling various aspects of their complex approach using neural networks.\n\n3. The fact that authors were able to make the complex optimization work is itself a good thing since the objective has a lot of moving parts.\n\n4. The presented evaluations also show some promise.\n\nThings to consider improving:\n=========================\n1. My basic question is regarding the motivation for such a framework. Why is this approach important? Is there any fundamentally new insight that can be obtained with a GFT framework that cannot be obtained using existing fair learning approaches.\n\n2. The argument that causal methods cannot benefit commonly used fairness metrics like demographic parity (DP) or equalized odds (EO) should be elaborated. Isn't it possible to create Structural causal models that subsume conditional independences like DP and EO?\n\n3. There are a number of issues with references:\n- There are a number of additional pre-, in-, and post-processing methods that can be cited. Looking into some recent fairness papers may give the authors an idea of the works.\n- Agarwal et al. 2018 is considered an in-processing approach and not post-processing or pre-processing since it imposes fairness criteria while training a fair model.\n- Demographic parity is a really old concept and has been discussed in many papers before Madras et al. 2018.\n- The authors should also consider citing and perhaps comparing with https://homes.cs.washington.edu/~suciu/sigmod-2019-fairness.pdf if it makes sense, since it also does \"database repair\" or pre-processing.\n- Barring some exceptions, many early works in fairness are not cited. For example https://ieeexplore.ieee.org/document/6175897 is a classic work that discusses a lot of concepts in fairness that we use today (they call it discrimination). I recommend the authors to do a thorough literature survey and include at least the important works.\n\n5. Many crucial algorithmic details are missing. While  the complex optimization is important to cover all the aspects of the framework for data generation, it should also be motivated and explained better. \n- In Sec. 3.2.1, how valid is it to assume U to be independent of A? Technically A is a part of the data that is generated so it may be reasonable to surmise that it should depend on the latent. In general the authors should provide a DAG which encodes their assumptions and justify them.\n- How would this architecture be modified for DP since the authors discuss both DP and EO in the beginning?\n- What is L_att trying to optimize?\n- What does it mean to have attribute labels as auxiliary tasks for D (in L_cls)? What are attribute labels?\n- It may make sense to summarize in a few sentences what each term in the objective does. If space  is less, the authors can move some contents to the appendix.\n- In Sec. 3.3, could the authors provide some insights on why such a training works?\n- Please also discuss what REINCFORCE is, and provide reference/more details.\n\n6. Generally in experiments, cross-validated results are needed. It is also crucial to provide sufficient pre-processing and hyper-parameter details to help reproduction of results.\n\n7. There are also a number of recent post-processing methods that can be compared with besides Hardt et al, 2016 (see http://auai.org/uai2019/proceedings/papers/315.pdf, http://proceedings.mlr.press/v108/wei20a.html)\n\n8. More details needed on how Agarwal et al., 2018 was run for example. In https://arxiv.org/pdf/1906.00066.pdf, pg. 30  Agarwal seems to be more competitive than what is shown here. Look at adult-gender-LR-EO and  adult-gender-GBM-EO, Agarwal et al. 2018 (named \"red\"\")  at EO=0.04 has higher  accuracies than shown here. Similar comments apply to COMPAS.\n\n9.There are  also a few other pre-processing approaches that use GANs. \nhttps://arxiv.org/pdf/1805.11202.pdf\nhttps://arxiv.org/pdf/1805.09910.pdf\n\n10. The authors must identify the caveats  of training a model on CelebA which has a \"western\" and \"celebrity\" bias. Models trained there may not transfer to other general settings.\n\n11. In page 8, the analysis of teacher model needs more details. How can we say original images dominate if only 7% image is chosen at max at any iteration? Why is the teacher behavior of choosing real samples in the beginning and synthesized samples later justified?\n\nIn summary, the paper has identified an interesting direction but this needs to be taken forward a bit more.\n\nPost-rebuttal\n===========\nThanks to the  authors for  their detailed response to  my  questions. Some of the answers are indeed satisfactory, but some questions remain - such as extensive comparisons to other methods (probably using more datasets), how  the method would  behave (practically)  with  a different fairness measure like DP, and more  carefully situating the method in  the  fairness literature. I encourage the authors  to keep pursuing this interesting  direction.", "extra_scores": {}, "extra_text": {}}, {"review_id": "-60AhWcnsDx", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes to pair a GAN based model for generating counterfactual samples given protected attribute labels and a reinforced data sampler for choosing whether to let a model train on generated data or original data.   The generative model in of itself is interesting, combining a gan component, and VAE component for analyzing an input image and a mutual information penalty for the VAE hidden vector and protected attribute.  The key to the method seems to be a reinforced data sampler, which picks, when to use a counterfactual sample versus the original sample. Given that the core of what is making this work is the data sampler, I wish there were more details. \n\nHow is the policy parameterized beyond the inputs? \nHow many episodes run out to tune the sampler?\nThe paper says the reward was the fairness measure on the held out set. Do you mean a different held out set or the same final evaluation set?\nI am confused by Figure 6. Is the y-axis the % of samples that left unmodified? If so, then seems like the sampler is essentially saying always use a counterfactual? So I would expect results very similar to all-fake baseline, but in terms of EO, the results are very different. If only a sample number of samples are different than that baseline, how is this possible?\n\nPositives:\n+ the paper proposes an interesting way to generate counterfactual samples\n+ the results seem promising\n\nNegatives:\n- mostly I am left confused how the reinforced data selector is making the method work. Answering some of the above questions would help. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses counterfactual fairness learning using generative approach. While acknowledging the importance and potential usefulness of generative approach, the reviewers and AC raised several important concerns that place this paper below the acceptance bar: 

(1) low degree of novelty – see multiple concerns and suggestions by R2, R3, R4;

(2) the model is not justified by a causal mechanism (R4), and it remains unclear under which condition the proposed GAN approach is ensured to obtain unbiased counterfactual samples (R2); 

 (3) lack of technical rigor when presenting the model – see R4’s request to relate to the DAG models, see R1 multiple questions regarding the reinforced data sampler; 

(4) lack of empirical evidence (R3) and evaluation details, e.g. on cross validation and more recent methods (see R4’s recommendations); 

 (5) related work is not discussed in sufficient details – see R4’s elaborate comment.

Among these, (4,5) did not have a substantial impact on the decision but would be helpful to address in a subsequent revision. However, (1), (2) and (3) make it very difficult to assess the benefits of the proposed approach and were viewed by AC as critical issues.
In the rebuttal it is stated that ‘Our counterfactual examples are generated using a powerful generator rather than a fixed synthesizer in Kusner et al. (2017)’ – more rigorous comparison has to be provided to support such statement. AC would urge the authors to contrast and compare their synthetic counterfactual examples with Kusner et al on the datasets where causal graph has been built. [Razieh Nabi and Ilya Shpitser. Fair inference on outcomes., AAAI2018], Figure 2 postulates causal graphs for the Compas and Adult Income datasets evaluated in this paper.

A general consensus among reviewers and AC suggests, in its current state the manuscript is not ready for a publication. We hope the detailed reviews and encouragements are useful for revising the paper.

Decision: Reject