FR-GAN: F AIR AND R OBUST T RAINING
Anonymous authors
Paper under double-blind review
A BSTRACT
We consider the problem of fair and robust model training in the presence of data poisoning.
Ensuring fairness usually involves a tradeoff against accuracy, so if the data poisoning is mistakenly viewed as additional bias to be ﬁxed, the accuracy will be sacriﬁced even more.
We demonstrate that this phenomenon indeed holds for state-of-the-art model fairness techniques.
We then propose FR-GAN, which holistically performs fair and robust model training using generative adversarial networks (GANs).
We ﬁrst use a generator that attempts to classify examples as accurately as possible.
In addition, we deploy two discriminators: (1) a fairness discriminator that predicts the sensitive attribute from classiﬁcation results and (2) a robustness discriminator that distinguishes examples and predictions from a clean validation set.
Our framework respects all the prominent fairness measures: disparate impact, equalized odds, and equal opportunity.
Also, FR-GAN optimizes fairness without requiring the knowledge of prior statistics of the sensitive attributes.
In our experiments, FR-GAN shows almost no decrease in fairness and accuracy in the presence of data poisoning unlike other state-of-the-art fairness methods, which are vulnerable.
In addition, FR-GAN can be adjusted using parameters to maintain reasonable accuracy and fairness even if the validation set is too small or unavailable.
I NTRODUCTION
As machine learning becomes widespread in the Software 2.0 era (Karpathy, 2017), there is an urgent need to address the issues of fairness and robustness.
For sensitive applications like healthcare, ﬁnance, and self-driving cars, a trained model must not discriminate customers based on sensitive attributes including age, sex, or religion.
In addition, as applications often rely on external data sets for their training data, the model training must be resilient against noisy or even adversarial data.
Traditionally, model fairness research (Venkatasubramanian, 2019; Chouldechova & Roth, 2018;
Verma & Rubin, 2018) has focused on developing metrics such as disparate impact (Feldman et al., 2015), equalized odds (Hardt et al., 2016), and equal opportunity (Hardt et al., 2016), which capture various notions of discrimination.
More recently, there has been a surge in unfairness mitigation techniques (Bellamy et al., 2018), which improve the model fairness by either ﬁxing the training data, training process, or trained model.
Unfairness mitigation usually involves some tradeoff between the model’s accuracy and fairness.
Most recently, generative adversarial networks (GANs) are being adapted to a fairness setting (Zhang et al., 2018).
The architecture of GANs is suitable because it can be used to compete the training between two objectives: accuracy and fairness.
Robust model training is also important and needs to be concurrently taken into consideration.
There has been a proliferation of algorithms that make model training resilient against noisy and even adversarial data (Natarajan et al., 2013; Biggio et al., 2011; Fr´enay & Verleysen, 2014).
The latter scenario may occur when the source of the training data is external, and we cannot control the data that is being added.
For example, any public dataset on the Web may contain intentionallyfalse information.
Moreover, one must have a robust training algorithm if the data set comes from diverse users’ inputs that cannot be fully trusted (Koneˇcn`y et al., 2016).
Recently, data poisoning attacks have become increasingly sophisticated where it is becoming difﬁcult to sanitize the data against all of them (Koh et al., 2018).
In addition, dataset searching is becoming mainstream as demonstrated by Google Dataset Search (Goods)
(Halevy et al., 2016) and its public version for searching scientiﬁc datasets (Noy et al., 2019).
While data lakes within companies may consist of reﬁned datasets, datasets in the public are easy to poison.
And anyone can poison public datasets using attacks in the literature and share them.
We thus believe that it is essential to address both bias and poisoning as a preventive measure.
Solving model fairness without addressing data poisoning may lead to a worse tradeoff between accuracy and fairness.
For example, consider a banking system that is giving out loans where there are two sensitive groups: men and women.
According to disparate impact, the two groups must have similar ratios of positive predictions.
A perfect disparate impact would have a value of 1.
Suppose the clean training data has no bias and a perfect (or high) disparate impact score.
Now suppose this data is poisoned where half of the positive labels of men are ﬂipped to negative labels.
If the model is ﬁtted so as to make more negative predictions on men than women, then disparate impact would worsen.
To address this problem, one may suggest using data sanitization techniques prior to the model training, but it is challenging to do so without any knowledge of the model.
Even though many data sanitization methods are proposed, there have been newer attacks that can easily evade such defenses (Koh et al., 2018).
Our main contribution is the development of a new framework called FR-GAN, which trains accurate models that are also fair and robust to poisoning.
FR-GAN consists of a generator used for classiﬁcation, a discriminator that distinguishes predictions from one sensitive group against others, and a second discriminator that distinguishes { examples, predictions } of the training data from { examples, labels } of a separate and clean validation set.
The ﬁrst discriminator ensures that the prediction ˆ y is independent of the sensitive attribute z .
We show that variations of this approach can be used to maximize any of the following prominent fairness measures: disparate impact, equalized odds, and equal opportunity.
The second discriminator ensures that the model predictions on the training data are “consistent” with labels on clean data.
We theoretically show that FR-GAN optimizes fairness without requiring the knowledge of prior statistics of sensitive attributes.
In addition, the parameters of FR-GAN can be adjusted to maintain reasonable accuracy and fairness even if the validation set is too small or unavailable.
In the following sections, we present the related work, demonstrate the weaknesses of current fairness methods, and propose FR-GAN.
R ELATED W ORK
Model fairness
The notion of discrimination has many deﬁnitions and usually comes from certain social goals that one wants to guarantee.
Fairness research has been focused on deﬁning fairness measures that capture a variety of notions of fairness (Verma & Rubin, 2018). There are largely two types of fairness measures: (1) group fairness (Barocas & Selbst, 2016; Hardt et al., 2016), which ensures similar statistics between two sensitive groups; (2) individual fairness (Dwork et al., 2012), which guarantees similar prediction results across nearby examples. Group fairness measures are widely studied in general applications, even though these cannot be simultaneously satisﬁed in certain circumstances (Kleinberg et al., 2017). In addition, there are also recent signiﬁcant works that focus on individual fairness (Garg et al., 2018; Yurochkin et al., 2019; Kearns et al., 2019; Jung et al., 2019).
Among the two types of fairness with different aspects, we focus on group fairness as our initial research.
Recently, there has also been a surge of research on unfairness mitigation techniques (Bellamy et al., 2018).
Depending on where a ﬁx occurs, there are mainly three approaches: (1) pre -processing techniques (Kamiran & Calders, 2011; du Pin Calmon et al., 2017; Zemel et al., 2013;
Feldman et al., 2015) that ﬁx the training data; (2) in -processing techniques (Zafar et al., 2017; Jiang & Nachum, 2019;
Zhang et al., 2018; Kamishima et al., 2012; Cotter et al., 2019; 2018;
Agarwal et al., 2018) that address the issue during model training; and (3) post -processing techniques (Hardt et al., 2016; Pleiss et al., 2017; Kamiran et al., 2012) that manipulate predictions while maintaining the model.
Among the three, the in-processing techniques have the advantages that one can work with any data and that there is more control on model training (Venkatasubramanian, 2019).
F AIRNESS CONSTRAINTS (Zafar et al., 2017) incorporates a regularization term that reﬂects fairness constraints in the context of convex margin-based classiﬁers such as logistic regression and support vector machines (SVMs).
The main hyperparameter is the λ term, which serves to balance the accuracy and fairness objectives.
L ABEL BIAS CORRECTION (Jiang & Nachum, 2019) assumes the existence of true yet possibly biased labels.
A reweighting algorithm is proposed with theoretical guarantees that training on the resulting loss corresponds to training on the true unbiased labels, which yields a fair model.
Here the hyperparameters include the number of training iterations.
Finally, A DVERSARIAL
[Figure] DEBIASING (Zhang et al., 2018) trains a classiﬁer so that an adversary cannot predict the sensitive attributes based on the output of the classiﬁer.
Hence, the classiﬁer achieves fairness when its predictions do not have any correlation with sensitive attributes.
This idea of adversarial training is similar to GANs (Goodfellow et al., 2014).
A DVERSARIAL DEBIASING provides insights of using adversarial training to minimize mutual information between the prediction and sensitive attributes.
In this paper, we strengthen the theoretical results of A DVERSARIAL DEBIASING using information theory, and this motivates us to provide systematic methodology for various fairness metrics (Section 4.1).
As we demonstrate in Section 3, the existing techniques are not tailored for robust training, so are vulnerable to data poisoning attacks.
In comparison, FR-GAN addresses both model fairness and robust training within the same model training process.
Robust training There is a heavy literature on how to make the model training robust against noisy or even adversarial data (Natarajan et al., 2013; Biggio et al., 2011; Fr ´ enay & Verleysen, 2014;
Kurakin et al., 2017).
A major challenge is that there can be a wide range of data poisoning attacks that keep on evolving.
One defense approach is to sanitize the training data and thus ﬁx the root cause.
A fundamental limitation is that we do not know in general what kind of poisoning will occur, so a universal defense against all possible attacks does not seem feasible as demonstrated by Koh et al.
A more recent trend is to develop general defense algorithms for any attack during model training using meta learning (Veit et al., 2017; Li et al., 2017; Xiao et al., 2015;
Hendrycks et al., 2018).
Ren et al.
(2018) propose a meta-learning framework that employs a clean validation set for the purpose of preventing the model training from being inﬂuenced by poisoned data.
By using the validation loss as a meta objective, it intends to defend against any poisoning attack.
Our FR-GAN framework is inspired by this approach, but employs a GAN-based model to support fair and robust training without using meta learning.
In particular, the design of FR-GAN’s robustness discriminator is based on mutual-information-based theoretical insights (Section 4.2).
Another line of research is defending against adversarial attacks during test time (Biggio et al., 2013; Goodfellow et al., 2015; Wong & Kolter, 2018).
In comparison, our focus is on defending against data poisoning on the training data.
V ULNERABILITY OF FAIRNESS METHODS
We perform experiments to demonstrate that state-of-the-art fairness methods are indeed vulnerable even to simple poisoning attacks.
We generate a synthetic dataset as shown in Figure 1a (see the generation details in Section 5.1).
There are two non-sensitive attributes a and b , which are reﬂected in the x-axis and y-axis, respectively.
The examples are further divided into two classes based on the sensitive attribute z .
To generate poisoned data, we measure accuracy degradation for a ﬂipping of each z . We then choose the top-10% of the z values with the highest degradation and ﬂip them.
This approach is similar to the existing label ﬂipping method (Paudice et al., 2018), except that the [Figure] z value is being ﬂipped instead of the label so that the model fairness is also affected.
To make a validation set, we randomly select clean examples that amount to 10% of the entire training data.
We use disparate impact (Feldman et al., 2015) as a fairness measure and evaluate one fairness method, called F AIRNESS CONSTRAINTS (Zafar et al., 2017).
As this method involves a regularization factor λ that balances the accuracy and fairness objectives, we can obtain a tradeoff curve by adjusting its value.
Figure 1b shows two accuracy-fairness tradeoff curves obtained with the clean and poisoned synthetic datasets.
Notice that adding data poisoning clearly shifts the curve to the left, which means both accuracy and fairness decrease.
This coincides with our intuition.
The poisoning confuses the model so that there are more biased examples to ﬁx, which in turn makes it overreact and thus sacriﬁce more on accuracy.
In the supplementary, we show similar trends when the data is poisoned via label ﬂipping.
In Section 5, we will show how data poisoning affects L ABEL BIAS
CORRECTION (Jiang & Nachum, 2019) and A DVERSARIAL DEBIASING (Zhang et al., 2018).
FR-GAN
We now describe FR-GAN, which is shown in Figure 2.
Unlike traditional GANs, the generator is a classiﬁer that receives an example x ∈ X and returns a prediction ˆ y .
The other two discriminators optimize fairness and robustness as we describe in the following sections.
4. F AIRNESS
We denote by D tr the training data set.
Suppose D tr has m examples { ( x ( i ) , z ( i ) , y ( i ) ) } mi =1 where x ( i ) contains the non-sensitive attributes, z ( i ) contains the sensitive attributes, and y ( i ) is the label.
Both the sensitive attribute and label can be multi-class, i.e., they can have one of multiple values.
For notational simplicity, we assume there is one sensitive attribute, which can be viewed as a merged result of multiple sensitive attributes with a larger alphabet size.
For illustrative purposes, we focus on disparate impact, leaving in the supplementary our formulation for equalized odds and equal opportunity.
The ﬁrst discriminator in FR-GAN distinguishes predictions w.r.t.
one sensitive group from those in the others.
Disparate impact intends the sensitive attribute to be independent of the model’s prediction, i.e.,
I ( Z ; ˆ Y ) = 0 .
We explain how FR-GAN can enforce the above constraint.
Let P Z ( z ) be the distribution of Z where z ∈ Z and Z is the set of possible sensitive attribute values.
Let ˆ Y | Z
=
z ∼ P ˆ Y |
z ( · ) and ˆ Y ∼ P
ˆ Y ( · ) .
Then P ˆ Y ( · ) =
(cid:80) z ∈Z P Z ( z ) P ˆ Y | z ( · ) .
We show that mutual information is equivalent to the following expression where the optimal discriminator D (cid:63)z
[Equation]
= P
[Equation] ,Z ( [Equation] ,z ) P ˆ Y (ˆ
[Equation]
and
(cid:80) z ∈Z D (cid:63)z (ˆ y ) =
1 ∀ ˆ y ∈ Y .
Theorem 1.
I
[Equation]
[Equation]
While deferring the detailed proof to the supplemental materials, we provide a brief overview of the proof.
As the optimization problem in the RHS is convex, we ﬁnd the optimal dicriminator by solving the KKT conditions.
We then show that the maximum value attained by the optimal discriminator is equal to the mutual information by using the properties of mutual information and the generalized Jensen-Shannon divergence (Lin, 1991).
What is more involved than showing the above equality is designing a right optimization problem.
One needs to carefully handcraft a plausible optimization problem so that its unique solution matches the desired quantity.
Here, we design the above optimization problem via a ‘guess-&-check’ approach aided by the structural insights across the KL divergences that appear in an alternative expression of mutual information.
We now discuss how to implement the above expression.
Since we do not know P ˆ Y | z ( · ) exactly, we compute the following empirical version:
[Equation]
Now for sufﬁciently large m , the number m z of examples with z ( i ) =
z is approximately the same as P Z ( z ) m .
Therefore, the above expression becomes:
[Equation]
Interestingly, this formulation is exactly the same as that in the original GAN (Goodfellow et al., 2014) when |Z| = 2 .
We also remark that our formulation does not require a prior knowledge on
P Z ( z ) .
4. R OBUSTNESS
We utilize a clean yet small validation set to ensure robust training.
In particular, we add a second discriminator that distinguishes the training data with predictions { ( x ( i ) , z
( i ) , ˆ y ( i ) ) } mi =1 from the validation set { ( x ( i ) val , z ( i ) val , y
( i ) val ) } m val
i =1 .
Intuitively, if the classiﬁer is confused by data poisoning in the training data, then its predictions will not be consistent with the labels of the clean data, and the discriminator would be able to detect that difference.
While using a validation set is inspired by the meta learning approach by Ren et al.
(2018), we take an adversarial learning approach.
A key motivation behind this choice is that our GAN approach introduces a knob that controls the emphasis of robust training.
We ﬁnd that this knob enables FR-GAN to be more robust to the validation set size.
See details in Section 5.1.
Our formulation is as follows.
Let V ∼ Bern ( β ) denote whether an example comes from training data ( V = 1 ) or from the validation set ( V = 0 ).
We then want to ensure that V is independent of the training data and predictions, i.e., I ( V ; X, Z, ˆ Y )
= 0 , which means the predictions on the training data are indistinguishable from labels of the validation set.
This way we can mimic the clean data set while expecting an indirect sanitization effect.
We optimize this independence using a formulation similar to that of fairness.
Another possible formulation is to use the hard decision value from D ( ˆ Y ) , say ˆ Z , instead of Z .
However, this discriminator can only be trained after the fairness discriminator is trained.
Instead, we take an approach that allows us to train the two discriminators at the same time.
4. A RCHITECTURE
Figure 2 illustrates the architecture of FR-GAN.
For the loss function in the generator, we employ the cross entropy loss:
[Equation]
As per Theorem 1, we set the loss function w.r.t.
the fairness discriminator as:
[Equation]
where D ( · ) :
= ( D 1 ( · ) , . . .
, D |Z| ( · )) .
The condition (cid:80) z ∈Z D (cid:63)z ( ˆ Y )
= 1 can be enforced by adding a softmax layer to the discriminator. [Table]
0.763 (7.18% ↓ ) 0.651 (19.2% ↓ )
LBC+LD 0.
0.513 (37.4% ↓ ) 0.859 (13.0% ↑ )
AD+LD 0. 0.730 (9.54% ↓ ) 0.818 (0.86% ↑ ) Both FR-GAN 0.
0.830 (2.09% ↑ ) 0.808 (0.12% ↓ )
Finally, implementing I ( V ; X, Z, ˆ Y ) as advised by Theorem 1, we set the loss function w.r.t.
the robustness discriminator as:
[Equation]
The ﬁnal objective function is to minimize the weighted sum of these value functions:
[Equation]
Here λ 1 and λ 2 are tuning knobs that play roles to emphasize fair and robust training, respectively.
E XPERIMENTS
We provide experimental results for disparate impact, while leaving in the supplementary those for equalized odds and equal opportunity. All of our results are on a separate test set.
More implementation details are also in the supplementary.
5. S YNTHETIC DATA RESULTS
For the synthetic data, we generate 2,000 examples with two non-sensitive attributes a and b , a sensitive attribute z , and a label y , using a method similar to the algorithm proposed by Zafar et al.
Both z and y are binary and generated uniformly at random.
For the four possible ( z , y ) combinations, we generate different normal distributions for a and b .
Finally for each example, the a and b values are sampled as per the normal distribution associated with the ( z , y ) pair.
For data poisoning, we ﬂip the z values of examples so as to lower the model’s accuracy the most as described in Section 3.
To make a fair comparison in the context of robust training, we also improve the existing fairness methods with data sanitization and compare them with FR-GAN.
Since FR-GAN is one of the ﬁrst works to address both fairness and robustness, there are few baselines to compare with.
Hence, we employed one reasonable baseline, which ﬁrst sanitizes the poisoned data using a well-known sanitization technique and then performs each fairness algorithm.
There are several sanitization techniques surveyed in Koh et al.
and we choose the loss defense method.
Here examples that [Figure] are not well ﬁt by the model and result in a high loss are considered poisoned and are subsequently discarded.
Loss defense is the only method that utilizes the trained model and thus has the best chance of defending poisoning attacks.
Accuracy and fairness We compare FR-GAN with F AIRNESS CONSTRAINTS (Zafar et al., 2017), L ABEL BIAS CORRECTION (Jiang & Nachum, 2019), A DVERSARIAL DEBIASING (Zhang et al., 2018), and a plain non-fairness method, logistic regression.
We also compare with the meta learning approach by Ren et al.
(2018), which focuses on robust training.
For this comparison, we tailor our framework to a setting where λ 1 =
(no fairness constraint), and name it R-GAN.
For
FR-GAN, we use a validation set that amounts to 10% of D tr .
When setting λ 1 and λ 2 , we usually ﬁx λ 2 to some value and then adjust λ 1 .
For the clean data, we apply proper hyperparameters so that the disparate impacts are similar (around 0 . 8 ) across all distinct methods.
We do the same for the poisoned data.
There is no such hyperparameter tuning for logistic regression and the meta learning by Ren et al.
(2018), as the frameworks have no knob for adjusting fairness.
We ﬁnd that this leads to poor disparate impacts.
For the three fairness methods, data poisoning aggravates performances in accuracy, fairness or both.
For example, the disparate impact and accuracy of F AIRNESS CONSTRAINTS fall by 8.03% and 18.7%, respectively.
On the other hand, the performance changes for
FR-GAN are negligible: disparate impact increases by 2.09%, and accuracy decreases by 0.12%.
Table 1 also shows that combining the fairness methods with loss defense (rows 6–8) does not always yield better accuracy and fairness.
In fact, using the loss defense may lower the accuracy or fairness (e.g., LBC+LD has a disparate impact of 0.513 on poisoned data while LBC has 0.816).
Perhaps this is because the data sanitization sometimes mistakenly discards clean data as well.
The results demonstrate that even a data sanitization technique may not help much.
In the supplementary, we also perform experiments using equalized odds and show that FR-GAN has similar beneﬁts.
We observe how accuracy trades off with fairness on clean and poisoned datasets.
The results for
F AIRNESS CONSTRAINTS are shown in Figure 1b.
For L ABEL BIAS CORRECTION , we employ the number of training iterations as a knob to trade accuracy off fairness.
As shown in Figure 3a, the tradeoff curve shifts to the left, which demonstrates a clear tradeoff degradation.
For A DVERSARIAL
DEBIASING , we employ the α parameter (Zhang et al., 2018) analogous to λ 1 as a knob to trade accuracy off fairness.
We see in Figure 3b that the tradeoff curve again shifts to the left.
Validation set size Figures 3c to 3e show how the validation set size affects the robustness of FRGAN.
In particular, we compare the accuracy-fairness tradeoff of FR-GAN on clean data and that on poisoned data while varying the size of the validation set.
When running on poisoned data, we ﬁxed λ 2 = 0 .
2 and varied λ 1 .
We see that even 2% validation set (Figure 3d) is sufﬁcient to maintain the accuracy and fairness obtained on the clean data.
When using 0.5% (Figure 3e), the validation set is too small and has an adverse effect on the training.
However, our framework includes a tuning knob λ 2 , so by decreasing λ 2 down to 0.1, we could de-emphasize robust training, thereby avoiding the adverse effect (Figure 3f).
This is in contrast to the meta learning approach, which suffers from a non-negligible performance degradation for a very small validation set.
See details in the supplementary.
The results show that only a holistic framework like FR-GAN can achieve both excellent model fairness and training robustness.
In comparison, other methods tailored for only one of these objectives lose either accuracy, fairness or both.
Ablation study (Without ‘R’)
For a small value of λ 2 , we observe in Figure 3f that the accuracyfairness tradeoff curve shifts to the left, just like other fairness-only methods.
To make the comparison clearer, we also plot them together in the supplementary.
(Without ‘F’)
When λ 1 = 0 , FR-GAN is expected to behave similarly to the other non-fair algorithms.
The performance of FR-GAN with λ 1 = 0 , which we dub as R-GAN, is reported in Table 1.
One can observe that R-GAN achieves a poor fairness performance just like the other non-fair algorithms.
5. R EAL DATA RESULTS
We use the following benchmark datasets: ProPublica COMPAS (Angwin et al., 2016) and Adult
Census (Kohavi, 1996).
For sensitive attributes, we use SEX for both datasets.
We do not employ the German Credit data set (Dua & Graff, 2017) as its highly biased towards positive labels.
With such dataset, it is nearly impossible to observe any meaningful tradeoff.
For data poisoning, we use the same method employed on synthetic data: ﬂipping the z values so as to maximize the accuracy performance degradation.
While there may be stronger attacks, this poisoning attack is effective enough to fool the three fairness methods.
In the supplementary, we also evaluate FR-GAN against label ﬂipping attacks and show similar results.
We use the same constructions for the validation set and poisoning as described in Section 3.
Accuracy and fairness See Table 2.
As in Table 1, we apply proper hyperparameters so that disparate impacts are similar across all distinct methods, both for the clean and poisoned datasets.
The results are similar to Table 1: logistic regression and meta learning exhibit poor disparate impacts; the three fairness methods have worse disparate impact and accuracy due to data poisoning; and FR-GAN again shows little degradation both in fairness and accuracy.
Table 2 also shows that combining the fairness methods with loss defense (rows 6–8 and 15–17) does not always yield better accuracy and fairness and may even lower them, which is consistent to the results on synthetic data.
D ISCUSSION
In this section, we discuss possible ways to extend FR-GAN. Subjective labels FR-GAN can be generalized to a setting where labels are subjective instead of being clean or poisoned. Here the dataset without the undesirable human biases becomes the clean validation set.
In general, given two datasets with different distributions where one of them is desirable, FR-GAN can train robustly against the other distribution.
Constructing real datasets We propose a general methodology for constructing a real dataset that has both bias and poisoning, as well as a “clean” validation set.
Suppose that we are performing evaluation tasks (say loan decisions) where there is a high chance of human bias (i.e., poisoning) due to the high workload.
We can construct a clean validation set by selecting a few tasks, assigning more evaluators, and taking a majority vote of the evaluations to minimize the bias.
While employing more evaluators can be expensive, constructing a small validation set is sufﬁcient for robust training, which makes this method practical.
Generalizing to all attacks An interesting question is whether FR-GAN can defend against all poisoning attacks.
If we know all the possible attacks, we can construct a training set containing these attacks.
In the more challenging case where we do not know which attacks even exist, there seems to be a fundamental limitation in protecting against the attacks.
Generalization is a universal problem in machine learning where a model trained on one dataset is not guaranteed to perform well in another dataset with a different distribution.
Although the generalization is a critical issue to address, it is currently beyond the scope of this paper.
[Table]
We proposed FR-GAN, which is a holistic framework that enables both unfairness mitigation and robust training.
Our key contribution is proposing a novel GAN architecture that enjoys the synergistic effect of combining two approaches: (1) employing a fairness discriminator that distinguishes predictions w.r.t.
one sensitive group from others and (2) employing a robust discriminator that distinguishes training data with predictions from a clean yet small validation set.
We showed that existing fairness methods are vulnerable to data poisoning, even when combined with data sanitization techniques.
In comparison, FR-GAN is robust to the poisoning and can be adjusted to maintain reasonable accuracy and fairness even if the validation set is too small or unavailable.
We showed that FR-GAN is guaranteed to maximize fairness and supports all the prominent fairness measures: disparate impact, equalized odds, and equal opportunity.
A A PPENDIX A. P ROOF FOR T HEOREM
Before we present the proof of the main theorem
[Equation]
we ﬁrst recall our notation.
Let P Z ( z ) be the distribution of Z where z ∈ Z and Z is the set of possible sensitive attribute values.
Let ˆ Y | Z
=
z ∼ P ˆ Y |
z ( · ) and ˆ Y ∼ P
ˆ Y ( · ) .
Then P ˆ Y ( · ) =
(cid:80) z ∈Z P Z ( z ) P ˆ Y | z ( · ) .
Also, let Y ∼ P Y ( · ) .
For convenience
[Equation]
let us repeat the statement of Theorem 1 here:
[Equation]
We now prove the theorem.
Proof.
Denote by D the collection of D z (ˆ y ) for all possible values of z and ˆ y
[Equation]
and by ν the collection of ν ˆ y for all values of ˆ y .
We can construct the Lagrangian function as follows:
[Equation]
We use the following KKT conditions:
[Equation]
∀ ˆ y ∈ Y .
Solving the two equations, we obtain ν (cid:63) ˆ y
=
P ˆ Y (ˆ y ) for all ˆ y .
Thus,
[Equation]
Putting this to the above optimization,
[Equation]
Here, the second last equality is due to the deﬁnition of the generalized JS divergence
[Equation]
and the last equality is due to its equivalence to the mutual information (Lin, 1991).
A. E XTENSIONS TO OTHER FAIRNESS MEASURES
The following theorem relates the conditional mutual information I ( Z ; ˆ Y | Y ) to the solution of an optimization problem.
Theorem 2.
[Equation]
Recall that this conditional mutual information term can be used to capture equalized odds , which is another important fairness metric.
We also note that the following theorem can be modiﬁed in a straightforward manner so that it can handle I ( Z ; ˆ Y | Y = 1) , which can be used to capture equal opportunity .
We now prove the theorem.
Proof.
Denote by D the collection of D z | y
(ˆ y ) for all possible values of ( z , ˆ y , and y ) and by ν the collection of ν y, ˆ y for all values of y and ˆ y .
We can construct the Lagrangian function as follows:
[Equation]
We use the following KKT conditions:
[Equation]
Solving the two equations
[Equation]
we obtain ν (cid:63)y, ˆ y =
P Y, ˆ Y ( y, ˆ y ) for all ( y, ˆ y
) ∈
Y × Y .
Thus,
[Equation]
Y × Y .
Putting this to the above optimization,
[Equation]
The third last equality is due to the deﬁnition of the generalized JS divergence; the second last equality is due to its equivalence to the mutual information (Lin, 1991); and the last equality is due to the deﬁnition of conditional mutual information.
We now discuss how to actually compute the mutual information.
We compute the following empirical version using the example { ( x ( i ) , z ( i ) , y ( i ) ) } mi =1 .
[Equation]
[Figure] [Figure]
Now for a sufﬁciently large value of m
[Figure]
m y,z ≈ P Y,Z ( y, z ) m .
Therefore, the above expression is approximated as:
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
Hence, we can set L 2 (i.e., the loss w.r.t.
the fairness discriminator) to the above expression.
The rest of the objective function is the same.
Figure 4 shows the resulting FR-GAN architecture.
A. A DDITIONAL EXPERIMENTS A.3. S YNTHETIC DATA
We continue our experiments from Sections 3 and 5.1.
Label ﬂipping for data poisoning Figure 5a to 5d show the experimental results when the data is poisoned via label ﬂipping.
We used the same synthetic data, as illustrated in Figure 1a, and ﬂipped 7% of labels from the clean data to make a poisoned data.
Figures 5a, 5b, and 5c show the accuracy-fairness tradeoff curves of F AIRNESS C ONSTRAINTS , L ABEL B IAS C ORRECTION , and
A DVERSARIAL D EBIASING , respectively.
While the tradeoff curves in Figure 5a to 5c considerably shift to the left, Figure 5d shows that FR-GAN is noticeably robust against label ﬂipping.
[Figure]
[Table]
[Figure]
Meta learning with different validation set sizes Table 3 shows the accuracy and fairness results for the meta learning method by Ren et al.
(2018) (ML) for different validation set sizes.
We observe a drastic decrease of accuracy when the validation set size is 0.1% of the training data.
Ablation study for fairness Figure 6 shows the accuracy-fairness tradeoff curves for A DVERSARIAL D EBIASING and FR-GAN on the synthetic dataset.
A DVERSARIAL D EBIASING tradeoff curves are from Figure 3b
[Figure]
and the FR-GAN curve comes from Figure 3f.
We observe in Figure 6 that the accuracy-fairness tradeoff of FR-GAN using a small value of λ 2 on the poisoned data is similar to the results of fairness-only methods.
A.3. R EAL DATA
We continue our experiments from Section 5.2.
Label ﬂipping for data poisoning Table 4 uses the same experimental setting as in Table 2, except that data poisoning is label ﬂipping (not w.r.t.
sensitive attributes).
We observe similar trends as in Table 2.
For the non-fairness methods, the fairness results are signiﬁcantly worse than other methods.
For the fairness-only methods, both fairness and accuracy tend to be high on the clean data, but at least one of them decreases signiﬁcantly on the poisoned data.
On the other hand,
FR-GAN yields high fairness and accuracy on the clean data and is also the most resilient against poisoned data.
The results emphasize our holistic technique that takes both fairness and accuracy into consideration.
A.3. FR-GAN USING OTHER FAIRNESS MEASURES
As we showed in Appendix A.2, FR-GAN respects equalized odds and equal opportunity.
Table 5 shows the experimental results on the synthetic and real datasets for equalized odds.
We see that
FR-GAN signiﬁcantly improves equalized odds with reasonable accuracy.
The results w.r.t.
equal opportunity are similar and thus not shown here.
[Table]
[Table]
A. T RAINING METHODOLOGY
We implement all algorithms in PyTorch (Paszke et al., 2017)
[Table]
and all experiments are performed on a server with Intel i7-6850 CPUs.
The generator G is a neural network with zero or one hidden layer.
The discriminator D f is a single layer neural network
[Table]
and the discriminator D r is a neural network with one hidden layer.
We usedor 16 nodes in the hidden layers.
We set an Adam optimizer (Kingma & Ba, 2014) for the generator, [Table]
[Table]
and a stochastic gradient descent (SGD) optimizer for each discriminator.
We empirically observe that one can stabilize the training procedure by freezing the parameters of the fairness discriminator D f for the initial phase of training.
Thus, we choose to freeze the parameters of the fairness discriminator D f for the ﬁrst few epochs until the generator achieves a certain accuracy.
We use the generator/discriminator update ratio of 1:1 for the ﬁrst 300 epochs and use the update ratio of 1: (or 1:5) for the rest of training.
Also, we use the following details for choosing the values of λ 1 and λ 2 .
For clean data, we set λas a small value (e.g., 0 . 01 ) and vary λ 1 from 0 to 0 .
For poisoned data, we set λ 2 as 0 . 2 , 0 . 3 , or 0 . 4 , and vary λ 1 from 0 to 0 .
95 − λ 2 .
Given the values of λ 1 and λ 2 , we also normalize L 1 (the generator loss) by multiplying it with (1 − λ 1 − λ 2 ) .
A. A DDITIONAL EXPERIMENTS ON A DULT DATASET
We compare FR-GAN and AD using the AdultCensus dataset and the same poisoning in Section 3.
For a fair comparison, we use loss defense (LD) as data sanitization prior to running AD.
We generate confusion matrices both for disparate impact (DI) and equalized odds (EO).
Overall, FRGAN performs better than AD because of its robustness discriminator.
The robustness discriminator successfully ignores the poisoned distribution in the training data, and FR-GAN’s TPR is higher than AD with sanitization.
Table 6 shows confusion matrices as results of disparate-impact-focused experiment.
The results are reported when AD achieves (Acc, DI) = (0.728, 0.821), and FR-GAN achieves (Acc, DI) =
In addition, Table 7 represents confusion matrices as results of equalized-odds-focused experiment.
The results are reported when AD achieves (Acc, EO
when y = 1, EO when y = 0) = (0.788, 0.844, 0.750), and FR-GAN achieves (Acc, EO
when y = 1, EO when y = 0) = (0.838, 0.824, 0.773).