FR-GAN: Fair and Robust Training

1 INTRODUCTION . As machine learning becomes widespread in the Software 2.0 era ( Karpathy , 2017 ) , there is an urgent need to address the issues of fairness and robustness . For sensitive applications like healthcare , finance , and self-driving cars , a trained model must not discriminate customers based on sensitive attributes including age , sex , or religion . In addition , as applications often rely on external data sets for their training data , the model training must be resilient against noisy or even adversarial data . Traditionally , model fairness research ( Venkatasubramanian , 2019 ; Chouldechova & Roth , 2018 ; Verma & Rubin , 2018 ) has focused on developing metrics such as disparate impact ( Feldman et al. , 2015 ) , equalized odds ( Hardt et al. , 2016 ) , and equal opportunity ( Hardt et al. , 2016 ) , which capture various notions of discrimination . More recently , there has been a surge in unfairness mitigation techniques ( Bellamy et al. , 2018 ) , which improve the model fairness by either fixing the training data , training process , or trained model . Unfairness mitigation usually involves some tradeoff between the model ’ s accuracy and fairness . Most recently , generative adversarial networks ( GANs ) are being adapted to a fairness setting ( Zhang et al. , 2018 ) . The architecture of GANs is suitable because it can be used to compete the training between two objectives : accuracy and fairness . Robust model training is also important and needs to be concurrently taken into consideration . There has been a proliferation of algorithms that make model training resilient against noisy and even adversarial data ( Natarajan et al. , 2013 ; Biggio et al. , 2011 ; Frénay & Verleysen , 2014 ) . The latter scenario may occur when the source of the training data is external , and we can not control the data that is being added . For example , any public dataset on the Web may contain intentionallyfalse information . Moreover , one must have a robust training algorithm if the data set comes from diverse users ’ inputs that can not be fully trusted ( Konečnỳ et al. , 2016 ) . Recently , data poisoning attacks have become increasingly sophisticated where it is becoming difficult to sanitize the data against all of them ( Koh et al. , 2018 ) . In addition , dataset searching is becoming mainstream as demonstrated by Google Dataset Search ( Goods ) ( Halevy et al. , 2016 ) and its public version for searching scientific datasets ( Noy et al. , 2019 ) . While data lakes within companies may consist of refined datasets , datasets in the public are easy to poison . And anyone can poison public datasets using attacks in the literature and share them . We thus believe that it is essential to address both bias and poisoning as a preventive measure . Solving model fairness without addressing data poisoning may lead to a worse tradeoff between accuracy and fairness . For example , consider a banking system that is giving out loans where there are two sensitive groups : men and women . According to disparate impact , the two groups must have similar ratios of positive predictions . A perfect disparate impact would have a value of 1 . Suppose the clean training data has no bias and a perfect ( or high ) disparate impact score . Now suppose this data is poisoned where half of the positive labels of men are flipped to negative labels . If the model is fitted so as to make more negative predictions on men than women , then disparate impact would worsen . To address this problem , one may suggest using data sanitization techniques prior to the model training , but it is challenging to do so without any knowledge of the model . Even though many data sanitization methods are proposed , there have been newer attacks that can easily evade such defenses ( Koh et al. , 2018 ) . Our main contribution is the development of a new framework called FR-GAN , which trains accurate models that are also fair and robust to poisoning . FR-GAN consists of a generator used for classification , a discriminator that distinguishes predictions from one sensitive group against others , and a second discriminator that distinguishes { examples , predictions } of the training data from { examples , labels } of a separate and clean validation set . The first discriminator ensures that the prediction ŷ is independent of the sensitive attribute z . We show that variations of this approach can be used to maximize any of the following prominent fairness measures : disparate impact , equalized odds , and equal opportunity . The second discriminator ensures that the model predictions on the training data are “ consistent ” with labels on clean data . We theoretically show that FR-GAN optimizes fairness without requiring the knowledge of prior statistics of sensitive attributes . In addition , the parameters of FR-GAN can be adjusted to maintain reasonable accuracy and fairness even if the validation set is too small or unavailable . In the following sections , we present the related work , demonstrate the weaknesses of current fairness methods , and propose FR-GAN . 2 RELATED WORK . Model fairness The notion of discrimination has many definitions and usually comes from certain social goals that one wants to guarantee . Fairness research has been focused on defining fairness measures that capture a variety of notions of fairness ( Verma & Rubin , 2018 ) . There are largely two types of fairness measures : ( 1 ) group fairness ( Barocas & Selbst , 2016 ; Hardt et al. , 2016 ) , which ensures similar statistics between two sensitive groups ; ( 2 ) individual fairness ( Dwork et al. , 2012 ) , which guarantees similar prediction results across nearby examples . Group fairness measures are widely studied in general applications , even though these can not be simultaneously satisfied in certain circumstances ( Kleinberg et al. , 2017 ) . In addition , there are also recent significant works that focus on individual fairness ( Garg et al. , 2018 ; Yurochkin et al. , 2019 ; Kearns et al. , 2019 ; Jung et al. , 2019 ) . Among the two types of fairness with different aspects , we focus on group fairness as our initial research . Recently , there has also been a surge of research on unfairness mitigation techniques ( Bellamy et al. , 2018 ) . Depending on where a fix occurs , there are mainly three approaches : ( 1 ) pre-processing techniques ( Kamiran & Calders , 2011 ; du Pin Calmon et al. , 2017 ; Zemel et al. , 2013 ; Feldman et al. , 2015 ) that fix the training data ; ( 2 ) in-processing techniques ( Zafar et al. , 2017 ; Jiang & Nachum , 2019 ; Zhang et al. , 2018 ; Kamishima et al. , 2012 ; Cotter et al. , 2019 ; 2018 ; Agarwal et al. , 2018 ) that address the issue during model training ; and ( 3 ) post-processing techniques ( Hardt et al. , 2016 ; Pleiss et al. , 2017 ; Kamiran et al. , 2012 ) that manipulate predictions while maintaining the model . Among the three , the in-processing techniques have the advantages that one can work with any data and that there is more control on model training ( Venkatasubramanian , 2019 ) . FAIRNESS CONSTRAINTS ( Zafar et al. , 2017 ) incorporates a regularization term that reflects fairness constraints in the context of convex margin-based classifiers such as logistic regression and support vector machines ( SVMs ) . The main hyperparameter is the λ term , which serves to balance the accuracy and fairness objectives . LABEL BIAS CORRECTION ( Jiang & Nachum , 2019 ) assumes the existence of true yet possibly biased labels . A reweighting algorithm is proposed with theoretical guarantees that training on the resulting loss corresponds to training on the true unbiased labels , which yields a fair model . Here the hyperparameters include the number of training iterations . Finally , ADVERSARIAL DEBIASING ( Zhang et al. , 2018 ) trains a classifier so that an adversary can not predict the sensitive attributes based on the output of the classifier . Hence , the classifier achieves fairness when its predictions do not have any correlation with sensitive attributes . This idea of adversarial training is similar to GANs ( Goodfellow et al. , 2014 ) . ADVERSARIAL DEBIASING provides insights of using adversarial training to minimize mutual information between the prediction and sensitive attributes . In this paper , we strengthen the theoretical results of ADVERSARIAL DEBIASING using information theory , and this motivates us to provide systematic methodology for various fairness metrics ( Section 4.1 ) . As we demonstrate in Section 3 , the existing techniques are not tailored for robust training , so are vulnerable to data poisoning attacks . In comparison , FR-GAN addresses both model fairness and robust training within the same model training process . Robust training There is a heavy literature on how to make the model training robust against noisy or even adversarial data ( Natarajan et al. , 2013 ; Biggio et al. , 2011 ; Frénay & Verleysen , 2014 ; Kurakin et al. , 2017 ) . A major challenge is that there can be a wide range of data poisoning attacks that keep on evolving . One defense approach is to sanitize the training data and thus fix the root cause . A fundamental limitation is that we do not know in general what kind of poisoning will occur , so a universal defense against all possible attacks does not seem feasible as demonstrated by Koh et al . ( 2018 ) . A more recent trend is to develop general defense algorithms for any attack during model training using meta learning ( Veit et al. , 2017 ; Li et al. , 2017 ; Xiao et al. , 2015 ; Hendrycks et al. , 2018 ) . Ren et al . ( 2018 ) propose a meta-learning framework that employs a clean validation set for the purpose of preventing the model training from being influenced by poisoned data . By using the validation loss as a meta objective , it intends to defend against any poisoning attack . Our FR-GAN framework is inspired by this approach , but employs a GAN-based model to support fair and robust training without using meta learning . In particular , the design of FR-GAN ’ s robustness discriminator is based on mutual-information-based theoretical insights ( Section 4.2 ) . Another line of research is defending against adversarial attacks during test time ( Biggio et al. , 2013 ; Goodfellow et al. , 2015 ; Wong & Kolter , 2018 ) . In comparison , our focus is on defending against data poisoning on the training data . 3 VULNERABILITY OF FAIRNESS METHODS . We perform experiments to demonstrate that state-of-the-art fairness methods are indeed vulnerable even to simple poisoning attacks . We generate a synthetic dataset as shown in Figure 1a ( see the generation details in Section 5.1 ) . There are two non-sensitive attributes a and b , which are reflected in the x-axis and y-axis , respectively . The examples are further divided into two classes based on the sensitive attribute z . To generate poisoned data , we measure accuracy degradation for a flipping of each z . We then choose the top-10 % of the z values with the highest degradation and flip them . This approach is similar to the existing label flipping method ( Paudice et al. , 2018 ) , except that the z value is being flipped instead of the label so that the model fairness is also affected . To make a validation set , we randomly select clean examples that amount to 10 % of the entire training data . We use disparate impact ( Feldman et al. , 2015 ) as a fairness measure and evaluate one fairness method , called FAIRNESS CONSTRAINTS ( Zafar et al. , 2017 ) . As this method involves a regularization factor λ that balances the accuracy and fairness objectives , we can obtain a tradeoff curve by adjusting its value . Figure 1b shows two accuracy-fairness tradeoff curves obtained with the clean and poisoned synthetic datasets . Notice that adding data poisoning clearly shifts the curve to the left , which means both accuracy and fairness decrease . This coincides with our intuition . The poisoning confuses the model so that there are more biased examples to fix , which in turn makes it overreact and thus sacrifice more on accuracy . In the supplementary , we show similar trends when the data is poisoned via label flipping . In Section 5 , we will show how data poisoning affects LABEL BIAS CORRECTION ( Jiang & Nachum , 2019 ) and ADVERSARIAL DEBIASING ( Zhang et al. , 2018 ) .

Summary: This paper combines adversarial fair training with adversarial robust training. The basic idea is that a classifier is combined with two adversaries: one tries to predict the sensitive attribute $Z$ from the output of the classifier (essentially the approach by Edwards&Storkey 2015) and the other adversary tries to recognize if a label was predicted or is from a clean hold-out dataset. The latter is intended to harden the classifier against data-poisoning of the training set.