FR-GAN: Fair and Robust Training

1 INTRODUCTION . As machine learning becomes widespread in the Software 2.0 era ( Karpathy , 2017 ) , there is an urgent need to address the issues of fairness and robustness . For sensitive applications like healthcare , finance , and self-driving cars , a trained model must not discriminate customers based on sensitive attributes including age , sex , or religion . In addition , as applications often rely on external data sets for their training data , the model training must be resilient against noisy or even adversarial data . Traditionally , model fairness research ( Venkatasubramanian , 2019 ; Chouldechova & Roth , 2018 ; Verma & Rubin , 2018 ) has focused on developing metrics such as disparate impact ( Feldman et al. , 2015 ) , equalized odds ( Hardt et al. , 2016 ) , and equal opportunity ( Hardt et al. , 2016 ) , which capture various notions of discrimination . More recently , there has been a surge in unfairness mitigation techniques ( Bellamy et al. , 2018 ) , which improve the model fairness by either fixing the training data , training process , or trained model . Unfairness mitigation usually involves some tradeoff between the model ’ s accuracy and fairness . Most recently , generative adversarial networks ( GANs ) are being adapted to a fairness setting ( Zhang et al. , 2018 ) . The architecture of GANs is suitable because it can be used to compete the training between two objectives : accuracy and fairness . Robust model training is also important and needs to be concurrently taken into consideration . There has been a proliferation of algorithms that make model training resilient against noisy and even adversarial data ( Natarajan et al. , 2013 ; Biggio et al. , 2011 ; Frénay & Verleysen , 2014 ) . The latter scenario may occur when the source of the training data is external , and we can not control the data that is being added . For example , any public dataset on the Web may contain intentionallyfalse information . Moreover , one must have a robust training algorithm if the data set comes from diverse users ’ inputs that can not be fully trusted ( Konečnỳ et al. , 2016 ) . Recently , data poisoning attacks have become increasingly sophisticated where it is becoming difficult to sanitize the data against all of them ( Koh et al. , 2018 ) . In addition , dataset searching is becoming mainstream as demonstrated by Google Dataset Search ( Goods ) ( Halevy et al. , 2016 ) and its public version for searching scientific datasets ( Noy et al. , 2019 ) . While data lakes within companies may consist of refined datasets , datasets in the public are easy to poison . And anyone can poison public datasets using attacks in the literature and share them . We thus believe that it is essential to address both bias and poisoning as a preventive measure . Solving model fairness without addressing data poisoning may lead to a worse tradeoff between accuracy and fairness . For example , consider a banking system that is giving out loans where there are two sensitive groups : men and women . According to disparate impact , the two groups must have similar ratios of positive predictions . A perfect disparate impact would have a value of 1 . Suppose the clean training data has no bias and a perfect ( or high ) disparate impact score . Now suppose this data is poisoned where half of the positive labels of men are flipped to negative labels . If the model is fitted so as to make more negative predictions on men than women , then disparate impact would worsen . To address this problem , one may suggest using data sanitization techniques prior to the model training , but it is challenging to do so without any knowledge of the model . Even though many data sanitization methods are proposed , there have been newer attacks that can easily evade such defenses ( Koh et al. , 2018 ) . Our main contribution is the development of a new framework called FR-GAN , which trains accurate models that are also fair and robust to poisoning . FR-GAN consists of a generator used for classification , a discriminator that distinguishes predictions from one sensitive group against others , and a second discriminator that distinguishes { examples , predictions } of the training data from { examples , labels } of a separate and clean validation set . The first discriminator ensures that the prediction ŷ is independent of the sensitive attribute z . We show that variations of this approach can be used to maximize any of the following prominent fairness measures : disparate impact , equalized odds , and equal opportunity . The second discriminator ensures that the model predictions on the training data are “ consistent ” with labels on clean data . We theoretically show that FR-GAN optimizes fairness without requiring the knowledge of prior statistics of sensitive attributes . In addition , the parameters of FR-GAN can be adjusted to maintain reasonable accuracy and fairness even if the validation set is too small or unavailable . In the following sections , we present the related work , demonstrate the weaknesses of current fairness methods , and propose FR-GAN . 2 RELATED WORK . Model fairness The notion of discrimination has many definitions and usually comes from certain social goals that one wants to guarantee . Fairness research has been focused on defining fairness measures that capture a variety of notions of fairness ( Verma & Rubin , 2018 ) . There are largely two types of fairness measures : ( 1 ) group fairness ( Barocas & Selbst , 2016 ; Hardt et al. , 2016 ) , which ensures similar statistics between two sensitive groups ; ( 2 ) individual fairness ( Dwork et al. , 2012 ) , which guarantees similar prediction results across nearby examples . Group fairness measures are widely studied in general applications , even though these can not be simultaneously satisfied in certain circumstances ( Kleinberg et al. , 2017 ) . In addition , there are also recent significant works that focus on individual fairness ( Garg et al. , 2018 ; Yurochkin et al. , 2019 ; Kearns et al. , 2019 ; Jung et al. , 2019 ) . Among the two types of fairness with different aspects , we focus on group fairness as our initial research . Recently , there has also been a surge of research on unfairness mitigation techniques ( Bellamy et al. , 2018 ) . Depending on where a fix occurs , there are mainly three approaches : ( 1 ) pre-processing techniques ( Kamiran & Calders , 2011 ; du Pin Calmon et al. , 2017 ; Zemel et al. , 2013 ; Feldman et al. , 2015 ) that fix the training data ; ( 2 ) in-processing techniques ( Zafar et al. , 2017 ; Jiang & Nachum , 2019 ; Zhang et al. , 2018 ; Kamishima et al. , 2012 ; Cotter et al. , 2019 ; 2018 ; Agarwal et al. , 2018 ) that address the issue during model training ; and ( 3 ) post-processing techniques ( Hardt et al. , 2016 ; Pleiss et al. , 2017 ; Kamiran et al. , 2012 ) that manipulate predictions while maintaining the model . Among the three , the in-processing techniques have the advantages that one can work with any data and that there is more control on model training ( Venkatasubramanian , 2019 ) . FAIRNESS CONSTRAINTS ( Zafar et al. , 2017 ) incorporates a regularization term that reflects fairness constraints in the context of convex margin-based classifiers such as logistic regression and support vector machines ( SVMs ) . The main hyperparameter is the λ term , which serves to balance the accuracy and fairness objectives . LABEL BIAS CORRECTION ( Jiang & Nachum , 2019 ) assumes the existence of true yet possibly biased labels . A reweighting algorithm is proposed with theoretical guarantees that training on the resulting loss corresponds to training on the true unbiased labels , which yields a fair model . Here the hyperparameters include the number of training iterations . Finally , ADVERSARIAL DEBIASING ( Zhang et al. , 2018 ) trains a classifier so that an adversary can not predict the sensitive attributes based on the output of the classifier . Hence , the classifier achieves fairness when its predictions do not have any correlation with sensitive attributes . This idea of adversarial training is similar to GANs ( Goodfellow et al. , 2014 ) . ADVERSARIAL DEBIASING provides insights of using adversarial training to minimize mutual information between the prediction and sensitive attributes . In this paper , we strengthen the theoretical results of ADVERSARIAL DEBIASING using information theory , and this motivates us to provide systematic methodology for various fairness metrics ( Section 4.1 ) . As we demonstrate in Section 3 , the existing techniques are not tailored for robust training , so are vulnerable to data poisoning attacks . In comparison , FR-GAN addresses both model fairness and robust training within the same model training process . Robust training There is a heavy literature on how to make the model training robust against noisy or even adversarial data ( Natarajan et al. , 2013 ; Biggio et al. , 2011 ; Frénay & Verleysen , 2014 ; Kurakin et al. , 2017 ) . A major challenge is that there can be a wide range of data poisoning attacks that keep on evolving . One defense approach is to sanitize the training data and thus fix the root cause . A fundamental limitation is that we do not know in general what kind of poisoning will occur , so a universal defense against all possible attacks does not seem feasible as demonstrated by Koh et al . ( 2018 ) . A more recent trend is to develop general defense algorithms for any attack during model training using meta learning ( Veit et al. , 2017 ; Li et al. , 2017 ; Xiao et al. , 2015 ; Hendrycks et al. , 2018 ) . Ren et al . ( 2018 ) propose a meta-learning framework that employs a clean validation set for the purpose of preventing the model training from being influenced by poisoned data . By using the validation loss as a meta objective , it intends to defend against any poisoning attack . Our FR-GAN framework is inspired by this approach , but employs a GAN-based model to support fair and robust training without using meta learning . In particular , the design of FR-GAN ’ s robustness discriminator is based on mutual-information-based theoretical insights ( Section 4.2 ) . Another line of research is defending against adversarial attacks during test time ( Biggio et al. , 2013 ; Goodfellow et al. , 2015 ; Wong & Kolter , 2018 ) . In comparison , our focus is on defending against data poisoning on the training data . 3 VULNERABILITY OF FAIRNESS METHODS . We perform experiments to demonstrate that state-of-the-art fairness methods are indeed vulnerable even to simple poisoning attacks . We generate a synthetic dataset as shown in Figure 1a ( see the generation details in Section 5.1 ) . There are two non-sensitive attributes a and b , which are reflected in the x-axis and y-axis , respectively . The examples are further divided into two classes based on the sensitive attribute z . To generate poisoned data , we measure accuracy degradation for a flipping of each z . We then choose the top-10 % of the z values with the highest degradation and flip them . This approach is similar to the existing label flipping method ( Paudice et al. , 2018 ) , except that the z value is being flipped instead of the label so that the model fairness is also affected . To make a validation set , we randomly select clean examples that amount to 10 % of the entire training data . We use disparate impact ( Feldman et al. , 2015 ) as a fairness measure and evaluate one fairness method , called FAIRNESS CONSTRAINTS ( Zafar et al. , 2017 ) . As this method involves a regularization factor λ that balances the accuracy and fairness objectives , we can obtain a tradeoff curve by adjusting its value . Figure 1b shows two accuracy-fairness tradeoff curves obtained with the clean and poisoned synthetic datasets . Notice that adding data poisoning clearly shifts the curve to the left , which means both accuracy and fairness decrease . This coincides with our intuition . The poisoning confuses the model so that there are more biased examples to fix , which in turn makes it overreact and thus sacrifice more on accuracy . In the supplementary , we show similar trends when the data is poisoned via label flipping . In Section 5 , we will show how data poisoning affects LABEL BIAS CORRECTION ( Jiang & Nachum , 2019 ) and ADVERSARIAL DEBIASING ( Zhang et al. , 2018 ) .

Summary: This paper introduces a new method for training a classifier that simultaneously optimizes for a fairness criterion and robustness to data poisoning. The method is shown to increase measures of fairness and reduce inaccuracy on poisoned data relative to classifiers that only consider accuracy or fairness. Extensive results are shown for both synthetic and real benchmark data sets.