To be Robust or to be Fair: Towards Fairness in Adversarial Training

1 INTRODUCTION . The existence of adversarial examples ( Goodfellow et al. , 2014 ; Szegedy et al. , 2013 ) causes huge concerns when applying deep neural networks on safety-critical tasks , such as autonomous driving vehicles and face identification ( Morgulis et al. , 2019 ; Sharif et al. , 2016 ) . These adversarial examples are artificially crafted samples which do not change the semantic meaning of the natural samples , but can misguide the model to give wrong predictions . As countermeasures against the attack from adversarial examples , adversarial training algorithms aim to train classifier that can classify the input samples correctly even when they are adversarially perturbed . Namely , they optimize the model to have minimum adversarial risk of that a sample can be perturbed to be wrongly classified : min f E x [ max ||δ|≤ L ( f ( x+ δ ) , y ) ] These adversarial training methods ( Kurakin et al. , 2016 ; Madry et al. , 2017 ; Zhang et al. , 2019b ) have been shown to be one type of the most effective and reliable ways to improve the model robustness against adversarial attacks . Although promising to improve model ’ s robustness , recent studies show side-effects of adversarial training : it usually degrades model ’ s clean accuracy ( Tsipras et al. , 2018 ) . In our work , we find a new intriguing property about adversarial training algorithms : they usually result in a large disparity of accuracy and robustness between different classes . As a preliminary study in Section 2 , we apply natural training and PGD adversarial training ( Madry et al. , 2017 ) on the CIFAR10 dataset ( Krizhevsky et al. , 2009 ) using a ResNet18 ( He et al. , 2016 ) architecture . For a naturally trained model , the model performance in each class is similar . However , in the adversarially trained model , there is a severe performance discrepancy ( both accuracy and robustness ) of the model for data in different classes . For example , the model has high clean accuracy and robust accuracy ( 93 % and 67 % successful rate , separately ) on the samples from the class “ car ” , but much poorer performance on those “ cat ” images ( 59 % and 17 % successful rate ) . More preliminary results in Section 2 further show the similar “ unfair ” phenomenon from other datasets and models . Meanwhile , we find that this fairness issue does not appear in natural models which are trained on clean data . This fact demonstrates that adversarial training algorithms can indeed unequally help to improve model robustness for different data groups and unequally degrade their clean accuracy . 1The model ’ s accuracy on the input samples that have been adversarially perturbed . In this work we first define this problem as the unfairness problem of adversarial training algorithms . If this phenomenon happens in real-world applications , it can raise huge concerns about safety or even social ethics . Imagine that an adversarially trained traffic sign recognizer has overall high robustness , but it is very inaccurate and vulnerable to perturbations for some specific signs such as stop signs . The safety of this autonomous driving car is still not guaranteed . In such case , the safety of this recognizer depends on the worst class performance . Therefore , in addition to achieving overall performance , it is also essential to achieve fair accuracy and robustness among different classes , which can guarantee the worst performance . Meanwhile , this problem may also lead to the issues from social ethics perspectives , which are similar to traditional ML fairness problems ( Buolamwini & Gebru , 2018 ) . For example , a robustly trained face identification system might provide different qualitative levels of service safety for different ethnic communities . In this paper , we first explore the potential reason which may cause this unfair accuracy / unfair robustness problem . In particular , we aim to answer the question - “ Will adversarial training algorithms naturally cause unfairness problems , such as the disparity of clean accuracy and adversarial robustness between different classes ? ” To answer this question , we first propose a conceptual example under a mixture of two spherical Gaussian distributions which resembles to the previous work ( Tsipras et al. , 2018 ) but with different variances . In this setting , we hypothesize that adversarial training tends to only use robust features for model prediction , whose dimension is much lower than the non-robust feature space . In the lower dimensional space , an optimal linear model is more sensitive to the inherent data distributional difference and be biased when making predictions . Motivated by these empirical and theoretical findings , we then propose a Fair Robust Learning ( FRL ) framework to mitigate this unfairness issue , which is inspired from the traditional debiasing strategy to solve a series of cost-sensitive classification problems but we make specific effort to achieve the fairness goal in adversarial setting . Our main contributions can be summarized as following : ( a ) We discover the phenomenon of “ unfairness ” problem of adversarial training algorithms and implement empirical studies to present this problem can be general ; ( b ) We build a conceptual example to theoretically investigate the main reasons that cause this unfairness problem ; and ( c ) We propose a Fair Robust Learning ( FRL ) framework to mitigate the unfairness issue in adversarial setting . 2 PRELIMINARY STUDIES . CIFAR10 In this section , we present our preliminary studies to show that adversarial training algorithms usually present the unfairness issues , which are related to the strong disparity of clean accuracy and robustness among different classes . We implement algorithms including PGD adversarial training ( Madry et al. , 2017 ) and TRADES ( Zhang et al. , 2019b ) on the CIFAR10 dataset ( Krizhevsky et al. , 2009 ) . In CIFAR10 , we both naturally and adversarially train ResNet18 ( He & Garcia , 2009 ) models . In Figure 1 , we present list the the model ’ s accuracy and robustness performance ( under PGD attack by intensity 4/255 and 8/255 ) for each individual class . From the Figure 1 , we can observe that – for the naturally trained models , every class has similar clean accuracy ( around 90 ± 5 % ) and adversarial accuracy ( close to 0 % ) under the PGD attack . It suggests that naturally trained models do not have strong disparity of both clean and robustness performance among classes . However , for adversarially trained models ( under PGD Adv . Training or TRADES ) , the disparity phenomenon becomes severe . For example , a PGD-adversarially trained model has 59.1 % clean accuracy and 17.4 % adversarial accuracy for the samples in the class “ cat ” , which are much lower than the model ’ s overall performance . This phenomenon demonstrates that adversarial training algorithms can not provide the same help for the robustness for the samples in class “ cat ” as other classes , and unfairly degrades too much clean accuracy for “ cat ” . We list our empirical studies under more model architectures in Table 3 and more datasets ( GTRSB ( Stallkamp et al. , 2011 ) ) in Appendix A.2 , where we can find the similar observations . GTSRB We also investigate the fairness issue in German Traffic Sign Recognition Benchmark ( GTRSB ) ( Stallkamp et al. , 2011 ) . It consists of 43 classes of images from different traffic signs , with image sizes 32 × 32 × 3 . In this dataset we also both naturally and adversarially train a 3-Layer CNN classifier . We list the model ’ s performance and sort the classes in the order of decreasing clean accuracy and adv . accuracy . From the Figure 2 , we can see that for the naturally trained model ( left ) , most classes have high accuracy which is over 90 % , but for adversarial training , some classes ’ accuracy drops by a large margin . Meanwhile , adversarial training also unequally improves the model ’ s robustness against PGD attacks given that some classes have very low adversarial accuracy . In this dataset , both natural model and robust model have clear distinguished adversairal accuracy ( robustness ) among classes . 3 THEORETICAL ANALYSIS BASED ON A CONCEPTUAL EXAMPLE . From our preliminary studies , we always observe that adversarially trained models have huge performance disparity ( clean and adversarial accuracy ) between different groups . In this section , we try to understand the unfairness problem via theoretical analysis based on a binary classification problem on a mixture-Gaussian distribution , which is similar to ( Tsipras et al. , 2018 ) . We first state the necessary notions in this paper . Notations . In the following , we use f to denote the classification model which is a mapping f : X → Y from input data space X and output labels Y . Generally , naturally training will find the optimal f to minimize the overall clean error Rnat ( f ) = Pr . ( f ( x ) 6= y ) ; and adversarially training will minimize the overall robust errorRrob ( f ) = Pr . ( ∃δ , ||δ|| ≤ , s.t.f ( x+ δ ) 6= y ) . Specifically in the following binary classification problem , Y = { −1 , +1 } and each class ’ s clean error and robust error are denoted as the conditional probabilities : Rnat ( f , −1 ) = Pr . ( f ( x ) = +1|y = −1 ) , and Rrob ( f , −1 ) = Pr . ( ∃δ , ||δ|| ≤ , s.t.f ( x+ δ ) = +1|y = −1 ) , respectively . 3.1 A BINARY CLASSIFICATION TASK . Our study is motivated by ( Tsipras et al. , 2018 ) which uncovers one key behavior of adversarial training : it excludes high-dimensional non-robust features ( which are vulnerable to attack ) and only preserves lower-dimensional robust features for prediction . Thus , in our case , we assume our conceptual dataset has the data-label pairs ( x , y ) sampled from a distribution D follows : y u.a.r∼ { −1 , +1 } , θ = ( dim = m︷ ︸︸ ︷ γ , ... , γ , dim = d︷ ︸︸ ︷ η , ... , η ) , x ∼ { N ( θ , σ2+1I ) if y = +1 N ( −θ , σ2−1I ) if y = −1 ( 1 ) whereN ( θ , σ2+1I ) is a normal distribution with mean vector θ and covariance matrix σ2+1I and same for class “ -1 ” . Following the work ( Tsipras et al. , 2018 ) , we suppose that the feature space consists of two kinds of features : ( a ) robust features with center γ and dimension m ; and ( b ) non-robust features with center η and dimension d. We assume η < < γ , so an adversarial perturbation δ with intensity ||δ||∞ ≤ can manipulate a non-robust feature to have a different sign in expectation , but δ can not attack a robust feature . Usually , the non-robust features ’ dimension d is far higher than the robust features ’ dimension d , i.e. , ( m < < d ) . In our case , we assume that the 2 classes have a key difference between their variances : σ+1 : σ−1 = K : 1 , where K > 1 . In this theoretical example , our main hypothesis is that : the variance difference between 2 classes will not lead to strong disparity of model performance for a naturally trained model whose prediction is based on a high dimensional feature space . However , the variance difference can cause large performance gap ( both accuracy and robustness ) for adversarially trained models which are based on low-dimensional robust features . To illustrate this fact , we will explicitly calculate the 2 classes ’ clean and robust errors in the proposed distribution for both clean models and robust models .

Summary: The authors study adversarially trained classifiers and observe that the accuracy discrepancy between classes is larger than that of standard models. They then propose a theoretical model where this phenomenon provably arises. Finally, they propose a method to reduce the (standard and robust) accuracy discrepancy between classes, by adapting existing methods from the non-adversarial setting.