To be Robust or to be Fair: Towards Fairness in Adversarial Training

1 INTRODUCTION . The existence of adversarial examples ( Goodfellow et al. , 2014 ; Szegedy et al. , 2013 ) causes huge concerns when applying deep neural networks on safety-critical tasks , such as autonomous driving vehicles and face identification ( Morgulis et al. , 2019 ; Sharif et al. , 2016 ) . These adversarial examples are artificially crafted samples which do not change the semantic meaning of the natural samples , but can misguide the model to give wrong predictions . As countermeasures against the attack from adversarial examples , adversarial training algorithms aim to train classifier that can classify the input samples correctly even when they are adversarially perturbed . Namely , they optimize the model to have minimum adversarial risk of that a sample can be perturbed to be wrongly classified : min f E x [ max ||δ|≤ L ( f ( x+ δ ) , y ) ] These adversarial training methods ( Kurakin et al. , 2016 ; Madry et al. , 2017 ; Zhang et al. , 2019b ) have been shown to be one type of the most effective and reliable ways to improve the model robustness against adversarial attacks . Although promising to improve model ’ s robustness , recent studies show side-effects of adversarial training : it usually degrades model ’ s clean accuracy ( Tsipras et al. , 2018 ) . In our work , we find a new intriguing property about adversarial training algorithms : they usually result in a large disparity of accuracy and robustness between different classes . As a preliminary study in Section 2 , we apply natural training and PGD adversarial training ( Madry et al. , 2017 ) on the CIFAR10 dataset ( Krizhevsky et al. , 2009 ) using a ResNet18 ( He et al. , 2016 ) architecture . For a naturally trained model , the model performance in each class is similar . However , in the adversarially trained model , there is a severe performance discrepancy ( both accuracy and robustness ) of the model for data in different classes . For example , the model has high clean accuracy and robust accuracy ( 93 % and 67 % successful rate , separately ) on the samples from the class “ car ” , but much poorer performance on those “ cat ” images ( 59 % and 17 % successful rate ) . More preliminary results in Section 2 further show the similar “ unfair ” phenomenon from other datasets and models . Meanwhile , we find that this fairness issue does not appear in natural models which are trained on clean data . This fact demonstrates that adversarial training algorithms can indeed unequally help to improve model robustness for different data groups and unequally degrade their clean accuracy . 1The model ’ s accuracy on the input samples that have been adversarially perturbed . In this work we first define this problem as the unfairness problem of adversarial training algorithms . If this phenomenon happens in real-world applications , it can raise huge concerns about safety or even social ethics . Imagine that an adversarially trained traffic sign recognizer has overall high robustness , but it is very inaccurate and vulnerable to perturbations for some specific signs such as stop signs . The safety of this autonomous driving car is still not guaranteed . In such case , the safety of this recognizer depends on the worst class performance . Therefore , in addition to achieving overall performance , it is also essential to achieve fair accuracy and robustness among different classes , which can guarantee the worst performance . Meanwhile , this problem may also lead to the issues from social ethics perspectives , which are similar to traditional ML fairness problems ( Buolamwini & Gebru , 2018 ) . For example , a robustly trained face identification system might provide different qualitative levels of service safety for different ethnic communities . In this paper , we first explore the potential reason which may cause this unfair accuracy / unfair robustness problem . In particular , we aim to answer the question - “ Will adversarial training algorithms naturally cause unfairness problems , such as the disparity of clean accuracy and adversarial robustness between different classes ? ” To answer this question , we first propose a conceptual example under a mixture of two spherical Gaussian distributions which resembles to the previous work ( Tsipras et al. , 2018 ) but with different variances . In this setting , we hypothesize that adversarial training tends to only use robust features for model prediction , whose dimension is much lower than the non-robust feature space . In the lower dimensional space , an optimal linear model is more sensitive to the inherent data distributional difference and be biased when making predictions . Motivated by these empirical and theoretical findings , we then propose a Fair Robust Learning ( FRL ) framework to mitigate this unfairness issue , which is inspired from the traditional debiasing strategy to solve a series of cost-sensitive classification problems but we make specific effort to achieve the fairness goal in adversarial setting . Our main contributions can be summarized as following : ( a ) We discover the phenomenon of “ unfairness ” problem of adversarial training algorithms and implement empirical studies to present this problem can be general ; ( b ) We build a conceptual example to theoretically investigate the main reasons that cause this unfairness problem ; and ( c ) We propose a Fair Robust Learning ( FRL ) framework to mitigate the unfairness issue in adversarial setting . 2 PRELIMINARY STUDIES . CIFAR10 In this section , we present our preliminary studies to show that adversarial training algorithms usually present the unfairness issues , which are related to the strong disparity of clean accuracy and robustness among different classes . We implement algorithms including PGD adversarial training ( Madry et al. , 2017 ) and TRADES ( Zhang et al. , 2019b ) on the CIFAR10 dataset ( Krizhevsky et al. , 2009 ) . In CIFAR10 , we both naturally and adversarially train ResNet18 ( He & Garcia , 2009 ) models . In Figure 1 , we present list the the model ’ s accuracy and robustness performance ( under PGD attack by intensity 4/255 and 8/255 ) for each individual class . From the Figure 1 , we can observe that – for the naturally trained models , every class has similar clean accuracy ( around 90 ± 5 % ) and adversarial accuracy ( close to 0 % ) under the PGD attack . It suggests that naturally trained models do not have strong disparity of both clean and robustness performance among classes . However , for adversarially trained models ( under PGD Adv . Training or TRADES ) , the disparity phenomenon becomes severe . For example , a PGD-adversarially trained model has 59.1 % clean accuracy and 17.4 % adversarial accuracy for the samples in the class “ cat ” , which are much lower than the model ’ s overall performance . This phenomenon demonstrates that adversarial training algorithms can not provide the same help for the robustness for the samples in class “ cat ” as other classes , and unfairly degrades too much clean accuracy for “ cat ” . We list our empirical studies under more model architectures in Table 3 and more datasets ( GTRSB ( Stallkamp et al. , 2011 ) ) in Appendix A.2 , where we can find the similar observations . GTSRB We also investigate the fairness issue in German Traffic Sign Recognition Benchmark ( GTRSB ) ( Stallkamp et al. , 2011 ) . It consists of 43 classes of images from different traffic signs , with image sizes 32 × 32 × 3 . In this dataset we also both naturally and adversarially train a 3-Layer CNN classifier . We list the model ’ s performance and sort the classes in the order of decreasing clean accuracy and adv . accuracy . From the Figure 2 , we can see that for the naturally trained model ( left ) , most classes have high accuracy which is over 90 % , but for adversarial training , some classes ’ accuracy drops by a large margin . Meanwhile , adversarial training also unequally improves the model ’ s robustness against PGD attacks given that some classes have very low adversarial accuracy . In this dataset , both natural model and robust model have clear distinguished adversairal accuracy ( robustness ) among classes . 3 THEORETICAL ANALYSIS BASED ON A CONCEPTUAL EXAMPLE . From our preliminary studies , we always observe that adversarially trained models have huge performance disparity ( clean and adversarial accuracy ) between different groups . In this section , we try to understand the unfairness problem via theoretical analysis based on a binary classification problem on a mixture-Gaussian distribution , which is similar to ( Tsipras et al. , 2018 ) . We first state the necessary notions in this paper . Notations . In the following , we use f to denote the classification model which is a mapping f : X → Y from input data space X and output labels Y . Generally , naturally training will find the optimal f to minimize the overall clean error Rnat ( f ) = Pr . ( f ( x ) 6= y ) ; and adversarially training will minimize the overall robust errorRrob ( f ) = Pr . ( ∃δ , ||δ|| ≤ , s.t.f ( x+ δ ) 6= y ) . Specifically in the following binary classification problem , Y = { −1 , +1 } and each class ’ s clean error and robust error are denoted as the conditional probabilities : Rnat ( f , −1 ) = Pr . ( f ( x ) = +1|y = −1 ) , and Rrob ( f , −1 ) = Pr . ( ∃δ , ||δ|| ≤ , s.t.f ( x+ δ ) = +1|y = −1 ) , respectively . 3.1 A BINARY CLASSIFICATION TASK . Our study is motivated by ( Tsipras et al. , 2018 ) which uncovers one key behavior of adversarial training : it excludes high-dimensional non-robust features ( which are vulnerable to attack ) and only preserves lower-dimensional robust features for prediction . Thus , in our case , we assume our conceptual dataset has the data-label pairs ( x , y ) sampled from a distribution D follows : y u.a.r∼ { −1 , +1 } , θ = ( dim = m︷ ︸︸ ︷ γ , ... , γ , dim = d︷ ︸︸ ︷ η , ... , η ) , x ∼ { N ( θ , σ2+1I ) if y = +1 N ( −θ , σ2−1I ) if y = −1 ( 1 ) whereN ( θ , σ2+1I ) is a normal distribution with mean vector θ and covariance matrix σ2+1I and same for class “ -1 ” . Following the work ( Tsipras et al. , 2018 ) , we suppose that the feature space consists of two kinds of features : ( a ) robust features with center γ and dimension m ; and ( b ) non-robust features with center η and dimension d. We assume η < < γ , so an adversarial perturbation δ with intensity ||δ||∞ ≤ can manipulate a non-robust feature to have a different sign in expectation , but δ can not attack a robust feature . Usually , the non-robust features ’ dimension d is far higher than the robust features ’ dimension d , i.e. , ( m < < d ) . In our case , we assume that the 2 classes have a key difference between their variances : σ+1 : σ−1 = K : 1 , where K > 1 . In this theoretical example , our main hypothesis is that : the variance difference between 2 classes will not lead to strong disparity of model performance for a naturally trained model whose prediction is based on a high dimensional feature space . However , the variance difference can cause large performance gap ( both accuracy and robustness ) for adversarially trained models which are based on low-dimensional robust features . To illustrate this fact , we will explicitly calculate the 2 classes ’ clean and robust errors in the proposed distribution for both clean models and robust models .

Summary: This paper introduces a fairness perspective on accuracy performance among distinct classes in the context of adversarial training. It makes an observation that adversarial training algorithms (Madry et al. 2017, Zhang et al. 2019) yield biased performances on CIFAR 10. It also offers a theoretical study under a Gaussian mixture setting that respects Eq. (1). Three versions of fair robust algorithms are proposed and evaluated on CIFAR 10.