On the Tradeoff Between Robustness and Fairness
Abstract
Interestingly, recent experimental results [2, 26] have identiﬁed a robust fairness phenomenon in adversarial training (AT), namely that a robust model well-trained by AT exhibits a remarkable disparity of standard accuracy and robust accuracy among different classes compared with natural training.
However, the effect of different perturbation radii in AT on robust fairness has not been studied, and one natural question is raised: does a tradeoff exist between average robustness and robust fairness?
Our extensive experimental results provide an afﬁrmative answer to this question: with an increasing perturbation radius, stronger AT will lead to a larger class-wise disparity of robust accuracy.
Theoretically, we analyze the class-wise performance of adversarially trained linear models with mixture Gaussian distribution.
Our theoretical results support our observations.
Moreover, our theory shows that adversarial training easily leads to more serious robust fairness issue than natural training.
Motivated by theoretical results, we propose a fairly adversarial training (FAT) method to mitigate the tradeoff between average robustness and robust fairness.
Experimental results validate the effectiveness of our proposed method.
Introduction
Deep neural networks (DNN) have achieved impressive performance on numerous challenging tasks ranging from computer vision [10, 16] and natural language processing [9, 11] to autonomous driving.
Despite this success, it turns out that DNN is vulnerable to adversarial examples.
In other words, very small and even imperceptible perturbations are sufﬁcient to deceive DNN models, resulting in erroneous predictions [7, 8, 12, 19].
As one of the most popular methods against adversarial examples, adversarial training (AT) can effectively improve model’s robustness [17, 21, 24, 29].
Recently, numerous authors [2, 26] have identiﬁed an interesting robust fairness phenomenon in
AT, speciﬁcally that a robust model well-trained by AT exhibits a remarkable disparity of standard accuracy and robust accuracy among different classes compared with natural training.
Usually, the performance of a naturally trained model is similar across classes.
However, adversarially trained models tend to perform well in some classes and poorly in others, which will lay serious hidden dangers for some applications.
For example, an autonomous driving system may achieve a satisfying average robust accuracy for recognizing objects in the road.
Nevertheless, this system is robust to the classes of inanimate objects (high robust accuracy) but vulnerable to some important classes
[Figure] [Figure]
such as human (low robust accuracy), which is a “snake” in the grass for drivers and pedestrians in the road.
Additionally, in many sensitive ﬁelds related to social ethics, it is crucial to ensure that the models have no discriminatory behaviors toward certain classes (groups or populations).
It is therefore imperative to conduct in-depth study on the robust fairness issue.
Notably, the effect of different perturbation radii in AT on robust fairness has not been studied.
When further considering the robust fairness phenomenon in AT, one natural question arises:
Is there a tradeoff between average robustness and robust fairness; speciﬁcally, as the perturbation radius increases, will stronger adversarially trained models lead to a larger class-wise disparity of robust accuracy among different classes?
This paper attempts to provide an understanding of the robust fairness phenomenon in AT and study the tradeoff between average robustness and robust fairness.
Following [5, 18], we use the variance of class-wise robust accuracy to quantitatively measure model’s robust fairness.
Our experimental results in Figures 1 and 2 provide an afﬁrmative answer to this question, suggesting that there is a tradeoff between average robustness and robust fairness: with the increasing perturbation radius,
AT is able to improve the average robust accuracy of models over classes, but AT also leads to a larger variance of class-wise robust accuracy.
Theoretically, we analyze the class-wise performance of adversarially trained linear models with mixture Gaussian distribution.
Our theoretical results support our observations.
Moreover, our theory shows that AT easily results in more serious fairness issue compared with natural training.
Motivated by our theoretical results, we propose a variancebased regularized method to mitigate the tradeoff between average robustness and robust fairness.
Our core contributions can be summarized as:
• We empirically ﬁnd the relation between the variance of class-wise robust accuracy and perturbation radius in AT, namely that AT with larger perturbation radius will lead to larger variance of class-wise robust accuracy while average robust accuracy is improved, which suggests that there exists a tradeoff between robustness and fairness; • We theoretically analyze this new phenomenon above
and provide a potential explanation for it through linear model with mixture Gaussian distribution.
Additionally, we theoretically prove that AT can lead to severer robust fairness issue compared with natural training;
• We propose FAT to mitigate the tradeoff between robustness and fairness, and experimental results validate the effectiveness of our proposed method.
Related Work
Adversarial training Madry [17] proposes adversarial training (AT) to improve the robustness of the model against adversarial examples.
Some variants of AT have also been presented.
For example, [21, 29] propose different regularization terms.
[4, 22] add unlabeled data to the training set.
[25] propose an algorithm to accelerate training process.
A large number of works [6, 14, 23] have studied the tradeoff between robustness and accuracy.
Among these, TRADES [29] is one of the most popular methods due to its promising experimental results.
We take the two classical methods,
Madry and TRADES, as our benchmarks.
Robust fairness Recently, [2, 20, 26] have identiﬁed the phenomenon of class-wise disparity of robustness and describe it as a robust fairness problem.
[26] point out that AT can lead to larger class-wise disparity of standard accuracy compared with natural training and focus on adversarial training algorithms.
Different from [2, 20, 26], this paper focuses on exploring the relation between the perturbation radius in AT and class-wise disparity of robust accuracy, and further studies whether there is a tradeoff between average robust accuracy and class-wise disparity of robust accuracy.
We provide an afﬁrmative answer to this question by comprehensive experiments and rigorous theoretical analysis.
Notation and Deﬁnition
We donote by X ⊆
R d the feature space and Y = { 1 , 2 , 3 , . . .
, L } the label space with unknown joint distribution D , and X has marginal distribution D x .
Let ( x, y ) be the feature-label pair, where instance x ∈ X and label y ∈ Y .
We denote the transpose of vector/matrix by the superscript ′ .
Denote by f θ : X → Y the classiﬁer (or hypothesis) which is parameterized by θ ∈ Θ , where Θ is the parameter space and maps x to a certain label y .
For simplicity, we denote ℓ ( θ, x, y ) =
ℓ ( f θ ( x ) , y ) where ℓ ( · ) is a certain loss function.
Natural training.
Recall that for natural training in the canonical classiﬁcation setting, the primary goal is to maximize the standard accuracy of models on the unseen examples from the underlying distribution.
Concretely, we hope to ﬁnd the classiﬁer f nat with the smallest natural risk:
[Equation]
Adversarial training.
AT [17, 21, 29] aims to build models that are robust against adversarial examples by solving a robust optimization problem.
Speciﬁcally, the task of AT is to ﬁnd the classiﬁer f adv with the smallest adversarial/robust risk:
[Equation]
where U ( x, ϵ ) represents the perturbation set.
This paper focuses on the set U ( x, ϵ ) =
{ z ∈ X : ||
z − x || ∞ ≤ ϵ } , where || · || ∞ is the l ∞ -norm.
Such type of set is also called l ∞ -attack in AT.
Empirical Exploration
This section explores the relations between the perturbation radius in AT and the variance of classwise robust accuracy, and studies whether there is a tradeoff between average robust accuracy and class-wise disparity of robust accuracy through a series of experiments.
We use Madry [17] with various perturbation radii ϵ train = { 4 / 255 , 8 / 255 , 12 / 255 , 16 / 255 } to train ResNet18, ResNet50 [13] and Wide ResNet-32-10 (WRN-32-10) [28] on the CIFAR-10 and CIFAR-100 datasets [15].
The evaluation metrics are the average robust accuracy and the variance of class-wise robust accuracy.
The adversarial testing examples are generated by FGSM [12], PGD-20 [17] and C & W [3] with different testing perturbation radii ϵ test =
{ 16 / 255 , 20 / 255 } , respectively.
The experimental settings(including optimizer, epoch, learning rate and so on) are same as Section 7.1 Here, we present only the experimental results of Madry on CIFAR-10.
[Table] [Table]
[Table] [Table]
The results in Figures 1 and 2 show that with increasing ϵ train , the average robust accuracy for
Madry using three neural network models on CIFAR-10 monotonously increases; moreover, their variance of class-wise robust accuracy monotonously increases as well, which leads to severer robust fairness issues.
For example, when ϵ train = 4 / 255 , the average robust accuracy and the variance of class-wise robust accuracy are 29 .
04% and 0 .
019 respectively for Madry using ResNet18 on CIFAR10 under FGSM attack.
When ϵ train = 16 / 255 , Madry using ResNet18 achieves 42 .
94% average robust accuracy, while the variance of class-wise robust accuracy of ResNet18 increases to 0 .on
CIFAR-10.
Additionally, the detailed testing results in Table 1 reveal that this trend consistently exists under different networks, attack algorithms and testing perturbation radii.
The results of Madry on CIFAR-100 are presented in Appendix A. We also use TRADES [29] with various perturbation radii ϵ train to train ResNet18, ResNet50 and WRN-32-10 on CIFAR-10 and
CIFAR-100.
These results are presented in Appendix A. From these results, we can observe the same trend above.
These observations suggest that the phenomenon in Figure 1, Figure 2 and Table 1 can be generalized to more AT algorithms, model architectures, datasets and adversarial attacks.
From the above empirical analysis, we ﬁnd that higher average robust accuracy is accompanied by a larger variance of class-wise robust accuracy, suggesting that there exists a tradeoff between average robustness and robust fairness.
[Table]
Why does AT results in this phenomenon?
Table 2 presents the robust accuracy of some classes, average robust accuracy and variance of class-wise robust accuracy for Madry using ResNet50 on
CIFAR-10.
The results in Table 2 may reveal the underlying rationale for our ﬁndings.
From Table 2, we can observe that model’s robust accuracy on classes “cat” and “deer” is signiﬁcantly lower than that on classes “horse” and “ship” for various values of ϵ train .
Additionally, with the increasing of ϵ train , the robust accuracy of the model on the classes “cat” and “deer” monotonously decreases; while the robust accuracy of the model on classes “horse” and “ship” monotonously increases.
This observation indicates that, to further improve the robustness of models, AT prefers to “sacriﬁce” model’s robust accuracy on relatively vulnerable classes (like “cat” and “deer”) and pay more atten- tion to learning the distribution of examples of more robust classes (like ‘horse” and “ship”).
This preference of AT increases the variance of class-wise robust accuracy and leads to a tradeoff between average robustness and robust fairness, although the average robustness of the model can be improved.
The following section presents a theoretical analysis of this phenomenon.
Theoretical Analysis
Following [4, 22, 26], we study a concrete mixture Gaussian distribution.
Inspired by the results in Table 2, our motivation is to design two classes, “ +1 ” and “ − 1 ” and then we analyze the performance of natural training and AT on each class.
We begin with the following deﬁnition.
Deﬁnition 5.1.
(Mixture Gaussian Distribution).
Let µ + , µ − > 0 be the per-class mean parameter and σ + , σ − > 0 be variance parameter of two classes.
The ( µ + , µ − , σ + , σ − ) -Gaussian mixture distribution D ∗ can be then deﬁned by the following distribution over ( x, y ) ∈
R d × {± 1 } :
[Equation]
where α is the prior probability of class “ +1 ” and µ + = µ + 1 , µ − = µ − 1 , 1 = (
[Equation]
|
[Equation]
′ , I is a d - dimension identity matrix.
In this section, we focus on studying the performance of the linear model on the distribution D ∗ as follows:
[Equation]
where the parameters w ∈ R d , b ∈ R , and sign(t) evaluates to 1 if scalar t ≥ 0 and to -1 otherwise.
5.1 Naturally Trained Linear Model
In this subsection, we analyze the performance of a naturally trained linear model on distribution D ∗ .
For simplicity, we denote
[Equation]
where 1 ( · ) is the indicator function that takes 1 when the statement in the braces is true andotherwise.
Theorem 5.2.
For the naturally trained linear classiﬁer f nat that minimizes the natural risk:
[Equation]
if σ + ̸ = σ − , the class-wise natural risk can be expressed as follows:
[Equation]
where Φ( · ) is the cumulative distribution function (c
[Equation]
d
[Equation]
f.) of standard Gaussian distribution N (0 , 1) and
[Equation]
where A = − d ( µ + σ 2 − + µ − σ 2+ ) and K is a positive constant.
If σ + = σ − =
σ , the class-wise natural risk can be expressed as follows:
[Equation]
A proof of Theorem 5.2 can be found in Appendix B.1.
Because Φ( · ) is a monotonously increasing function, Theorem 5.2 shows that if the variances of two classes are same, the naturally trained model performs better on class “ +1 ” than class “ − 1 ”; namely:
[Equation]
5.2 Adversarially Trained Linear Model
This subsection studies the performance of the adversarially trained linear model on distribution D ∗ .
Without loss of generality, we consider the l ∞ -attack in adversarial training and testing.
In this case, the perturbation set can be expressed as U ( x, ϵ ) =
{ z : ||
z − x || ∞ ≤ ϵ } .
The radius ϵ of the perturbation set can be used to adjust the level of attack.
Obviously, larger ϵ means that adversary can generate stronger adversarial examples which can easily trick the trained model into making incorrect predictions.
We use ϵ train and ϵ test to represent the radius of the perturbation set in the training and testing phases, respectively, and we further assume ϵ train ≤ ϵ test .
For simplicity, we denote
[Equation]
Theorem 5.3.
For the adversarial trained linear classiﬁer f adv that minimizes the adversarial risk:
[Equation]
if σ + ̸ = σ − , the class-wise adversarial risk can be expressed as follows:
[Equation]
where Φ( · ) is the c.d.f.
of standard Gaussian distribution N (0 , 1) and
[Equation]
where B = − d ( µ + σ 2 − + µ − σ 2+ − ϵ train ( σ 2+ + σ 2 − )) and K is a positive constant.
If σ + = σ − =
σ , the class-wise adversarial risk can be expressed as follows:
[Equation]
where C = d 2 ( µ + + µ − − 2 ϵ train )
( µ + + µ − − 2 ϵ test ) .
The proof of Theorem 5.3 can be seen in Appendix B.2.
Theorem 5.3 demonstrates that if the variances of two classes are the same, we have
[Equation]
which indicates that the examples in class “ − 1 ” are more vulnerable than those in class “ +1 ”.
In the following, we will present our core theoretical results based on Theorems 5.2 and 5.3.
5.3 Tradeoff between Average Robustness and Robust Fairness
Following [5, 18], we use variance of class-wise robust accuracy (VCRA) to quantitatively measure the model fairness.
The deﬁnition of VCRA is presented below.
Deﬁnition 5.4.
(VCRA).
Given a classiﬁer f : X → Y where Y = { 1 , 2 , 3 , . . .
, L } , the variance of class-wise robust accuracy of f is deﬁned as
[Equation]
where p adv ( i ) =
P {∀ z ∈ U ( x, ϵ ) , f ( z ) = i | y
= i } and ¯ p adv = 1 L P Li
=1 p adv ( i ) .
Additionally, we have
[Equation]
When calculatinng the variance of class-wise accuracy of f : V CA ( f ) , we just need to replace p adv ( i ) with p ( i )
where p ( i ) =
P ( f ( x ) = i | y
= i ) .
To obtain more concise analytical results, we focus on the situation in which σ + ≤ σ − .
However, our results can be generalized to other settings under some mild conditions.
Theorem 5.5.
Given an adversarially trained linear model f adv in Equation (7) , the variance of class-wise robust accuracy V CRA ( f adv ) is increasing with respect to ϵ train if σ − ≥ σ + .
The proof of Theorem 5.5 can be found in Appendix B.3.
A large radius ϵ test of the perturbation set U ( x, ϵ test ) in the testing phase enables adversary to generate strong adversarial examples that can easily trick the trained model into making wrong predictions.
To defend against these adversarial examples, we usually increase the radius ϵ train of U ( x, ϵ train )
( ϵ train ≤ ϵ test ) to generate strong adversarial examples in the training phase, which has been proved to effectively improve the average robust accuracy of models.
However, Theorem 5.5 shows that the variance of the class-wise robust accuracy of the adversarially trained model f adv increases monotonically with respect to the radius ϵ train .
In other words, a larger radius ϵ train will lead to a larger variance of class-wise robust accuracy of the adversarially trained model even though the average robust accuracy of the model might be improved.
Therefore, there exists a tradeoff between average robust accuracy and robust fairness.
This provides an conﬁrmative answer to our question in Section 1 and explains our experimental ﬁndings in Section 4.
Notably, when perturbation radius ϵ test = 0 , Theorem 5.5 suggests that AT( ϵ train > 0 ) results in larger class-wise disparity of standard accuracy compared with natural training( ϵ train = 0 ).
Hence, the results in [26] is a special case of Theorem 5.5.
Why does AT lead to this phenomenon?
For simplicity, we just consider the situation in which σ − =
σ + .
In this setting, R +1 adv ( f adv ) is monotonically decreasing, but R −
adv ( f adv ) is monotonically increasing with respect to ϵ train .
Therefore, improving model’s average robust accuracy by increasing ϵ train will result in a smaller R +1 adv ( f adv ) and
larger R − 1 adv ( f adv ) , meaning larger classwise disparity of robust accuracy.
Moreover, the examples in class “ − 1 ” are more vulnerable than those in class “ +1 ”.
These analysis suggests that, to further improve average robust accuracy, AT will “sacriﬁce” model’s robust accuracy on relatively vulnerable class “ − 1 ” and pay more attention to learning the distribution of examples in more robust class “ +1 ”, which explains our observations in Table 2.
This preference of AT will lead to a tradeoff between average robust accuracy and robust fairness.
Previous works [2, 26] empirically ﬁnd that a robust model well-trained by AT exhibits a remarkable disparity of standard accuracy and robust accuracy among different classes compared with natural training.
We will theoretically prove this phenomeno.
First, we give following theoretical result.
Theorem 5.6.
Given an adversarially trained linear model f adv in Equation (7) , and suppose σ + = σ − .
The variance of class-wise robust accuracy of f adv can be expressed as follows:
[Equation]
where ξ is a constant and K is a positive constant.
The proof of Theorem 5.6 can be found in Appendix B.3.
Theorem 5.7.
Given an adversarially trained linear model f adv in Equation (7) and a naturally trained linear model f nat in Equation (3) , and suppose σ + = σ − .
If ϵ train < µ + + µ − e and ϵ test < µ + + µ − − eK
[Equation]
[Equation]
µ + + µ − ) , we have
[Equation]
where K is a positive constant and ζ 1 , ζ 2 are two constants.
The proof of Theorem 5.7 can be found in Appendix B.3.
From Theorem 5.7, we can ﬁnd
V CRA ( f nat ) <
V CRA ( f adv ) .
Similar to Theorem 5.7, we also demonstrate V CA ( f nat ) ≤
V CRA ( f adv ) in Theorem B.6 of Appendix B.3.
The results of Theorems 5.7 and B.6 suggest that
AT easily leads to severer robust fairness issue than natural training.
Fairly Adversarial Training
In this section, we will propose a novel fairly adversarial training (FAT) to mitigate the tradeoff between average robustness and robust fairness.
The analytical results in Section 5 and Equation (10) show that we can control the variance of class-wise robust accuracy by controlling the variance of class-wise adversarial risk.
Theorem 6.1 provide a theoretical guarantee for using the variance of class-wise adversarial risk to trade off average robustness against robust fairness.
Denote ˜ ℓ ( θ, x, y ) =
sup z ∈U ( x,ϵ ) ℓ ( θ, z, y ) and the corresponding empirical adversarial risk
ˆ R adv ( f )
= 1 n P ni =1 ˜ ℓ ( θ, x i , y i ) .
In addition, we denote the variance of class-wise adversarial risk (VCAR) as
[Equation]
where R adv ( f, i ) =
E (˜ ℓ ( θ, x, y = i )) and ¯ R adv ( f ) =
1 L P Li =1 R adv ( f, i ) .
Theorem 6.1.
Let ( x 1 , y 1 ) , ( x 2 , y 2 ) , . . .
, ( x n , y n ) be drawn independent and identically distributed (i.i.d.)
from the unknown distribution D .
Under appropriate conditions on the loss ℓ ( · ) , parameter space Θ , with probability of at least 1 − δ , the following holds for all θ ∈ Θ :
[Equation]
The proof of Theorem 6.1 can be found in Appendix B.4.
Theorem 6.1 shows that the variance of class-wise adversarial risk can control the robust risk of classiﬁers and the class-wise disparity of robustness.
Inspired by Theorem 6.1, we propose to minimize the following object:
[Equation]
where the parameter λ in Equation (11) captures the tradeoff between average robustness and robust fairness.
We use ˆ R adv ( f, i ) to estimate R adv
( f, i ) where ˆ R adv ( f, i )
=
1 n i P n i j =1 ˜ ℓ ( θ, x j , i ) and n i is the number of examples that belong to class i .
The estimator of VCAR( f ) can be expressed as follows:
[Equation]
Usually, \ VCAR( f ) is a biased but consistent estimator of V CAR ( f ) .
FAT can thus be formulated as follows:
[Equation]
Experiments
In this section, we present the experimental results to validate the effectiveness of FAT for mitigating the tradeoff between average robustness and robust fairness.
7.1 Experimental Settings
We conduct our experiments on the benchmark datasets CIFAR-10 and CIFAR-100 [15].
Based on ResNet18 [13], we compare our proposed FAT with Madry [17] on two datasets with different
[Table]
[Table]
perturbation radii ϵ train ∈ { 4 / 255 , 8 / 255 , 12 / 255 , 16 / 255 } .
The maximum PGD step and step size are set to 20 and ϵ train / 10 , respectively.
For optimization, we use SGD with 0 .
9 momentum for 120 epochs.
The initial learning rate is set to 0 .
and is divided by 10 at epoch 60 and epoch 80 , respectively.
The adversarial testing examples are generated by FGSM [12], PGD-20 [17] and
C & W [3] with different testing perturbation radii ϵ test =
{ 16 / 255 , 20 / 255 } , respectively.
Here, we present only the experimental results on CIFAR-10 in Tables 3 and 4.
The experiment results of FAT and Madry on CIFAR-100 can be found in Appendix C.1.
Besides, the sensitivity analysis of the regularization hyperparameter λ in FAT is presented in Appendix C.2.
Our code can be found on
GitHub at https://github.com/wzekai99/FAT .
7.2 Experimental Results
Under the attack of FGSM, PGD-20 and C & W with different ϵ test , Table 3 presents the variance of class-wise robust accuracy for FAT and Madry using ResNet18 on CIFAR-10.
From table 3, we can observe that FAT signiﬁcantly reduces the variance of class-wise robust accuracy of Madry for various perturbation radii ϵ train .
The results from Tables 3 and 4 further show that with increasing ϵ train , the average robust accuracy and the variance of class-wise robust accuracy of FAT increase, which is consistent with our theory in Section 5.
However, compared with Madry, FAT consistently obtains smaller variance of class-wise robust accuracy while achieving comparable average robust accuracy on different ϵ train and attack algorithms.
For example, with the increasing of ϵ train from 4 / 255 to 16 / 255 , FAT reduces the variance of class-wise robust accuracy of Madry by around 16% under the C & W attack with ϵ test = 16 / 255 .
These experimental results suggest that FAT can effectively suppress the growth of variance of class-wise robust accuracy while achieving an competitive average robust accuracy compared with Madry with the increasing of ϵ train .
The results of FAT and
TRADES in Appendix C.1 also present conclusions consistent with those drawn here.
Therefore,
FAT can alleviate the robust fairness problem and mitigate the tradeoff between average robustness and robust fairness.
Conclusion
In this work, we empirically ﬁnd that there is a tradeoff between average robustness and robust fairness: higher average robust accuracy of models is accompanied by severer robust fairness issues with the increasing of training perturbation radius.
Our theoretical results explain our observations.
We then theoretically demonstrate that AT indeed leads to severer robust fairness issue compared with natural training.
Moreover, a novel method is proposed to mitigate the tradeoff between average robustness and robust fairness.
Our experimental results verify the superiority of our proposals.
Acknowledgements
This work is supported by the National Natural Science Foundation of China under Grant 61976161.
[-If σ − ≥ σ + , we can easily verify that R +
-]
[-adv ( f adv ) is decreasing and R − 
-]