On the Tradeoff Between Robustness and Fairness
Authors: Xinsong Ma
Zekai Wang
Weiwei Liu
Venue: NeurIPS 2022 Accept (2022)
Keywords: Adversarial training
robust fairness

Abstract: Interestingly, recent experimental results [2, 26, 22] have identified a robust fairness phenomenon in adversarial training (AT), namely that a robust model well-trained by AT exhibits a remarkable disparity of standard accuracy and robust accuracy among different classes compared with natural training. However, the effect of different perturbation radii in AT on robust fairness has not been studied, and one natural question is raised: does a tradeoff exist between average robustness and robust fairness? Our extensive experimental results provide an affirmative answer to this question: with an increasing perturbation radius, stronger AT will lead to a larger class-wise disparity of robust accuracy. Theoretically, we analyze the class-wise performance of adversarially trained linear models with mixture Gaussian distribution. Our theoretical results support our observations. Moreover, our theory shows that  adversarial training easily leads to more serious robust fairness issue than natural training. Motivated by theoretical results, we propose a fairly adversarial training (FAT) method to mitigate the tradeoff between average robustness and robust fairness. Experimental results validate the effectiveness of our proposed method.


On the Tradeoff Between Robustness and Fairness
Xinsong Ma School of Computer Science Wuhan University
Zekai Wang
School of Computer Science
Wuhan University
maxinsong101800maa... com
wzzka1198gmail.comm
Weiwei Liu'
School of Computer Science
Wuhan University
1iuxeiive1163@@mai... com
Abstract
Interestingly, recent experimental results 26] have identified robust faimess phenomenon in adversarial training (AT), namely that a robust model well-trained by AT exhibits remarkable disparity of standard accuracy and robust accuracy among different classes compared with natural training. However, the effect different perturbation radii in AT robust fairness has not been studied, and natural question is raised: does a tradeoff exist between average robustness and robust fairness? Our extensive experimental results provide an affirmative answer to this question: with an increasing perturbation radius, stronger AT will lead to larger class-wise disparity of robust accuracy. Theoretically, we analyze the class-wise performance of adversarially trained linear models with mixture Gaus- sian distribution. Our theoretical results support our observations. Moreover, our theory shows that adversarial training easily leads to more serious robust faimess issue than natural training. Motivated by theoretical results, we propose fairly adversarial training (FAT) method to mitigate the tradeoff between average robust- ness and robust fairness. Experimental results validate the effectiveness of our proposed method.
Introduction
Deep neural networks (DNN) have achieved impressive performance on numerous challenging tasks ranging from computer vision [τι [6] and natural language processing [9] [ل]] to autonomous driv- ing. Despite this success, it turns out that DNN is vulnerable to adversarial examples. In other words, very small and even imperceptible perturbations
sufficient to deceive DNN models, resulting in erroneous predictions [1], & [12, [9]. As one of the most popular methods against adversarial exam- ples, adversarial training (AT) can effectively improve model's 's robustness
2 [0]
Recently, numerous authors [e. [6] have identified an interesting robust fairesss phenomenon in AT, specifically that robust model well-trained by AT exhibits remarkable disparity of standard accuracy and robust accuracy among different classes compared with natural training. Usually, the performance of a naturally trained model is similar across classes. However, adversarially trained models tend to perform well some classes and poorly in others, which will lay serious hidden dangers for some applications. For example, an autonomous driving system may achieve satisfying average robust accuracy for recognizing objects in the road. Nevertheless, this system is robust to the classes of inanimate objects (high robust accuracy) but vulnerable some important classes
"Correponniigg author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

4,25
4/255
B223
6/233
22/225
120233
12/233
Vanance
20/255
Varaacee
26/235
Variance
10/235
100
120
20
100
120
20
40
60
80
100
120
Epoch
Epach
Epach
ResNet18
(b) ResNet50
(c) WRN-32-10
Figure 1: The variance of class-wise robust accuracy for Madry using ResNer18, ResNet50 and WRN-32-10 CIFAR-10. The perturbation radii for AT are chosen from Etrain (4/255, 8/255, 12/255, 16/255). The adversarial testing examples generated by FGSM with
testing perturbation radius Exewt 16/255. 45
Robust Accuracye
35
40
35
40
2
45
30
30
25
25
20
20
4/255
15
6255
4015
15
8/255
8255
20
RP15
12/35
18223
15
12235
18455
18/23
19255
60 Epoch
80
20
40
60
100 120
Epach
Epoch
(a) ResNet18
(b) ResNet50
(c) WRN-32-10
Figure 2: The average robust accuracy for Madry using ResNer18, ResNet50 and WRN-32-10 10 CIFAR-1O. The perturbation radii for are chosen from Etrain = {4/255, 8/ 8/255, 12/255, 16/255]. The adversarial testing examples are generated by FGSM with testing perturbation radius Great = 16/255.
such human (low robust accuracy), which is "snake" in the grass for drivers and pedestrians in the road. Additionally, in many sensitive fields related to social ethics, it is crucial to ensure that the models have no discriminatory behaviors toward certain classes (groups or populations). It is therefore imperative to conduct in-depth study on the robust fairness issue. Notably, the effect of different perturbation radii in AT on robust fairness has not been studied. When further considering the robust fairness phenomenon in AT, one natural question arises:
Is there a tradeoff between average robustness and robust fairness; specifically, the perturbation radius increases, will stronger ad adversarially trained models lead to a larger class-wise disparity of robust accuracy among different classes?
This paper attempts to provide an understanding of the robust fairness phenomenon in AT and study the tradeoff between average robustness and robust fairness. Following [5, [8), we use the variance of class-wise robust accuracy to quantitatively measure model's robust fairness. Our experimental results in Figures II and E provide an affirmative answer to this question, suggesting that there is a tradcoff between average robustness and robust fairness: with the increasing perturbation radius, AT is able to improve the average robust accuracy of models over classes, but AT also leads to larger variance of class-wise robust accuracy. Theoretically, we analyze the class-wise performance of adversarially trained linear models with mixture Gaussian distribution. Our theoretical results support our observations. Moreover, our theory shows that AT easily results in more serious fairness issue compared with natural training. Motivated by our theoretical results, propose variance- based regularized method to mitigate the tradeoff between average robustness and robust fairness. Our core contributions be summarized as:
We empirically find the relation between the variance of class wise robust accuracy and perturbation radius in AT, namely that AT with larger perturbation radius will lead to larger variance of class-wise robust accuracy while average robust accuracy is improved, which suggests that there exists tradeoff between robustness and fairness;
We theoretically analyze this new phenomenon above and provide potential explanation for it through linear model with mixture Gaussian distribution. Additionally, we theoreti- cally prove that AT can lead to severer robust fairness issue compared with natural training;

---

We propose FAT to mitigate the tradeoff between robustness and faimess, and experimental results validate the effectiveness of our proposed method.
2 Related Work
Adversarial training Madry || proposes adversarial training (AT) to improve the robustness of the model against adversarial examples. Some variants of AT have also been presented. For example,
[2T, 2 propose different regularization terms. [. [2]] add unlabeled data to the training set. [2S]
propose an algorithm to accelerate training process. A large number of works [[..
14. 23 have
studied the tradeoff between robustness and accuracy. Among these, TRADES [29] is of the most popular methods due to its promising experimental results. We take the two classical methods, Madry and TRADES, our benchmarks.
Robust fairness Recently, [2, 20 26] have identified the phenomenon class-wise disparity of robustness and describe it as a robust fairness problem. [26] point out that AT can lead to larger class-wise disparity of standard accuracy compared with natural training and focus on adversarial training algorithms. Different from [2] 20, g6].. this paper focuses exploring the relation between the perturbation radius in AT and class- wise disparity of robust accuracy, and further studies whether there is tradeoff between average robust accuracy and class-wise disparity of robust accuracy. We provide an affirmative answer to this question by comprehensive experiments and rigorous theoreti- cal analysis.
3 Notation and Definition
We donote by X' C IRd the feature space and y = (1, .}3 the label space with unknown joint distribution D, and X has marginal distribution D,, Let (x, be the feature-label pair, where instance 2 E X' and label y E y. We denote the transpose of vector/matrix by the superscript ' Denote by fo X' - y the classifier (or hypothesis) which is parameterized by 9 O where θ is the parameter space and maps x to a certain label y. For simplicity, we denote E(0,, y) = ((f(())) y) where E(-) is certain loss function.
Natural training. Recall that for natural training in the canonical classification setting, the primary goal is maximize the standard accuracy of models the unseen examples from the underlying distribution. Concretely, we hope to find the classifier fnat with the smallest natural risk:
Bmot (Jart) = min (x.))=D (0, xr y)
8E8
Adversarial training. AT [1 21, 29 aims to build models that are robust against adversarial ex- amples by solving robust optimization problem. Specifically, the task of AT is find the classifier fado with the smallest adversarial/robust risk: risk:
Radv (Jadv) min Beayyll sup { 2, sEU((.,))
where U(x e) represents the perturbation set. This paper focuses on the set U((x,)) fs X || 2: olloc S c), where || I1so is the -nomm. Such type of set is also called -actack in AT.
4 Empirical Exploration
This section explores the relations between the perturbation radius in AT and the variance of class- wise robust accuracy, and studies whether there is a tradcoff between average robust accuracy and class-wise disparity of robust accuracy through a series of experiments. We use Madry [[Z]] with var- ious perturbation radii Grain (4/255, 8/ 8/255, 12/255. 16 16/255) to train ResNet18 88, ResNet50 [[3]] and Wide ResNet-32-11 10 (WRN-32-10 [S]] on the CIFAR-10 10 and CIFAR- datasets [US].. The evaluation metrics are the average robust accuracy and the variance of class robust accuracy.
The adversarial testing examples are generated by FGSM [I22), PGD-20 [[7]] and C&W [3] with different testing perturbation radii étest
(66/255, 20/25 (0/255), }. respectively. The experimental set-
tings(in optimizer, epoch, learning rate and so on) are same as Section [.]l Here, we present only the experimental results of Madry on CIFAR-10.
3

---

Table 1: The average robust accuracy (%) and variance of class-wise robust accuracy (10-²) for Madry us- ing ResNet18, ResNet50 and WRN-32 on CIFAR-IO. The perturbation radii for AT are chosen from Cirain {4/255, 8/255, 12 2/255 16 16/255). The adversarial testing examples are generated by FGSM. PGD-20 20 and C&W with different testing perturbation radii Creat = {16/255, 20/255). We use var and rob.acc to denote the variance of class-wise robust accuracy, and average robust accuracy, respectively.
atlack algorithm
FGSM
PGD-20
C&&W
Crest
16/255
20V255
16/255
20/255
16/255
20V255
modee
Feroin
rob.ace
rob.acc
rob.acc
Tob.acc
rob,ace
robacc
4/255
1,90
29.04
1.31
21.71
0.10
3.54
0.02
1.12
0.11
3.61
0.02
1,05
ResNet18
8/255
2.34
39,00
32.13
11.23
4,96
12.14
0.19
5.35
12/255
3.12
41.99
2.53
35.71
0.98
19.43
0.35
10.03
1.09
19.39
0.42
10.53
16/255
3.52
42.94
2.99
37.34
22.50
1.38
22.03
0.60
4/255
1.24
28.89
0.84
21.14
0.07
3.34
0.01
1.13
0.07
3.47
0.01
1,08
8/255
2.36
37.74
1.81
30.68
0.56
13.13
0.22
6.10
0.68
14,06
0.29
6.90
ResNet50
12/255
3.15
39.28
33.95
20.88
11.59
1.16
20.19
16/255
3.89
41.26
3.20
35.95
1.54
23.92
0.66
14.44
1.61
22.56
0.75
14.11
4/255
2.30
39.21
0.16
5.15
0.16
5.15
0.02
1.79
0.09
3.97
0.01
1.10
WRN 10
8/255
2.46
40.68
0.93
13.62
0.93
13.62
0.43
7.91
0.74
12.76
0.25
6.58
12/255
2.58
41.51
0.97
15.95
0.96
15.63
0.48
9.09
0.96
15.69
0.46
8,87
16/255
2.73
43.36
17.05
17.05
9.87
1.00
17.09
0.53
9.76
The results in Figures M and @ show that with increasing Étrain, the average robust accuracy for Madry using three neural network models on CIFAR-10 monotonously increases; moreover, their variance of class-wise robust accuracy monotonously increases well, which leads to severer robust fairness issues. For example, when Etrain = 4/255, the average robust accuracy and the variance of class-wise robust accuracy are 29.04% and 0.019 respectively for Madry using ResNet18 on CIFAR- 10 under FGSM attack. When Etrain = 16/255, Madry using ResNet18 achieves 42.94% average robust accuracy, while the variance of class-wise robust accuracy of ResNet18 increases to 0.035 on CIFAR-10. Additionally, the detailed testing results in Table I1 reveal that this trend consistently exists under different networks, attack algorithms and testing perturbation radii.
The results of Madry on CIFAR-100 are presented in Appendix AL We also use TRADES [ with various perturbation radii étrain train ResNet18, ResNet50 and WRN-32-10 on CIFAR-10 and CIFAR-100. These results are presented in Appendix AL From these results, we can observe the same trend above. These observations suggest that the phenomenon in Figure M. Figure D and Table M can be generalized to more AT algorithms,, model architectures. datasets and adversarial attacks. From the above empirical analysis, we find that higher average robust accuracy accompanied by a larger variance of class- class-wise robust accuracy, suggesting that there exists a tradeoff between average robustness and robust fairness.
Table 2: The robust accuracy (%), average robust accuracy (%) and variance of class-wise robust accuracy (10 (10-2) of some classes for Madry using ResNer50 on CIFAR-1O. The perturbation radii for AT are chosen from étrain = [1/255, 8/255, 12/255, 16/255). The adversarial testing examples are gen- crated by FGSM with testing perturbation radius clett 16/255. We use var and rob.acc to denote the variance of class-wis robust accuracy, and average robust accuracy, respectively. Moreover, diff means the difference between testing results where Ctrain is 16/255 and 4/255, respectively.
Ctrain
cat
deer
horse
ship
var
rob.acc.
4/255
12.0
17.5
39.9
34.2
1.24
28.89
8/255
10.9
16.7
49.6
53.5
2.36
37.74
12/255
8.9
13.7
50.4
60.4
3.15
39.42
16/255
7.8
13.7
51.4
62.4
3.89
41.26
diff
4.2
-3.8
11.5
28.2
2.65
12.37
Why does AT results in this phenomenon? Table @ presents the robust accuracy of some classes, average robust accuracy and variance of class-wise robust accuracy for Madry using ResNet50 on CIFAR-10. 10. The results in Table 2 may reveal the underlying rationale for our findings. From Table Q we can observe that model's 's robust accuracy on classes "cat" and "deer" is significantly lower than that on classes "horse" and "ship" for various values of Ctrain- Additionally, with the increasing of étrain, the robust accuracy of the model on the classes "cat" and "deer" monotonously decreases; while the robust accuracy of the model on classes "horse" and "ship" monotonously increases. This observation indicates that, to further improve the robustness of models, AT prefers to "sacrifice" model's 's robust accuracy on relatively vulnerable classes (like "cat" and "deer") and pay more atten-

---

tion to learning the distribution of examples of more robust classes (like 'horse" and "ship"). This preference of AT increases the variance of class-wise robust accuracy and leads tradeoff be- tween average robustness and robust fairness, although the average robustness of the model can be improved. The following section presents theoretical analysis of this phenomenon.
5 Theoretical Analysis
Following [+L 22 26], we study a concrete mixture Gaussian distribution. Inspired by the results in Table E motivation is to design two classes, "+1" and "-1" and then analyze the perfor- mance of natural training and AT on each class. We begin with the following definition.
Definition 5.1. (Mixture Gaussian Distribution). Let p / > 0 be the per-class mean parameter and 0+; 0- 0 be variance parameter of two classes. The (,+, 0,, --),,,,------ distribution D can be then defined by the following distribution over (2. E IR x {+1}:
o-)-auusssaaa J-Gaussin mixture
+1, y 1, p= p=1 x
SW (u+, of [ (-M-_,021
y = +1
(1)
if =-1
dim
where or is the prior probability of class "+1" and p4 = μ μ p 1.1 d- dimension identity matrix.
is
In this section, we focus on studying the performance of the linear model on the distribution D as follows:
f(x) = sign ((w, x) + b)
where the parameters w E IR,, b E IR, and sign(t) evaluates to 1 if scalar 2 0 and to otherwise.
5.1 Naturally Trained Linear Model
In this subsection, we analyze the performance of a naturally trained linear model on distribution
D* For simplicity, we denote
R (f) = (a (1((0)01()))))))0)))) -1)y-=11
R (f) (a.)) (1f((x) +1)ly=
where 1() is the indicator function that takes 1 when the statement in the braces true and 0 otherwise.
Theorem 5.2. For the naturally trained linear classifier fnot that minimizes the natural risk: fnat(x)) arg (i) (1(((x)) & y)).
(3)
f
1 # o_, the class-wise natural risk can be expressed as follows:
Bed (Jmat) = = (= )
R (Frat) = + (1) -d-d
Vda_ ("
djs
(-ooo+
(4)
Vdo+
where (-) is the cumulative distribution function (c.df.) of standard Gaussian distribution N(0 1) and
02 o2-03
A + A+040-(M4 (p+ + +
1 2K
(1i+*
11"
-2-2
where = -dd40240 x_) and K is positive constant. If 04 = natural risk can be expressed as follows:
the class-wise
124 + 11 2Ko2
Rt1 (Fnat) = ②b
(5)
2d3/2 ( (-)
(-8(14 + +1 2Ko
Rede (nnat)
2d3/20(144 + n-)
proof of Theorem 5.2 can be found in Appendix B.I. Because +(-) is monotonously increasing
function, Theorem 5.2 shows that if the variances of two classes are same, the naturally trained model performs better on class "+1" than class "=1"; namely:
(fnat) < Rara (fnat).
5

---

5.2 Adversarially Trained Linear Model
This subsection studies the performance of the adversarially trained linear model on distribution Dr. Without loss of generality, we consider the l.0 -atackk in adversarial training and testing. In this case, the perturbation set can expressed as (d e) = {z : || - x|| I <}. The radius of the perturbation set can be used to adjust the level of attack. Obviously, larger means that adversary can generate stronger adversarial examples which can casily trick the trained model into making incorrect predictions.
We use Ctrain and Clest to represent the radius of the perturbation set in the training and testing phases, respectively, and we further assume Etrain S Étest For simplicity, we denote
R- (f) ((aa))DD sup 1(f(z) 11lly +1] zEU(æ.e)
Rydo (f) (x.y))
sup =El((a,e)
1(f(2) ++)ly -1].
Theorem 5.3. For the adversarial trained linear classifier fadu that minimizes the adversarial risk:
fado (x) arg min
sup (1(f(z) # v)l.
=cU((a,))
0+ # o_, the class-wise adversarial risk can be expressed as follows:
Rt1 Rade (Sado)= (-x² d(u+ Etest) -test))
R- (Sado)= (7" 77--d(p- d(p- Ecest)
(8)
vdo+
vdo_
where (-) is the c.d.f. of standard Gaussian distribution N(, 1) and
B B+040_ (M4 + - -2Erraia) 14 2K(o2-o2 (M-tH -2erran
2 02
(vain (0 + +))) and K is positive constant. Ifo+ =0_ = the class-wise adversarial risk can expressed as follows:
where B= -d + p_
2222Ka
adv -
2d332 (1 (1= + -2errain)
-C+2Ko"
R Redo (fadv)= =
2033/2 ---- +-- -2Etrain)
where C = (1+ + p -2 2etrain + pr_ 26test)
The proof of Theorem R can be seen in Appendix variances of two classes are the same, we have
Theorem
demonstrates that the
Rtdo (Sadv) Redo (Jadv);
which indicates that the examples in class "-1" are more vulnerable than those in class "+1". In the following, we will present our core theoretical results based on Theorems 5,2) and 5.3.
5.3 Tradeoff between Average Robustness and Robust Fairness
Following [5. 18), we use variance of class-wise robust accuracy (VCRA) to quantitatively measure the model fairness. The definition of VCRA is presented below.
Definition 5.4. (VCRA). Given a classifier class-wise robust accuracy of / is defined as
X where y 2, 3........ L), the variance of
1.
VCRA(J) =
Σ
(podo (i) - padv))
(9)

---

where (ado(1) = P(vz ((22 f(2) = ily = i and Pade = E Pade (t). Additionally, we have
Pudo (1) = P(3e E ((, f()) + ily
(10)
= - (1, sup 1(f(z) + i)ly = i].
=El((ae)
When calculatinng the variance of class-wise accuracy of f: VCA(f), we just need to replace (t) with p(i) where p(i) = (zf = ily = i). To obtain more concise analytical results, we focus on the situation in which o S o_. However, our results can be generalized to other settings under some mild conditions.
Theorem 5.5. Given an adversarially trained linear model fado in Equation (2), the variance of class-wise robust accuracy V VCRA(fade) is increasing with respect to Etrain if o 0+.
The proof of Theorem 5.5 can be found in Appendix B A large radius Ctest of the perturbation set (x, Ctest) the testing phase enables adversary to generate strong adversarial examples that can easily trick the trained model into making wrong predictions. To defend against these adversarial examples, we usually increase the radius étrain of ((r,, Etrain) (erain((enn) (Etrain S étest) to generate strong adversarial examples in the training phase, which has been proved to effectively improve the aver- age robust accuracy of models. However, Theorem 5.5 shows that the variance of the class-wise robust accuracy of the adversarially trained model fadv increases monotonically with respect to the radius Etroin- In other words, a larger radius Etrain will lead to a larger variance of class-wise robust accuracy of the adversarially trained model even though the average robust accuracy of the model might be improved. Therefore, there exists a tradeoff between average robust accuracy and robust faireess. This provides confirmative answer to question in Section M and explains exper- imental findings in Section Notably, when perturbation radius Cleat = O, Theorem 55.5 suggests that AT(Ctrain 0) results in larger class wise disparity of standard accuracy compared with natural training(Etrain = 0). Hence, the results in [26] is a special case of Theorem 5..5.
Why does AT lead to this phenomenon? For simplicity, we just consider the situation in which a_ = 0+. In this setting, (add is monotonically decreasing, but Rado (fade) is monotoni- cally increasing with respect to Ctrain- Therefore, improving model's average robust accuracy by increasing étrain will result in smaller Rtt.
(ad (Sadv) and larger Rædo (Jadu), meaning larger class- wise disparity of robust accuracy. Moreover, the examples in class 111" are more vulnerable than those in class "+1". These analysis suggests that, to further improve average robust accuracy, AT will "sacrifice" model's robust accuracy on relatively vuinerable class "-1" and pay more attention learning the distribution of examples in more robust class "+1", which explains our observations in Table D. This preference of AT will lead to a tradeoff' between average robust accuracy and robust fairness.
Previous works [2] 26] empirically find that a robust model well-trained by AT exhibits a remarkable disparity of standard accuracy and robust accuracy among different classes compared with natural training. We will theoretically prove this phenomeno. First, we give following theoretical result.
Theorem 5.6. Given an adversarially trained linear model Sado in Equation (α), and suppose a.
o_. The variance of class-wis robust accuracy of Jadt can be expressed as follows:
Ko
(aRA(
T expl
144tu-2eetrdd -2train
where & a constant and is positive constant.
The proof of Theorem 5.6 can be found in Appendix
Theorem 5.7. Given an adversarially trained linear model fadv in Equation
(w) and a naturally Pi-p and Etest <
trained linear model fnat in Equation (F), and suppose 0+ T_. If Ctrain
+p
3
we have
2
(14 +.
VCRA(Jado) c(33-19) (
14 + p.
2
2
VCRA(fnot)
14 + F_ 22train
where K. is positive constant and S1, S2 two constants.
7

---

The proof of Theorem 1 can be found in Appendix H From Theorem 5 we can find (na)(f fado) Similar Theorem 5. we also demonstrate VCA((nat) S VCRA(Sado) in Theorem B.G of Appendix B.3. The results of Theorems 5..7 and B.d suggest that AT easily leads to severer robust fairness issue than natural training.
Fairly Adversarial Training
In this section, will propose novel fairly adversarial training (FAT) to mitigate the tradcoff between average robustness and robust fairness. The analytical results in Section E and Equation (IT)) show that control the variance of class-wise robust accuracy by controlling the variance of class-wise adversarial risk. Theorem 6. provide a theoretical guarantee for using the variance of class-wise adversarial risk to trade off average robustness against robust faimesss.
Denote E(0,2r,)) su,zeu((,e)) e 20, and the corresponding empirical adversarial risk Rodv (f) 1 Ei= E(0, xz yt). .. In addition, we denote the variance of class-wise adversarial risk (VCAR)
VCAR(f) = E (Rado (f Radv (adv(f))²
where Rado (S.i) TE(E(0, x.
2, x.y= = i)) and Radv = ( + Radv (S,
Theorem 6.1. Let (Ii, (522 yy), ..., (Jn, yn) ) be drawn independent and identically distributed (i..dd) from the unknown distribution D. Under uppropriate conditions on the loss E(-), parameter space Q with probability of least 6, the following holds for all E A:
VCAR(F) C
Radv S (fd +
The proof of Theorem 6. can be found in Appendix BEL Theorem b. shows that the variance of class-wise adversarial risk can control the robust risk of classifiers and the class-wise disparity of robustness. Inspired by Theorem 6.1. we propose to minimize the following object:
minl Rad (f) +VCCR(())
(11)
disparity
where the parameter À in Equation (II) captures the tradeoff between average robustness and robust fairness.
use Rado if to estimate ( where Roav(ff)) = 1 L i(0, x), and n; is
number of examples that belong to class i. The estimator of VCAR(J) can be expressed as follows:
VCAR()) - Σ (Bade Radv (f))
1=1
Usually, VCAR(f) is a biased but consistent estimator of VCAR(f). FAT can thus be formulated as follows:
min Radd (1) +VCAR((R)))
(12)
7 Experiments
In this section, we present the experimental results to validate the effectiveness of FAT for mitigating the tradeoff between average robustness and robust faimess.
7.1 Experimental Settings
We conduct our experiments on the benchmark datasets CIFAR-10 CIFAR-100 [[S]. Based
on ResNet18 [E]] compare proposed FAT with Madry [ on two datascts with different

---

Table 3: The variance of class-wise robust accuracy (10 3) for Madry and FAT using ResNet18 CIFAR-IO. The perturbation radii are chosen from Etrain {4/255, 8/255, 12/255, 16/ 16/255). The adversarial testing examples are generated by FGSM, PGD-20 and C&W with different testing perturbation radii
étest {16/255 20/255).
attack algorithm
FGSM
PGD-20
C&
16/255
20/25
16/255
20/255
16/255
20/255
Serain
PGD
FAI
PGD
FAT
PGD
FAT
PGD
FAI
PGD
FAC
PDG
FAT
4/255
1.90
1.31
0.10
0.02
0.01
0.08
0.02
0.01
8/255
2.34
2.18
1.99
1.69
0.50
0.46
0.16
0.12
0.61
0.52
0.19
0.14
12/258
3.12
2.99
2.53
2.47
0.98
0.84
0.35
0.35
0.90
0.42
16/255
3.52
3.32
2.99
2.90
1.28
1.06
0.48
0.40
1.38
1.22
0.60
0.51
Table The average robust accuracy (10-3) 3) for Madry FAT using ResNet18 on CIFAR-I0. The per- turbation radii are chosen from Frrain = [1/255, 8/255, 12/255, 66/255]. The adversarial test- ing examples are generated by FGSM, PGD-20 and C&W with different testing perturbation radii
Ctest (16/255, 20/255). We use bold to denote higher robust accuracy.
algorithm
FGSM
PGD-20
C&WW
16/255
20255
16/255
20W25
16/25
2W255
PGD
FAT
PGD
FAT
PGD
FAT
PGD
FAT
PGD
FAC
PDG
FAT
47255
29.04
29.33
21.74
22.65
3.54
1.12
3.61
1.05
0.99
8/255
39.00
38.57
32.13
32.26
11.23
11.29
4.96
5.26
12.14
12.32
5.35
5,48
12/255
41.99
42.77
35.71
36.71
19.43
19.74
10.03
10.97
19.39
19.88
10.53
11.48
16/255
42.97
43.40
37.34
37.98
22.50
22.79
12.86
13.11
22.03
22.21
12.65
12.99
perturbation radii étrain E {4/255, 8/255, 12/255, 16/255j. The maximum PGD step and step size are set to 20 and Etrain/ /10, respectively. For optimization, we use SGD with 0.9 momentum for 120 epochs. The initial learning rate is set to 0.1 and is divided by 10 at epoch 60 and epoch 80, respectively. The adversarial testing examples are generated by FGSM [I22]. PGD-20 [[Z]] and C&W [3]] with different testing perturbation radii étest = (16/255, 20/25 12/55 respectively. Here, present only the experimental results on CIFAR-10 in Tables @ and HL The experiment results of FAT and Madry CIFAR-100 can be found in Appendix C.I. Besides, the sensitivity analysis of the regularization hyperparameter in FAT is presented in Appendix
Our code can be found on
GitHub at tttps://githube wttp.//wwwwwaaa/////..a.
7.2 Experimental Results
Under the attack of FGSM, PGD-20 and C&W with different Gests Table & presents the variance of class-wise robust accuracy for FAT and Madry using ResNet18 on CIFAR-1O. From table B. we can observe that FAT significantly reduces the variance of class-wise robust accuracy of Madry for various perturbation radii étrain. The results from Tables E and @| further show that with increasing Etrain, the average robust accuracy and the variance of class wise robust accuracy of FAT increase, which is consistent with our theory in Section EL However, compared with Madry, FAT consistently obtains smaller variance of class-wiss wise robust accuracy while achieving comparable average robust accuracy different Grrin and attack algorithms. For example, with the increasing of Crain from 4/255 to 16 /255, FAT reduces the variance of class-wise robust accuracy of Madry by around 16% under the C&W attack with Grest 16 16/255. These experimental results suggest that FAT effec- tively suppress the growth of variance of class-wise wise robust accuracy while achieving an competitive average robust accuracy compared with Madry with the increasing of étrain. The results of FAT and TRADES in Appendix [ also present conclusions consistent with those drawn here. Therefore, FAT can alleviate the robust fairness problem and mitigate the tradeoff between average robustness and robust fairness.
8 Conclusion
In this work, we empirically find that there tradeoff between average robustness and robust fairness: higher average robust accuracy of models is accompanied by severer robust fairess issues with the increasing of training perturbation radius. Our theoretical results explain our observations. We then theoretically demonstrate that AT indeed leads to severer robust fairness issue compared with natural training. Moreover, a novel method is proposed to mitigate the tradeoff between average robustness and robust faimess. Our experimental results verify the superiority of our proposals.

---

Acknowledgements
This work is supported by the National Natural Science Foundation of China under Grant 61976161.
References
[1] Peter L.. Bartlett, Olivier Bousquet, and Shahar Mendelson. Local rademacher complexities. The Annals of Statistics, 33:1497-15337 2005.
[2] Philipp Benz, Chaoning Zhang, Adil Karjauv, and In So Kweon. Robustness may be odds with faimess: An empirical study on class-wise accuracy. arXiv preprint arXiv:2010. 13365, 2020.
[3] Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy(SP), pages 17-57,20 2017.
[4] Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C. Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In NewrtPS, pages 11190- 11201, 2019.
[5] Simon Caton and Christian Haas.
Fairness in machine lcarning: A survey. arXiv preprint
arXiv:2010.0444 0,00000,,,00 2020.
[6] Edgar Dobriban, Hamed Hassani, David Hong, and Alexander Robey. Provable tradeoffs in adversarially robust classification. arXiv preprint arXiv:200.00161,, 2020.
[7] Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks deep learning visual classification. In CVPR, pages 1625-1634, 2018.
[8] Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Analysis of classifiers' robustness to adversarial perturbations. Machine Learning, 107-3):81--088 2018.
[9] Xiuwen Gong, Jiahui Yang. Dong Yuan, and Wei Bao. Generalized large margin Skšan for partial label learning. IEEETransactions Multimedia, 24:1055- 2022.
[10] Xiuwen Gong, Dong Yuan, and Wci Bao. Discriminative metric learning for partial label learning. IEEE Transactions on Neural Networks and Learning Systems, 2021.
[11] Xiuwen Gong, Dong Yuan, and Wei Bao. Top-k partial label machine. IEEE Trunsactions on Neurul Networks and Learning Systems, 2021.
[12] lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial exam- ples. In ICLR, 2015.
[13] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.
[14] Adel Javanmard, Mahdi Soltanolkotabi, and Hamed Hassani. Precise tradeoff's adversarial training for linear regression. In Jacob D. Abernethy and Shivani Agarwal, editors, COLT, volume 125, pages 2034-2078 2020.
[15] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[16] Weiwei Liu, Donna Xu, Ivor Tsang, and Wenjic Zhang. Metric learning for multi-outpun tasks. JEEE Transactions on Pattern Analysis and Machine Intelligence, 41:408-422, 2019.
[17] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018
[18] Ninarch Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and faireess in machine learning. ACM Computing Surveys, 54(6):115:) 15::1111::111355 2021.
[19]
Soyed-Mohsen Moosavi-Dezie Dezfooli, Alhusscin Fawzi, and Pascal Frossard. Decpfool: A simple and accu- rate method fool deep neural networks. CVPR, pages 2574-2582, 2016.
[20] Vedant Nanda, Samuel Dooley, Sahil Singla, Soheil Feizi, and John P. Diekerson. Faimess through robustness: Investigating robustness disparity in deep learning. In FAceT, pages 466 2021.
[21] Tianyu Pang, Xiao Yang. Yinpeng Dong. Taufik Xu, Jun Zhu, and Hang Su. Boosting adversarial training with hypersphere embedding. In NeurIPS, 2020.
10

---

[22] Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversari- ally robust generalization requires more data. In NeurlPS, pages 5019-5031, 2018.
[23] Dimitris Tsipras, Shihani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robust- may be odds with accuracy. In ICLR, 2019.
[24] Zekai Wang and Weiwei Liu. Robustness verification for contrastive learning. In ICML, volume 162, pages 22865-22883, 2022.
[25] Eric Wong, Lestie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In ICLR, 2020.
[26] Han Xu, Xiaorui Liu, Yaxin Li, Anil K. Jain, and Jiliing Tang. To be robust or to be fair: Towards faimess in adversarial training. In JCML,, volume 139, pages 11492-11501, 2021.
[27] Dong Yin, Kannan Ramchandran, and Peter L. Bartlett. Rademacher complexity for adversarially robust generalization. In ICML, volume 97, pages 7085-7094, 2019.
[28] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv: 1605.07146. 2016.
[29] Hongyang Zhang. Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, volume 97, pages 7472- 7482, 2019.

---

Checklist
1. For all authors...
(a) Do the main claims made the abstract and introduction accurately reflect the paper's contri- butions and scope? [Yes]
(b) Did you describe the limitations your work? [N/A]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethies review guidelines and ensured that your paper conforms to them?
[Yes]
2. If you are including theoretical results...
(a) Did you the full of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed reproduce the main experimental results (either in the supplemental material or as URL)? [Yes]
(h) Did you specify all the training details (e.g., . data splits, hyperparameters, how they were cho-
sen)? [Yes]
(c) Did you report error bars (e.g., with respect to random seed after running experiments mul- tiple times)? [N/A]
(d) Did you include total amount of compute and the type resources used (e.g.. type GPUs, internal cluster, or cloud provider)? [N/A]
4. If you using existing assets (e.g.. code, data, models) curating/releasing assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention license of the assets? [N/A]
(c) Did you include any new either in the supplemental material or as URL? [Yes]
(d) Did you discuss whether and how consent obtained from people whose data you're us-
ing/curating? [N/A]
(c) Did you discuss whether the data you using/curating contains personally identifiable infor- mation or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you deseribe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(C) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "qNH9jfU3Zv7", "reviewer": "Reviewer_zHuG", "summary": "This paper studies an important and interesting topic in adversarial training, i.e., robust fairness issue. The robust fairness issue indicates that the adversarially trained model tends to perform well in some classes and poorly in other classes, creating a disparity in the robust accuracies for different classes. This paper focuses on the influence of perturbation radii on the robust fairness problem and finds that there is a tradeoff between average robust accuracy and robust fairness. To mitigate this tradeoff, the authors present a method to correct the lack of fairness in adversarial training.", "questions": "Does the phenomenon in Table 2 still exist for other datasets?", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper empirically finds a new phenomenon that AT with a larger perturbation radius will lead to a larger variance of class-wise robust accuracy while average robust accuracy is improved. Concretely, the authors use Madry and TRADES with different neural network architectures and attack algorithms to conduct extensive experiments to prove the generality of this new phenomenon. Experimental results consistently support their finding.\n2. The authors provide a theoretical explanation for their experimental results through a linear model. Then, this paper also provides evidence for the existence of a tradeoff between robust accuracy and robust fairness.\n3. The authors present a method to correct the lack of fairness of the AT. Simultaneously, the authors empirically and theoretically show the effectiveness of the proposed method to obtain better robust fairness without significantly altering the robust accuracy of the model.\n\nWeaknesses:\n1. The paper is well-written overall, but it would benefit from a background section (before Section 3) that introduce some of the terminology and definitions for readers who are not familiar with the adversarial training literature.\n2. There is too little space devoted to explaining experimental results, it is expected to proivide a more detailed discussion and analysis.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ITJJADWLpP", "reviewer": "Reviewer_8YA2", "summary": "This paper studies the relation between the variance of class-wise robust accuracy and perturbation radius in adversarial training. It finds that a tradeoff exists between average robustness and robust fairness. Theoretically, this paper analyzes this phenomenon of the tradeoff and provides an understanding. Finally, this paper proposes a variance based regularized method to mitigate the tradeoff. The idea of this paper is direct and reasonable, and the experimental results demonstrate the effectiveness of the proposed method. ", "questions": "The theorems in this paper built on a simple linear model. How to extend them to DNN model with multi classification?\n\nFor the Theorem 4.5, does the situation σ_+ 〖>σ〗_- equal to the situation σ_+ 〖<σ〗_-? How the variance VCRA changes if σ_+ 〖>σ〗_-?\n\nCan the methods in [1][2] also mitigate the variance of class-wise robust accuracy?\n\n[1] Han Xu, Xiaorui Liu, Yaxin Li, Anil K. Jain, and Jiliang Tang. To be robust or to be fair: Towards fairness in adversarial training. In ICML, 2021.\n[2] Qi Tian, Kun Kuang, Kelu Jiang, Fei Wu, and Yisen Wang. Analysis and applications of class-wise robustness in adversarial training. In KDD, 2021.\n", "limitations": "N.A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper is clearly written and is easy to follow.\n\n2.\tThis paper reveals fairness of robust accuracy among different classes from a theoretical point of view.\n\n3.\tTheoretical proof and analysis are sufficient.\n\n4.\tExtensive experiments are shown to demonstrate the effectiveness of the proposed framework.\n\nWeakness:\n\n1.\tFigures 1 (a), (b) and figure 2 (a), (b) are incomplete, and some contents are not displayed well. \n\n3.\tThere are some grammatical errors, like “a understanding …” in line 34. \n\n4.\tIf I understand correctly, the “sign(t)” should take “-1” for “otherwise” rather than “0” in line 145.\n\n5.\tFor the equation between line 245 and 246 in Theorem 5.1, there is an extra plus sign.\n\n6.\tI am not sure that if Φ(·) in Theorem 4.2 is the probability distribution function. In the line 419 in appendix, the authors describe it as a cumulative distribution function.\n\n7.\tThere are some typos in appendix: “P(f(x) = +1|y = +1)” in equation 19 should be “P(f(x) = +1|y = -1)”, “Rnat (f,1)” in line 411 should be “Rnat (f,-1 )”.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2uT-W_DAtRV", "reviewer": "Reviewer_F5Kr", "summary": "The authors investigate the robust fairness phenomenon in adversarial training and study the tradeoff between average robustness and robust fairness.  They find that with increasing perturbation radii the average robust accuracy improves over classes while the class-wise robust accuracy suffers from increased variability.  The authors propose a Fairly Adversarial Training (FAT) approach to mitigate the tradeoff between robustness and fairness in adversarial training.", "questions": "What is the significance of the results and how can this work impact other areas?", "limitations": "see above", "rating": 3, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The general problem is clear and seems an important problem to address.\n\nThe authors should provide more motivation for why the problem is important to explore.  An example would be helpful here.\n\nSection 3 is difficult to read and the definitions in section 4 should come before this section.  The experiments used to generate the results in this section also need much more information in regards to the data, application, models and training/evaluation strategy for this to be self-contained and fully convincing.\n\nMore detail on explaining FAT and how it is used in section 5 is needed.  This section is not clear.\n\nThe experiment section should be expanded in the paper or appendix to examine in more detail the significance of the results.  This is not currently clear.\n\nIt is not clear how large the impact this work will have in its current state.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept