Title: Revisiting Adversarial Robustness Distillation from the Perspective of Robust Fairness
Venue: NeurIPS 2023
Accepted: yes
Keywords: Deep Learning

Knowledge Distillation

Adversarial Training

Fairness
Abstract: Adversarial Robustness Distillation (ARD) aims to transfer the robustness of large teacher models to small student models, facilitating the attainment of robust performance on resource-limited devices. However, existing research on ARD primarily focuses on the overall robustness of student models, overlooking the crucial aspect of $\textit{robust fairness}$. Specifically, these models may demonstrate strong robustness on some classes of data while exhibiting high vulnerability on other classes. Unfortunately, the "buckets effect" implies that the robustness of the deployed model depends on the classes with the lowest level of robustness. In this paper, we first investigate the inheritance of robust fairness during ARD and reveal that student models only partially inherit robust fairness from teacher models. We further validate this issue through fine-grained experiments with various model capacities and find that it may arise due to the gap in capacity between teacher and student models, as well as the existing methods treating each class equally during distillation. Based on these observations, we propose $\textbf{Fair}$ $\textbf{A}$dversarial $\textbf{R}$obustness $\textbf{D}$istillation (Fair-ARD), a novel framework for enhancing the robust fairness of student models by increasing the weights of difficult classes, and design a geometric perspective-based method to quantify the difficulty of different classes for determining the weights. Extensive experiments show that Fair-ARD surpasses both state-of-the-art ARD methods and existing robust fairness algorithms in terms of robust fairness (e.g., the worst-class robustness under AutoAttack is improved by at most 12.3\% and 5.3\% using ResNet18 on CIFAR10, respectively), while also slightly improving overall robustness. Our code is available at: [https://github.com/NISP-official/Fair-ARD](https://github.com/NISP-official/Fair-ARD).

Revisiting Adversarial Robustness Distillation from the Perspective of Robust Fairness

Abstract

Adversarial Robustness Distillation (ARD) aims to transfer the robustness of large teacher models to small student models, facilitating the attainment of robust performance on resource-limited devices. However, existing research on ARD primarily focuses on the overall robustness of student models, overlooking the crucial aspect of robust fairness . Specifically, these models may demonstrate strong robustness on some classes of data while exhibiting high vulnerability on other classes. Unfortunately, the "buckets effect" implies that the robustness of the deployed model depends on the classes with the lowest level of robustness. In this paper, we first investigate the inheritance of robust fairness during ARD and reveal that student models only partially inherit robust fairness from teacher models. We further validate this issue through fine-grained experiments with various model capacities and find that it may arise due to the gap in capacity between teacher and student models, as well as the existing methods treating each class equally during distillation. Based on these observations, we propose Fair A dversarial R obustness D istillation (FairARD), a novel framework for enhancing the robust fairness of student models by increasing the weights of difficult classes, and design a geometric perspective-based method to quantify the difficulty of different classes for determining the weights. Extensive experiments show that Fair-ARD surpasses both state-of-the-art ARD methods and existing robust fairness algorithms in terms of robust fairness (e.g., the worst-class robustness under AutoAttack is improved by at most 12.3% and 5.3% using ResNet18 on CIFAR10, respectively), while also slightly improving overall robustness. Our code is available at: https://github.com/NISP-official/Fair-ARD.

1 Introduction

Knowledge distillation [15, 29, 19, 5] is currently a popular technique for transferring knowledge from large models (teacher models) to small models (student models). This practice is particularly vital in resource-constrained environments such as mobile or embedded devices. However, as neural networks are notably vulnerable to adversarial examples [31, 10], where the addition of small perturbations to input examples can potentially lead to misclassification, student models obtained through knowledge distillation are also susceptible to these adversarial attacks [4, 7, 2, 8]. Therefore, Adversarial Robustness Distillation (ARD) [9, 44, 45, 43] have been proposed to train robust student models by integrating adversarial training (AT) [24, 40, 34, 37].

However, some recent works [38, 33] have revealed the issue of robust fairness in AT, where adversarially trained models may exhibit high robustness on some classes while demonstrating significantly low robustness on others. Due to the "buckets effect," the security of a system often depends on the security of its weakest component. Therefore, even if the overall robustness of a

model is high, poor robustness on a specific class of data can pose security issues. For instance, an autonomous driving system that demonstrates high robustness for inanimate objects on the road but lacks robustness when detecting pedestrians may be misled by adversarial examples, leading to traffic accidents.

Thus, we raise a question: how about the robust fairness of ARD that also incorporates adversarial examples into training like AT? Previous works have only concentrated on the overall robustness of ARD, overlooking the aspect of robust fairness. In this paper, we, for the first time, investigate whether ARD can transfer robust fairness from teacher models to student models. Unfortunately, experimental results provide a negative answer to this question, revealing that student models can only partially inherit robust fairness from their teacher counterparts.

Then, we reveal that the main reason lies in the capacity gap between the teacher model and the student model, and the common strategy of the prior works that treats the knowledge about each class in the same way during the distillation process. For instance, in the CIFAR-10 dataset [20], the 'car' class is relatively easy for the teacher model to learn, and the student model can fit it well. By contrast, for the 'cat' class, learning its knowledge is already challenging for the teacher model. Therefore, expecting a student model with significantly reduced capacity to attain the same level of robustness as the teacher model becomes even more difficult. This implies that the equitable treatment of each class in previous methods [9, 44, 45, 43] may be inappropriate regarding robust fairness. It becomes challenging for the student model to effectively learn the knowledge from the teacher model regarding difficult classes.

In light of these observations, we argue for treating different classes distinctively during distillation, focusing more on difficult classes and less on easy ones (those student models can fit relatively easily). While previous methods [38] have incorporated re-weighting schemes to address robust fairness in AT, a closer examination of their quantification of example difficulty suggests that their metric for measuring class difficulty is intuitively coarse-grained, potentially lacking precision in assessing the actual difficulty of classes. From a geometric perspective, examples farther from the decision boundary are less likely to be misclassified, thereby implying lower difficulty. Therefore, the distance from an example to the decision boundary can serve as a metric for example difficulty. Comparative analysis indicates that this metric provides finer granularity. Additionally, the average difficulty of examples within a class can be employed to measure the class's difficulty. Drawing inspiration from these insights, we introduce a novel, more fine-grained class difficulty metric. We further devise a unique re-weighting scheme, adaptively assigning different weights to classes with varying levels of difficulty during distillation, thereby mitigating the bias of student models towards knowledge of different classes.

Our contributions can be summarized as follows:

To the best of our knowledge, we are the first to investigate the issue of robust fairness in adversarial robustness distillation. We reveal that the student model only partially inherits the robust fairness of the teacher model and present the internal reasons.

We present the Fair Adversarial Robustness Distillation (Fair-ARD) framework, which employs a more refined difficulty metric and an adaptive class re-weighting approach, enabling the student model to learn the knowledge about each class from the teacher model in a fairer way. Fair-ARD is a versatile method: existing ARD techniques such as IAD [44], RSLAD [45], and MTARD [43] can also be modified into Fair-IAD, Fair-RSLAD, and Fair-MTARD, respectively.

Extensive experiments validate that Fair-ARD can significantly improve the robust fairness of the student model and slightly improve the overall robustness, outperforming state-of-theart ARD methods and existing robust fairness algorithms.

2 Can the Student Model Inherit the Robust Fairness of the Teacher Model?

In this section, we investigate the capability of the student model to inherit robust fairness from the teacher model. Due to the space limitation, we extensively introduce the related works in Appendix B, including AT, ARD, robust fairness, etc.

FGSM

PGD

C&W

AA

Attack

0.20

0.25

0.30

0.35

0.40

NSD

Teacher

ARD

IAD

RSLAD

MTARD

(a) ResNet18

2

4

6

8

10

W

0.25

0.30

0.35

0.40

0.45

NSD

Teacher

ARD

IAD

RSLAD

MTARD

(b) WRN-34-W

10

16

22

28

34

D

0.26

0.28

0.30

0.32

0.34

NSD

(c) WRN-D-10

18 34 50

101

152

X

0.26

0.28

0.30

0.32

0.34

NSD

(d) ResNet-X

2.1 The Degree of Robust Fairness Inheritance

Prior works about ARD [9, 44, 45, 43] mainly focused on improving the overall robustness of the student model. However, following the observation of robust fairness concerns with AT, the usability of ARD should be carefully evaluated to avoid potential security risks. In this section, we present a detailed analysis of the ARD methods in robust fairness.

Specifically, we evaluate the Normalized Standard Deviation (NSD 2 ) of the class-wise robustness for the teacher model and the student models trained by four ARD methods: ARD [9], IAD [44], RSLAD [45], and MTARD [43]. The results are shown in Fig. 1(a). The details about hyperparameters and the methods for generating adversarial examples are in Section 4.1. The results show that all the student models consistently achieve higher NSD than the teacher model, i.e., the robust fairness of the student models is worse than the teacher model. In other words, student models can only partially inherit the teacher model's robust fairness.

2.2 Influential Factors in Robust Fairness Inheritance

It is well-known that the gap in model capacity accounts for the inferior performance of the student model compared to the teacher model in knowledge distillation [25, 5]. Naturally, this leads us to consider whether the robust fairness inheritance is also related to this gap. We vary the capacity of the student model (by adjusting the depth or width of the model [28]) and analyze the change of NSD. As shown in Fig. 1 (b) to (d), for a given teacher model, the NSD decreases with the increasing of the student model's capacity, approaching or even falling below the teacher model's NSD. This confirms our hypothesis that the capacity gap between models leads to the inferior robust fairness of the student model compared to the teacher model.

Next, we conduct a more detailed evaluation of the relationship between the capability of the student model and the robust fairness it inherits. We calculate the proportion of robustness inherited by the student model for each class from the teacher model. Table 1 presents the results for the least robust class (cat) and the most robust class (car) in the teacher model. We can observe that the hardest class

(cat) inherits less robustness than the easiest class (car). It reveals that the learning difficulty for the student model varies among different classes. For instance, the 'car' class, whose knowledge is relatively easy, is well fitted by the student model with smaller capacity, inheriting about 94% of the teacher model's robustness under AutoAttack (AA) [8] for these four distillation methods. In contrast, the 'cat' class presents increased difficulty, with the teacher model only achieving 26.8% robustness under PGD-20 [24]. This decline amplifies the limitation for the student model, which only inherits less than 80% of the teacher's robustness under AA for ARD, IAD, and MTARD. More results about other classes are available in Appendix C.2. Therefore, considering the inherent difference in difficulty among different classes, the practice of prior ARD methods that treat all classes equally poses an obstacle to the inheritance of robust fairness. This could potentially result in a case where student models may effectively learn knowledge of easy classes while hard to learn knowledge of difficult classes. Consequently, we should assign larger (smaller) weights to harder (easier) classes, allowing student models to focus more on the teacher model's knowledge about hard classes.

3 Methodology: Fair Adversarial Robustness Distillation

In Section 2, we observe that treating all classes equally poses an obstacle to the inheritance of robust fairness. An intuitive way to mitigate this issue is to assign different weights to different classes. Let ω i be the weight for the i -th class; we formulate a fair adversarial robustness distillation (Fair-ARD) method as follows:

where x j i is the j -th example of the i -th class, ˜ x j i is the adversarial example of x j i , y i is the label of the i -th class, n i is the number of examples of the i -th class, C is the total number of classes, L ARD is the loss of ARD [9], S is the student model with parameter θ S , T is the teacher model, τ is the temperature constant added to the softmax operation, α is a hyperparameter in ARD, CE is the Cross-Entropy loss, and KL is the Kullback-Leibler divergence loss.

3.1 Metrics to Measure Class Difficulty

The main challenge is determining the optimal weights. Hence, it is necessary to propose a metric for quantifying class difficulty to calculate the weights. Prior works about improving the robust fairness of AT, such as Fair Robust Learning (FRL) [38], measure the class difficulty by the class-wise robustness, i.e., the expectation of the successful defense of adversarial examples within a given class. Let r i represent the robustness of the i -th class, and it can be formulated as

where ˜ x is the adversarial example of the natural example x , y is the label of x . 1 ( · ) is the indicator function, which outputs 1 when S (˜ x ) = y and 0 otherwise. Thus, the difficulty of an example, denoted by φ ( x, y ) , can only be 1 or 0, representing defense success or failure, respectively. And we refer to this metric of example difficulty as FRL Example Difficulty ( FED ).

However, we are concerned that such a coarse-grained method of quantifying example difficulty can be further refined to obtain more precise results. In Support Vector Machines (SVMs) [6], the distance of an example to the decision boundary can be utilized to measure the difficulty of that example, offering a broader range and finer granularity in difficulty measurement. For linear SVMs, where the decision boundary is a hyperplane, distance computation is relatively straightforward. However, in DNNs, the decision boundary in high-dimensional space might constitute a complex, nonlinear surface, rendering the calculation of example-to-boundary distance challenging. Inspired by GAIRAT [41], the least PGD steps ( LPS ), representing the steps needed for adversarial perturbations of a natural example to cross the decision boundary, can be employed as an approximation for the

1

2

3

4

5

8

6

7

9

10

FED=0

LPS=1

FED=0

LPS=6

FED=0

LPS=10

FED=1

LPS=10

Natural Example

Adversarial Example

Adversarial Intermediate Variant

1~10

PGD Step

Adversarial Direction

1

2

3

4

5

8

6

7

9

10

1

2

3

4

5

8

6

7

9

10

1

2

3

4

5

8

6

7

9

10

Decision Boundary

x

1

x

1

x

2

x

3

x

4

x

2

x

3

x

4

distance from the example to the decision boundary. Specifically, for data point ( x, y ) , the distance d ( x, y ) from the data point ( x, y ) to the decision boundary is calculated as follows:

̸

where ˜ x ( t ) is the adversarial example of the natural example x at PGD step t , K is the number of steps, γ is the step size, ℓ is the loss function, e.g. CE loss, and Π B ϵ [ x ] ( · ) is the projection function that projects the adversarial data back into the ϵ -ball centered at x when necessary.

Fig. 2 presents an example to show the difference between FED and LPS. For three different natural examples, x 1 , x 2 , and x 3 , their FED values are all 0, but their LPS values are 1, 6, and 10, respectively. Thus, LPS can effectively distinguish difficult levels between different examples, providing a more fine-grained measure for quantifying example difficulty than FED. We will provide more experimental results to show the differences between the two metrics in Section 5.

Then, the average LPS of all examples within a class can serve as a metric of class difficulty. Therefore, let κ i denote the average LPS for all examples in the i -th class, and it can be calculated as follows:

3.2 The Re-weighting Strategy

After defining the metric for measuring class difficulty, we need to design a re-weighting function to allocate appropriate weights for each class based on its difficulty. Following the principle of giving larger (smaller) weights to harder (easier) classes, the weight ω i should decrease w.r.t. κ i . Therefore, we heuristically design several re-weighting functions, and here we provide an example. More examples of re-weighting functions are discussed in Appendix C.8.

where β is a hyperparameter controlling the smoothness of the re-weighting function. To make the total loss roughly in the same scale when applying ω i , we normalize ω i so that 1 C ∑ C i =1 ω i = 1 .

Formally, given an example from the i -th class, we propose to add a weight factor of 1 /κ i β in the loss function, where the hyperparameter β ∈ [0 , ∞ ) . If β = 0 , then ω i = 1 , corresponding to no re-weighting, i.e., Fair-ARD recovers to vanilla ARD with equal weights for all classes.

ARD

IAD

RSLAD

MTARD

0.20

0.25

0.30

0.35

0.40

NSD

FGSM

ARD

IAD

RSLAD

MTARD

0.20

0.25

0.30

0.35

0.40

PGD

ARD

IAD

RSLAD

MTARD

0.20

0.25

0.30

0.35

0.40

C&W

ARD

IAD

RSLAD

MTARD

0.20

0.25

0.30

0.35

0.40

AA

Teacher

Vanilla

Fair

Due to the requirement of computing the weights for each class based on the entire dataset, ω i cannot be updated by mini-batch like GAIRAT. Instead, it is calculated based on the model from the previous epoch. To demonstrate the generality of the proposed re-weighting strategy, we apply it to four ARD methods: ARD [9], IAD [44], RSLAD [45], and MTARD [43]. And the algorithms of their fair version Fair-ARD, Fair-IAD, Fair-RSLAD, and Fair-MTARD are in Appendix A.

4 Experiments

4.1 Experimental Setup

Baselines. We conduct the experiments on four benchmark datasets, CIFAR-10, CIFAR-100 [20], SVHN [27], and Tiny-ImageNet [21]. Due to the space limitation, results for datasets beyond CIFAR10 are presented in Appendix C. We consider seven baseline methods, including natural training, two AT methods: SAT [24] and TRADES [40], and four distillation methods: ARD[9], IAD[44], RSLAD[45], and MTARD[43].

Teacher-Student Pairs. Following [45, 43], the used teacher models include a naturally trained ResNet-56 [13] and a robust WideResNet34-10 [39] trained with TRADES. Student models include ResNet-18 and MobileNetV2 [30].

Training Settings. We use the stochastic gradient descent (SGD) optimizer with an initial learning rate of 0.1, a momentum of 0.9, and a weight decay of 2e-4 to train the networks. The batch size is set as 128. For the baseline methods, i.e., SAT, TRADES, ARD, IAD, RSLAD, and MTARD, we

ARD

IAD

RSLAD

MTARD

0.20

0.25

0.30

0.35

0.40

NSD

FGSM

ARD

IAD

RSLAD

MTARD

0.20

0.25

0.30

0.35

0.40

PGD

ARD

IAD

RSLAD

MTARD

0.20

0.25

0.30

0.35

0.40

C&W

ARD

IAD

RSLAD

MTARD

0.20

0.25

0.30

0.35

0.40

AA

Teacher

Vanilla

Fair

strictly follow their original settings. For the version improved by our proposed method, namely Fair-ARD, Fair-IAD, Fair-RSLAD, and Fair-MTARD, we also follow the original settings of the non-fair training version. Additionally, we search for the hyperparameter β on CIFAR10 using Fair-ARD and determine β = 2 . We adopt β = 2 for all other fair adversarial robustness distillation methods and other datasets. All the results are reported as the average of five runs, and standard deviations are omitted because they are too small.

Evaluation Metrics. We evaluate the model using four typical attacks: FGSM [10], PGD [24], C&W[4], and AutoAttack (AA) [8]. For both datasets, the maximum perturbation is set to ϵ = 8 / 255 , and the perturbation steps for both PGD and C&W are set to 20 following [45]. And following [38], we use the worst-class robustness and NSD to measure the robust fairness of the student model. The average and worst-class robustness of teacher models under the four attacks can be found in Appendix C.1.

4.2 Robustness and Fairness

As shown in Table 2 and Table 3, after applying Fair-ARD, the models have a significant improvement in robust fairness. In addition, in most of the results, there is a slight improvement in the average accuracy and robustness as well. Our method achieves state-of-the-art worst-class robustness under all attacks, surpassing even the worst-class robustness of the teacher model. These results confirm that Fair-ARD outperforms all baseline methods in inheriting robust fairness from the teacher model. Considering that we fine-tune the hyperparameter using ResNet-18 on Fair-ARD, it demonstrates the most significant enhancement, with the worst-class robustness under AA experiencing a substantial

1

4

7

10

0.00

0.25

0.50

0.75

1.00

= 0

= 0.1

= 0.5

= 1

= 2

= 3

0

0.1 0.5

1

2

3

51.00

51.25

51.50

51.75

52.00

0

0.1 0.5

1

2

3

20

25

30

35

80

81

82

83

84

85

Accuracy(%)

Avg. Acc

60

62

64

66

68

70

Accuracy(%)

Worst Acc

12.3% improvement. If the hyperparameter were fine-tuned on other fair methods, e.g., Fair-IAD, they could potentially exhibit further improvements.

For MTARD, we observe that while Fair-ARD can improve its robust fairness, it may decrease clean accuracy. This is due to the workflow of MTARD. The student model is simultaneously guided by a clean model and a teacher model, where the weights of these two models sum up to 1. And Fair-ARD aims to enhance robust fairness. Consequently, the robust teacher's weight is potentially increased during the training process, naturally leading to a decrease in the clean teacher's weight, thereby reducing the clean accuracy.

Then, we present the NSD of both the vanilla ARD and Fair-ARD across various models in Fig. 3 and Fig. 4. The results show that Fair-ARD effectively reduces the NSD of the student model, especially in most experimental results where the NSD of the student model even falls below that of the teacher model. This further demonstrates the superiority of our method in enhancing robust fairness.

4.3 Ablation Studies

To obtain a comprehensive understanding of Fair-ARD, we investigate the effect of β . Additional ablation studies on different re-weighting functions and teacher models are presented in Appendix C.8 and C.9, respectively. Specifically, we choose ResNet-18 as the student model and ARD as the baseline method; other settings are the same as in Section 4.1. We first visualize the value of ω with different β in the left panel of Fig. 5. With the increase of β , the re-weighting magnitude and the weight gap among classes also increase, and student models will pay more attention to the knowledge of difficult classes.

In the middle and right panels of Fig. 5, we present the average and worst-class performance of the student model obtained with different β , respectively. Note that when β = 0 , all classes are considered equally, i.e., Fair-ARD is the same as ARD in this case. We can observe that the average robustness of Fair-ARD is worse than ARD when β is too small. However, when β > 1 , Fair-ARD exhibits higher robustness compared to ARD while almost not sacrificing accuracy. Additionally, the worst-class robustness and accuracy of Fair-ARD across various β values surpass those of ARD, highlighting the superiority of our method. In particular, when β = 2 , the student model achieves the highest worst-class robustness. Therefore, we adopt this setting in our experiments. Moreover, the worst-class robustness of Fair-ARD initially increases and then decreases as β increases. The reason might be that an excessively large β leads to a significant weight disparity among classes, resulting in unstable training.

5 Discussion

In this section, we discuss the differences between our Fair-ARD and related works about (fair) adversarial training with detailed experimental results. More discussions are provided in Appendix C.10.

5.1 Comparison with GAIRAT

GAIRAT [41] is an instance-reweighted adversarial training algorithm, and our metric for measuring class difficulty is designed based on the LPS proposed by GAIRAT. We highlight several differences between our Fair-ARD and GAIRAT. Firstly, Fair-ARD focuses on both overall robustness and robust fairness, while GAIRAT only concerns overall robustness. Additionally, GAIRAT can be viewed as finding solutions for each individual example, whereas Fair-ARD seeks solutions for an entire class of examples. Finally, the re-weighting functions of the two are also different: GAIRAT' function is a tanh-type function, while ours is a power function. And the comparison of different re-weighting functions is provided in Appendix C.8.

For a fair comparison, we applied the GAIRAT method to ARD, resulting in a variant we refer to as GAIR-ARD. As shown in Table 4, although GAIR-ARD demonstrates higher worst-class robustness under the PGD attack than ARD, it still does not outperform our Fair-ARD. Furthermore, under the AutoAttack, the average robustness and robust fairness of GAIR-ARD are both significantly lower than our Fair-ARD. Upon investigating the reason, we observed the weights assigned to each example during the training process of GAIR-ARD and found that the disparities among different examples' weights were substantial, even differing by several orders of magnitude. This may lead to subtle gradient masking [3, 16], resulting in a substantial drop in robustness when facing AA. Instead, our Fair-ARD employs a re-weighting function that acts on the classes, and the hyperparameter β can control the re-weighting amplitude within a smaller range. Consequently, Fair-ARD does not suffer from gradient masking, as evidenced by the results under AA.

5.2 Comparison with Robust Fairness Algorithms

FRL [38], FAT [23], and CFA [35] are the state-of-the-art methods for improving the robust fairness of AT. FAT enhances robust fairness by controlling the variance of class-wise adversarial risk, which offers a distinct perspective compared to methods like FRL, CFA, and Fair-ARD. Hence, we mainly focus on comparing our Fair-ARD with both FRL and CFA, as they all employ the re-weighting approach. Firstly, they use different metrics to quantify the class difficulty. FRL and CFA use the class-wise robustness on the validation set, while we utilize the average distance between examples and the decision boundary, which offers a finer granularity, as shown in Fig. 2. Additionally, their re-weighting functions differ from ours. FRL's function is an iterative algorithm based on the difficulty metric, with more than four hyperparameters, and CFA applies a fractional function based on class difficulty, whereas we mainly use power functions for re-weighting. We make more discussions about several types of re-weighting functions in Appendix C.8. Finally, CFA utilizes the exponential moving average (EMA), while FRL and Fair-ARD do not employ it.

To show the advantages of our proposed method compared to FRL, FAT, and CFA, we also employed them in ARD, namely FRL-ARD, FAT-ARD, and CFA-ARD, respectively. The results in Table 4 demonstrate that while FRL and FAT can improve the worst-class robustness of ARD, they also lead to a slight decrease in the average robustness. On the other hand, Fair-ARD outperforms both FRL-ARD and FAT-ARD in terms of both average and worst-class robustness. Most notably, Fair-ARD improves the worst-class robustness under AA by 5.3% compared to FRL-ARD. This also provides rough evidence that the metric LPS is superior to FED. Regarding CFA, while CFA-ARD outperforms ARD in terms of robust fairness, it falls short when compared to Fair-ARD. This distinction highlights the superiority of Fair-ARD in enhancing robust fairness.

6 Conclusion

In this paper, we conducted an in-depth investigation into the issue of robust fairness in ARD. We revealed that the student model obtained through ARD fails to fully inherit the robust fairness from the teacher model. We discovered that this issue may arise due to the capacity gap between the teacher and student models, and previous ARD methods treated the knowledge of all classes equally. Based on these observations, we proposed Fair-ARD, which utilizes a refined class difficulty metric from a geometric perspective and a re-weighting strategy for distinct classes to enhance the robust fairness of the student model. Extensive experiments showed that Fair-ARD significantly improves the robust fairness of student models compared to existing methods while maintaining high overall robustness. Our work can help in establishing adversarially robust and fair lightweight deep learning models.

References

Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In NeurIPS , 2019.

Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In ECCV , 2020.

Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML , 2018.

Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In S&P , 2017.

Jang Hyun Cho and Bharath Hariharan. On the efficacy of knowledge distillation. In ICCV , 2019.

Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning , 1995.

Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In ICML , 2020.

Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML , 2020.

Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In AAAI , 2020.

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR , 2015.

Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593 , 2020.

Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In NeurIPS , 2016.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR , 2016.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV , 2016.

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 , 2015.

Dorjan Hitaj, Giulio Pagnotta, Iacopo Masi, and Luigi V Mancini. Evaluating the robustness of geometry-aware instance-reweighted adversarial training. arXiv preprint arXiv:2103.01914 , 2021.

Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. Las-at: adversarial training with learnable attack strategy. In CVPR , 2022.

Sangwon Jung, Donggyu Lee, Taeeon Park, and Taesup Moon. Fair feature distillation for visual recognition. In CVPR , 2021.

Nikos Komodakis and Sergey Zagoruyko. Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. In ICLR , 2017.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report , 2009.

Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N , 2015.

Chen Liu, Mathieu Salzmann, Tao Lin, Ryota Tomioka, and Sabine Süsstrunk. On the loss landscape of adversarial training: Identifying challenges and how to overcome them. In Advances in Neural Information Processing Systems , 2020.

Xinsong Ma, Zekai Wang, and Weiwei Liu. On the tradeoff between robustness and fairness. In NeurIPS , 2022.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR , 2018.

Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh. Improved knowledge distillation via teacher assistant. In AAAI , 2020.

Vedant Nanda, Samuel Dooley, Sahil Singla, Soheil Feizi, and John P Dickerson. Fairness through robustness: Investigating robustness disparity in deep learning. In ACM FAccT , 2021.

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.

Thao Nguyen, Maithra Raghu, and Simon Kornblith. Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth. In ICLR , 2021.

Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In ICLR , 2015.

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR , 2018.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR , 2014.

Mingxing Tan and Quoc Le. Efficientnetv2: Smaller models and faster training. In ICML , 2021.

Qi Tian, Kun Kuang, Kelu Jiang, Fei Wu, and Yisen Wang. Analysis and applications of class-wise robustness in adversarial training. In KDD , 2021.

Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR , 2020.

Zeming Wei, Yifei Wang, Yiwen Guo, and Yisen Wang. Cfa: Class-wise calibrated fair adversarial training. In CVPR , 2023.

Boxi Wu, Jinghui Chen, Deng Cai, Xiaofei He, and Quanquan Gu. Do wider neural networks really help adversarial robustness? In NeurIPS , 2021.

Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. In NeurIPS , 2020.

Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust or to be fair: Towards fairness in adversarial training. In ICML , 2021.

Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC , 2016.

Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML , 2019.

Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-aware instance-reweighted adversarial training. In ICLR , 2021.

Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In CVPR , 2018.

Shiji Zhao, Jie Yu, Zhenlong Sun, Bo Zhang, and Xingxing Wei. Enhanced accuracy and robustness via multi-teacher adversarial distillation. In ECCV , 2022.

Jianing Zhu, Jiangchao Yao, Bo Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, and Hongxia Yang. Reliable adversarial distillation with unreliable teachers. In ICLR , 2021.

Bojia Zi, Shihao Zhao, Xingjun Ma, and Yu-Gang Jiang. Revisiting adversarial robustness distillation: Robust soft labels make student better. In ICCV , 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the issue of amplified class-imbalanced performance during adversarial robust distillation (ARD). The authors propose a fair-oriented ARD method that incorporates a class re-weighting mechanism for adversarial training. The method is applicable to various ARD techniques and consistently achieves success across different datasets.\n\nSTRENGTHS:\n1. The paper provides strong evidence (Fig. 1 and Tab. 1) to highlight the severity of the robust fairness (class-imbalanced adversarial robustness) issue.\n2. The proposed method is simple yet effective, as validated through extensive experiments.\n\nWEAKNESSES:\n1. The reviewer suggests investigating whether existing fair-oriented distillation methods can mitigate the robust fairness problem with slight modifications. For example, [A] addressed biased issues in standard knowledge distillation and conducted experiments on the CIFAR-10S dataset. It would be beneficial to discuss and compare these existing methods in this submission.\n\n2. Prior work [17, 29] has emphasized that average accuracy can be odd to fairness in both standard and adversarial robust settings. However, the experiments presented in Tab. 2, Tab. 4 of the main paper, as well as Tab. 7 and Tab. 8 in the supplementary material, show that the proposed method consistently achieves significantly better average scores while maintaining the best fairness scores, contrary to established theories. This raises concerns about the credibility of the experiments, and the reviewer requests an explanation for this phenomenon.\n\n[A] Jung, Sangwon, et al. \"Fair feature distillation for visual recognition.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.\n\nQUESTIONS:\n1. The term \"fairness\" has been overloaded in recent years. The authors seem to refer to the \"class-imbalance\" issue and evaluate it using \"minimax fairness.\" However, it is important to note that conventional fairness problems typically involve specified sensitive groups, which are orthogonal to predicted classes, and require multiple evaluation metrics such as demographic parity and equalized odds. It is recommended that the authors address more fairness topics in the paper, perhaps in the related work section, to provide clarity.\n\n2. The authors appear to have missed relevant related work, such as [B], which discusses fairness through robustness.\n\n[B] Nanda, Vedant, et al. \"Fairness through robustness: Investigating robustness disparity in deep learning.\" Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 2021.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the issue of robust fairness in adversarial robustness distillation. This paper proposes the Fair Adversarial Robustness Distillation (Fair-ARD) framework, which utilizes a more refined difficulty metric and an adaptive class re-weighting approach, enabling the student model to learn the knowledge about each class in a fairer way. Extensive experiments confirm that Fair-ARD can significantly improve the robust fairness of the student model and slightly improve overall robustness.\n\nSTRENGTHS:\n+The main strength paper firstly investigates the issue of robust fairness in adversarial robustness distillation, and furthermore proposed a reweighting method to get a class-fair student model. \n\n+The paper is well-writing and easy to follow.\n\nWEAKNESSES:\n-The novelty is somewhat limited.  Robust fairness is widely studied in the adversarial training method (adversarial distillation is a type of variant to improve the robustness based on the adversarial training). In this regard, the originality of the research is limited.\n\n-The reweighting method toward different class is widely applied in adversarial training ([1], [2] as mentioned in article). More advanced state-of-the-art method (such as [3]) should be discussed and compared.\n\n-The metric to measure class difficulty is lack of theoretical basis. Directly applying the least PGD steps seem to not be a good estimate method. Different step size and steps num will lead to different results, e.g., Auto-PGD [4], and the similar idea is applied in AT (e.g. [2], [5]). If the least PGD steps is applied in reweighting and can really be helpful, a theoretical analysis is supposed to provide.\n\n[1] Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. 360 Geometry-aware instance-reweighted adversarial training. In ICLR, 2021.\n\n[2] Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust or to be fair: Towards 355 fairness in adversarial training. In ICML, 2021.\n\n[3] Wei, Zeming, et al. \"Cfa: Class-wise calibrated fair adversarial training.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[4] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an  ensemble of diverse parameter-free attacks. In International conference on machine learning, 362 pages 2206–2216. PMLR, 2020. \n\n[5] Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, Xiaochun Cao; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.\n\nQUESTIONS:\nsee weakness\n\nLIMITATIONS:\nsee weakness", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAdversarial robustness distillation transfers knowledge from a teacher model to a student model and improves the overall robustness of the student in terms of resisting adversarial attacks. This work studies robust fairness in the task of adversarial robustness distillation, which focuses on the adversarial robustness of different classes. This work first makes an observation that the student may only partially inherit the teacher's robust fairness, due to the capacity gap between the teacher and student, and the varying class difficulties. To address this issue of robust fairness, this work proposes Fair-ARD, which uses a geometric perspective to quantify the per-class difficulty and adjust the weight of each class accordingly. Extensive experiments demonstrate the efficacy of Fair-ARD in improving the robust fairness in knowledge distillation.\n\nSTRENGTHS:\n1. This work, for the first time, explores how to transfer robust fairness in adversarial robustness distillation. Robust fairness can be of vital importance in ML applications with social and/or safety concerns. This research initiates an important direction for the community.\n2. The proposed method, Fair-ARD, is a general framework that can be readily incorporated into existing ARD methods.\n3. The experiment evaluation is thorough.\n4. The writing and math notations are clear.\n\nWEAKNESSES:\n1. The experiments are mostly done on CIFAR-10/100, a relatively curated and easy dataset. Following prior work like IAD [35], it is encouraged to include more realistic and challenging datasets like Tiny-ImageNet. In the sense of robust fairness, a large number of classes and/or unbalanced data distribution would introduce some more practical challenges.\n2. The re-weighting strategy in proposed method requires re-estimating the difficulty of every sample in every epoch. This process involves multiple PGD steps. This method may introduce a large quantity of computation overhead to the distillation procedure.\n3. Citations should be given when previous methods first appear in the paper (e.g., IAD, RSLAD, and MTARD in Line 75).\n\nQUESTIONS:\n1. In Figure 2, the 10-th adversarial example of $\\tilde x_4$ is still correctly classified. Why the LPS value of $x_4$ is 10? This seems to contradict with Equation (5).\n2. Also in Equation (5), how to determine $d(x,y)$ if the adversarial examples within the $\\epsilon$-ball never move across the decision boundary?\n\nLIMITATIONS:\nPlease check 1. in Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper deals with the issue of fairness in adversarial training in image classification with distillation. The idea is that due to model capacity gap between the teacher and the student model, not all classes are equally capable of retaining the teacher model's robustness and thus the distilled student model shows a larger variation among different classes in terms of adversarial robustness which is not desired.\nThe proposed solution is a simple class-wise re-weighting of the training objective which increases the importance of such difficult classes and reduces that of the easier classes. The proposed weights are heuristically motivated according to the (inverse) of the number of PGD steps needed to find an adversarial example averaged over each class (weighted with hyper-parameter beta).\n\nNumerical results on CIFAR 10 and CIFAR100 show empirical improvements of the average and worst-class robustness.\n\nSTRENGTHS:\nThe problem is interesting and well-motivated. The paper is written well and easy to follow. The proposed solution is intuitive and simple.\n\nEmpirical results show improvement in terms of average and worst-class adversarial robustness against state-of-the-art on cifar 10 and cifar 100.\n\nWEAKNESSES:\nThe paper proposed a simple re-weighting approach, and lacks any foundational contribution to the problem. The performance is thus heavily judged by empirical tests.\n\nMy concern on this issue is the following:\n\n1- Due to lack of theoretical grounding, how reliable are the experimental results? This is specially important in CIFAR100 results given the low robustness (<5%). One could for instance report the standard deviation across different training runs to show such sensitivity.\n\n2- The formulation is heuristic, especially as the proposed class weights are not predetermined and change according to the latest model obtained in the last iteration. This means the objective changes in each epoch. How stable is this approach? Also, what are these weights calculated over (training set or validation set)? As one of the differences with GAIRAT it is mentioned that least PGD steps (LPS) is calculated over the validation set in there. How important is this choice and how much difference does it make?\n\n3- The metric to measure class difficulty is borrowed from GAIRAT, with some modifications. What is the sensitivity to the pgd parameters such as the step-size.\n\nQUESTIONS:\n1- Given the empirical nature of the results, how sensitive is the model versus the PGD step-size in LPS? How would the results change if $gamma$ was 10 times smaller for instance?\n\n2- As motivating examples, it is mentioned that the gap between student and teacher is the reason behind worse fairness. However, tests show (see Fig1 for instance) that teacher models performance are often surpasses by the student model once the student model size reaches that of the teacher. This means that the size of the model is only partly responsible for the performance deterioration and contradicts the statement in my opinion.\n\n3- In the abstract two relative improvements are mentioned (12.3% and 5.3%) while both refer to CIFAR10 with Resnet18. Could you please clarify.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the problem of fairness in the context of distillation of adversarially robust models. Specifically, it is shown that the accuracy disparity between classes (leading to “unfairness”) is _further exacerbated_ for student models, attributed to the a) capacity gap between the student and teacher model and b) equal importance given to classes during the distillation process. \nThis essentially leads to the need for fair adversarial distillation. This is achieved via modelling the “difficulty of learning a class” via the Least PDG Steps (LPS) measure to model the ease by which a sample can cross the decision boundary. Based on this, a flexible and general reweighting scheme is proposed which can be applied “post-hoc” on top of any distillation mechanisms by allowing for the reweighting. A thorough empirical evaluation is presented showing consistent improvements in the average and worst-group accuracy over existing methods as well as superiority over other fair adversarial training methods.\n\nSTRENGTHS:\nThis paper studies an important and relevant problem of the fairness-robustness trade-off in the relevant setting of knowledge distillation. I like the motivation behind the need to study the \"fine-grained\" effect of adversarial distillation w.r.t accuracy gap between different classes. The approach to analyse the problem is concise and well motivated: and has important implications in both the \"inductive bias\" of adversarial robustness towards different classes as well as the generality of reweighting via class difficulty. Further, the empirical evaluation is thorough and comparisons are made with comprehensive baselines, both w.r.t original robust-distillation methods as well as different \"reweighting\" / fairness methods.\n\nWEAKNESSES:\nFollowing are some questions which I believe should be addressed:\n1. An essential baseline assumption is that _all classes are balanced_ i.e. equally represented in the training data. How do you foresee the generality of your method in the scenario where we not only consider class difficulty but also the class-imbalance? It looks like the estimator on Eq(6) would now be biased.\n2. The results seem to be evaluated only on the CIFAR datasets, it would be interesting to see how this observation holds beyond CIFAR.\n\nQUESTIONS:\nPlease see section on Weakenesses.\n\nLIMITATIONS:\nWhile discussion on the choice of $\\beta$ is presented, it would be nice if any limitations of the existing method (such as reweighting) are discussed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-10"}