NEO-KD: Knowledge-Distillation-Based Adversarial Training for Robust Multi-Exit Neural Networks
Authors: Seokil Ham
Jungwuk Park
Dong-Jun Han
Jaekyun Moon
Venue: NeurIPS 2023 poster (2023)
Keywords: Multi-exit Neural Network
Adversarial Training
Knowledge Distillation
Adversarial Transferability
TL;DR: We propose a novel knowledge disillation based adversarial training considering the property of multi-exit neural network.

Abstract: While multi-exit neural networks are regarded as a promising solution for making efficient inference via early exits, combating adversarial attacks remains a challenging problem. In multi-exit networks, due to the high dependency among different submodels, an adversarial example targeting a specific exit not only degrades the performance of the target exit but also reduces the performance of all other exits concurrently. This makes multi-exit networks highly vulnerable to simple adversarial attacks. In this paper, we propose NEO-KD, a knowledge-distillation-based adversarial training strategy that tackles this fundamental challenge based on two key contributions. NEO-KD first resorts to neighbor knowledge distillation to guide the output of the adversarial examples to tend to the ensemble outputs of neighbor exits of clean data. NEO-KD also employs exit-wise orthogonal knowledge distillation for reducing adversarial transferability across different submodels. The result is a significantly improved robustness against adversarial attacks. Experimental results on various datasets/models show that our method achieves the best adversarial accuracy with reduced computation budgets, compared to the baselines relying on existing adversarial training or knowledge distillation techniques for multi-exit networks.

NEO-KD: Knowledge-DistilatinnBasd Adversarial Training for Robust Multi-Exit Neural Networks
Seokil Ham'
Jungwuk Park'
Dong-Jun Han Jaekyun Moon
KKAIST Purruue University
(gkatjrdif, savertm)Okaist. ac. kr, han762@purdue edu, jmoonOkaist.edd
Abstract
While multi-exit neural networks regarded promising solution for making efficient inference via early exits, combating adversarial attacks remains a challeng- ing problem. In multi-exit networks, due the high dependency among different submodels, an adversarial example targeting a specific exit not only degrades the performance of the target exit but also reduces the performance of all other exits concurrently. This makes multi-exit networks highly vulnerable to simple adver- sarial attacks, In this paper, we propose NEO-KD, a cowllege-distillation based adversarial training strategy that tackles this fundamental challenge based on two key contributions. NEO-KD first resorts neighbor knowledge distillation guide the output of the adversarial examples to tend to the ensemble outputs of neighbor exits of clean data. NEO-KD also employs exit-wise orthogonal knowl- edge distillation for reducing adversarial transferability across different submodels. The result a significantly improved robustness against adversarial attacks. Exper- imental results on various datasets/models show that our method achieves the best adversarial accuracy with reduced computation budgets, compared the basclines relying on existing adversarial training knowledge distillation techniques for multi-exit networks.
Introduction
Multi-exit neural networks are receiving significant attention 9 13 to make dynamic predictions in resource-coe coooureeconstraieed applications. Instead of making predictions at the final output of the full model, faster prediction can be made at earlier exit depending on the current time budget or computing budget. In this sense, a multi-exit network can be viewed as an architecture having multiple submodels, where each submodel consists of parameters from the input of the model to the output of specific exit. These submodels are highly correlated as they share some model parameters. It is also well-known that the performance of all submodels can be improved by distilling the knowledge of the last exit to other exits, i.c., via self-distillation have also been efforts to address the adversarial attack issues in the context of multi-exit networks 12].
28, 32 for their ability 26 27
[15] 20 24 27). There
Providing robustness against adversarial attacks is especially challenging in multi-exit networks: since different submodels have high correlations by sharing parameters, an adversarial example targeting specific exit can significantly degrade the performance of other submodels. In other words, an adversarial example can have strong adversarial transferability across different submodels, making the model highly vulnerable to simple adversarial attacks (e.g., adversarial attack targeting a single exit).
"Corresoonding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Motivation. Only a few prior works have focused on adversarial defense strategies for multi-exit networks [2 The authors of [[2] focused on generating adversarial examples tailored to multi-
exit networks (e.g., generate samples via max max-average attack), and trained the model to minimize the sum of clean and adversarial losses of all exits. Given the adversarial example constructed in [[], the authors of ( proposed regularization term to reduce the weights of the classifier at each exit during training. However, existing adversarial defense strategies [3 do not directly handle the high correlations among different submodels, resulting in high adversarial transferability and limited robustness in multi-exit networks. To tackle this difficulty, we take a www.llggeedistillatiin based approach in a fashion orthogonal to prior works [3] [2). Some previous studies [8] 23 33 [4
have shown that knowledge distillation can be utilized for improving the robustness of the model in conventional single-exit networks. However, although there are extensive existing works on self-distillation for training multi-exit networks using clean data [13 20 24 27 it is currently unknown how distillation techniques should be utilized for adversarial training of multi-exit networks. Moreover, when the existing distillation- based schemes are applied to multi-exit networks, the dependencies among submodels become higher since the same output (e.g.. the knowledge of the last exit) is distilled to all sub-models. Motivated by these limitations, we pose the following questions: How can we take advantage of knowledge networks? At the same time, how can we reduce adversarial transferability across different submodels in multi-exit networks?
improve adversarial robustness of multi-exit
To handle these questions, we propose NEO-KD, a nhowledgeddsstilatton based adversarial training strategy highly tailored to robust multi-exit neural networks. Our solution is two-pronged, neighbor knowledge distillation and exit-wise orthogonal knowledge distillation.
Main contributions.
Given a specific exit, the first part of our solution, neighbor knowledge distillation (NKD), distills the ensembled prediction of neighbor exits of clean data to the prediction of the adversarial example at the corresponding exit, as shown in Figure Ta This method guides the output of adversarial examples to follow the outputs of clean data, improving robustness against adversarial attacks. By ensembling the neighbor predictions of clean data before distillation, NKD provides higher quality features to the corresponding exits compared to the scheme distilling with only one exit in the same position.
The second focus of our solution, exit-wise orthogonal knowledge distillation (EOKD), mainly aims at reducing adversarial transferability across different submodels. This part is another unique contribution of our work compared to existing methods on robust multi-exit networks (that suffer from high adversarial transferability) or salf-ditiillation-are
3 [2] networks [5] 20 24 27 (that further increase adversarial transfcrability). In our EOKD, the output of clean data the i-th exit is distilled to the output of the adversarial sample at the i-th exit, in an exit-wise manner. During this exit-wise distillation process, we encourage the aon-ground-trrn orthogonal soft labels to each exit as described in Figure [b] By weakening the dependencies among different exit outputs, EOKD reduces the adversarial transferability across all submodels in the network, which leads to an improved robustness against adversarial attacks.
multi-exit
predictions of individual exits to be mutually orthogonal, by providing
The NKD and EOKD components of our architectural solution work together to reduce adversarial transferability across different submodels in the network while correctly guiding the predictions of the adversarial examples at each exit. Experimental results on various datasets show that the proposed strategy achieves the best adversarial accuracy with reduced computation budgets, compared to existing adversarial training methods for multi-exit networks. Our solution is a plug-and-play method, which can be used in conjunction with existing training strategies tailored to multi-exit networks.
2 Related Works
Knowledge distillation for multi-exit networks. Multi-exit neural networks 9 13 26 27 28 aim at making efficient inference via early exits in resource-constrann network literature, it is well-know that distilling the knowledge of the last exit to others significantly improves the overall performance on clean data without an external teacher network, i.e., via self- However, it is currently unclear how adversarial training can benefit from self-distillation multi-exit networks. One challenge is that simply applying existing self-distillation techniques increases ad adversarial transferability across different submodels, since the same knowledge
applications. In the multi-exit
distillation 15 20 24 27

---

from the last exit is distilled to all other exits, increasing dependency among different submodels in the network. Compared to the existing ideas, our contribution is develop a self-distillation strategy that does not increase the dependency of submodels as much; this helps reduce adversarial transferability of the multi-exit network for better robustness.
Improving adversarial robustness. Most existing defense methods 66 7 31 have mainly focused on creating new adversarial training losses tailored to single-exit networks. Several other works have showing that distilling the knowledge of the teacher network can improve robustness of the student network. Especially in [8) given a teacher network, it is shown that robustness of the teacher network can be distilled to the student network during adversarial training. Compared these works, our approach can be viewed as a new self-distillation strategy for multi-exit networks where teacher/student models are trained together. More importantly, adversarial transferability across different submodels has not been an issue in previous works as the focus there has been on the single-exit network. In contrast, in our multi-exit setup, all submodels sharing some model parameters require extra robustness against adversarial attacks; this motivates to propose exit-wise orthogonal knowledge distillation, reduce adversarial transferability among different submodels.
utilized the concept of knowledge distillation 88] 23 33 3
30) aim at improving adversarial robustness of the ensemble model, by reducing adversarial transferability across individual models. Specifically, the adaptive diversity- promoting regularizer proposed in [22] regularizes the non -oxxmmal predictions of individual models to be mutually orthogonal, and the maximal term is used compute the loss as usual. While the previous work focuses on reducing the transferability among different models having independent parameters, in multi-exit network setup, the problem becomes more challenging in that all submodels have some shared parameters, making the models to be highly correlated. To handle this issue, we specifically take advantage of knowledge distillation in an exit-wise manner, which can further reduce the dependency among different submodels in the multi-exit network.
Some prior works 19
Adversarial training for multi-exit networks. When focused on multi-exit networks, only a few prior works considered the adversarial attack issue in the literature [3] 10 II [2]. The authors of 10 focused on generating slowdown attacks in multi-exit networks rather than defense strategies. In [12], the authors proposed an adversarial training strategy by generating adversarial examples targeting specific exit (single attack) or multiple exits (average attack and max-average
attack). However, (i) [12] does not take advantage of knowledge distillation during training and (ii) 12 does not directly handle the high correlations among different submodels, which can result in high adversarial transferability. Our solution overcomes these limitations by reducing adversarial transferability while correctly guiding the predictions of adversarial examples at each exit, via self knowledge distillation.
3 Proposed NEO-KD Algorithm
Consider a multi- network with L exits, which is composed of L blocks { and L clas- sifiers [ Given the input data x, the output of the i-th exit is denoted as fo, (z). which is parameterized by the i-th submodel 0; = [oi...... Di, wy] that consists of i blocks and one classifier. fe,
Note that all submodels produce different predictions [fa, (z)..... fez (I)]. Here, since each
submodel shares several blocks with other submodels, the predictions of any two submodels are highly correlated.
3.1 Problem Setup: Adversarial Training in Multi-Exit Networks
The first step for adversarial training is to generate adversarial examples. Given L. different submodels {0,)<= clean data x. and the corresponding label y, the adversarial example add can be generated based on single attack, max-average attack or average attack, following the process of [12] specifically, we have
More
Ezingies
arg max of = Sr)
(((fo, (2)),
2. Fmaz
ade IF
where 1" = arg max
I e(Jo, (ooth
(2)
adv Kang
arg max =l:
Σ E(fo, (x)),
(3)
3

---

Cle
Dar
AA
Dox1
DM03
1224
LORD
(a) Neighbor knowledge distillation
(b) Exit-wise orthogonal knowledge distillation
Figure NEO-KD consists of two parts that together improve the adversarial robustness: NKD and FOKD. (a) NKD guides the output of the adversarial data to mimic the ensemble outputs of neighbor exits clean data. (b) EOKD reduces adversarial transfcrability of the network by distilling orthogonal knowledge of the clean data to adversarial data for the non-ground-trutt predictions, in an exit-wise manner. Although omitted in this figure, EOKD normalizes the likelihood before distilling the soft labels. The overall process operates in single model, although we consider two cases depending on the input (clean or adversarial example) for clear presentation.
which correspond to the adversarial example generated by single attack targeting exit é, max-average
attack, and average attack, respectively. denotes the perturbation degree. In the single attack of (1. the adversarial example u: is generated to maximize the cross-entropy loss f(- of the target exit utilizing an attacker algorithm (e.g.. PGD (2T). In the max-average attack of (2. among the adversarial examples generated by the single attack for all exits i = 1, 2. L. the sample that maximizes the average loss of all exits is selected. Finally, the average attack (3) directly generates
an adversarial sample that maximizes the average loss of all exits. Based on the generated x addo
typical strategy to update the model considering both the clean and adversarial losses of all exits as follows:
C 1 M (e(,,,15)).)) (15). + E(Jo, (1, v,y)). J=1
(4)
where N is the number of samples in the training set and radu is the adversarial example correspond- ing clean sample zy generated by one of the attacks described above. However, the loss in (4) does not directly consider the correlation among submodels, which can potentially increase adversarial transferability of the multi-exit network.
3.2 Algorithm Deseription
To tackle the limitations of prior works 3 12), we propose neighbor exit-wise orthogonal knowledge distillation (NEO-KD), self-distillation strategy tailored to robust multi-exit networks. To gain insights, we divide our solution into two distinct components with different roles neighbor knowledge distillation and exit-wise orthogonal knowledge distillation and first describe each component separately, and then put together into the overall NEO-KD method. A high-level description of our approach is given in Figure [
Neighbor knowledge distillation (NKD). The first component of our solution, NKD, guides the output feature of adversarial data at each exit to mimic the output feature of clean data, Specifically, the proposed NKD loss of the j-th train sample at the i-th exit is written as follows:
(agdd) E(So, (agd e(So, (1 (ardo)
Σ for (I1)) L
i=1
NKD.J-
)
k==--1 Jo, (x²))
i=
(5)
Ek Jos (xz))
otherwise,
which can be visualized with the colored arrows as in Figure la Different from previous self- knowledge distillation methods, for each exit NKD generates teacher prediction by ensembling (averaging) the neighbor predictions (i.e., from exit and exit + 1) of clean data and distills it to each prediction of adversarial examples.
Compared other distillation strategies, NKD takes advantage of only these neighbors during distillation, which has the following key advantages for improving adversarial robustness. First,

---

by ensembling the neighbor predictions of clean data before distillation, NKD provides higher quality feature of the original data to the corresponding exit; compared to the naive baseline that is distilled with only one exit in the same position (without ensembling), NKD achieves better adversarial accuracy, where the results are provided in Table 7 Secondly, by considering only the neighbors during ensembling, we can distill different teacher predictions to each exit. Different teacher predictions of NKD also play a role of reducing adversarial transferability compared to the strategies that distill the same prediction (e.g., the last exit) to all exit; ensembling other exits (beyond the neighbors) increases the dependencies among submodels, resulting in higher adversarial transferability. The corresponding results are also shown via experiments in Section 4.3
However, a multi-exit network trained with only NKD loss still has a significant room for mitigating adversarial transferability further. In the following, we describe the second part of our solution that solely focuses on reducing adversarial transferability of multi-exit networks.
Exit-wise orthogonal knowledge distillation (EOKD). EOKD provides orthogonal soft labels to each exit for the non-ground-truth predictions, in an exit- manner. As can be seen from the red arrows in Figure Tb the output of clean data at the i-th exit is distilled to the output of adversarial example the i-th exit. During this exit-wise distillation process, some predictions are discarded to encourage the non-ground-truth predictions of individual exits to be mutually orthogonal. We randomly allocate the classes of non non-grooe con--rrond-trre prevents the classifier to be biased compared to the fixed allocation strategy. The proposed EOKD loss of the j-th sample the i-th exit is defined as follows:
predictions each exit for every epoch, which
F.K (x; O(Jo, (zz)))-
(6)
Here, (()) is the orthogonal labeling operation to make the non-ground-truth predictions to be orthogonal across all exits. For each exit, O(-) randomly selects L(C - L) labels among total of C' classes so that the selected labels are non-overlapping across different exits (except for the ground-truth label), as in Figure 1b Lastly, the probabilities of selected labels are normalized to sum to one. To gain clearer insights, consider a toy example with a 3-exit network (i.e.. L = 3) focusing on 4-way classification task (i.e., C = 4). Let [pi, p2. P/g p'] be the softmax output of the clean sample at the i-th exit, for i = 1, 3. If class is the ground-truth. the orthogonal labeling operation 0()) jointly produces the following results from each exit: [pt, pb, 0, 0) from exit 1, [oft O, pš. 0) from exit 2, I 0, O, p1] from exit 3, where ý indicates the normalized probability of so that the values in each vector sum to one.
Based on Eq. (6). the output of the orthogonal label operation Offe, (zz)) for the clean data zy is distilled to fe, (agde) which is the prediction of the adversarial example of the j-th sample at the i-th exit. This encourages the model to self-distill orthogonally distinct knowledge in exit-wise manner while keeping the essential knowledge of the ground-truth class. By taking this exit-wise orthogonal distillation approach, EOKD reduces the dependency among different submodels, reducing the adversarial transfcrability of the network.
Overall NEO-KD loss. Finally, considering the proposed loss functions in Eq. 5 and the original adversarial training loss, the overall objective function of our scheme is written as follows:
N L L = M j=l
+ E(fo, (1gdd)) ), + :a(a- NKD.,, + B EOKD.,)), (7)
where a, 8 control the weights for each component and 7/: is the knowledge distillation weight for each exit i. Since later exits have lower knowledge distillation loss than the earlier exits, impose slightly higher "y for the later exits than 7/8 of the earlier exits. More details regarding hyperparameters described in Appendix.
By introducing two unique components- NKD and EOKD - the overall NEO-KD KD loss function in Eq. reduces adversarial transfcrability in the multi-exit network while correctly guiding the output of the adversarial examples in each exit, significantly improving the adversarial robustness of multi-exit networks, as we will see in the next section.
4 Experiments
In this section, we evaluate our method on five datasets commonly adopted in multi-exit networks: MNIST [8), CIFAR-10, CIFAR-100 [16), Imageent (17), and ImageNet (25). For MNIST, we
5

---

Exit
Average
Exit
Average
Disiil L23
89,21%
95. 95.35%
96.82%
94.15%%
Adv. weo Distil
85.17%
95,87%
97.20%
92.7558
SKD 23
89.77%
96.24%
ARD
89.659
95 799
96.475
93,9758
ARD
079
97.
1W
87.08%
93.36%
95.42%
92.125
LW B
85 26%
94.54%
96.05%
91.95%
NEO
90.56%
96.39%
96.62%
94,49%
NEO-KD
86.17%
96.92%
4258
93.50%
(a) Max-average attack
(b) Average attack
Table 1: Anytime prediction setup: Adversarial test accuracy on MNIST.
Exit
Average
Exit
Average
Adv. wño Dishill
P
44.77%
46.10%
46.86%
45.91%
Adv. wio Distil
P
38,009
41,42%
40.70%
-00,04%
SKD 24
44.545
44.79%
44,0996
SKD 24
39.36%
41,39%
38.29%
39.7156
44.50%
45.85%
51.82%
47.
30.37%
41.98%
43.53%
41.63%
LWW
37.38%
55.3918
34.85%
35,8795
L.W
31.47%
31.41%
28.98%
30,0285
NBO KD
46.53%
47.05%
50.71%
48.30%
NBO-KD
41.67%
45.38%
45.54%
44.30%
(a) Max-average attack
(b) Average attack
Table 2: Anytime prediction setup: Adversarial test accuracy on CIFAR-10.
with 3 and 7 exits using CIFAR-10 and CIFAR- 100, respectively. For Lin-ImmaeNee and ImageNet, we trained the MSDNet with 5 exits. More implementation details are provided in Appendix.
use SmalICNN with exits. We trained the MSDNet 13
4.1 Experimental Setup
Generating adversarial examples. To generate adversarial examples during training and testing, we utilize max-average attack and average attack proposed in [2]. We perform adversarial training using adversarial examples generated by max-average attack, where the results for adversarial training via average attack are reported in Appendix. During training, we use PGD attack [2]] with 7 steps as attacker algorithm
generate max-average and average attack while PGD attack with 50 steps is adopted at test time for measuring robustness against stronger attack. We further consider other strong attacks in Section 4.3 each attacker algorithm, the perturbation degree is 0.3 for MNIST, datasets during adversarial training and when measuring the adversarial test accuracy. Other details for generating adversarial examples and additional experiments various attacker algorithms are described in Appendix.
and 8/255 for CIFAR- 10/00, and 2/255 for
Evaluation metric. We evaluate the adversarial test accuracy as in [2]. which is the classification accuracy on the corrupted test dataset compromised by an attacker (e.g., average attack). We also measure the clean test accuracy using the original clean test data and report the results in Appendix.
Baselines. We compare our NEO-KD with the following baselines. First, we consider the scheme based adversarial training without any knowledge distillation, where adversarial examples generated by max-average attack or average attack 12). The second baseline is the conventional self-knowledge distillation (SKD) strategy (20] 24! combined with adversarial training: during adversarial training. the prediction of the last exit for a given clean/adversarial data is distilled to the predictions of all the previous exits for the same clean/ad aecasraaveraaiial data. The third bascline the
knowledge distillation scheme for adversarial training [8), which distills the prediction of clean data to the prediction of adversarial examples in single-exit networks. As in g we distill the last output of clean data to the last output of adversarial data. The last baseline is regularizer- ggguaaiizerbbsed adversarial training strategy for multi-exit networks [3], where the regularizer restricts the weights of the fully connected layers (classifiers). In Appendix we compare our NEO-KD with the recent work TEAT general defense algorithm for single-exin networks.
Inference scenarios. At inference time, we consider two widely known setups for multi-exit networks: (i) anytime prediction setup and (ii) budgeted prediction setup. In the anytime prediction setup, an appropriate exit is selected depending on the current latency constraint, In this setup, for each exit, we report the average performance computed with all test samples. In the budgeted prediction setup, given fixed computational budget, each sample is predicted at different exits depending on the predetermined confidence threshold (which is determined by validation set). Starting from the first exit, given a test sample, when the confidence at the exit (defined as the maximum softmax value) larger than the threshold, prediction is made at this exit. Otherwise, the sample proceeds to the next exit. In this scenario, easier samples are predicted at earlier exits and harder samples are predicted at later exits, which leads to efficient inference. Given the fixed computation budget and confidence threshold, we measure the average accuracy of the test samples. We evaluate our method in these two

---

Top-1
aecuracy (S6)
top-S accuracy (%)
Avg.
Avg.
Distil
28.32
27
26.
27.12
61.15
57.44
58.95
SKD 24
27.36
27.68
25.79
24.74
24.10
20.86
19.28
24.26
60.50
60.23
57.56
55.98
55.11
52.71
52.17
56.32
27.60
27.42
29.08
60.51
59.37
LW
20.44
20.57
19.86
19.34
19.51
19.46
20.22
19.91
50.09
50.91
48.47
48.56
48.13
49.61
49.76
49.45
NEO KD (ours)
28.37
28.78
29.02
29.49
30.06
28.45
28.54
28.96
59.58
60.04
59.67
59.29
60.46
58.96
39.44
59.63
(a)
aax-aae atack
Top-1
necuracy
Top-5 accuracy
Exit
2
Avg.
5
Avg.
Adv. w/o Distil
12
16.74
17.33
19.05
19.47
19.06
18.15
17.12
18.13
47.31
48.34
50.78
50.75
50.80
49.89
47.10
49.28
SKD
24
18.45
19.54
14.21
18.06
50.72
51.73
47.32
49.41
ARD
16.63
17.16
19.13
19.74
19.70
18.92
19.83
18.73
46.90
48.60
50.44
51.28
50.93
49.14
49.01
49.47
13.95
15.16
15.86
13.01
13.57
41.27
44.08
42.09
NEO-KD
20.41
21.32
23.27
24.30
24,47
23,99
22.39
22.88
49.48
51.47
53.13
55.01
54.69
54.32
52.11
52.89
(b) Average attack
Table 3: Anytime prediction setup: Adversarial test accuracy CIFAR-IO0.
top. accuracy (%)
tap 5 accuracy (%)
Avg.
Avg.
Adv. wio Distiil
12
31.30
32.38
32.52
31.42
31.56
51.84
60,20
60.84
61.10
58.64
59.56
60.007
SKD
24
33.04
13.24
30.50
28.40
28.50
30.74
63.68
62.72
60.04
58.50
58.28
60.64
ARD
30.08
31.84
30.56
31.22
32.34
31.21
59.88
60.74
59.62
59.28
59.72
59.85
LW X
26.06
26.00
24.54
24.42
24.88
25. 18
53.24
51.90
51.02
51.02
31.14
51,66
NEO-KD (ours)
32.96
35.08
33,42
32.40
32.64
33.30
62.48
62.82
61.76
60.70
60.92
61.74
(a) Max- ave-average attack
accuracy (%)
top-5 accuracy
Exit
2
3
4
5
Avg.
Avg.
Adv. wio Distill 12
25.20
27.34
28.94
28.06
28.48
27.61
53.68
55.46
58.18
57.46
51.38
56.43
SKD 24
28.40
29.26
28.74
28.1
26.82
28.27
58.34
59.30
58.38
57.72
55.48
$7.84
ARD g
24.48
26.76
27.78
28.46
29. 14
27.32
53.44
56.10
57.20
57.32
57.08
56.23
23.12
23.58
22.
NEO-KD (OWNS)
28.24
31,14
30.58
31,58
31.24
30.56
57.34
59.82
60.16
60,04
59.06
59,28
(b) Average attack
Table 4: Anytime prediction setup: Adversarial test accuracy on Tmageem
top-1 accuracy (3)
top-5 accuracy (%)
Avg.
Adv. wio Disrill
Pp
26.10
31.71
31.94
30.22
32.30
30.45
55.40
59.84
59.23
57.17
59.90
58.31
SKD
27.34
31.44
29.63
27.03
26.71
28.43
56,96
59,94
57.49
53.98
55.24
56.72
ARD
g
25.81
32.00
32,93
32.00
31.72
30.89
54,46
59.34
59.24
57.93
58.29
57.85
NEO KD (ours)
27,89
32.61
32.99
32.74
35.63
32.37
55.46
59.64
59.79
59.61
62.46
59.39
(a) Max-average attack
accuracy (%)
top-5 accuracy (%)
Exit
2
3
4
5
Avg.
Avg.
Adv. W/o Distill 02]
18.20
24.21
28.24
28.73
28.50
25.58
44.74
52.57
56.82
57.55
57.14
53.77
SKD
24
19.04
24.45
26.45
25.72
22.17
23.57
46.31
52.92
55.33
54.71
50.18
$1.89
ARD
17.56
23.68
27.98
28.52
25.83
24.71
43.76
52.07
56.42
57.30
53.48
$2.60
KD (ours)
28.62
31.77
32.78
34.30
Average attack
Table Anytime prediction setup: Adversarial test accuracy on ImageNet.
settings and show that our method outperforms the baselines in both settings. More detailed settings for our inference scenarios are provided in Appendix.
4.2 Main Experimental Results
compare the adversarial test accuracy of different schemes under max-average attack and average attack using MNIST, CIFAR- 10/110//0000 ImageNe, and ImageNet, respectively. Note that we achieve adversarial accuracies between 40% 50% for CIFAR- 10, which is standard considering the prior works on robust multi-exit networks [1] Our first observation from the results indicates that the performance of SKD 24 generally lower than that of Adv. w/o Distill, whereas ARD S outperforms Adv. w/o Distill. This suggests that the naive application of self knowledge distillation can either increase or decrease the adversarial robustness of multi-exit networks. Consequently, the method of knowledge distillation
Result 1:
Anytime prediction setup. Tables I 2
7

---

Budget (MRLOFF)
PITLOPS
mAde AcT Dedil
-nAd an Diet
=M5 mWARD -M10040
-ith wol m5d0 modd -
(c) Timy-ImageNet
(a) CIFAR-10
(b) CIFAR-100
Figure 2: Budgeted prediction setup: Adversarial test accuracy under average attack. The result for LW is excluded since the performance is too low and thus hinders comparison between baselines and NEO-KD.
100
Sudget Usege AMPCCTT
Endyet Juge 24110RS
Exdget Cvege
-MDD-D
(a) CIFAR-10
(b) CIFAR-100
(c) Tiny-ImageNer
Figure 3: Budgeted prediction setup: Adversarial lest accuracy under max-average atlack. The result for LW
is excluded since the performance is too low and thus hinders the comparison between baselines and NEO-KD.
significantly influences the robustness of multi-exit networks (i.e., determining which knowledge to distill and which exit to target). To further enhance robustness, we investigate strategies for distilling high-quality knowledge and mitigating adversarial transferability.
By combining EOKD with NKD to mitigate dependency across submodels while guiding multi-esi network to extract high quality features from adversarial examples original data, NEO-KD achieves the highest adversarial test accuracy at most exits compared to the baselines for all datasets/attacks. The overall results confirm the advantage of NEO-KD for robust multi-exit networks.
Result 2: Budgeted prediction setup. Different from the anytime prediction setup where the pure performance of each exit is measured, in this setup, we adopt ensemble strategy at inference time where the predictions from the selected exit (according the confidence threshold) and the previous exits are ensembled. From the results in anytime prediction setup. it is observed that various schemes tend to show low performance at the later exits compared to earlier exits in the model, where more details are discussed in Appendix. Therefore, this ensemble strategy can boost the performance of the later exits. With the ensemble scheme, given a fixed computation budget, we compare adversarial test accuracies of our method with the baselines.
Figures 2 and B show the results in budgeted prediction setup under average attack and max-average attack. respectively. NEO-KD achieves the best adversarial test accuracy against both average and max-average attacks in all budget setups. Our scheme also achieves the target accuracy with significantly smaller computing budget compared to the baselines. For example, to achieve 41.21% of accuracy against average attack using CIFAR- 10 (which is the maximum accuracy of Adv. w/o Distill), the proposed NEO-KD needs 10.59 MFlops compared to Adv. w/o Distill that requires 20.46 MFlops, saving 48. 24% of computing budget. Compared ARD, our NEO-KD saves 25.2 25% of computation, while SKD and LW are unable to achieve this target accuracy. For CIFAR-100 and Tmay- NEO-KD saves 81.60 81.60% and 50.27% of computing budgets compared to Adv. w/o Distill. The overall results are consistent with the results in anytime prediction setup, confirming the advantage of our solution in practical settings with limited computing budget.
Result 3: Adversarial transterability. We also compare the adversarial transferability of our NEO- KD and different baselines among exits in a multi-exit neural network. When measuring adversarial transferability. as in (29) we initially gather all clean test samples for which all exits produce correct predictions. Subsequently, we generate adversarial examples targeting each exit using the collected clean samples (We use PGD-50 based single attack). Finally, we assess the adversarial transferability as the attack success rate of these adversarial examples at each exit. Figure shows the adversarial transferability map of each scheme on CIFAR-100. Here, each row corresponds to the target exit for generating adversarial examples, and each column corresponds the exit where attack success rate measured. For example, the (i.) --th element in the map is adversarial transferability measured at exit
8

---

23.54
00227
D119
18.6 1827
1541
2220
0121
24.04
+1.00
D710
2451
00.00
31.34
26.40
1544
0221
1923
14.24
2856
055-
$2.00
40.40
1004
MAT
1841
15.9
Y7AR 21:1
00.00
06.00
BATY
1882
15.09
20.4
22.00
29.03
1279
$1:18
1632
13.04
14:00
1571
19.00
$1.00
12122
13.37
12:00
12:05
18:08
1841
1854
6837
(c) NKD (Avg. w/o Diag.: 23.68%) (Avg. w/o Diag.: : 33.36%) (Avg. w/o Diag.: 22.76%) (Avg. w/o Diag.: 20.12%)
(a) Adv. w/o Distill
12
(b) SKD 124
(d) NEO-KD (ours)
Figure 4: Adversarial transferability map of each method on CIFAR- 100. Diag. indicates the diagonal of the matrix. Row: Target exit for generating adversarial examples. Columm: Exit where adversarial transferability is measured, Adopting NKD solely already achieves better adversarial transferabilitty compared to the existing baselines. Applying FOKD to NKD can further improve adversarial transferability by reducing the dependency among different submodels in the multi-exit network.
BaR
Average
Type of ensembles
3
Average
Distil
33.00%
Adv. Distill
35002
41.12%
40.70%
4000S
NKD
40.01%
42,67%
40.23%
No ensembling
39.55%
41.77%
20.15%
40.49%
BOKD
36.
41.10%
38.60 681
39.01%
Ensemble neightees (NKD)
41.67%
45.38
45.54%
44.20%
NEO-KD (Ours)
41.67%
45.38%
45.54%
44.20%
Ensemble all exits
38.63%
41.61%
39.95%
40.06%
Table Effect of each component of NEO-KD:
Table 7: Results with different number of
Adversarial lest accuracy against average attack on CIFAR-IO. NKD and BOKD work in complementary attack on CIFAR-10. The neighbor-wis ensemble fashion and have implicit synergies.
ensembles: Adversarial test accuracy against average
case corresponds our NKD.
j. generated by the adversarial examples targeting exit i. The values and the brightness in the map indicate success rates of attacks; lower value (dark color) means lower adversarial transferability.
We have the following key observations from adversarial transferability map. First, as observed in Figure 4al compared to Adv. w/o Distill (12). SKD 24 in Figure 4b exhibits higher adversarial transferability. This indicates that distilling the same teacher prediction to every exit leads to a high dependency across exits. Thus, is essential to consider distilling non-overlapping knowledge across different exits. Second, when compared to the baselines [12] [24,, our proposed NKD in Figure 4c demonstrates low adversarial transferability. This can be attributed the fact that NKD takes into account the quality of the distilled features and ensures that the features are not overlapping among exits. Third, as seen in Figure 44 the adversarial transferability is further mitigated by incorporating EOKD, which distills orthogonal class predictions to different exits, into NKD. Comparing the average of attack success rate across all exits (excluding the values of the target exits shown in the diagonal), it becomes evident that NEO-KD yields 3.56% and 13.24% gains compared to Adv. w/o Distill and SKD, respectively. The overall results confirm the advantage of our solution to reduce the adversarial transferability in multi-exit networks. These results support the improved adversarial test accuracy of NEO-KD reported in Section 4.2
4.3 Ablation Studies and Discussions
Effect of each component of NEO-KD. In Table 6 we observe the effects of our individual components, NKD and EOKD. It shows that combining NKD and EOKD boosts up the performance beyond the sum of their original gains. Given different roles, the combination of NKD and BOKD enables multi-exit networks to achieve the state-of-the-ar performance under adversarial attacks.
Effect of the type of ensembles in NKD. In the proposed NKD, we consider only the neighbor exits to distill the knowledge of clean data. What if we consider fewer or more exits than neighboring exits? If the number of ensembles is too small, the scheme does not distill high-quality features. If the number of ensembles is too large, the dependencies among submodels increase, resulting in high adversarial transferabilityy To see this effect, in T we measure adversarial test accuracy of three types of ensembling methods depending on the number of exits used for constructing ensembles: no ensembling, ensemble neighbors (NKD), and ensemble all exits. In no enesmbling approach, we distill the knowledge of each exit from clean data to the output at the same position of exit for adversarial examples. In contrast, the ensemble all exits scheme averages the knowledge of all exits from clean data and provides to all exits of adversarial examples. The ensemble neighhors approach corresponds to our NKD. The results show that the proposed NEO-KD with neighbor ensembling

---

Atlacks
Exit
2
5
6
7
Average
PGD-100
network while Adv. Distill T2]
16.82%
17.24%
19.03%
19.41%
18.97%
17.90%
16.9%
18.05%
NEO-KD (ours)
20.38%
21.29%
23.22%
24.38%
24.44%
23,82%
22.21%
22.82%
Adv. w/o Distill [12]
35.31%
35.72%
36.47%
36.85%
37.20%
37.19%
36.68%
36.49%
CW
KD (ours)
31.56%
34.55%
38.20%
44.03%
43.01%
39.35%
Adv. w/o Distill [12
31.32%
34.24%
37.27%
39.67%
41.20%
41.73%
40.45%
37.98%
AutoAitack
NEO KD (ours)
31.56%
34.55%
38.20%
40.60%
43.48%
44.03%
43.01%
39.35%
Table Results with stronger attacker algorithms: Adversarial test accuracy against average attack using CIFAR-100 dataset.
enables to distill high-quality features while lowering dependencies among submodels, confirming our intuition.
Robustness against stronger adversarial attack. We evaluate NEO-KD against stronger adversarial attacks; we perform average attack based on PGD-100 2TT. Carlini and Wagner (CW) [2], and AutoAttack [5]. Table shows that NEO-KD achieves higher adversarial test accuracy than Adv. w/o Distill [12] in most of cases. Typically, CW attack and AutoAttack are stronger attacks than the PGD attack in single-exit networks. However, in the context of multi-exit networks, these attacks become weaker than the PGD attack when taking all exits into account. Details for generating stronger adversarial attacks are described in Appendix.
Additional results. Other results including clean test accuracy, results with average attack based adversarial training, results with varying hyperparameters, and results with another baseline used in single-exit network, are provided in Appendix.
Conclusion
In this paper, we proposed new knowledge distillation based adversarial training strategy for robust multi-exit networks. Our solution, NEO-KD, reduces adversarial transferability guiding the output of the adversarial examples to closely follow the ensemble outputs of the neighbor exits of the clean data, significantly improving the overall adversarial test accuracy. Extensive experimental results both anytime and budgeted prediction setups using various datasets confirmed the effectiveness of our method, compared to baselines relying on existing adversarial training or knowledge distillation techniques for multi-exit networks.
Acknowledgement
This work was supported by the National Research Foundation Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2019R, AIAA20000 135), by the Center for Applied Research in Artificial Intelligence (CARAD) grant funded by DAPA and ADD (UD230017TD), and by IITP funds from MSIT of Korea (No. 2020-0-00626).
References
[1] Maksym Andriushchenko, Francesco Croce. Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII, pages 484-501. Springer, 2020.
[2] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks, In 2017 ieee symposium on security and privacy (sp). pages 39-57. leee, 2017.
[3] Sihong Chen, Haojing Shen, Ran Wang, and Xizhao Wang. Towards improving fast adversarial training in multi-exit network. Neural Newworks, 150:1-11, 2022.
[4] Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In International Conference on Machine Learning, pages 2196-2205. PMLR, 2020.
[5] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pages 2206-2216, PMLR, 2020.
10

---

[6] Junhao Dong, Seyed- Meyee-Mohsen Moosavi-Dezfodd Jianhuang Lai. and Xiaohua Xie. The enemy of my enemy is my friend: Exploring inverse adversaries for improving adversarial training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14678-24681 2023.
[7] Yinpeng Dong, Ke Xu, Xiao Yang. Tianyu Pang, Zhijie Deng, Hang Su, and Jun Zhu. Exploring memorization adversarial training. arXiv preprint arXiv:2106.01660 2021.
[8] Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 3996 4003, 2020.
[9] Dong-Jun Han, Jungwuk Park, Scokil Ham, Namjin Lec, and Jackyun Moon. Improving low- latency predictions in multi-exit neural networks via block-dependent losses. IEEE Transactions on Neural Networks and Learning Systems, 2023.
[10] Mirazul Haque, Anki Chauhan, Cong Liu, and Wei Yang. Ilfo: Adversarial attack on adaptive neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14264 14273, 2020.
[11] Sanghyun Hong. Yigitcan Kaya, Ionut- Modoranu, and Tudor Dumitras. A panda? no, it's sloth: Slowdown attacks adaptive multi-exit neural network inference. In International Conference on Learning Representations, 2020.
[12] Ting-Kuei Hu, Tianlong Chen, Haotao Wang, and Zhangyang Wang. Triple wins: Boosting ac- curacy, robustness and efficiency together by enabling input-adaptive inference. In International Conference Learning Representations, 2020.
[13] Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens Van Der Maaten, and Kilian Q Weinberger. Multi-scale dense networks for resource efficient image classification. arXiv preprint arXiv: (eee::100004444 2017.
[14] Hoki Kim. Torchattacks: A pytorch repository for adversarial attacks. arXiv:2010.01950, 2020.
arXiv preprint
[15] Alexandros Kouris, Stylianos Venieris, Stefanos Laskaridis, and Nicholas Lane. Multi-exit semantic segmentation networks. In European Conference on Computer Vision, pages 330-349, Springer, 2022.
[16] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[17] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
[18] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278:222233 2324,, 1998.
[19] Sungyoon Lee, Hoki Kim, and Jaewook Lee. Graddiv: Adversarial robustness of randomized neural networks via gradient diversity regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022.
[20] Hao Li, Hong Zhang, Xiaojuan Qi. Ruigang Yang. and Gao Huang. Improved techniques for training adaptive deep networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1891-1900, 2019.
[21] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
[22] Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. International Conference on Machine Learning, pages 4970-4979. PMLR, 2019.

---

[23] Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense adversarial perturbations against deep neural networks, In 2016 IEEE symposium on security and privacy (SP), pages 582-597. IEEE, 2016.
[24] Mary Phuong and Christoph H Lampert. Distillation bassdd training for multi-exit architectures. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1355-
1364, 2019.
[25] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 11::211-252, 2015.
[26] Surat Teerapittayanon, Bradley McDanel, and Aisiang-ssum Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd International Conference on Pattern Recognition (ICPR), pages 2464 2469. IEEE, 2016.
[27] Xinglu Wang and Yingming Li. Harmonized dense knowledge distillation training for multi-exit architectures. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pages 10218-10226, 2021.
[28] Qunliang Xing, Mai Xu, Tianyi Li, and Zhenyu Guan. Early exit or not: Resource-efficient blind quality enhancement for compressed images. In European Conference Computer Vision, pages 275-292. Springer, 2020.
[29] Huanrui Yang, Jingyang Zhang, Hongliang Dong, Nathan Inkawhich, Andrew Gardner, Andrew Touchet, Wesley Wilkes, Heath Berry, and Hai Li. Dverge: diversifying vulnerabilities for enhanced robust generation of ensembles. Advances in Neural Information Processing Systems, 33:5505-5515, 2020.
[30] Zhuolin Yang, Linyi Li, Xiaojun Xu, Shiliang Zuo, Qian Chen, Pan Zhou, Benjamin Rubinstein, Ce Zhang, and Bo Li. Trs: Transferability reduced ensemble via promoting gradient diversity and model smoothness. Advances in Neural Information Processing Systems, 34, 2021.
[31] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pages 7472-7482. PMLR, 2019.
[32] Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, and Furu Wei. Bert loses patience: Fast and robust inference with early exit. Advances in Neural Information Processing Systems, 33:18330-18341 2020.
[33] Jianing Zhu, Jiangchao Yao, Bo Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, and Hongxia Yang. Reliable adversarial distillation with unreliable teachers. arXiv preprini arXiv: 106.04988, 2021.
[34] Bojia Zi Shihao Zhao, Xingjun Ma. and Yu-Gang Jiang. Revisiting adversarial robustness dis- tillation: Robust soft labels make student better. Proceedings of the IEEE/CVE International Conference Computer Vision, pages 16443-16455 2021.
12

---

A Experiment Details
We provide additional implementation details that were not described in the main manuseript.
A.1 Model training
We train a SmallCNN 12 for 150 epochs with batch size 128 on MNIST 18 a MSDNet [3] for
150 epochs with batch size 128 on CIFAR-IO/100 [16] and Tiny-ImageNet [17] For ImageNet [25], since it takes a long time to train from the beginning, we finetune the pretrained model with our NEO-KD loss function for 10 epochs. All experiments are implemented with two RTX3090 GPUs.
Other settings follow [ for SmalICNN and 13 for MSDNet except for a channel, which is set for our experiments on CIFAR-I0/100 For the optimizer, SGD used with a momentum of 0.9 and a weight decay of 5 x 10 10-4 For the MNIST dataset, the initial learning rate is set to 0.01 and the learning rate is decayed 10-fold at 50 epoch. For the CIFAR- 10/1R000000 dataset, the initial learning rate is 0.1, and is decayed 10-fold at 75-th epoch and 115-th epoch. For learning rate is set 0.1, and is decayed 10-fold at 50-th epoch and 100-th epoch. For ImageNet, the learning rate is constant with 0.001.
the initial
A.2 Adversarial training
During adversarial training, we use max-average attack and average attack [12] for generating adversarial examples via PGD attacker algorithm [21] with 7-steps, while the PGD attacker algorithm with 50-step is used for measuring robustness against a stronger attack during test time. With PGD- 50 attack, we measure adversarial test accuracy on 3 random seeds and average the results. For PGD attack, the perturbation degree 0.3 is used for MNIST, 10/100, and e = 2/255 Miny-ImgeNeet/mmgeeNe The step size ó is set to 20/255 for MNIST, 2/255 for CIFAR- ImageNet//mageNN
8/255 is used for CIFAR- during both adversarial training and inference time.
and &e (0.0052) for Tiny- The number of iterations is commonly 50-steps. Similarly, when measuring adversarial transferability, we also use PGD attack with 50 steps, 2/255 step size, and 8/ 8/255 epsilon value to generate adversarial attacks. With PGD attack, the attack success rate is utilized as the metric measure adversarial transferability. We perform each experiment on 5 random seeds and average the results.
Additionally, the hyperparameter for NKD is set to 3, and | for EOKD is set to 1 across all experiments. On the other hand, the exit- balancina parameter y is set to [1, 1] for MNIST and CIFAR-10, and [1, 1, 1, 1.5, 1.5], [1, 1, 1.5, 1.5, 1.5, 1.5] for Tiny- Lanp-maaaeNe//mmeeeet CIFAR-100, respectively.
A.3 How to determine confidence threshold in budgeted prediction setup
We provide a detailed explanation about how to determine confidence threshold for each exit using validation set before the testing phase. First, in order to obtain confidence thresholds for various budget scenarios, we allocate the number of validation samples for each exit. For simplicity, consider toy example with 3-exit network (i.e., L 3) and assume the number of validation set is 3000. Then, each exit can be assigned a different number of samples: for instance, (2000, 500, 500), (1000, 1000, 1000) and (500, 1000, 1500). As more samples are allocated to the early exits, scenario with a smaller budget can be obtained, while allocating more data to the later exits can lead to a scenario with larger budget. More specifically, to see how obtain the confidence threshold for each exit, consider the low-budget case of (2000, 500, 500). The model first makes predictions on all 3000 samples exit 1 and sorts the samples based on their confidence. Then, the 2000-th largest confidence value is set as the confidence threshold for the exit 1. Likewise, the model performs predictions on remaining 1000 samples exit 2 and the 500-th largest confidence is determined as the threshold for exit 2. Following this process, all thresholds for each exit are determined. During the testing phase, we perform predictions on test samples based on the predefined thresholds for each exit, and calculate the total computational budget for the combination of (2000, 500, 500). In this way, we can obtain accuracy and computational budget for different combinations of data numbers
13

---

Exit
2
3
Average
Adv. w/o Distill [12
94%
99.36%
99.55%
99.02%
NEO-KD (ours)
97.82%
99.29%
99.48%
98.86%
(a) MNIST
top-1 accuracy (%)
1op-5 accuracy (%)
Exit
Avg.
3
5
Avg.
Adv. w/o Distill [22
43.10
47.64
50.36
50.58 50.70
48.48
68.26
72.30
74.86
75.58
75.36
73.27
NEO-KD (ours)
45.56
48.32
49.88
50.56 50.52
48.97
70.56
73.12
74.52
74.22
74.98
73.48
(b) Ima-lmm
Table Al: Clean test accuracy in the anytime prediction setup: NEO-KD's advantage in terms of adversarial test accuracy (as shown in the main mansuscript) can be achieved without largely compromising the clean test accuracy.
Top- nocuracy (%)
Top-5 accuracy (5))
Exit
Avg.
2
5
Avg.
Adv. w/o Distali
12
21.14
20.49
20.68
20.18
20.58
20.56
20.24
20.55
48
48.24
46.75
46.41
46.31
46.96
45.64
46.94
22.17
18.82
19.20
51.43
46.84
ARD
20.55
19.84
20.95
19.61
19.41
20.52
21.72
20.37
48.18
46.94
46.96
45.37
45.06
46.17
48.42
46.73
LW
17.65
18.80
16.97
16.64
17.15
17.08
16.79
17.30
44.33
44.03
41.49
41.48
41.37
42.06
41.64
42.34
NEO-KD (ours)
22.01
21.99
49.12
48.00
47.23
48.46
Max-average attack
10p-1
accuracy (%6
top-5 accuracy (98)
Avg.
Avg.
Adv. wio Distil 12
12.73
12.10
13.33
13.27
13.14
13.17
13.27
13.00
36.22
35,66
37.07
36.57
36.47
37.26
36.51
36.34
SKD
24
15.96
15.90
15.31
15.32
15.70
14.84
10.33
14.77
42.19
41.72
41.29
41.46
42.03
40.08
32.68
40.21
ARD
12.74
12.53
13.80
13.04
12.79
13.56
14.73
13.31
36.12
35.67
37.21
36.28
28
35.81
37.09
38.82
36.71
1W
12.82
13.33
12.28
11.96
12.10
12.25
12.55
12.47
36.49
36.23
34.46
34.17
34.09
34.78
34.72
34.99
(ours)
Average attack
Table A2: Adversarial training via Average attack: Adversarial test accuracy in the anytime prediction setup on CIFAR-100.
(i.e., various budget scenarios), Figures 2 and in the main manuscript show the results for 100 cases of different budget scenarios.
B Clean Test Accuracy
Table shows the clean accuracy results using the model built upon adversarial training via max- average attack. We observe that NEO-KD generally shows comparable clean test accuracy with Adv. w/o Distill [12), especially on the more complicated dataset Tiny-ImageNe [17] while achieving much better adversarial test accuracy as reported in the main manuscript.
C Adversarial Training via Average Attack
In the main manuscript, we presented experimental results using the model trained based on max- average attack. Here, we also adversarially train the model via average attack [12] and measure adversarial test accuracy on CIFAR- -00 dataset. Table A2 compares adversarial test accuracies of NEO-KD and other baselines against max-average attack and average attack. The overall results are consistent with the ones in the main manuscript with adversarial training via max max-average attack, further confirming the advantage of NEO-KD.
D Hyperparameter Tuning
In the NEO-KD objective function, there are three hyperparameters (0,3, 3),,,, where 3 control the amount distilling knowledge from NKD, HOKD and y increases the amount of knowledge distilled to later exits.
14

---

D.1 Hyperparameter (a. .3)
The extreme value of a and 3 can destroy ideal adversarial training. Too large a makes strong NKD, which results in high dependency among submodels and too small a makes weak NKD, which cannot distill enough knowledge to student exits. In contrast, too large makes strong BOKD, which can interrupt adversarial training by distilling only sparse knowledge (likelihoods of majority classes zero) and too small 3 makes weak EOKD, which cannot mitigate dependency among submodels. We select a, B values in the range of [0.35, 3] and measure the adversarial test accuracy value by averaging adversarial test accuracy from all exits. The candidate (ax, (3) pairs are (0.35, 1), (1, 0.35), (0.35, 0.35). (0.5, 1); (1, 0.5). (0.5, 0.5). (1, 1), (2, (1, (2, (3, 1). (1,3), and (3,3). When (a. is (3. 1). NEO-KD achieves 28.96% of adversarial test accuracy against max-average attack and 22. 28. against average attack, which is the highest adversarial test accuracy among the various candidate (a, 3) pairs. Therefore, we use (3, 1) as (a, 3) pair in our experiments.
D.2 Hyperparameter y
Since the prediction difference between the last exit (teacher prediction) and later exits is smaller than the prediction difference between the last exit and early exits, later exits are less effective for taking advantage of knowledge distillation. Therefore, we provide slightly larger weights to later exits for distilling more knowledge to later exits than early exits. The candidate 7 values are [1, 1, 1, 1, 1, 1, 1], 1,, 1.5, 1.5, 1.5). and [1, 1, 1.7, 1.7, 1.7, 1.7]. As result, when we distill 1.5 times more knowledge to later exits, NEO-KD achieves 28. 19.96% of adversarial test accuracy against max- avx-average attack and 22.88% against average attack, which the highest adversarial test accuracy compared to providing same weights with earlier exits to later exits (28.13% for max-average and 21.66% for average attack) or distilling 1.7 times more knowledge to later exits than earlier exits (28.68% for max-average and 22.58% for average attack). The adversarial test accuracy value is the average of adversarial test accuracies from all exits. Therefore, we use y [1, 1, 1, 1.5, 1.5, 1.5, 1.5] in our experiments. This result proves that the exit-balancing parameter with appropriate value is needed for high performance.
E Discussions on Performance Degradation at Later Exits
As can be seen from the results for the anytime prediction in the main manuscript, the adversarial test accuracy of the later exits sometimes lower than the performance of earlier exits. This phenomenon can be explained as follows: In general, we observed via experiments that adversarial examples targeting later exits has the higher sum of losses from all exits compared adversarial examples targeting earlier exits. This makes max-average or average attack mainly focus on attacking the later exits, leading to low adversarial test accuracy at later exits. The performance of later exits can be improved by adopting the ensemble strategy as in the main manuscript for the budgeted prediction setup.
F Comparison with Recent Defense Methods for Single-Exit Networks
The baselines in the main paper were generally the adversarial defense methods designed for multi- exit networks. In this section, we conduct additional experiments with recent defense method, TEAT [7]. and compare with our method. Since TEAT was originally designed for the singlc-exit network, we first adapt TEAT the multi-exit network setting. Instead of the original TEAT that generates the adversarial examples considering the final output of the network, we modify TEAT to generate adversarial examples that maximizes the average loss of all exits in the multi-exit network. Table A3 below shows the results using max-average attack on CIFAR- 10/100. It can be seen that our NEO-KD, which designed for multi-exit networks, achieves higher adversarial test accuracy compared to the TEAT methods (PGD-TE and TRADES-TE) designed for single-exit networks. The results highlight the necessity of developing adversarial defense techniques geared to multi-exit networks rather than adapting general defense methods used for single-cxit networks.
15

---

Exit
2
3
Average
PGD-TE 7
48.73%
46.00%
46.85%
47.19%
TRADES-TE
45.05%
39.6 39.64%
42.10%
42.26%
NEO-KD (ours)
46.53%
47.65 %
50.71%
48.30%
(a) CIFAR-10
Exit
2
4
5
7
Avg.
PGD-TE 7
24.07%
24.39%
25.14%
25.35%
26.29%
25.57%
24.60%
25.06%
TRADE-TE
7 17.62%
18.52%
18.61%
18.98%
18.95%
19.67%
20.35%
18.96%
NEO-KD (ours)
28.37% %
28.78% %
29.02%
29.49 %
30.06%
28.45%
28.54 %
28.96 %
(h) CIFAR-100
Table A3:
Comparison of adversarial test accuracy against max-average attack between TEAT methods and our NEO-KD.
Exit
2
3
Avg.
SKD (exit I)
32.27%
36.92%
38.57%
35.92%
SKD (exit 2)
35.33%
35.10%
37.82%
36.08%
SKD (exit 3)
39.36%
41.39%
38.39%
39.71%
SKD (ensemble)
38.63%
41.80%
40.13%
40.19%
ARD (exit I)
35.64%
38.10%
42.12%
38.62%
ARD (exit
35.35%
38.24%
40.00%
37.86%
ARD (exit 3)
39.37%
41.98%
43.53%
41.63%
ARD (ensemble)
35.22%
38.35%
40.76%
38. 18%
NEO-KD (ours)
41.67%
45.38%
45.54%
44.20%
Table A4: Adversarial test accuracy of SKD and ARD according to exit selection as a teacher prediction.
G Comparison with SKD and ARD
for multi-exit networks improve the performance on clean samples by self-distilling the knowledge of the last exit, as the last exit has the best prediction quality. Therefore, following the original philosophy, we also used the last exit implementing the SKD baseline. Regarding ARD S. since it was proposed for single-exit networks, we also utilized the last exit with high performance when applying ARD to multi-exit networks. Nevertheless, we perform additional experiments to consider comprehensive baselines using various exits for distillation. Table above shows the results of SKD and ARD using specific exit or an ensemble of all exits for distillation. The results show that our scheme consistently outperforms all baselines.
Existing eeff-istilllatinn schemes 20 24
H Implementations of Stronger Attacker Algorithms
In Section 4.3 of the main manuscript, during inference, we replaced the Projected Gradient Descent (PGD) attack with other attacker algorithms (PGD-100 attack, Carlini and Wagner (CW) attack (2], and AutoAttack (5) to generate stronger attacks for multi-exit neural networks. This section provides explanation on how these stronger attacks are implemented tailored multi-exit neural networks.
H.1 Carlini and Wagner (CW) attack
The Carlini and Wagner (CW) attack is a method of generating adversarial examples designed to reduce the difference between the logits of the correct label and the largest logits among incorrect labels. In alignment with this attack strategy, we modify the CW attack for multi-exit neural networks. the process of minimizing this difference, our modification aims to minimize the average difference across all exits of the multi-exit neural network. Moreover, when deciding whether sample has been successfully converted into an adversarial example, we consider a sample adversarial if misleads all exits in the multi-exit neural network.
16

---

H.2 AutoAttack
AutoAttack produces adversarial attacks by ensembling various attacker algorithms. For our experi- ment, we sequentially use APGD [S], APGD-T [5], FAB [4, and Square [1] algorithms to generate adversarial attacks, as they are commonly used.
H.2.1 APGD and APGD-T
The APGD attack is a modified version of the PGD attack, which is limited by its fixed step size, suboptimal choice. The APGD attack overcomes this limitation by introducing an adaptive step size and momentum term. Similarly, the APGD-T attack is variation of the APGD attack where the attack perturbs a sample to change to a specific class. this process, we use the average loss of all exits in the multi-exit neural network as the loss for computing the gradient for adversarial example updates. Moreover, we define sample as adversarial it misleads all exits in the multi-exit neural network.
H.2.2 FAB
The FAB attack creates adversarial attacks through process involving linear approximation of classifiers, projection to the classifier hyperplane, convex combinations, and extrapolation. The FAB attack first defines hyperplane classifier separating two classes, then finds new adversarial example through convex combination of the crapoolation-rooeettee current adversarial example and the extrapolation projected original sample with the minimum perturbation norm. Here, we the average gradient of all exits in the multi-exit neural network as the gradient for updating adversarial examples. Similar to above, we label a sample as adversarial if can mislead all exits in the multi-exit neural network.
H.2.3
Square
The Square attack generates adversarial attacks via random searches of adversarial patches with variable degrees of perturbation and position. The Square attack algorithm iteratively samples the perturbation degree and the positions of patches while reducing the size of patches. The sampled adversarial patches added sample, and the patches that maximizes the loss of the target model are selected. Here, we use the average loss of all exits in the multi-exit neural network as the loss for determining whether sampled perturbation increases decreases the loss of the target model. Additionally, we determine a sample as adversarial if it misleads all exits in the multi-exit neural network.
H.3 Experiment details
For all attacks, we commonly use = 0.03 as the perturbation degree and generate adversarial examples over 50 steps. All the attacks based Tox norm. For the APGD attack, we employ cross entropy loss for computing the gradient to update adversarial examples. In both the APGD-T and FAB attacks, all class pairs are considered when generating adversarial attacks. For the Square attack, the random search operation is conducted 5000 times (the number of queries). Other settings follow [14). In terms of performance comparison against stronger attacker algorithms, we adopt adversarial training via average attack in both Adv. w/o Distill [2] and NEO-KD (our approach). However, since the original CW attack algorithm and AutoAttack algorithm were designed for single-exit neural networks, this adapted versions targeting multi-exit neural networks are relatively weak.
17


Reviews:
[{"review_id": "oes1ORvVwy", "reviewer": "Reviewer_iEEe", "summary": "The paper presents a novel method called Neighbor Exitwise Orthogonal Knowledge Distillation (NEO-KD) for improving the adversarial robustness of multi-exit networks. The method's motivation lies in addressing the issue that existing knowledge distillation schemes are not ideal for multi-exit networks as they either increase or decrease adversarial robustness. The authors argue that the choice of the knowledge to distill and the specific exit to target significantly influence the robustness of multi-exit networks. \n\n- NKD improves the network's defense by ensuring the outputs of adversarial examples mimic the outputs of clean data, by distilling the combined predictions of clean data neighbor exits, leading to enhanced robustness and superior feature quality for corresponding exits. \n\n- EOKD concentrates on minimizing adversarial transferability between different network submodels, by distilling the output of clean data to the output of adversarial samples in an exit-by-exit manner, while promoting orthogonality in non-maximal predictions of individual exits. \n\n- Experiments demonstrate that NEO-KD outperforms existing solutions across a range of commonly adopted datasets, including MNIST, CIFAR-10/100, and Tiny-ImageNet. \n- NEO-KD demonstrates superior performance under different prediction setups (anytime and budgeted), and exhibits reduced adversarial transferability. \n- Ablation studies demonstrated the benefits of its individual components and its robustness against stronger adversarial attacks. \nOverall, the paper offers a substantial contribution to improving the adversarial robustness of multi-exit networks.", "questions": "The paper incorporates the application of ensemble strategy at inference time in the budgeted prediction setup. It would be interesting to learn more about your decision process for using this approach. Specifically, how did you determine the confidence threshold for selecting the exit? Are there any trade-offs or implications if the threshold was set differently?", "limitations": "- The experiments were performed using relatively smaller datasets (CIFAR-10, CIFAR-100, MNIST, and Tiny-ImageNet), and the performance on larger datasets such as ImageNet remains unexplored.\n- The choice and implications of the confidence threshold used in the ensemble strategy during the budgeted prediction setup are not clearly explained.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1) The paper presents a novel approach called NEO-KD that uses Neighbor Knowledge Distillation (NKD) and Exitwise Orthogonal Knowledge Distillation (EOKD) for enhancing adversarial robustness in multi-exit networks. This approach is distinct in its application of knowledge distillation techniques to the unique challenges posed by multi-exit networks.\n\n2) An extensive set of experiments were conducted to validate the proposed method using different datasets and adversarial attacks. This includes the use of anytime and budgeted prediction setups, and analysis of adversarial transferability. The results show the NEO-KD method's performance relative to baseline techniques.\n\n3) The paper(motivation) addresses an important problem in the area of multi-exit networks: how to enhance adversarial robustness. The proposed solution, NEO-KD, not only improves robustness, but also optimizes computation costs and reduces adversarial transferability.", "weaknesses": "The approach is tested on four standard datasets: MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet. While these are common benchmarks, the effectiveness of NEO-KD on larger datasets such as ImageNet is not demonstrated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for the time and efforts. In the response below, we provide answers to the comments raised by the reviewer.\n\n### **Results on larger datasets**\n\nWe appreciate the suggestion. We conducted additional experiments using ImageNet with 1000 object classes. Due to the strict timeline, we considered ImageNet(100) where 100 samples are selected from each class in the ImageNet to construct the training set. Considering that there are 1000   classes in ImageNet, the number of train samples we use is 100,000. This dataset has been adopted in many multi-exit network literature to prove the effectiveness of the algorithm [ICCV'19], [AAAI'21]. The results in the below table show that the proposed NEO-KD performs the best in the larger-scale dataset with 1000 classes, further confirming its advantage.\n\n| Exit | 1 | 2 | 3 | 4 | 5 | Average |\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|\n| Adv. w/o Distill | 18.54% | 24.66% | 27.63% | 28.28% | 29.71% | 25.76% |\n| NEO-KD (ours) | 21.86% | 28.04% | 31.15% | 31.93% | 33.87% | 29.37% |\n\n### **Accuracy-computation trade-off controlled by the confidence threshold** \n\nIn a budgeted prediction setup, given a limited computing budget, the trained model has to make predictions for all samples within the budget. For instance, if the budget is sufficiently provided, we can obtain high performance by classifying many samples at the later exits. On the other hand, if the budget is significantly small, the model should classify most of the samples at early exits while compromising performance. In other words, the model should make predictions efficiently within limited budget constraints by classifying 'easy' samples at early exits and 'hard' samples at later exits. At this time, _confidence threshold_, which is the maximum value of the prediction probability vector by softmax, is a criteria for determining whether to perform prediction on a sample at the current exit or the next exit. Note that each exit has a different confidence threshold, which is determined using validation set (this will be clarified  soon). Specifically, given a sample, if the confidence of the sample at a specific exit is greater than the predefined threshold at the exit (easy sample), the prediction is made on the current exit, and if it is lower than the threshold (hard sample), the feature of the sample is passed to the next exit for prediction at the next exit. Therefore, confidence threshold controls the trade-off between computational cost and accuracy. As the threshold increases, more samples are predicted at the earlier exits, reducing the number of samples forwarded to the later exits.  Consequently, fewer resources are used for classifying test samples while compromising the performance. Conversely, when the threshold is decreased, more samples are predicted at later exit, which consumes more resources and yields higher performance.\n\n### **How to determine confidence threshold** \n\nAs per the reviewer's suggestion, we would like to provide a detailed explanation about how to determine confidence threshold for each exit using validation set before the testing phase. First, in order to obtain confidence thresholds for various budget scenarios, we allocate the number of validation samples for each exit. For simplicity, consider a toy example with 3-exit network (i.e., $L = 3$) and assume the number of validation set is 3000. Then, each exit can be assigned a different number of samples: for instance, (2000, 500, 500), (1000, 1000, 1000) and (500, 1000, 1500). As more samples are allocated to the early exits, a scenario with a smaller budget can be obtained, while allocating more data to the later exits can lead to a scenario with a larger budget. More specifically, to see how to obtain the confidence threshold for each exit, consider the low-budget case of (2000, 500, 500). The model first makes predictions on all 3000 samples at exit 1 and sorts the samples based on their confidence. Then, the 2000th largest confidence value is set as the confidence threshold for the exit 1. Likewise, the model performs predictions on remaining 1000 samples at exit 2 and the 500th largest confidence is determined as the threshold for the exit 2. Following this process, all thresholds for each exit are determined. During the testing phase, we perform predictions on test samples based on the predefined thresholds for each exit, and calculate the total computational budget for the combination of (2000, 500, 500). In this way, we can obtain accuracy and computational budget for different combinations of data numbers (i.e., various budget scenarios). Fig. 2 and 3 in the main paper show the results for 100 cases of different budget scenarios. \n\nAgain, we appreciate the reviewer for the insightful comments. We would be happy to address any remaining questions the reviewer might have.\n\n[ICCV'19] Phuong et al., ``Distillation-Based Training for Multi-Exit Architectures,'' ICCV 2019.\n\n[AAAI'21] Wang et al., ``Harmonized Dense Knowledge Distillation Training for Multi-Exit Architectures,'' AAAI 2021.\n\nAgain, we appreciate the reviewer for the positive comments with valuable feedback. We would appreciate further opportunities to answer any remaining concerns you might have."}, {"review_id": "8wh34KQIfu", "reviewer": "Reviewer_PqqG", "summary": "This paper proposed a knowledge-distillation based adversarial training method, which is designed for multi-exit neural networks. The authors propose neighbor knowledge distillation to improve the robustness against adversarial attacks, and propose exit-wise orthogonal knowledge distillation to reduce the adversarial transferability across different submodels. Moreover, the proposed method is a plug-and-play method, which can be used in prevailing training strategies of multi-exit networks.", "questions": "1.\tI hope the authors can clarify the motivation of EOKD and the orthogonal labeling operation more clearly.\n2.\tPlease explain the setting of baselines which only use the prediction of last exit to conduct knowledge distillation.\n3.\tI would like to know the training speed of the proposed NEO-KD, as there are many extra self-distillation terms, which will complicate the computational graph.\n", "limitations": "As a new adversarial training method, this paper builds a comprehensive solution, and I think this paper have discussed their limitations enough.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThis paper brings self-distillation into adversarial training, and make a good combination with multi-exit networks.\n2.\tThe experimental results on several datasets and attacks demonstrate its superiority, and the considered scenarios are sufficient.\n", "weaknesses": "1.\tThough the experimental result is extraordinary, the motivation and operation of EOKD makes me confused. I did not really get the meaning of orthogonal labeling operation $O(\\cdot)$, and the paper does not clearly elaborate the relation between orthogonal labeling and adversarial transferability reduction. Moreover, I would like to know whether EOKD will change the behavior of the output from every exit on clean examples.\n2.\tThe settings of baselines need to be discussed. The author conduct experiments of distillation-based baselines (SKD and ARD) by utilizing the prediction of the last exit, instead of every exit or random exit or another specific exit. Is there any reason to choose the last exit in baseline methods?\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for the helpful comments, especially on the unclear aspects in the paper. In the response below, we would like to clarify all the ambiguous points raised by the reviewer.\n\n### **Motivation of EOKD**\nMulti-exit networks are highly vulnerable to simple attacks (e.g., an adversarial attack targeting a single exit) as the submodels in the network are highly correlated by sharing some model parameters. The motivation of EOKD is to _reduce the dependencies among exits_ while taking advantage of knowledge-distillation during adversarial training of multi-exit networks. To achieve this, EOKD is equipped with two key components: (i) exit-wise distillation and (ii) orthogonal distillation. The idea of the first component is to distill the knowledge of clean data to the output of the adversarial sample in an exit-wise manner, which intuitively reduces the interdependencies between exits by distilling a different knowledge to each exit (this has a clear difference compared to prior self-distillation strategies [ICCV'19a], [ICCV'19b] that distill the same knowledge to all exits). The second component of EOKD is based on the orthogonal labeling operation $O(\\cdot)$ to encourage the predictions for the non-ground-truth classes of individual exits to be mutually orthogonal, by providing orthogonal soft labels. \n\n### **Motivation of orthogonal labeling and its operation.**\nThe EOKD loss function is defined as $EOKD_{i,j}=\\ell(f_{\\theta_i}(x^{adv}_j), O(f _{\\theta_i}(x_j)))$ as in Eq. (6) of our main paper. To see how the orthogonal labeling operation $O(\\cdot)$ works, consider a toy example with 3-exit network (i.e., $L=3$) focusing on a 4-way classification task (i.e., $C=4$). Let  $[p^i_1, p^i_2, p^i_3, p^i_4]$ be the softmax output of the clean sample at the $i$-th exit, for $i=1,2,3$. If class 1 is the ground-truth, the orthogonal labeling operation $O(\\cdot)$ jointly produces the following results from each exit: $[\\hat{p}^1_1, \\hat{p}^1_2, 0, 0]$ from exit 1, $[\\hat{p}^2_1, 0, \\hat{p}^2_3, 0]$ from exit 2, $[\\hat{p}^3_1, 0, 0, \\hat{p}^3_4]$ from exit 3, where $\\hat{p}$ indicates the normalized probability of $p$ so that the values in each vector sum to one. Here, it can be seen that except the prediction $\\hat{p}_1^i$ for class 1, the non-ground-truth predictions become orthogonal across different exits with no overlappings. This strategy enables the model to distill the ground-truth information to all exits while maximizing the distinction of distilled knowledge for each exit, reducing the dependencies among exits (and thus improving adversarial transferability).   \n\nMore generally speaking, for each exit, $O(\\cdot)$ randomly selects $\\lfloor (C-1)/L \\rfloor$ labels among total of $C$ classes so that   the selected labels are non-overlapping across different exits (except for the answer label), where the probabilities of selected labels are normalized to sum to one. This makes the predictions  for the non-ground-truth classes to be orthogonal across all exits. By doing so, the essential knowledge of the ground-truth class is preserved while the knowledge of other classes is orthogonally distilled, promoting the reduction of dependencies between exits. In Fig. 4 of the main paper, it can be seen that EOKD can effectively reduce the adversarial transferability. This also leads to an improved adversarial test accuracy compared with the baselines, as can be seen in Tables 1,2,3,4 of our main manuscript.\n\n### **Effect of EOKD on clean examples**\nTable R1 & R2 above shows the comparison between NKD and NEOKD in terms of clean/adversarial test accuracy using CIFAR-10. As can seen from the results, it is confirmed that applying EOKD slightly compromises the clean accuracy but yields large performance gain in adversarial test accuracy.\n\n### **Why do baselines (SKD, ARD) use the last exit for distillation?**\nWe would like to first clarify the reason why the last exit is utilized in implementing the baselines (SKD [ICCV'19a], ARD [AAAI'20]), and then provide additional experimental results using different distillation strategies for the baselines (e.g., using another exit or ensemble of exits instead of the last one). Existing self-distillation schemes [ICCV'19a, ICCV'19b] for multi-exit network improve the performance on clean samples by self-distilling the knowledge of the last exit, as the last exit has the best prediction quality. Therefore, following the original philosophy, we also used the last exit in implementing the SKD baseline. Regarding ARD [AAAI'20], since it was proposed for single-exit network, we also utilized the last exit with high performance when applying ARD to multi-exit networks. Nevertheless, as per the reviewer's suggestion, we performed additional experiments to consider comprehensive baselines using various exits for distillation. Table R3 above shows the results of SKD and ARD using a specific exit or an ensemble of all exits for distillation. The results show that our scheme consistently outperforms all of the baselines.\n\n### **Training speed**\nIn Table R4, we compare the training time of our scheme and the considered baselines for one epoch on CIFAR-100 dataset. It is observed that our scheme requires 4\\% more time than the basic adversarial training (i.e., Adv. w/o Distill), and only 1.6\\% more time than other KD based baselines (SKD, ARD). Considering that the main focus of multi-exit networks is the latency during inference rather than training, this small additional computation during training can be seen as a reasonable cost for achieving an improved adversarial test accuracy compared to the baselines.\n \nAgain, thank you for your time and efforts in reviewing our paper. Your raised concerns made us think deep and wide; and we feel we have managed to clarify all the issues raised. We would appreciate further opportunities to answer any remaining concerns you might have."}, {"review_id": "BivcKAM1Y8", "reviewer": "Reviewer_KNG2", "summary": "This paper proposed a know-distillation-based method to improve the adversarial robustness of multi-exit neural networks. Extensive experiments are conducted to show the effectiveness of the proposed method.", "questions": "1. \"Considering our distillation method, NEO-KD is currently not directly applicable to object detection.\"\nCan the author briefly explain why it cannot be applied to object detection?\n", "limitations": "The limitations are clearly stated.", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. Extensive experiments are conducted to show the effectiveness of the proposed method.\n2. This method is a novel combination of previous methods.\n3. The paper is well-organized and easy to follow.\n4. Technical details are provided in the supplementary materials.\n5. Ablation study is provided.", "weaknesses": "1. This method is to improve the adversarial robustness of DNNs, but this paper lacks a comparison with some general defense methods, e.g., TEAT[1]. \n2. What is the advantage of distillation methods over other adversarial defense methods on this task?\n3. \"One challenge is that simply applying existing self-distillation techniques increases the adversarial transferability across different submodels, since the same knowledge from the last exit is distilled to all the other exits, increasing the dependency among different submodels in the network. \"\nI can find any support for this claim in this paper. \n4. What is the difference between the proposed method and other related work? This paper lacks a detailed/formal comparison with other related methods. \n\nDong, Yinpeng, et al. \"Exploring memorization in adversarial training.\" ICLR 2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for the positive comments and valuable feedback. Below, we provide answers to the comments raised by the reviewer.\n\n### **Comparison with TEAT**\nThe baselines in the main paper were generally the adversarial defense methods designed for multi-exit network. As the reviewer suggested, we conducted additional experiments with TEAT [ICLR'22] and compare with our method. Since TEAT was originally designed for the single-exit network, we first adapted TEAT to the multi-exit network setting. Instead of the original TEAT that generates the adversarial examples considering the final output of the network, we modified TEAT to generate adversarial examples that maximizes the average loss of all exits in the multi-exit network. Table R5 above shows the results using max-average attack on CIFAR-10/100 dataset. It can be seen that our NEO-KD, which is designed for multi-exit network, achieves higher adversarial test accuracy compared to the TEAT methods (PGD-TE and TRADES-TE) designed for single-exit networks. The results highlights the necessity of developing adversarial defense techniques geared to multi-exit networks rather than adapting general defense methods  used for single-exit network.\n\n### **Advantages of distillation-based approaches over adversarial defense methods**\nOur distillation-based approach for robust multi-exit networks has the following key advantages over conventional adversarial defense methods:\n\nFirst, it directly tackles the unique challenges that arise in multi-exit networks. Multi-exit networks are highly vulnerable to simple attacks (e.g., an attack targeting a single exit) since it has high dependencies between exits. Existing adversarial defense approaches cannot directly handle this unique problem of multi-exit network, as they focus on the single-exit network. Even when these methods are adapted to multi-exit networks, they face the same challenges as can be also seen in our results with TEAT above. On the other hand, distillation-based methods provide a great platform to tackle this issue, as each exit can be distilled with different knowledge to reduce the dependencies among exits. Our NEO-KD strategically utilizes distillation based on neighbor ensembling, exit-wise, and orthogonal distillation.\n\nAnother advantage of the distillation method in defending against adversarial attacks is its compatibility with other defense strategies. Specifically, our distillation-based scheme is an orthogonal approach to existing defense methods, as the proposed distillation losses can be used with any classification loss (including adversarial training loss). As can be seen from experimental results of the main paper, our NEO-KD combined with existing adversarial loss (via max-average attack [ICLR'20]) brings large performance gains, showing its good compatibility.\n\n### **Do the prior self-distillation methods really increase the adversarial transferability?**\nThis claim can be supported by the results in Fig. 4 of our main paper. The average adversarial transferability of SKD (which adopts self-distillation) is 33.36%, which is significantly larger than the transferability of the baseline without distillation, which is 23.68%. This makes sense since the same knowledge from the last exit is distilled to all other exits, which results in increased dependencies among all submodels. On the other hand, our NEO-KD achieves the lowest adversarial transferability (20.12%) based on the proposed neighbor distillation and exit-wise orthogonal distillation, that are designed to reduce the transferability among submodels. This advantage of NEO-KD in adversarial transferability contributes to a better adversarial test accuracy compared to the baselines, as shown in the main paper.\n\n### **More detailed comparison with related works**\nSince this paper focuses on the robustness of multi-exit neural networks, we mainly compared our NEO-KD approach with other self-distillation and defense methods used in the context of multi-exit networks, as discussed in the related work section (Section 2) of the main paper. Below, we provide a more detailed comparison with existing works relevant to conventional adversarial training.\n\nIn the context of defense schemes against adversarial examples, conventional adversarial training methods [ICML'19, ICLR'22, CVPR'23] proposed for single-exit network have mainly focused on creating new adversarial training losses restricted to single-exit network. Therefore, adversarial transferability between exits has not been a key issue in the prior works for single-exit network. Recent methods targeting multi-exit network [ICLR'20] have also proposed new adversarial training losses, but they still do not directly handle the adversarial transferability, which is a inherent problem in multi-exit networks. Our NEO-KD is an orthogonal approach focusing on directly mitigating adversarial transferability across exits and can be combined with the existing adversarial training losses.\n\nIn summary, our work provides distinct advantages compared to prior works by focusing on mitigating adversarial transferability which is a unique challenge of multi-exit network, improving robustness of multi-exit network.\n\n### **Application to object detection task**\nWhen applying distillation to object detection in multi-exit network, we need to distill predictions of box regression and classification from the teacher (last exit) to the students (early exits). However, since the box regression predictions can differ between the teacher and the students, the student may fail to detect an object box that the teacher can identify, making it difficult to apply to object detection. We believe our work could pave the way to develop distillation-based adversarial training for multi-exit network in more complicated tasks, such as object detection.\n\nAgain, thank you for the positive and helpful comments. We will make all of these points clearer in the revised manuscript."}, {"review_id": "5gAVCGeZGG", "reviewer": "Reviewer_grwc", "summary": "This paper proposes a knowledge distillation approach for defending against adversarial attacks in multi-exit networks. They have two different objectives: (i) leveraging self-distillation to improve adversarial robustness, (ii) reducing adversarial transferability among the submodules of the network. Since multi-exit networks have submodules that are correlated with each other, distilling knowledge from the output of clean data at the last exit to all submodules  increases the chances of adversarial transferability. To avoid this, this paper proposes a two-fold approach: (i) Neighbor knowledge distillation (NKD): generates teacher prediction at exit i by averaging the predictions of clean data from exits i-1, i and i+1 and distills it to adversarial predictions at exit i; (ii) Exit-wise orthogonal knowledge distillation (EOKD): output of clean data at exit i is distilled to adversarial example at exit i. Orthogonal labeling operation on clean predictions makes teacher predictions orthogonal across all exits. NKD helps create high quality teacher predictions improving adversarial robustness of multi-exit networks. At the same time, NKD creates different teacher predictions for each exit reducing the risk of adversarial transferability. EOKD further reduces chances of adversarial transferability.", "questions": "Please demonstrate results with ViT-tiny or small.\n\nPlease use large scale dataset like ImageNet to show the efficacy of the method.", "limitations": "Large scale demonstration missing, given that it is an empirical paper.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written. The approach is well explained and the claims have been substantiated with results.\n\n2. NEO-KD achieves best adversarial accuracy against max-average and average attacks in all budget setups.\n\n3. The paper successfully demonstrates reduction in adversarial transferability among exits in multi-exit networks", "weaknesses": "1. In anytime prediction setup, NEO-KD shows low performance at the later exits for small datasets like MNIST, Cifar-10 \n\n2. NEO-KD shows low performance at early exit for mid scale datasets like TinyImageNet.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for the time and efforts, and providing  helpful comments that are also very clear. Below, we provide responses to the reviewer's comments.\n\n### **NEO-KD shows low performance at the specific exits in some cases.**\n\nWe agree with the reviewer that in some datasets, NEO-KD occasionally does not achieve the best performance in specific exits. However, we respectfilly disagree that NEO-KD achieves low performance:  NEO-KD performs the best in all 7 exits for CIFAR-100 considering top-1 accuracy, performs the best in 4 out of 5 exits on Tiny-ImageNet. For small datasets (e.g., CIFAR-10), NEO-KD only looses against the baseline in 0 or 1 exit. Even in these exits, NEO-KD still achieves the second best performance most of the time, where the gap with  the best performance is marginal. NEO-KD also achieves the best average accuracy in all cases. Hence, when a system designer has  to decide which algorithm  to use for constructing a robust multi-exit neural network, it is easy to answer that the proposed NEO-KD is the best.  Overall, we believe that this strength of the proposed scheme as well as the fact that NEO-KD is the very first work to strategically integrate (i) _multi-exit networks_, (ii) _self-distillation_, and (iii) _adversarial training_, deserve merits in both multi-exit networks and adversarial training communities.\n\n### **Additional experiments.**\n\nWe appreciate the suggestion. To comply with the reviewer's comment, we conducted additional experiments using ImageNet with 1000 object classes. Due to the strict timeline, we considered ImageNet(100) where 100 samples are selected from each class in the ImageNet to construct the training set. Considering that there are 1000   classes in ImageNet, the number of train samples we use is 100,000. This dataset has been adopted in many multi-exit network literature to prove the effectiveness of the algorithm [ICCV'19], [AAAI'21]. The results in the below table show that the proposed NEO-KD performs the best in the larger-scale dataset with 1000 classes, further confirming its advantage.\n\n| Exit | 1 | 2 | 3 | 4 | 5 | Average |\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|\n| Adv. w/o Distill | 18.54% | 24.66% | 27.63% | 28.28% | 29.71% | 25.76% |\n| NEO-KD (ours) | 21.86% | 28.04% | 31.15% | 31.93% | 33.87% | 29.37% |\n\nWe also appreciate the reviewer for suggesting experiments with ViT models. However, due to strict timeline for  implementing and training  the baselines as well as our scheme with ViT models, we were not able to provide the corresponding results.  We agree that  there  exists some works that aim to combine multi-exit networks and  transformers, which itself is a challenging research topic   in the multi-exit network  community. However,  since our solution integrates   multi-exit network, self-distillation,  and adversarial training altogether (which has not been considered in the literature), adding another transformer-side dimension beyond these components   makes the problem setup extremely complicated and  causes  significant resource issues, compared to the case with only combining transformer with multi-exit network (with no adversarial training and no self-distillation).  Please  also consider that we  already used the most  commonly adopted model in multi-exit network literature  i.e., MSDNet.\n\nOverall, we   have now considered   5 datasets -- MNIST, CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet -- and trained SmallCNN and 3 different variations of MSDNet (with 3, 5, 7 exits).\n\n[ICCV'19] Phuong et al., ``Distillation-Based Training for Multi-Exit Architectures,'' ICCV 2019.\n\n[AAAI'21] Wang et al., ``Harmonized Dense Knowledge Distillation Training for Multi-Exit Architectures,'' AAAI 2021.\n\nAgain, thank you for your time and efforts, providing helpful comments to improve our paper. We would appreciate further opportunities to answer any remaining concerns you might have."}]

Author rebuttal:
We appreciate all reviewers for providing constructive comments, which have greatly helped us to improve the paper.

Due to the limited content we can provide in each response, we would like to share additional experimental results that **Reviewer PqqG** and  **Reviewer KNG2** suggested here. For the other reviewers (Reviewer iEEe and Reviewer grwc), all results are provided in our response corresponding to each reviewer.

In general, all approaches, including the baselines and our NEO-KD, are trained using adversarial examples generated through the max-average attack.


&nbsp;

### Tables \& References for **Reviewer PqqG**



* **Table R1:** Clean test accuracy of NKD and NEO-KD.
|Exit|1|2|3|Average Clean Accuracy|
|:---|:---:|:---:|:---:|:---:|
|NKD|76.81%|79.03%|81.98%|79.27%|
|NEO-KD|75.77%|78.37%|81.37%|78.50%|

* **Table R2:** Adversarial test accuracy of NKD and NEO-KD.
|Exit|1|2|3|Average Adversarial Accuracy|
|:---|:---:|:---:|:---:|:---:|
|NKD|46.48%|46.63%|44.64%|45.92%|
|NEO-KD|46.53%|47.65%|50.71%|48.30%|


* **Table R3:** Adversarial test accuracy of SKD and ARD according to exit selection as a teacher prediction.
|Exit|1|2|3|Average|
|:---|:---:|:---:|:---:|:---:|
|SKD (exit 1)|32.27%|36.92%|38.57%|35.92%|
|SKD (exit 2)|35.33%|35.10%|37.82%|36.08%|
|SKD (exit 3)|39.36%|41.39%|38.39%|39.71%|
|SKD (ensemble)|38.63%|41.80%|40.13%|40.19%|
|ARD (exit 1)|35.64%|38.10%|42.12%|38.62%|
|ARD (exit 2)|35.35%|38.24%|40.00%|37.86%|
|ARD (exit 3)|39.37%|41.98%|43.53%|41.63%|
|ARD (ensemble)|35.22%|38.35%|40.76%|38.11%|
|NEO-KD (ours)|41.67%|45.38%|45.54%|44.20%|


* **Table R4:** Comparison of training time between our NEO-KD and baselines.
|Method|Adv. w/o Distill|SKD|ARD|NEO-KD|
|:---|:---:|:---:|:---:|:---:|
|Training time (min/epoch) |13.40|13.80|13.80|14.02|



[ICCV'19a] Phuong et al., Distillation-based training for multi-exit architectures.

[ICCV'19b] Li et al., Improved techniques for training adaptive deep networks.

[AAAI'20] Goldblum et al,. Adversarially robust distillation.





&nbsp;


### Tables \& References for **Reviewer KNG2**

**Table R5:** Comparison of adversarial test accuracy against max-average attack between TEAT methods and our NEO-KD.

* (a) CIFAR-10
|Exit|1|2|3|Average|
|:---|:---:|:---:|:---:|:---:|
|PGD-TE [ICLR'22]|**48.73%**|46.00%|46.85%|47.19%|
|TRADES-TE [ICLR'22]|45.05%|39.64%|42.10%|42.26%|
|NEO-KD (ours)|46.53%|**47.65%**|**50.71%**|**48.30%**|

* (b) CIFAR-100
|Exit|1|2|3|4|5|6|7|Average|
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|PGD-TE [ICLR'22]|24.07%|24.39%|25.14%|25.35%|26.29%|25.57%|24.60%|25.06%|
|TRADES-TE [ICLR'22]|17.62%|18.52%|18.61%|18.98%|18.95%|19.67%|20.35%|18.96%|
|NEO-KD (ours)|**28.37%**|**28.78%**|**29.02%**|**29.49%**|**30.06%**|**28.45%**|**28.54%**|**28.96%**|


[ICLR'22] Dong, Yinpeng, et al. ``Exploring memorization in adversarial training,'' ICLR 2022.

[ICLR'20] Hu et al., Triple wins: Boosting accuracy, robustness and efficiency together by enabling input-adaptive inference.

[ICML'19] Zhang et al. Theoretically principled trade-off between robustness and accuracy.

[ICLR'22] Dong et al. Exploring memorization in adversarial training.

[CVPR'23] Dong et al., The Enemy of My Enemy is My Friend: Exploring Inverse Adversaries for Improving Adversarial Training.

[ICLR'20] Hu et al., Triple wins: Boosting accuracy, robustness and efficiency together by enabling input-adaptive inference.

[ICCV'19a] Phuong et al., Distillation-based training for multi-exit architectures.

[ICCV'19b] Li et al., Improved techniques for training adaptive deep networks.

Meta-review: The paper discusses an adversarial training approach based on knowledge distillation for multi-exit networks. The initial reviews raised questions regarding experiments on larger datasets, the role of hyper-parameters such as the confidence threshold,  motivation of components such as EOKD, training time, clarification on baseline, and comparison with TEAT. The AC feels the combination of adversarial training with self-distillation in the context of multi-exit networks is interesting and novel and the results are encouraging. Therefore, the AC recommends acceptance and would remind authors to include promised details and results in the revised version.
Decision: Accept (poster)