SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Generating adversarial examples. To generate adversarial examples during training and testing, we utilize max-average attack and average attack proposed in [12]. We perform adversarial training using adversarial examples generated by max-average attack, where the results for adversarial training via average attack are reported in Appendix. During training, we use PGD attack [21] with 7 steps as attacker algorithm to generate max-average and average attack while PGD attack with 50 steps is adopted at test time for measuring robustness against a stronger attack. We further consider other strong attacks in Section 4.3. In each attacker algorithm, the perturbation degree  $\epsilon$  is 0.3 for MNIST, and 8/255 for CIFAR-10/100, and 2/255 for Tiny-ImageNet/ImageNet datasets during adversarial training and when measuring the adversarial test accuracy. Other details for generating adversarial examples and additional experiments on various attacker algorithms are described in Appendix.
Evaluation metric. We evaluate the adversarial test accuracy as in [12], which is the classification accuracy on the corrupted test dataset compromised by an attacker (e.g., average attack). We also measure the clean test accuracy using the original clean test data and report the results in Appendix.
Baselines. We compare our NEO-KD with the following baselines. First, we consider the scheme based on adversarial training without any knowledge distillation, where adversarial examples are generated by max-average attack or average attack [12]. The second baseline is the conventional self-knowledge distillation (SKD) strategy [20, 24] combined with adversarial training: during adversarial training, the prediction of the last exit for a given clean/adversarial data is distilled to the predictions of all the previous exits for the same clean/adversarial data. The third baseline is the knowledge distillation scheme for adversarial training [8], which distills the prediction of clean data to the prediction of adversarial examples in single-exit networks. As in [8], we distill the last output of clean data to the last output of adversarial data. The last baseline is a regularizer-based adversarial training strategy for multi-exit networks [3], where the regularizer restricts the weights of the fully connected layers (classifiers). In Appendix we compare our NEO-KD with the recent work TEAT [7], a general defense algorithm for single-exit networks.
Inference scenarios. At inference time, we consider two widely known setups for multi-exit networks: (i) anytime prediction setup and (ii) budgeted prediction setup. In the anytime prediction setup, an appropriate exit is selected depending on the current latency constraint. In this setup, for each exit, we report the average performance computed with all test samples. In the budgeted prediction setup, given a fixed computational budget, each sample is predicted at different exits depending on the predetermined confidence threshold (which is determined by validation set). Starting from the first exit, given a test sample, when the confidence at the exit (defined as the maximum softmax value) is larger than the threshold, prediction is made at this exit. Otherwise, the sample proceeds to the next exit. In this scenario, easier samples are predicted at earlier exits and harder samples are predicted at later exits, which leads to efficient inference. Given the fixed computation budget and confidence threshold, we measure the average accuracy of the test samples. We evaluate our method in these two
top-1 accuracy (%)	top-5 accuracy (%)
Exit	1	2	3	4	5	6	7	Avg.	1	2	3	4	5	6	7	Avg.
Adv. w/o Distill [12]	28.04	28.32	28.34	27.64	26.78	25.93	24.77	27.12	60.64	61.15	59.13	58.46	58.38	57.46	57.44	58.95
SKD [24]	27.36	27.68	25.79	24.74	24.10	20.86	19.28	24.26	60.50	60.23	57.56	55.98	55.11	52.71	52.17	56.32
ARD [8]	27.60	28.00	27.99	27.42	28.32	27.25	29.08	27.95	60.26	60.51	59.31	58.40	58.40	58.09	60.62	59.37
LW [3]	20.44	20.57	19.86	19.34	19.51	19.46	20.22	19.91	50.69	50.91	48.47	48.56	48.13	49.61	49.76	49.45
NEO-KD (ours)	28.37	28.78	29.02	29.49	30.06	28.45	28.54	28.96	59.58	60.04	59.67	59.29	60.46	58.96	59.44	59.63
(a) Max-average attack
top-1 accuracy (%)	top-5 accuracy (%)
Exit	1	2	3	4	5	6	7	Avg.	1	2	3	4	5	6	7	Avg.
Adv. w/o Distill [12]	16.74	17.33	19.05	19.47	19.06	18.15	17.12	18.13	47.31	48.34	50.78	50.75	50.80	49.89	47.10	49.28
SKD [24]	18.13	18.45	19.53	19.87	19.54	16.67	14.21	18.06	49.85	50.72	51.17	51.91	51.73	47.32	43.18	49.41
ARD [8]	16.63	17.16	19.13	19.74	19.70	18.92	19.83	18.73	46.90	48.60	50.44	51.28	50.93	49.14	49.01	49.47
LW [3]	13.78	13.95	15.16	15.62	15.86	13.01	13.57	14.42	41.14	41.27	43.29	43.67	44.08	40.49	40.66	42.09
NEO-KD (ours)	20.41	21.32	23.27	24.30	24.47	23.99	22.39	22.88	49.48	51.47	53.13	55.01	54.69	54.32	52.11	52.89
(b) Average attack
Table 3: Anytime prediction setup: Adversarial test accuracy on CIFAR-100.
top-1 accuracy (%)	top-5 accuracy (%)
Exit	1	2	3	4	5	Avg.	1	2	3	4	5	Avg.
Adv. w/o Distill [12]	31.30	32.38	32.52	31.42	31.56	31.84	60.20	60.84	61.10	58.64	59.56	60.07
SKD [24]	33.04	33.24	30.50	28.40	28.50	30.74	63.68	62.72	60.04	58.50	58.28	60.64
ARD [8]	30.08	31.84	30.56	31.22	32.34	31.21	59.88	60.74	59.62	59.28	59.72	59.85
LW [3]	26.06	26.00	24.54	24.42	24.88	25.18	53.24	51.90	51.02	51.02	51.14	51.66
NEO-KD (ours)	32.96	35.08	33.42	32.40	32.64	33.30	62.48	62.82	61.76	60.70	60.92	61.74
(a) Max-average attack
top-1 accuracy (%)	top-5 accuracy (%)
Exit	1	2	3	4	5	Avg.	1	2	3	4	5	Avg.
Adv. w/o Distill [12]	25.22	27.34	28.94	28.06	28.48	27.61	53.68	55.46	58.18	57.46	57.38	56.43
SKD [24]	28.40	29.26	28.74	28.14	26.82	28.27	58.34	59.30	58.38	57.72	55.48	57.84
ARD [8]	24.48	26.76	27.78	28.46	29.14	27.32	53.44	56.10	57.20	57.32	57.08	56.23
LW [3]	22.34	23.12	23.58	22.76	23.30	23.02	47.56	48.16	49.86	48.64	49.72	48.79
NEO-KD (ours)	28.24	31.14	30.58	31.58	31.24	30.56	57.34	59.82	60.16	60.04	59.06	59.28
(b) Average attack
Table 4: Anytime prediction setup: Adversarial test accuracy on Tiny-ImageNet.
top-1 accuracy (%)	top-5 accuracy (%)
Exit	1	2	3	4	5	Avg.	1	2	3	4	5	Avg.
Adv. w/o Distill [12]	26.10	31.71	31.94	30.22	32.30	30.45	55.40	59.84	59.23	57.17	59.90	58.31
SKD [24]	27.34	31.44	29.63	27.03	26.71	28.43	56.96	59.94	57.49	53.98	55.24	56.72
ARD [8]	25.81	32.00	32.93	32.00	31.72	30.89	54.46	59.34	59.24	57.93	58.29	57.85
NEO-KD (ours)	27.89	32.61	32.99	32.74	35.63	32.37	55.46	59.64	59.79	59.61	62.46	59.39
(a) Max-average attack
top-1 accuracy (%)	top-5 accuracy (%)
Exit	1	2	3	4	5	Avg.	1	2	3	4	5	Avg.
Adv. w/o Distill [12]	18.20	24.21	28.24	28.73	28.50	25.58	44.74	52.57	56.82	57.55	57.14	53.77
SKD [24]	19.04	24.45	26.45	25.72	22.17	23.57	46.31	52.92	55.33	54.71	50.18	51.89
ARD [8]	17.56	23.68	27.98	28.52	25.83	24.71	43.76	52.07	56.42	57.30	53.48	52.60
NEO-KD (ours)	22.42	28.62	31.77	32.78	34.30	29.98	48.02	55.30	58.76	60.07	61.35	56.70
Table 5: Anytime prediction setup: Adversarial test accuracy on ImageNet.
settings and show that our method outperforms the baselines in both settings. More detailed settings for our inference scenarios are provided in Appendix.
4.2 Main Experimental Results
Result 1: Anytime prediction setup. Tables 1, 2, 3, 4, 5 compare the adversarial test accuracy of different schemes under max-average attack and average attack using MNIST, CIFAR-10/100, Tiny-ImageNet, and ImageNet, respectively. Note that we achieve adversarial accuracies between  $40\%$  -  $50\%$  for CIFAR-10, which is standard considering the prior works on robust multi-exit networks [12]. Our first observation from the results indicates that the performance of SKD [24] is generally lower than that of Adv. w/o Distill, whereas ARD [8] outperforms Adv. w/o Distill. This suggests that the naive application of self-knowledge distillation can either increase or decrease the adversarial robustness of multi-exit networks. Consequently, the method of knowledge distillation
Figure 2: Budgeted prediction setup: Adversarial test accuracy under average attack. The result for LW is excluded since the performance is too low and thus hinders the comparison between baselines and NEO-KD.
(a) CIFAR-10
Figure 3: Budgeted prediction setup: Adversarial test accuracy under max-average attack. The result for LW is excluded since the performance is too low and thus hinders the comparison between baselines and NEO-KD.
significantly influences the robustness of multi-exit networks (i.e., determining which knowledge to distill and which exit to target). To further enhance robustness, we investigate strategies for distilling high-quality knowledge and mitigating adversarial transferability.
By combining EOKD with NKD to mitigate dependency across submodels while guiding a multi-exit network to extract high quality features from adversarial examples as original data, NEO-KD achieves the highest adversarial test accuracy at most exits compared to the baselines for all datasets/attacks. The overall results confirm the advantage of NEO-KD for robust multi-exit networks.
Result 2: Budgeted prediction setup. Different from the anytime prediction setup where the pure performance of each exit is measured, in this setup, we adopt ensemble strategy at inference time where the predictions from the selected exit (according to the confidence threshold) and the previous exits are ensembled. From the results in anytime prediction setup, it is observed that various schemes tend to show low performance at the later exits compared to earlier exits in the model, where more details are discussed in Appendix. Therefore, this ensemble strategy can boost the performance of the later exits. With the ensemble scheme, given a fixed computation budget, we compare adversarial test accuracies of our method with the baselines.
Figures 2 and 3 show the results in budgeted prediction setup under average attack and max-average attack, respectively. NEO-KD achieves the best adversarial test accuracy against both average and max-average attacks in all budget setups. Our scheme also achieves the target accuracy with significantly smaller computing budget compared to the baselines. For example, to achieve  $41.21\%$  of accuracy against average attack using CIFAR-10 (which is the maximum accuracy of Adv. w/o Distill), the proposed NEO-KD needs 10.59 MFlops compared to Adv. w/o Distill that requires 20.46 MFlops, saving  $48.24\%$  of computing budget. Compared to ARD, our NEO-KD saves  $25.27\%$  of computation, while SKD and LW are unable to achieve this target accuracy. For CIFAR-100 and Tiny-ImageNet, NEO-KD saves  $81.60\%$  and  $50.27\%$  of computing budgets compared to Adv. w/o Distill. The overall results are consistent with the results in anytime prediction setup, confirming the advantage of our solution in practical settings with limited computing budget.
Result 3: Adversarial transferability. We also compare the adversarial transferability of our NEO-KD and different baselines among exits in a multi-exit neural network. When measuring adversarial transferability, as in [29], we initially gather all clean test samples for which all exits produce correct predictions. Subsequently, we generate adversarial examples targeting each exit using the collected clean samples (We use PGD-50 based single attack). Finally, we assess the adversarial transferability as the attack success rate of these adversarial examples at each exit. Figure 4 shows the adversarial transferability map of each scheme on CIFAR-100. Here, each row corresponds to the target exit for generating adversarial examples, and each column corresponds the exit where attack success rate is measured. For example, the  $(i,j)$ -th element in the map is adversarial transferability measured at exit
(a) Adv. w/o Distill [12]
(b) SKD [24]
(Avg. w/o Diag.: 23.68%)
w/o Diag.:  $33.36\%$
(c) NKD
(Avg. w/o Diag.: 22.76%)
Figure 4: Adversarial transferability map of each method on CIFAR-100. Diag. indicates the diagonal of the matrix. Row: Target exit for generating adversarial examples. Column: Exit where adversarial transferability is measured. Adopting NKD solely already achieves better adversarial transferability compared to the existing baselines. Applying EOKD to NKD can further improve adversarial transferability by reducing the dependency among different submodels in the multi-exit network.
(d) NEO-KD (ours)
(Avg. w/o Diag.: 20.12%)
Exit	1	2	3	Average
Adv. w/o Distill	38.00%	41.42%	40.70%	40.04%
NKD	40.01%	42.67%	40.23%	40.97%
EOKD	37.26%	41.10%	38.68%	39.01%
NEO-KD (ours)	41.67%	45.38%	45.54%	44.20%
Adversarial test accuracy against average attack on CIFAR-10. NKD and EOKD work in a complementary fashion and have implicit synergies.
Type of ensembles	1	2	3	Average
Adv. w/o Distill	38.00%	41.42%	40.70%	40.04%
No ensembling	39.54%	41.77%	40.15%	40.49%
Ensemble neighbors (NKD)	41.67%	45.38%	45.54%	44.20%
Ensemble all exits	38.63%	41.61%	39.93%	40.06%
Table 6: Effect of each component of NEO-KD:
Table 7: Results with different number of ensembles: Adversarial test accuracy against average attack on CIFAR-10. The neighbor-wise ensemble case corresponds to our NKD.
$j$ , generated by the adversarial examples targeting exit  $i$ . The values and the brightness in the map indicate success rates of attacks; lower value (dark color) means lower adversarial transferability.
We have the following key observations from adversarial transferability map. First, as observed in Figure 4a, compared to Adv. w/o Distill [12], SKD [24] in Figure 4b exhibits higher adversarial transferability. This indicates that distilling the same teacher prediction to every exit leads to a high dependency across exits. Thus, it is essential to consider distilling non-overlapping knowledge across different exits. Second, when compared to the baselines [12, 24], our proposed NKD in Figure 4c demonstrates low adversarial transferability. This can be attributed to the fact that NKD takes into account the quality of the distilled features and ensures that the features are not overlapping among exits. Third, as seen in Figure 4d, the adversarial transferability is further mitigated by incorporating EOKD, which distills orthogonal class predictions to different exits, into NKD. Comparing the average of attack success rate across all exits (excluding the values of the target exits shown in the diagonal), it becomes evident that NEO-KD yields  $3.56\%$  and  $13.24\%$  gains compared to Adv. w/o Distill and SKD, respectively. The overall results confirm the advantage of our solution to reduce the adversarial transferability in multi-exit networks. These results support the improved adversarial test accuracy of NEO-KD reported in Section 4.2.
4.3 Ablation Studies and Discussions
Effect of each component of NEO-KD. In Table 6, we observe the effects of our individual components, NKD and EOKD. It shows that combining NKD and EOKD boosts up the performance beyond the sum of their original gains. Given different roles, the combination of NKD and EOKD enables multi-entry networks to achieve the state-of-the-art performance under adversarial attacks.
Effect of the type of ensembles in NKD. In the proposed NKD, we consider only the neighbor exits to distill the knowledge of clean data. What if we consider fewer or more exits than neighboring exits? If the number of ensembles is too small, the scheme does not distill high-quality features. If the number of ensembles is too large, the dependencies among submodels increase, resulting in high adversarial transferability. To see this effect, in Table 7, we measure adversarial test accuracy of three types of ensembling methods depending on the number of exits used for constructing ensembles: no ensembling, ensemble neighbors (NKD), and ensemble all exits. In no enesbling approach, we distill the knowledge of each exit from clean data to the output at the same position of exit for adversarial examples. In contrast, the ensemble all exits scheme averages the knowledge of all exits from clean data and provides it to all exits of adversarial examples. The ensemble neighbors approach corresponds to our NKD. The results show that the proposed NEO-KD with neighbor ensembling
Attacks	Exit	1	2	3	4	5	6	7	Average
PGD-100	Adv. w/o Distill [12]	16.82%	17.24%	19.03%	19.41%	18.97%	17.90%	16.96%	18.05%
NEO-KD (ours)	20.38%	21.29%	23.22%	24.38%	24.44%	23.82%	22.21%	22.82%
CW	Adv. w/o Distill [12]	35.31%	35.72%	36.47%	36.85%	37.20%	37.19%	36.68%	36.49%
NEO-KD (ours)	31.56%	34.55%	38.20%	40.60%	43.48%	44.03%	43.01%	39.35%
AutoAttack	Adv. w/o Distill [12]	31.32%	34.24%	37.27%	39.67%	41.20%	41.73%	40.45%	37.98%
NEO-KD (ours)	31.56%	34.55%	38.20%	40.60%	43.48%	44.03%	43.01%	39.35%
Table 8: Results with stronger attacker algorithms: Adversarial test accuracy against average attack using CIFAR-100 dataset.
enables to distill high-quality features while lowering dependencies among submodels, confirming our intuition.
Robustness against stronger adversarial attack. We evaluate NEO-KD against stronger adversarial attacks; we perform average attack based on PGD-100 [21], Carlini and Wagner (CW) [2], and AutoAttack [5]. Table 8 shows that NEO-KD achieves higher adversarial test accuracy than Adv. w/o Distill [12] in most of cases. Typically, CW attack and AutoAttack are stronger attacks than the PGD attack in single-exit networks. However, in the context of multi-exit networks, these attacks become weaker than the PGD attack when taking all exits into account. Details for generating stronger adversarial attacks are described in Appendix.
Additional results. Other results including clean test accuracy, results with average attack based adversarial training, results with varying hyperparameters, and results with another baseline used in single-exit network, are provided in Appendix.
In this paper, we proposed a new knowledge distillation based adversarial training strategy for robust multi-exit networks. Our solution, NEO-KD, reduces adversarial transferability in the network while guiding the output of the adversarial examples to closely follow the ensemble outputs of the neighbor exits of the clean data, significantly improving the overall adversarial test accuracy. Extensive experimental results on both anytime and budgeted prediction setups using various datasets confirmed the effectiveness of our method, compared to baselines relying on existing adversarial training or knowledge distillation techniques for multi-exit networks.
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2019R1I1A2A02061135), by the Center for Applied Research in Artificial Intelligence (CARAI) grant funded by DAPA and ADD (UD230017TD), and by IITP funds from MSIT of Korea (No. 2020-0-00626).
[1] Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII, pages 484-501. Springer, 2020.
[2] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE symposium on security and privacy (sp), pages 39-57. IEEE, 2017.
[3] Sihong Chen, Haojing Shen, Ran Wang, and Xizhao Wang. Towards improving fast adversarial training in multi-exit network. Neural Networks, 150:1-11, 2022.
[4] Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference on Machine Learning, pages 2196-2205. PMLR, 2020.
[5] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pages 2206-2216. PMLR, 2020.
[23] Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP), pages 582-597. IEEE, 2016.
[24] Mary Phuong and Christoph H Lampert. Distillation-based training for multi-exit architectures. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1355-1364, 2019.
[25] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-252, 2015.
[26] Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd International Conference on Pattern Recognition (ICPR), pages 2464-2469. IEEE, 2016.
[27] Xinglu Wang and Yingming Li. Harmonized dense knowledge distillation training for multi-exit architectures. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10218–10226, 2021.
[28] Qunliang Xing, Mai Xu, Tianyi Li, and Zhenyu Guan. Early exit or not: Resource-efficient blind quality enhancement for compressed images. In European Conference on Computer Vision, pages 275-292. Springer, 2020.
[29] Huanrui Yang, Jingyang Zhang, Hongliang Dong, Nathan Inkawhich, Andrew Gardner, Andrew Touchet, Wesley Wilkes, Heath Berry, and Hai Li. Diverse: diversifying vulnerabilities for enhanced robust generation of ensembles. Advances in Neural Information Processing Systems, 33:5505-5515, 2020.
[30] Zhuolin Yang, Linyi Li, Xiaojun Xu, Shiliang Zuo, Qian Chen, Pan Zhou, Benjamin Rubinstein, Ce Zhang, and Bo Li. Trs: Transferability reduced ensemble via promoting gradient diversity and model smoothness. Advances in Neural Information Processing Systems, 34, 2021.
[31] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pages 7472-7482. PMLR, 2019.
[32] Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, and Furu Wei. Bert loses patience: Fast and robust inference with early exit. Advances in Neural Information Processing Systems, 33:18330-18341, 2020.
[33] Jianing Zhu, Jiangchao Yao, Bo Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, and Hongxia Yang. Reliable adversarial distillation with unreliable teachers. arXiv preprint arXiv:2106.04928, 2021.
[34] Bojia Zi, Shihao Zhao, Xingjun Ma, and Yu-Gang Jiang. Revisiting adversarial robustness distillation: Robust soft labels make student better. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16443-16452, 2021.
We provide additional implementation details that were not described in the main manuscript.
A.1 Model training
We train a SmallCNN [12] for 150 epochs with batch size 128 on MNIST [18], a MSDNet [13] for 150 epochs with batch size 128 on CIFAR-10/100 [16] and Tiny-IIImageNet [17]. For ImageNet [25], since it takes a long time to train from the beginning, we finetune the pretrained model with our NEO-KD loss function for 10 epochs. All experiments are implemented with two RTX3090 GPUs. Other settings follow  $[12]^2$  for SmallCNN and  $[13]^3$  for MSDNet except for a channel, which is set to 8 for our experiments on CIFAR-10/100. For the optimizer, SGD is used with a momentum of 0.9 and a weight decay of  $5\times 10^{-4}$ . For the MNIST dataset, the initial learning rate is set to 0.01 and the learning rate is decayed 10-fold at 50 epoch. For the CIFAR-10/100 dataset, the initial learning rate is 0.1, and is decayed 10-fold at 75-th epoch and 115-th epoch. For Tiny-IIImageNet, the initial learning rate is set to 0.1, and is decayed 10-fold at 50-th epoch and 100-th epoch. For ImageNet, the learning rate is constant with 0.001.
A.2 Adversarial training