SlowGuess/NeurIPS_2023_OCR huggingface dataset:
During adversarial training, we use max-average attack and average attack [12] for generating adversarial examples via PGD attacker algorithm [21] with 7-steps, while the PGD attacker algorithm with 50-step is used for measuring robustness against a stronger attack during test time. With PGD-50 attack, we measure adversarial test accuracy on 3 random seeds and average the results. For PGD attack, the perturbation degree  $\epsilon = 0.3$  is used for MNIST,  $\epsilon = 8 / 255$  is used for CIFAR-10/100, and  $\epsilon = 2 / 255$  Tiny-ImageNet/ImageNet during both adversarial training and inference time. The step size  $\delta$  is set to  $20 / 255$  for MNIST,  $2 / 255$  for CIFAR-10/100, and  $\frac{2}{3}\epsilon$  (0.0052) for Tiny-ImageNet/ImageNet. The number of iterations is commonly 50-steps. Similarly, when measuring adversarial transferability, we also use PGD attack with 50 steps,  $2 / 255$  step size, and  $8 / 255$  epsilon value to generate adversarial attacks. With PGD attack, the attack success rate is utilized as the metric to measure adversarial transferability. We perform each experiment on 5 random seeds and average the results.
Additionally, the hyperparameter  $\alpha$  for NKD is set to 3, and  $\beta$  for EOKD is set to 1 across all experiments. On the other hand, the exit-balancing parameter  $\gamma$  is set to [1, 1, 1] for MNIST and CIFAR-10, and [1, 1, 1, 1.5, 1.5], [1, 1, 1, 1.5, 1.5, 1.5, 1.5] for Tiny-ImageNet/ImageNet, CIFAR-100, respectively.
A.3 How to determine confidence threshold in budgeted prediction setup
We provide a detailed explanation about how to determine confidence threshold for each exit using validation set before the testing phase. First, in order to obtain confidence thresholds for various budget scenarios, we allocate the number of validation samples for each exit. For simplicity, consider a toy example with 3-exit network (i.e.,  $L = 3$ ) and assume the number of validation set is 3000. Then, each exit can be assigned a different number of samples: for instance, (2000, 500, 500), (1000, 1000, 1000) and (500, 1000, 1500). As more samples are allocated to the early exits, a scenario with a smaller budget can be obtained, while allocating more data to the later exits can lead to a scenario with a larger budget. More specifically, to see how to obtain the confidence threshold for each exit, consider the low-budget case of (2000, 500, 500). The model first makes predictions on all 3000 samples at exit 1 and sorts the samples based on their confidence. Then, the 2000-th largest confidence value is set as the confidence threshold for the exit 1. Likewise, the model performs predictions on remaining 1000 samples at exit 2 and the 500-th largest confidence is determined as the threshold for exit 2. Following this process, all thresholds for each exit are determined. During the testing phase, we perform predictions on test samples based on the predefined thresholds for each exit, and calculate the total computational budget for the combination of (2000, 500, 500). In this way, we can obtain accuracy and computational budget for different combinations of data numbers
$^{2}$ https://github.com/VITA-Group/triple-wins
<sup>3</sup>https://github.com/kalviny/MSDNet-PyTorch
Exit	1	2	3	Average
Adv. w/o Distill [12]	98.14%	99.36%	99.55%	99.02%
NEO-KD (ours)	97.82%	99.29%	99.48%	98.86%
(a) MNIST
top-1 accuracy (%)	top-5 accuracy (%)
Exit	1	2	3	4	5	Avg.	1	2	3	4	5	Avg.
Adv. w/o Distill [12]	43.10	47.64	50.36	50.58	50.70	48.48	68.26	72.30	74.86	75.58	75.36	73.27
NEO-KD (ours)	45.56	48.32	49.88	50.56	50.52	48.97	70.56	73.12	74.52	74.22	74.98	73.48
(b) Tiny-ImageNet
Table A1: Clean test accuracy in the anytime prediction setup: NEO-KD's advantage in terms of adversarial test accuracy (as shown in the main manuscript) can be achieved without largely compromising the clean test accuracy.
top-1 accuracy (%)	top-5 accuracy (%)
Exit	1	2	3	4	5	6	7	Avg.	1	2	3	4	5	6	7	Avg.
Adv. w/o Distill [12]	21.14	20.49	20.68	20.18	20.58	20.56	20.24	20.55	48.28	48.24	46.75	46.41	46.31	46.96	45.64	46.94
SKD [24]	22.39	22.17	19.89	18.81	18.82	17.48	14.84	19.20	51.73	51.43	47.88	45.92	45.90	43.65	41.38	46.84
ARD [8]	20.55	19.84	20.95	19.61	19.41	20.52	21.72	20.37	48.18	46.94	46.96	45.37	45.06	46.17	48.42	46.73
LW [3]	17.65	18.80	16.97	16.64	17.15	17.08	16.79	17.30	44.33	44.03	41.49	41.48	41.37	42.06	41.64	42.34
NEO-KD (ours)	21.31	22.01	21.46	22.10	21.89	21.64	20.78	21.60	49.16	48.88	49.12	48.76	48.00	48.08	47.23	48.46
(a) Max-average attack
top-1 accuracy (%)	top-5 accuracy (%)
Exit	1	2	3	4	5	6	7	Avg.	1	2	3	4	5	6	7	Avg.
Adv. w/o Distill [12]	12.73	12.10	13.33	13.27	13.14	13.17	13.27	13.00	36.22	35.66	37.07	36.57	36.47	37.26	36.51	36.54
SKD [24]	15.96	15.90	15.31	15.32	15.70	14.84	10.33	14.77	42.19	41.72	41.29	41.46	42.03	40.08	32.68	40.21
ARD [8]	12.74	12.53	13.80	13.04	12.79	13.56	14.73	13.31	36.12	35.67	37.21	36.28	35.81	37.09	38.82	36.71
LW [3]	12.82	13.33	12.28	11.96	12.10	12.25	12.55	12.47	36.49	36.23	34.46	34.17	34.09	34.78	34.72	34.99
NEO-KD (ours)	14.64	15.15	15.42	16.07	16.42	16.11	15.59	15.63	38.88	39.56	40.60	40.57	41.13	40.91	39.85	40.21
Table A2: Adversarial training via Average attack: Adversarial test accuracy in the anytime prediction setup on CIFAR-100.
(i.e., various budget scenarios). Figures 2 and 3 in the main manuscript show the results for 100 cases of different budget scenarios.
B Clean Test Accuracy
Table A1 shows the clean accuracy results using the model built upon adversarial training via max-average attack. We observe that NEO-KD generally shows comparable clean test accuracy with Adv. w/o Distill [12], especially on the more complicated dataset Tiny-ImageNet [17] while achieving much better adversarial test accuracy as reported in the main manuscript.
C Adversarial Training via Average Attack
In the main manuscript, we presented experimental results using the model trained based on max-average attack. Here, we also adversarially train the model via average attack [12] and measure adversarial test accuracy on CIFAR-100 dataset. Table A2 compares adversarial test accuracies of NEO-KD and other baselines against max-average attack and average attack. The overall results are consistent with the ones in the main manuscript with adversarial training via max-average attack, further confirming the advantage of NEO-KD.
In the NEO-KD objective function, there are three hyperparameters  $(\alpha, \beta, \gamma)$ , where  $\alpha, \beta$  control the amount of distilling knowledge from NKD, EOKD and  $\gamma$  increases the amount of knowledge distilled to later exits.
D.1 Hyperparameter  $(\alpha, \beta)$
The extreme value of  $\alpha$  and  $\beta$  can destroy ideal adversarial training. Too large  $\alpha$  makes strong NKD, which results in high dependency among submodels and too small  $\alpha$  makes weak NKD, which cannot distill enough knowledge to student exits. In contrast, too large  $\beta$  makes strong EOKD, which can interrupt adversarial training by distilling only sparse knowledge (likelihoods of majority classes are zero) and too small  $\beta$  makes weak EOKD, which cannot mitigate dependency among submodels. We select  $\alpha$ ,  $\beta$  values in the range of [0.35, 3] and measure the adversarial test accuracy value by averaging adversarial test accuracy from all exits. The candidate  $(\alpha, \beta)$  pairs are (0.35, 1), (1, 0.35), (0.35, 0.35), (0.5, 1), (1, 0.5), (0.5, 0.5), (1, 1), (2, 1), (1, 2), (2, 2), (3, 1), (1, 3), and (3, 3). When  $(\alpha, \beta)$  is (3, 1), NEO-KD achieves  $28.96\%$  of adversarial test accuracy against max-average attack and  $22.88\%$  against average attack, which is the highest adversarial test accuracy among the various candidate  $(\alpha, \beta)$  pairs. Therefore, we use (3, 1) as  $(\alpha, \beta)$  pair in our experiments.
D.2 Hyperparameter  $\gamma$
Since the prediction difference between the last exit (teacher prediction) and later exits is smaller than the prediction difference between the last exit and early exits, later exits are less effective for taking advantage of knowledge distillation. Therefore, we provide slightly larger weights to later exits for distilling more knowledge to later exits than early exits. The candidate  $\gamma$  values are  $[1, 1, 1, 1, 1, 1, 1]$ ,  $[1, 1, 1, 1.5, 1.5, 1.5, 1.5]$ , and  $[1, 1, 1, 1.7, 1.7, 1.7, 1.7]$ . As a result, when we distill 1.5 times more knowledge to later exits, NEO-KD achieves  $28.96\%$  of adversarial test accuracy against max-average attack and  $22.88\%$  against average attack, which is the highest adversarial test accuracy compared to providing same weights with earlier exits to later exits ( $28.13\%$  for max-average and  $21.66\%$  for average attack) or distilling 1.7 times more knowledge to later exits than earlier exits ( $28.68\%$  for max-average and  $22.58\%$  for average attack). The adversarial test accuracy value is the average of adversarial test accuracies from all exits. Therefore, we use  $\gamma = [1, 1, 1, 1.5, 1.5, 1.5, 1.5]$  in our experiments. This result proves that the exit-balancing parameter  $\gamma$  with an appropriate value is needed for high performance.
E Discussions on Performance Degradation at Later Exits
As can be seen from the results for the anytime prediction in the main manuscript, the adversarial test accuracy of the later exits is sometimes lower than the performance of earlier exits. This phenomenon can be explained as follows: In general, we observed via experiments that adversarial examples targeting later exits has the higher sum of losses from all exits compared to adversarial examples targeting earlier exits. This makes max-average or average attack mainly focus on attacking the later exits, leading to low adversarial test accuracy at later exits. The performance of later exits can be improved by adopting the ensemble strategy as in the main manuscript for the budgeted prediction setup.
F Comparison with Recent Defense Methods for Single-Exit Networks
The baselines in the main paper were generally the adversarial defense methods designed for multi-exit networks. In this section, we conduct additional experiments with a recent defense method, TEAT [7], and compare with our method. Since TEAT was originally designed for the single-exit network, we first adapt TEAT to the multi-exit network setting. Instead of the original TEAT that generates the adversarial examples considering the final output of the network, we modify TEAT to generate adversarial examples that maximizes the average loss of all exits in the multi-exit network. Table A3 below shows the results using max-average attack on CIFAR-10/100. It can be seen that our NEO-KD, which is designed for multi-exit networks, achieves higher adversarial test accuracy compared to the TEAT methods (PGD-TE and TRADES-TE) designed for single-exit networks. The results highlight the necessity of developing adversarial defense techniques geared to multi-exit networks rather than adapting general defense methods used for single-exit networks.
Exit	1	2	3	Average
PGD-TE [7]	48.73%	46.00%	46.85%	47.19%
TRADES-TE [7]	45.05%	39.64%	42.10%	42.26%
NEO-KD (ours)	46.53%	47.65%	50.71%	48.30%
(a) CIFAR-10
Exit	1	2	3	4	5	6	7	Avg.
PGD-TE [7]	24.07%	24.39%	25.14%	25.35%	26.29%	25.57%	24.60%	25.06%
TRADE-TE [7]	17.62%	18.52%	18.61%	18.98%	18.95%	19.67%	20.35%	18.96%
NEO-KD (ours)	28.37%	28.78%	29.02%	29.49%	30.06%	28.45%	28.54%	28.96%
(b) CIFAR-100
Table A3: Comparison of adversarial test accuracy against max-average attack between TEAT methods and our NEO-KD.
Exit	1	2	3	Avg.
SKD (exit 1)	32.27%	36.92%	38.57%	35.92%
SKD (exit 2)	35.33%	35.10%	37.82%	36.08%
SKD (exit 3)	39.36%	41.39%	38.39%	39.71%
SKD (ensemble)	38.63%	41.80%	40.13%	40.19%
ARD (exit 1)	35.64%	38.10%	42.12%	38.62%
ARD (exit 2)	35.35%	38.24%	40.00%	37.86%
ARD (exit 3)	39.37%	41.98%	43.53%	41.63%
ARD (ensemble)	35.22%	38.35%	40.76%	38.11%
NEO-KD (ours)	41.67%	45.38%	45.54%	44.20%
Table A4: Adversarial test accuracy of SKD and ARD according to exit selection as a teacher prediction.
G Comparison with SKD and ARD
Existing self-distillation schemes [20, 24] for multi-exit networks improve the performance on clean samples by self-distilling the knowledge of the last exit, as the last exit has the best prediction quality. Therefore, following the original philosophy, we also used the last exit in implementing the SKD baseline. Regarding ARD [8], since it was proposed for single-exit networks, we also utilized the last exit with high performance when applying ARD to multi-exit networks. Nevertheless, we perform additional experiments to consider comprehensive baselines using various exits for distillation. Table A4 above shows the results of SKD and ARD using a specific exit or an ensemble of all exits for distillation. The results show that our scheme consistently outperforms all baselines.
H Implementations of Stronger Attacker Algorithms
In Section 4.3 of the main manuscript, during inference, we replaced the Projected Gradient Descent (PGD) attack with other attacker algorithms (PGD-100 attack, Carlini and Wagner (CW) attack [2], and AutoAttack [5]) to generate stronger attacks for multi-exit neural networks. This section provides explanation on how these stronger attacks are implemented tailored to multi-exit neural networks.
H.1 Carlini and Wagner (CW) attack
The Carlini and Wagner (CW) attack is a method of generating adversarial examples designed to reduce the difference between the logits of the correct label and the largest logits among incorrect labels. In alignment with this attack strategy, we modify the CW attack for multi-exit neural networks. In the process of minimizing this difference, our modification aims to minimize the average difference across all exits of the multi-exit neural network. Moreover, when deciding whether a sample has been successfully converted into an adversarial example, we consider a sample adversarial if it misleads all exits in the multi-exit neural network.
H.2 AutoAttack
AutoAttack produces adversarial attacks by assembling various attacker algorithms. For our experiment, we sequentially use APGD [5], APGD-T [5], FAB [4], and Square [1] algorithms to generate adversarial attacks, as they are commonly used.
H.2.1 APGD and APGD-T
The APGD attack is a modified version of the PGD attack, which is limited by its fixed step size, a suboptimal choice. The APGD attack overcomes this limitation by introducing an adaptive step size and a momentum term. Similarly, the APGD-T attack is a variation of the APGD attack where the attack perturbs a sample to change to a specific class. In this process, we use the average loss of all exits in the multi-exit neural network as the loss for computing the gradient for adversarial example updates. Moreover, we define a sample as adversarial if it misleads all exits in the multi-exit neural network.
H.2.2 FAB
The FAB attack creates adversarial attacks through a process involving linear approximation of classifiers, projection to the classifier hyperplane, convex combinations, and extrapolation. The FAB attack first defines a hyperplane classifier separating two classes, then finds a new adversarial example through a convex combination of the extrapolation-projected current adversarial example and the extrapolation-projected original sample with the minimum perturbation norm. Here, we use the average gradient of all exits in the multi-exit neural network as the gradient for updating adversarial examples. Similar to above, we label a sample as adversarial if it can mislead all exits in the multi-exit neural network.
H.2.3 Square
The Square attack generates adversarial attacks via random searches of adversarial patches with variable degrees of perturbation and position. The Square attack algorithm iteratively samples the perturbation degree and the positions of patches while reducing the size of patches. The sampled adversarial patches are added to a sample, and the patches that maximizes the loss of the target model are selected. Here, we use the average loss of all exits in the multi-exit neural network as the loss for determining whether a sampled perturbation increases or decreases the loss of the target model. Additionally, we determine a sample as adversarial if it misleads all exits in the multi-exit neural network.
H.3 Experiment details
For all attacks, we commonly use  $\epsilon = 0.03$  as the perturbation degree and generate adversarial examples over 50 steps. All the attacks are based on  $L_{\infty}$  norm. For the APGD attack, we employ cross entropy loss for computing the gradient to update adversarial examples. In both the APGD-T and FAB attacks, all class pairs are considered when generating adversarial attacks. For the Square attack, the random search operation is conducted 5000 times (the number of queries). Other settings follow [14]. In terms of performance comparison against stronger attacker algorithms, we adopt adversarial training via average attack in both Adv. w/o Distill [12] and NEO-KD (our approach). However, since the original CW attack algorithm and AutoAttack algorithm were designed for single-exit neural networks, this adapted versions targeting multi-exit neural networks are relatively weak.