SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we provide a comprehensive evaluation to verify the effectiveness of AAER, including experiment settings (Section 4.1), performance evaluation (Section 4.2), ablation studies (Section 4.3) and time complexity study (Section 4.4).
Baselines. We compare our method with other SSAT methods, including RS-FGSM [39], FreeAT [30], N-FGSM [6], Grad Align [1], ZeroGrad and MultiGrad [10]. We also compare our method with multi-step AT, PGD-2 and PGD-10 (PGD-20 with 32/255 noise magnitude) [26], providing a reference for the ideal performance. The results of other competing baselines, including GAT [33], NuAT [34], PGI-FGSM [17], SDI-FGSM [18] and Kim [21], can be found in Appendix F. We report both the natural and robust accuracy results of the final model, which are obtained without early stopping and using the hyperparameters provided in the official repository. Please note that for FreeAT, we did not use the subset of training samples to keep the same training epochs across different methods.
Attack Methods. To report the robust accuracy of models, we attack these methods using the standard PGD adversarial attack with  $\alpha = \epsilon /4$  step size, 50 attack steps and 10 restarts. We also evaluate our methods on Auto Attack [5] as shown in Appendix C.
Datasets and Model Architectures. We evaluate our method on several benchmark datasets, including Cifar-10/100 [22], SVHN [28], Tiny-ImageNet [28] andImagenet-100 [7]. The standard data augmentation random cropping and horizontal flipping are applied for these datasets. The settings and results on SVHN, Tiny-ImageNet andImagenet-100 are provided in Appendix E. We use the PreactResNet-18 [14] and WideResNet-34 [43] architectures on these datasets to evaluate results. The results of WideResNet-34 can be found in Appendix D.
Setup for Our Proposed Method. In this work, we use the SGD optimizer with a momentum of 0.9, weight decay of  $5 \times 10^{-4}$  and  $L_{\infty}$  as the threat model. For the learning rate schedule, we use the cyclical learning rate schedule [32] with 30 epochs, which reaches its maximum learning rate
Table 1. The hyperparameter settings for Cifar-10/100 are divided by a slash. The top number represents  $\lambda_{2}$  while the bottom number represents  $\lambda_{3}$ . Throughout all settings,  $\lambda_{1}$  is fixed at 1.0.
CIFAR10/100	RS-AAER	N-AAER
8/255	12/255	16/255	32/355	8/255	12/255	16/255	32/355
λ2	2.5 / 3.5	5.0 / 3.5	7.0 / 6.0	5.75 / 5.0	1.5 / 1.5	5.0 / 3.5	8.5 / 6.0	2.75 / 3.5
λ3	1.5 / 1.5	2.75 / 2.75	3.25 / 2.25	1.5 / 0.75	0.15 / 0.15	0.55 / 0.3	1.5 / 0.5	0.75 / 0.5
Table 2. CIFAR10/100: Accuracy of different methods and different noise magnitudes using PreActResNet-18 under  $L_{\infty}$  threat model. The top number is the natural accuracy (\%), while the bottom number is the PGD-50-10 accuracy (\%). The results are averaged over 3 random seeds and reported with the standard deviation.
dataset	CIFAR10	CIFAR100
noise magnitude	8/255	12/255	16/255	32/255	8/255	12/255	16/255	32/255
FreeAT	76.20 ± 1.09	68.07 ± 0.38	45.84 ± 19.07	61.11 ± 8.41	47.41 ± 0.30	39.84 ± 0.40	3.32 ± 2.48	26.2 ± 15.54
43.74 ± 0.41	33.14 ± 0.62	0.00 ± 0.00	0.00 ± 0.00	22.27 ± 0.33	16.57 ± 0.20	0.00 ± 0.00	0.00 ± 0.00
ZeroGrad	81.60 ± 0.16	77.52 ± 0.21	79.65 ± 0.17	65.48 ± 6.26	53.83 ± 0.22	49.07 ± 0.14	50.76 ± 0.02	49.38 ± 1.39
47.56 ± 0.16	27.34 ± 0.09	6.37 ± 0.23	0.00 ± 0.00	25.02 ± 0.24	14.76 ± 0.26	5.23 ± 0.09	0.00 ± 0.00
MultiGrad	81.65 ± 0.16	81.09 ± 4.67	82.98 ± 3.30	70.84 ± 4.53	53.11 ± 0.34	46.81 ± 0.51	46.05 ± 8.68	28.33 ± 6.48
47.93 ± 0.18	9.95 ± 16.97	0.00 ± 0.00	0.00 ± 0.00	25.68 ± 0.21	16.56 ± 0.56	0.00 ± 0.00	0.00 ± 0.00
Grad Align	82.10 ± 0.78	74.17 ± 0.55	60.37 ± 0.95	25.23 ± 3.41	54.00 ± 0.44	45.83 ± 0.72	36.80 ± 0.10	15.05 ± 0.07
47.77 ± 0.58	34.87 ± 1.00	27.90 ± 1.01	11.53 ± 3.23	25.27 ± 0.68	18.13 ± 0.71	13.77 ± 0.76	2.85 ± 1.34
RS-FGSM	83.91 ± 0.21	66.46 ± 22.80	66.54 ± 12.25	36.43 ± 7.86	60.29 ± 1.51	18.19 ± 8.51	11.03 ± 5.24	11.40 ± 8.60
46.01 ± 0.18	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00	10.58 ± 13.10	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
N-FGSM	80.48 ± 0.21	71.30 ± 0.12	62.96 ± 0.74	29.79 ± 3.87	54.92 ± 0.28	46.16 ± 0.13	37.93 ± 0.22	18.18 ± 4.55
47.91 ± 0.29	36.23 ± 0.10	27.14 ± 1.44	8.30 ± 7.85	26.29 ± 0.41	18.75 ± 0.19	14.05 ± 0.07	0.00 ± 0.00
RS-AAER	83.83 ± 0.27	74.40 ± 0.79	64.56 ± 1.45	31.58 ± 1.13	57.71 ± 0.29	44.06 ± 0.93	33.10 ± 0.05	18.50 ± 1.68
46.14 ± 0.02	32.17 ± 0.16	23.87 ± 0.36	10.62 ± 0.51	25.31 ± 0.01	16.41 ± 0.13	11.80 ± 0.17	4.90 ± 0.50
N-AAER	80.56 ± 0.35	71.15 ± 0.18	61.84 ± 0.43	27.08 ± 0.02	54.47 ± 0.45	45.98 ± 0.13	36.80 ± 0.14	16.95 ± 0.44
48.31 ± 0.23	36.52 ± 0.10	28.20 ± 0.71	12.97 ± 0.57	26.81 ± 0.13	19.03 ± 0.04	14.31 ± 0.05	5.45 ± 0.14
PGD-2	85.07 ± 0.12	78.97 ± 0.23	72.31 ± 0.40	48.45 ± 0.71	60.09 ± 0.20	53.46 ± 0.27	47.50 ± 0.28	31.89 ± 0.69
45.27 ± 0.07	32.99 ± 0.46	24.32 ± 0.64	11.24 ± 0.40	24.58 ± 0.12	17.16 ± 0.21	12.69 ± 0.06	4.51 ± 0.21
PGD-10 (20)	80.55 ± 0.37	72.37 ± 0.31	67.20 ± 0.69	34.70 ± 0.67	55.05 ± 0.25	47.42 ± 0.29	42.39 ± 0.17	21.68 ± 0.18
50.67 ± 0.40	38.60 ± 0.39	29.34 ± 0.18	16.10 ± 0.20	27.87 ± 0.12	20.29 ± 0.18	15.01 ± 0.21	7.39 ± 0.38
(0.2) when half of the epochs (15) are passed. The results obtained by the long training schedule can be found in Appendix G. We superimpose our method on two baseline methods: RS-FGSM and N-FGSM, both of which use the vanilla min-max process. For RS-AAER, we follow the settings of [39] that set step size  $\alpha = 1.25\cdot \epsilon$  and random initialization  $\eta = \mathrm{Uniform}(-\epsilon ,\epsilon)$ . In accordance with the N-FGSM setting suggested by [6], we set  $\alpha = 1.0\cdot \epsilon$  and  $\eta = \mathrm{Uniform}(-2\cdot \epsilon ,2\cdot \epsilon)$  for N-AAER. The hyperparameter settings for Cifar-10/100 are summarized in the Table 1.
CIFAR10 Results. In Table 2 (left), we present a comparison of our proposed methods with the competing baselines. First, we can observe that CO occurs in all baseline methods except for Grad Align. However, Grad Align requires double backward propagation which notably reduces its efficiency. Compared to them, RS-AAER and N-AAER can effectively eliminate CO with all noise magnitudes and only incur negligible computational overhead. It is worth noting that our primary objective is to eliminate CO in SSAT, thus AAER will show significant performance enhancements under CO scenarios, exemplified by RS-AAER under 12, 16 and 32 noise magnitude, and N-AAER under 32 noise magnitude. Even without the CO scenario, our approach can consistently outperform the baselines under all experimental conditions. Additionally, we implemented Vanilla-AAER in Appendix A to further demonstrate the effectiveness of our method. For a fair comparison, we also
Figure 5. The number, the variation of prediction confidence and logits distribution (from left to right) for NAEs, AAEs and training samples in RS-AAER with 16/255 noise magnitude.
attempt to use the N-FGSM augmentation on Grad Align. However, we could not find suitable hyperparameter settings for Grad Align with the original step size. After reducing the step size to  $\alpha = 1.0\cdot \epsilon$  (consistent with N-AAER), the performance of Grad Align is similar to that reported in Table 2 (left). Besides, AAEs are not a unique phenomenon in SSAT, we also found them in the multi-step AT, indicating the existence of non-smooth points in their classifiers. However, we could not find competitive hyperparameter settings to enhance the robustness of multi-step AT using AAER.
CIFAR100 Results. We also conduct experiments on the CIFAR100 dataset, and the results are summarized in Table 2 (right). It is worth noting that CIFAR100 is more challenging as the number of classes/training images per class is ten times larger/smaller than that of CIFAR10. However, our proposed methods demonstrate their effectiveness in preventing CO and improving robust accuracy on CIFAR-100 as well. These results further validate that AAER is capable of reliably preventing CO and effectively improving robustness across different datasets.
In this part, we investigate the impacts of RS-AAER components with 16/255 noise magnitude using PreactResNet-18 on CIFAR10 under  $L_{\infty}$  threat model.
Figure 6. The role of  $\alpha$ .
**Optimization Objectives.** To validate the effectiveness of our proposed method, we illustrate the variation in test accuracy and three optimization objectives during the training process. Figure 1 demonstrates that our approach can successfully prevent CO and continuously improves robustness throughout the training. Additionally, from Figure 5, we can observe that the number, the variation of prediction confidence and logits distribution for AAEs are effectively constrained during the training. Specifically, when CO occurs in RS-FGSM (9th epoch), these three optimization objectives in RS-AAER are 29, 14, and 24 times smaller than it, respectively.
The Role of  $\alpha$ . From Figure 6, we can observe that enlarging the  $\alpha$  size will lead to an increase in the robust accuracy, and correspondingly a decrease in the natural accuracy. In light of this trade-off, we follow the baseline setting and do not adjust the  $\alpha$  size.
The impact of Regularization Term. We further investigate the interaction between the three parts of our regularization term as shown in Table 3. We find that neither part  $(i)$  number nor part  $(ii)$  prediction confidence can independently prevent CO, and only using part
Table 3. The impact of regularization term.
(i)	(ii)	(iii)	Natural Acc (%)	Robust Acc (%)
✓	75.14	0.00
✓	77.22	0.00
✓	13.61	9.77
✓	✓	76.19	0.00
✓	✓	56.65	23.29
✓	✓	16.68	12.56
✓	✓	✓	64.56	23.87
Table 4. CIFAR10 training time on a single NVIDIA RTX 4090 GPU using PreactResNet-18. The results are averaged over 30 epochs.
Method	FreeAT	ZeroGrad	MultiGrad	Grad Align	RS/N-FGSM	RS/N-AAER	PGD-2	PGD-10
Training Time (S)	43.8	11.0	21.7	36.1	11.0	11.2	16.4	59.1
(iii) logits distribution can partially mitigate CO. However, solely relying on part (iii) cannot accurately reflect the degree of classifier distortion as it lacks a comprehensive measure, resulting in poor performance. Additionally, part (i) also plays an important role in performance, without it, both natural and robust accuracy significantly drop. Meanwhile, part (ii) contributes to the stability and natural accuracy of the method. Therefore, to effectively and stably eliminate CO, all parts of the regularization term are necessary and critical. Further ablation studies on other regularization methods and  $\lambda$  selection can be found in Appendix B.
4.4 Time Complexity Study
Efficiency is a key advantage of SSAT over multi-step AT, as it can be readily scaled to large networks and datasets. Consequently, the computational overhead plays an important role in the SSAT overall performance. In Table 4, we present a time complexity comparison among various SSAT methods. It can be seen that AAER only imposes a minor training cost of 0.2 seconds, representing a mere  $1.8\%$  increase compared to FGSM. In contrast, Grad Align and PGD-10 are 3.2 and 5.3 times slower than our method.
In this paper, we find that the abnormal adversarial examples exhibit anomalous behaviour, i.e. they are further to the decision boundaries after adding perturbations generated by the inner maximization process. We empirically show the abnormal adversarial examples are closely related to the classifier distortion and catastrophic overfitting, by analyzing their number and outputs variation during the training process. Motivated by this, we propose a novel and effective method, Abnormal Adversarial Examples Regularization (AAER), through a regularizer to eliminate catastrophic overfitting by suppressing the generation of abnormal adversarial examples. Our approach can successfully resolve the catastrophic overfitting with different noise magnitudes and further boost adversarial robustness with negligible additional computational overhead.
Tongliang Liu is partially supported by the following Australian Research Council projects: FT220100318, DP220102121, LP220100527, LP220200949, and IC190100031.
[41] Chaojian Yu, Bo Han, Mingming Gong, Li Shen, Shiming Ge, Bo Du, and Tongliang Liu. Robust weight perturbation for adversarial training. arXiv preprint arXiv:2205.14826, 2022.
[42] Chaojian Yu, Bo Han, Li Shen, Jun Yu, Chen Gong, Mingming Gong, and Tongliang Liu. Understanding robust overfitting of adversarial training and beyond. In International Conference on Machine Learning, pages 25595-25610. PMLR, 2022.
[43] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016.
[44] Bohang Zhang, Du Jiang, Di He, and Liwei Wang. Rethinking lipschitz neural networks and certified robustness: A boolean function perspective. Advances in Neural Information Processing Systems, 35:19398-19413, 2022.
[45] Chaoning Zhang, Kang Zhang, Axi Niu, Chenshuang Zhang, Jiu Feng, Chang D Yoo, and In So Kweon. Noise augmentation is all you need for fgsm fast adversarial training: Catastrophic overfitting and robust overfitting require different augmentation. arXiv preprint arXiv:2202.05488, 2022.
[46] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pages 7472-7482. PMLR, 2019.
[47] Dawei Zhou, Nannan Wang, Xinbo Gao, Bo Han, Xiaoyu Wang, Yibing Zhan, and Tongliang Liu. Improving adversarial robustness via mutual information estimation. In International Conference on Machine Learning, pages 27338-27352. PMLR, 2022.
[48] Dawei Zhou, Nannan Wang, Heng Yang, Xinbo Gao, and Tongliang Liu. Phase-aware adversarial defense for improving adversarial robustness. 2023.
A Vanilla-AAER
To further validate the effectiveness of our method, we implement Vanilla-AAER to prevent CO. The Vanilla-AAER method follows the settings of Vanilla-FGSM [11], which does not use random initialization and sets the step size as  $\alpha = 1.0\cdot \epsilon$ . The Vanilla-AAER hyperparameters setting is shown in Table 5.
Table 5. The hyperparameters setting for different noise magnitudes. The top number is  $\lambda_{2}$  while the bottom number is  $\lambda_{3}$ . The  $\lambda_{1}$  is fixed as 1.0 in all settings.
Vanilla-AAER	8/255	12/255	16/255	32/255
λ2	5.5	6.5	7.0	4.8
λ3	2.0	3.5	3.5	0.7
Table 6. CIFAR10: Accuracy of Vanilla-FGSM and Vanilla-AAER with different noise magnitude using PreActResNet-18 under  $L_{\infty}$  threat model. The top number is the natural accuracy (\%), while the bottom number is the PGD-50-10 accuracy (\%).
noise magnitude	8/255	12/255	16/255	32/255
Vanilla-FGSM	84.16 ± 4.68	79.86 ± 2.05	72.51 ± 3.79	64.29 ± 3.83
0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
Vanilla-AAER	80.45 ± 0.25	64.97 ± 3.14	51.92 ± 2.90	18.78 ± 2.45
46.66 ± 0.74	32.44 ± 1.18	24.12 ± 0.76	12.19 ± 0.40
Based on the results presented in Table 6, we can observe that Vanilla-AAER achieves comparable or even superior robustness compared to RS-AAER. This outcome may be attributed to the fact that Vanilla-FGSM has a higher expectation of adversarial perturbation, as demonstrated in prior work [1]. However, the absence of random initialization in Vanilla-AAER may reduce the diversity of adversarial examples, potentially impacting the natural accuracy of the model. Nonetheless, the most significant finding is that Vanilla-AAER effectively eliminates CO across various noise magnitudes, which cannot be accomplished by Vanilla-FGSM.
B Impacts of Regularization Term
To showcase the distinct effectiveness of our method in eliminating CO, we conducted a comparison with other regularization methods used in multi-step AT, such as TRADES [46] and ALP [19]. In order to ensure a fair comparison, we set the iteration times for TRADES and ALP as 1, and the step size as  $\alpha = 1.25\cdot \epsilon$  and  $\alpha = 1.0\cdot \epsilon$  for superimposing RS-FGSM and N-FGSM, respectively.
From Table 7, we can observe that TRADES and ALP methods may improve adversarial robustness when CO is not present in the baseline methods. However, these methods are not effective in eliminating CO. As demonstrated by the RS-TRADES and RS-ALP methods, CO still occurs with larger noise magnitudes, similar to RS-FGSM. Furthermore, these methods can even harm the baseline method, as they break the N-FGSM robustness with 32/255 noise magnitude. Therefore, we conclude that the TRADES and ALP methods are not suitable for eliminating CO. Additionally, other multi-step and robust overfitting methods also prove ineffective against CO. Hence, CO has been identified as an independent phenomenon requiring distinct solutions. In contrast to these regularization methods, AAER explicitly reflects and prevents distortion of the classifier from the perspective of AAEs, which is the key factor enabling AAER to effectively eliminate CO.
We further investigate the impact of hyperparameters  $\lambda_{1}$ ,  $\lambda_{2}$ , and  $\lambda_{3}$  on the performance of AAER. Figure 7 (left) shows the effect of  $\lambda_{1}$  on the performance. It can be observed that when  $\lambda_{1}$  is small, AAER is unable to effectively suppress CO, and increasing  $\lambda_{1}$  improves both natural and robust accuracy. However, when  $\lambda_{1}$  increases from 1.0 to 2.0, the robust accuracy remains unchanged while the natural accuracy decreases. Therefore, we choose  $\lambda_{1} = 1$  to balance both natural and robust
Table 7. CIFAR10: Accuracy of TRADES and ALP methods with different noise magnitude using PreActResNet-18 under  $L_{\infty}$  threat model. The top number is the natural accuracy (\%), while the bottom number is the PGD-50-10 accuracy (\%).
noise magnitude	8/255	12/255	16/255	32/255
RS-TRADES (β = 1.0)	89.03	91.41	92.11	90.95
35.56	0.86	0.00	0.00
RS-TRADES (β = 6.0)	90.72	92.19	91.50	88.69
11.29	0.03	0.01	0.00
RS-ALP (logit pairing weight=0.5)	86.75	92.18	91.14	81.03
43.96	0.04	0.00	0.00
RS-ALP (logit pairing weight=1.0)	85.15	92.14	90.48	81.03
46.59	0.02	0.00	0.00
N-TRADES (β = 1.0)	86.82	81.12	85.62	81.81
41.24	16.40	0.12	0.00
N-TRADES (β = 6.0)	83.23	74.77	68.16	85.97
49.56	35.98	26.13	0.24
N-ALP (logit pairing weight=0.5)	84.63	81.57	72.05	79.41
46.03	27.85	0.32	0.00
N-ALP (logit pairing weight=1.0)	82.60	76.84	69.32	82.98
48.32	33.36	23.46	0.00
Figure 7. The role of  $\lambda_{1}$ ,  $\lambda_{2}$  and  $\lambda_{3}$  under 16/255 noise magnitude (from left to right).
accuracy. Figure 7 (middle) demonstrates the impact of  $\lambda_{2}$ . It can be observed that when  $\lambda_{2}$  is small, increasing  $\lambda_{2}$  is beneficial for both natural and robust accuracy. However, when  $\lambda_{2}$  increases from 7 to 14, the robust accuracy becomes flat while the natural accuracy decreases. Therefore, we select  $\lambda_{2} = 7$  considering both natural and robust accuracy. Figure 7 (right) shows the effect of  $\lambda_{3}$ . It can be observed that when  $\lambda_{3}$  is small (2.25), the model experiences CO. Increasing  $\lambda_{3}$  reduces the natural accuracy. From the variation of  $\lambda_{3}$  between 2.25 and 4.75, we choose  $\lambda_{3} = 3.25$  to balance the elimination of CO with natural and robustness performance.
C Evaluation Based on Auto Attack
Auto Attack [5] is regarded as the most reliable robustness evaluation to date. It is an ensemble of complementary attacks, consisting of three white-box attacks (APGD-CE, APGD-DLR, and FAB) and a black-box attack (Square Attack). In order to avoid the pseudo-robustness brought by gradient masking or gradient obfuscation, we report the Auto Attack results on Cifar10/100 in Table 8 and Table 9.
In Table 8 and Table 9, we observe that our method, AAER, consistently improves adversarial robustness under Auto Attack on both the CIFAR-10 and CIFAR-100 datasets. It demonstrates that
AAER is effective in preventing CO and enhancing robustness under various adversarial attacks. The results validate the robustness and comprehensiveness of our proposed method.
Table 8. CIFAR10: Accuracy of different methods and different noise magnitudes using PreactResNet-18 under  $L_{\infty}$  threat model. We only report the robust accuracy (\%) under Auto Attack while the natural accuracy is same as Table 2. The results are averaged over 3 random seeds and reported with the standard deviation.
noise magnitude	8/255	12/255	16/255	32/255
FreeAT	40.23 ± 0.33	28.04 ± 0.73	0.00 ± 0.00	0.00 ± 0.00
ZeroGrad	43.48	-	-	-
MulitGrad	44.39	-	-	-
Grad Align	44.82 ± 0.09	30.05 ± 0.17	19.60 ± 0.47	7.89 ± 2.62
RS-FGSM	43.17 ± 0.34	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
N-FGSM	44.43 ± 0.24	30.32 ± 0.08	19.06 ± 1.81	6.78 ± 0.75
RS-AAER	43.22 ± 0.20	26.75 ± 0.21	17.03 ± 0.51	5.37 ± 0.67
N-AAER	44.79 ± 0.23	30.76 ± 0.17	20.18 ± 0.15	8.46 ± 0.74
PGD-2	42.97 ± 0.65	28.63 ± 0.38	18.52 ± 0.55	3.77 ± 0.02
PGD-10 (20)	46.95 ± 0.54	33.30 ± 0.20	22.29 ± 0.27	11.48 ± 0.43
Table 9. CIFAR100: Accuracy of different methods and different noise magnitudes using PreactResNet-18 under  $L_{\infty}$  threat model. We only report the robust accuracy (\%) under Auto Attack while the natural accuracy is same as Table 2. The results are averaged over 3 random seeds and reported with the standard deviation.
noise magnitude	8/255	12/255	16/255	32/255
FreeAT	18.28 ± 0.20	12.37 ± 0.14	0.00 ± 0.00	0.00 ± 0.00
ZeroGrad	21.15	-	-	-
MulitGrad	21.62	-	-	-
Grad Align	21.87 ± 0.13	13.78 ± 0.11	9.64 ± 0.12	1.76 ± 0.70
RS-FGSM	7.98 ± 11.91	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
N-FGSM	22.68 ± 0.25	14.57 ± 0.09	10.30 ± 0.14	0.00 ± 0.00
RS-AAER	21.41 ±0.01	12.31± 0.28	8.56 ±0.02	2.93 ± 0.17
N-AAER	22.93 ± 0.10	14.73 ± 0.24	10.35 ± 0.11	3.46 ± 0.14
PGD-2	22.52 ± 0.14	13.69 ± 0.02	9.56 ± 0.07	1.76 ± 0.22
PGD-10 (20)	23.78 ± 0.08	15.61 ± 0.09	10.93 ± 0.05	4.13 ± 0.10
D Experiment with WideResNet Architecture
We also compare the performance of our method using WideResNet-34, which is more complex than PreActResNet. Since the baselines cannot adapt well to WideResNet-34, we also need to correspondingly adjust the hyperparameters. The  $\lambda_{1}$  is fixed as 1.0 in all settings. For CIFAR10, we set RS-AAER  $\lambda_{2} = 4.0$  and  $\lambda_{3} = 2.0$ , N-AAER  $\lambda_{2} = 2.5$  and  $\lambda_{3} = 0.6$ . For CIFAR100, we set RS-AAER  $\lambda_{2} = 2.5$  and  $\lambda_{3} = 1.0$ , N-AAER  $\lambda_{2} = 1.0$  and  $\lambda_{3} = 0.2$ . We report the results on Cifar10/100 in Table 10 and Table 11.
Table 10. CIFAR10: Accuracy of different methods with 8/255 noise magnitude using WideResNet-34 under  $L_{\infty}$  threat model. The results are averaged over 3 random seeds and reported with the standard deviation.
method	RS-FGSM	N-FGSM	RS-AAER	N-AAER	PGD-2	PGD-10
natural accuracy (%)	84.41 ± 0.45	84.67 ± 0.32	87.39 ± 0.14	84.47 ± 0.23	88.68 ± 0.14	85.53 ± 0.22
robust accuracy (%)	0.00 ± 0.00	49.72 ± 0.25	47.58 ± 0.42	50.07 ± 0.53	47.32 ± 0.50	53.70 ± 0.53
training time (S)	98.2	98.6	147.1	536.2
Table 11. CIFAR100: Accuracy of different methods with 8/255 noise magnitude using WideResNet-34 under  $L_{\infty}$  threat model. The results are averaged over 3 random seeds and reported with the standard deviation.
method	RS-FGSM	N-FGSM	RS-AAER	N-AAER	PGD-2	PGD-10
natural accuracy (%)	55.04 ± 1.24	59.02 ± 0.63	59.81 ± 0.38	57.76 ± 0.36	64.64 ± 0.27	60.34 ± 0.34
robust accuracy (%)	0.00 ± 0.00	28.49 ± 0.54	26.88 ± 0.30	29.09 ± 0.66	26.47 ± 0.10	30.02 ± 0.09
In Table 10 and Table 11, we observe that when using the WideResNet-34 architecture, RS-FGSM suffers from CO with a noise magnitude of 8/255, which is different from the results obtained with the PreActResNet-18 architecture. However, our method, AAER, can successfully prevent CO and achieve high robustness even with complex network architectures. This demonstrates the reliability of AAER in preventing CO and improving robustness across different network architectures. It is worth noting that complex networks can better reflect the efficiency of our method in terms of training time, while our method can achieve comparable robustness to multi-step AT.
E Settings and Results on SVHN, Tiny-ImageNet and Imagenet-100