SlowGuess/NeurIPS_2023_OCR huggingface dataset:
affected by adversarial perturbations. For fair comparison, we apply the same perturbation crafted in 5.2 for both methods. From Figure 3, we can observe that for TRADES, all the samples' prediction scores cluster closely to a small value before perturbation as it is working on an imbalanced dataset, while the prediction scores of our method scatter more evenly. With introduced perturbations, the ranking order predicted by TRADE is changed significantly with small changes in prediction scores, which leads to a considerable decrease in terms of AP. On the contrary, the perturbation does not affect the predicted ranking order of our method, despite the impact on their prediction scores. This helps demonstrate why our method is more robust than other baselines in terms of AP.
To investigate the significance of the new features in our proposed method, i.e., the listwise adversarial regularization and the non-zero-sum game approach, we conduct ablation studies to compare different approaches shown in Table 1 for adversarial AP maximization. Following the setting in 5.1, we
Figure 2: Visualization of the trade-off between robustness and average precision. For the sake of clarity in these figures, we defer the detailed values to Table 9 in the appendix to additionally demonstrate the correlation between robust AP/clean AP and  $\lambda$ .
Figure 3: Visualization of models' predictions after perturbation
run all these five approaches on Cifar10, CIFAR100 and BDD100K datasets. The experimental settings and the hyperparameters for AdAP_LN and AdAP_LPN are the same as those in 5.1. The hyperparameters we tune for AdAP_LZ are the same as those for AdAP_LN. For AdAP_PZ, we tune the weight parameter in  $\{1,4,8,10,40,80,100\}$ . For AdAP_MM, we tune  $\gamma_{1}$  in  $\{0.1,0.9\}$ . The results are presented in Table 5. We can see that (i) robust regularization approaches are better than the minimax approach (AdAP_MM) (ii) the non-zero-sum game approaches (AdAP_LPN, AdAP_LN) are usually better than the zero-sum game approach (AdAP_LZ); (iii) combining listwise and pointwise adversarial regularization in AdAP_LPN could bring significant boost in term of both robust AP and clean AP.
Insensitivity to Batch Size. We investigate the proposed approximation method for  $g(\mathbf{w})$  by varying the mini-batch size for AdAP_LN algorithm and report results in Figure 4 in the appendix. We can see that AdAP_LN does not require a very large batch size and is generally not sensitive to the mini-batch size, which implies the effectiveness of our approximation method.
Sensitivity to  $\gamma_{1},\gamma_{2}$ . We study the sensitivity of the hyper-parameters  $\gamma_{1},\gamma_{2}$  for proposed AdAP_LN algorithm. From the results in Table 8 in the appendix, we can observe that  $\gamma_{1},\gamma_{2}$  have a significant impact on the performance. However, when we tune  $\gamma_{1},\gamma_{2}$  in  $\{0.1,0.9\}$ , it is able to achieve relatively good performance but not always the optimal one.
Empirical convergence analysis. We report the convergence curves of proposed AdAP_LN algorithm in Figure5 in the appendix, which demonstrates the convergence of the algorithm.
Time efficiency. We compare the training time efficiency of the proposed method with different algorithms in Table 7 in the appendix. We can observe that (i) adversarial training methods are generally more time-consuming than natural training; (ii) our proposed AdAP_LN and AdAP_LPN methods cost a little more time than traditional PGD method but much less time than TRADES.
In this paper, we have proposed a novel solution for maximizing average precision with adversarial ranking robustness. The proposed formulation is robust in terms of listwise ranking performance against individual adversarial perturbations and is able to trade off between average precision on clean data and adversarial ranking robustness. We have developed an efficient stochastic algorithm to solve the resulting objective. Extensive experimental results on various datasets demonstrate that the proposed method can achieve significantly improved adversarial robustness in terms of AP, compared with other strong adversarial training baselines. It would be interesting to extend our method to construct more robust systems with average precision as the objective, such as object detection systems and medical diagnosis systems.
We thank anonymous reviewers for constructive comments. G. Li and T. Yang were partially supported by NSF Career Award 2246753, NSF Grant 2246757, NSF Grant 2246756, NSF Grant 2306572, and GM gift funding.
[52] Wentao Wang, Han Xu, Xiaorui Liu, Yaxin Li, Bhavani Thuraisingham, and Jiliang Tang. Imbalanced adversarial training with reweighting. In 2022 IEEE International Conference on Data Mining (ICDM), pages 1209-1214, 2022.
[53] Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2020.
[54] Zhengyang Wang, Meng Liu, Youzhi Luo, Zhao Xu, Yaochen Xie, Limei Wang, Lei Cai, Qi Qi, Zhuoning Yuan, Tianbao Yang, et al. Advanced graph and sequence neural networks for molecular property prediction and drug discovery. Bioinformatics, 38(9):2579-2586, 2022.
[55] Peisong Wen, Qianqian Xu, Zhiyong Yang, Yuan He, and Qingming Huang. Exploring the algorithm-dependent generalization of auprc optimization with list stability. Advances in Neural Information Processing Systems, 35:28335-28349, 2022.
[56] Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R Salakhutdinov, and Kamalika Chaudhuri. A closer look at accuracy vs. robustness. Advances in neural information processing systems, 33:8588-8601, 2020.
[57] Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2636-2645, 2020.
[58] Jianhe Yuan and Zhihai He. Ensemble generative cleaning with feedback loops for defending adversarial attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 581-590, 2020.
[59] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pages 7472-7482. PMLR, 2019.
[60] Mo Zhou, Zhenxing Niu, Le Wang, Qilin Zhang, and Gang Hua. Adversarial ranking attack and defense. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV 16, pages 781-799. Springer, 2020.
A More Experimental Results
All experiments in our paper are run across 16 NVIDIA A10 GPUs and 10 NVIDIA A30 GPUs. We present the statistics for each task in our experiments in Table 6.
Table 6: Datasets statistics. The percentage in parenthesis represents the proportion of positive samples.
Dataset	Train	Validation	Test
CIFAR-10	40000 (10%)	10000 (10%)	10000 (10%)
CIFAR-100	40000 (5%)	10000 (5%)	10000 (5%)
BDD100K(tunnel)	55890 (0.18%)	13973 (0.19%)	10000 (0.27%)
BDD100K(residential)	55890 (11.56%)	13973 (11.56%)	10000 (12.53%)
BDD100K(rainy)	55890 (7.26%)	13973 (7.26%)	10000 (7.38%)
BDD100K(Cloudy)	55890 (6.99%)	13973 (6.98%)	10000 (7.38%)
CelebA(gray_hair)	162770 (4.24%)	19867 (4.87%)	19962 (3.19%)
CelebA(mustache)	162770 (4.08%)	19867 (5.05%)	19962 (3.87%)
A.1 Time efficiency of our proposed algorithm
Time efficiency is typically a concern for adversarial training techniques. To order to investigate the time efficiency of our proposed method, we've conducted some experiments to show time efficiency comparison with different algorithms. In the experiment, we set the parameters, which could affect training time, exactly the same(e.g. batch size as 128, total epochs as 60, adversarial samples are generated with 6 projected gradient ascent steps) and run all the models over three times on the Class_0 task of CIFAR10. From the table 7, we can observe that (i) adversarial training methods are generally more time-consuming than natural training; (ii) our proposed AdAP_LN and AdAP_LPN methods cost a little more time than traditional PGD method but much less time than TRADES. This is because, to generate adversarial samples in training, TRADES is solving the maximization of KL divergence between the probabilities predicted with clean data and perturbed data(i.e.  $\max_{\| \delta \| \leq \epsilon} \sum_k h(x)_k \log h(x)_k - h(x)_k \log h(x + \delta)_k$ , where  $h(x)_k$  and  $h(x + \delta)_k$  are predicted probabilities for class k on clean data and perturbed data respectively, which requires two forward propagations and one backpropagation in each projected gradient ascent step. However, PGD and our proposed methods are directly solving maximization of Cross Entropy(i.e.  $\max_{\| \delta \| \leq \epsilon} - \log h(x + \delta)_y$ , which only needs one forward propagation and one backpropagation in each projected gradient ascent step. In adversarial training, since each gradient descent step w.r.t w requires multiple gradient ascent steps w.r.t  $\delta$ , the computational expense primarily stems from the projected gradient ascent steps, which can be also observed by comparing the efficiency of CE Min. with PGD.
Table 7: Training efficiency comparison
Methods	Run 1	Run 2	Run 3	Average
CE Min.	563s	566s	568s	565.67s
AP Max.	589s	590s	589s	589.33s
PGD	2833s	2804s	2803s	2813.33s
TRADES	4203s	4182s	4179s	4188.00s
MART	3192s	3205s	3194s	3197.00s
AdAP_LN	3227s	3213s	3211s	3217.00s
AdAP_LP	3234s	3219s	3218s	3223.67s
A.2 Sensitivity of proposed algorithm to batch size
In our proposed algorithm, one essential step is the approximation of  $g(\mathbf{w})$ . Due to the nonlinear dependence of  $\nabla_{\mathbf{w}}R(\mathbf{w},\delta ,\mathcal{D})$  on  $g(\mathbf{w})$ , we cannot simply use their mini-batch estimator in the calculation as it will yield a large optimization error [51]. Instead, we maintain a vector  $\mathbf{u} = [u_1,u_2,u_3]$  for tracking  $[g_{1}(\mathbf{w}),g_{2}(\mathbf{w}),g_{3}(\mathbf{w})]$  to reduce the optimization error. The vector  $\mathbf{u}$  is updated by Equation (11). To offer more empirical results on this, we have conducted some experiments to investigate the proposed AdAP_LN's sensitivity to batch sizes on CIFAR10 dataset. The results are shown in Figure 4. We can observe that our method does not require a very large batch size to achieve a good performance and is generally not sensitive to batch size, which implies the effectiveness of our approximation method.
Figure 4: Illustration for insensitivity of AdAP_LN to batch size
A.3 Sensitivity of proposed algorithm to  $\gamma_{1}$  and  $\gamma_{2}$
In our main experiments, the hyper-parameters  $\gamma_{1},\gamma_{2}$  for our proposed algorithms are tuned in  $\{0.1,0.9\}$  as mentioned in Section 5. To study the sensitivity of these parameters, we conducted some more experiments on CIFAR10 and BDD100K datasets. The results are summarized in Table 8. From the table, we can observe that  $\gamma_{1},\gamma_{2}$  have a significant impact on the performance. However, when we tune  $\gamma_{1},\gamma_{2}$  in  $\{0.1,0.9\}$ , it is able to achieve relatively good performance but not always the optimal one.
Table 8: Robust AP on validation for AdAP_LN with different  ${\gamma }_{1}$  and  ${\gamma }_{2}$
CIFAR10 (average)	γ2=0.1	γ2=0.3	γ2=0.5	γ2=0.7	γ2=0.9
γ1=0.1	0.2719(0.0116)	0.273(0.0079)	0.275(0.0105)	0.2756(0.0135)	0.2753(0.0114)
γ1=0.3	0.2701(0.012)	0.2696(0.01)	0.2753(0.0128)	0.2733(0.0101)	0.2745(0.0107)
γ1=0.5	0.2674(0.0088)	0.2741(0.011)	0.2688(0.0108)	0.2685(0.0135)	0.2741(0.0113)
γ1=0.7	0.2649(0.0103)	0.2713(0.007)	0.2686(0.0165)	0.2696(0.0138)	0.2658(0.0103)
γ1=0.9	0.2668(0.0137)	0.2766(0.0131)	0.2668(0.013)	0.2689(0.0139)	0.2641(0.0124)
BDD100K(rainy)	γ2=0.1	γ2=0.3	γ2=0.5	γ2=0.7	γ2=0.9
γ1=0.1	0.2433(0.0209)	0.2436(0.0205)	0.2404(0.0188)	0.2415(0.0220)	0.2423(0.0215)
γ1=0.3	0.2471(0.0219)	0.2440(0.0174)	0.2432(0.0169)	0.2473(0.0214)	0.2460(0.0188)
γ1=0.5	0.2479(0.0210)	0.2475(0.0209)	0.2476(0.0217)	0.2450(0.0189)	0.2472(0.0187)
γ1=0.7	0.2507(0.0203)	0.2453(0.0204)	0.2471(0.0179)	0.2461(0.0206)	0.2480(0.0176)
γ1=0.9	0.2522(0.0208)	0.2479(0.0200)	0.2485(0.0201)	0.2447(0.0193)	0.2478(0.0193)
A.4 Empirical convergence analysis
Since the theoretical convergence analysis of our proposed algorithm is very challenging, deriving the convergence would be a significant work by itself. Instead, we provide some empirical results to show the convergence of the proposed algorithm. Specifically, we set  $\lambda = 1$ ,  $\gamma_{1} = 0.1$ ,  $\gamma_{2} = 0.1$  and run AdAP_LN algorithm on CIFAR10 dataset and BDD100K dataset for a total of 120 epochs and 80 epochs, respectively. We evaluate the training loss after each epoch and report the loss values. We present the AP loss (i.e.,  $P(\mathbf{w})$  in Equation 9) and regularization term (i.e.,  $R(\mathbf{w},\delta,\mathcal{D})$  in Equation 9) separately, as well as the summation of the two losses. For each experiment, we repeat three times with different random seeds. The results are shown in Figure 5, which demonstrates the convergence of our algorithm.
Figure 5: Training convergence curves on CIFAR10 and BDD100K dataset
A.5 Detailed values for Figure 2
To clearly demonstrate the relationship between robust AP/clean AP and  $\lambda$ , corresponding to Fig.2, we provide the detailed values in Table 9. For TRADES, AdAP_LN and AdAP_LPN, the results clearly indicate that there is a positive correlation between robust AP and  $\lambda$  for relatively small values of  $\lambda$  and this correlation diminishes when  $\lambda$  continues increasing. Moreover, a negative correlation between clean AP and  $\lambda$  is consistently observed, in accordance with our expectations. But for MART, the trend is not clear as it is not a trade-off based approach.
Table 9: The Correlation between Robust AP/Clean AP and  $\lambda$  . The results are presented in the format of robust AP/clean AP, with the values representing the average of three runs with different seeds.
λ	0.01	0.4	0.8	1	4	8	10
CIFAR10(cls_1)
TRADES	0.0521/0.9609	0.4514/0.9135	0.4476/0.8846	0.4455/0.8727	0.4040/0.7930	0.3727/0.7479	0.3616/0.7305
MART	0.4511/0.8777	0.4548/0.8812	0.4519/0.8769	0.4349/0.8622	0.4247/0.8450	0.4016/0.8164	0.4030/0.8111
AdAP_LN	0.1153/0.9525	0.4264/0.9295	0.4590/0.9190	0.4464/0.9099	0.3181/0.8649	0.2695/0.8385	0.2522/0.8356
AdAP_LPN	0.1504/0.9358	0.5185/0.9314	0.5263/0.9146	0.5229/0.9067	0.4818/0.8468	0.4517/0.8102	0.4411/0.7968
CIFAR10(cls_2)
TRADES	0.0282/0.8823	0.3252/0.7107	0.3241/0.6571	0.3090/0.6324	0.2782/0.5213	0.2759/0.5040	0.2757/0.4994
MART	0.3263/0.6669	0.3207/0.6538	0.3216/0.6527	0.3183/0.6420	0.3075/0.6095	0.2990/0.5839	0.2950/0.5767
AdAP_LN	0.0811/0.8752	0.2782/0.7984	0.3184/0.7668	0.3238/0.7540	0.3644/0.6736	0.3437/0.6286	0.3436/0.6164
AdAP_LPN	0.1604/0.8332	0.3475/0.7896	0.3809/0.7558	0.3859/0.7441	0.3724/0.6470	0.3501/0.6011	0.3392/0.5827
BDD100K(Cloudy)
TRADES	0.0379/0.6429	0.2301/0.5554	0.2390/0.5260	0.2382/0.5153	0.2105/0.4251	0.1565/0.2443	0.1510/0.2193
MART	0.2130/0.4459	0.2130/0.4450	0.2145/0.4441	0.2129/0.4404	0.1993/0.3973	0.1974/0.3901	0.1887/0.3631
AdAP_LN	0.0621/0.6504	0.2221/0.5986	0.2742/0.5632	0.2815/0.5577	0.2796/0.4935	0.2347/0.4937	0.2103/0.4861
AdAP_LPN	0.0411/0.6570	0.2455/0.5985	0.2682/0.5681	0.2726/0.5600	0.2725/0.4886	0.2464/0.4366	0.2473/0.4295
CelebA(gray_hair)
TRADES	0.0161/0.7686	0.2341/0.5428	0.2581/0.4707	0.2571/0.4513	0.2447/0.3828	0.2206/0.3366	0.2265/0.3423
MART	0.2567/0.4910	0.2576/0.4882	0.2582/0.4865	0.2594/0.4864	0.2617/0.4784	0.2613/0.4696	0.2616/0.4672
AdAP_LN	0.0264/0.7037	0.1831/0.6282	0.2449/0.6061	0.2696/0.6010	0.2859/0.5221	0.2714/0.4662	0.2601/0.4506
AdAP_LPN	0.0284/0.7123	0.2280/0.6259	0.2555/0.5907	0.2644/0.5804	0.2799/0.4948	0.2676/0.4515	0.2676/0.4515
A.6 Detailed experimental results on CIFAR10 and CIFAR100 Dataset
Due to the limit of space, we include mAP results over 10 classes on CIFAR10 and CIFAR100 datasets in Section 5.1 and Section 5.2. Here, we present the detailed performance for each class on CIFAR10 and CIFAR100 datasets in Table 10, Table 11, Table 12, Table 13, Table 14, Table 15. From Table 10, Table 11, Table 12 and Table 13 about white-box robustness, we can observe that, in most cases, our methods improve other baselines significantly. There is one special case on CIFAR10 Class_3 in Table 10 that MART method behaves much better than others due to high variance (we can see it from the deviation in the parenthesis). When we look at Table 12 about white-box robustness against stronger APGD attacks, we can find that the superiority of MART disappears on CIFAR10 Class_3 since the advantage gained by high variance is not stable. From Table 14 and Table 15 about black-box robustness, we can observe that our methods outperform other baselines by a large margin consistently.
A.7 More plots for trade-off visualization
Due to page limitations, we present one single trade-off curve for each dataset in Section 5.3 to illustrate our findings. In this part, we include more trade-off curves on all the datasets in Figure 6, Figure 7, Figure 8, Figure 9. These additional figures provide further support for the superiority of our methods.
Table 10: Adversarial robustness against PGD white-box attacks on CIFAR10 dataset
CIFAR10
Methods	class_0	class_1	class_2	class_3	class_4
Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean	Robust
CE Min.	0.0519(0.0000)	0.9037(0.0031)	0.0558(0.0027)	0.9687(0.0009)	0.0518(0.0000)	0.8493(0.0057)	0.0518(0.0000)	0.7235(0.0064)	0.0519(0.0001)
AP Max.	0.0528(0.0010)	0.9056(0.0049)	0.0563(0.0063)	0.9707(0.0026)	0.0522(0.0005)	0.8602(0.0021)	0.0521(0.0003)	0.7398(0.0123)	0.0523(0.0004)
PGD	0.4315(0.0039)	0.6109(0.0131)	0.4584(0.0036)	0.8869(0.0024)	0.1419(0.0037)	0.3160(0.0131)	0.1629(0.0381)	0.1730(0.0071)	0.1892(0.0070)
TRADES	0.4372(0.0032)	0.6708(0.0033)	0.4523(0.0049)	0.9109(0.0011)	0.1798(0.0060)	0.3982(0.0094)	0.1293(0.0004)	0.2685(0.0029)	0.1854(0.0009)
MART	0.4295(0.0029)	0.6032(0.0066)	0.4559(0.0076)	0.8796(0.0061)	0.1528(0.0101)	0.3068(0.0212)	0.1942(0.0825)	0.1731(0.0143)	0.1838(0.0025)
AdAP_LN	0.4259(0.0042)	0.6498(0.0033)	0.4679(0.0315)	0.9143(0.0064)	0.1568(0.0152)	0.5538(0.0364)	0.1169(0.0037)	0.3342(0.0187)	0.1703(0.0031)
AdAP_LP	0.4572(0.0024)	0.7382(0.0018)	0.5262(0.0035)	0.9145(0.0016)	0.2041(0.0031)	0.5016(0.0047)	0.1420(0.0008)	0.2788(0.0038)	0.1856(0.0007)
Methods	class_5	class_6	class_7	class_8	class_9
Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean	Robust
CE Min.	0.0518(0.0000)	0.8347(0.0050)	0.0518(0.0000)	0.9403(0.0048)	0.0521(0.0002)	0.9372(0.0022)	0.0548(0.0031)	0.9559(0.0022)	0.0527(0.0005)
AP Max.	0.0518(0.0000)	0.8420(0.0040)	0.0519(0.0000)	0.9419(0.0022)	0.0519(0.0000)	0.9416(0.0021)	0.0545(0.0035)	0.9569(0.0014)	0.0518(0.0000)
PGD	0.2329(0.0004)	0.4524(0.0054)	0.2363(0.0026)	0.4111(0.0031)	0.2356(0.0169)	0.5298(0.0232)	0.4157(0.0355)	0.7390(0.0020)	0.2602(0.0070)
TRADES	0.2225(0.0017)	0.4068(0.0047)	0.2289(0.0016)	0.3492(0.0029)	0.3723(0.0022)	0.7893(0.0011)	0.3877(0.0054)	0.7961(0.0159)	0.2890(0.0028)
MART	0.2348(0.0010)	0.4510(0.0019)	0.2346(0.0023)	0.4059(0.0030)	0.2917(0.0092)	0.6667(0.0250)	0.4158(0.0020)	0.7397(0.0070)	0.2662(0.0054)
AdAP_LN	0.2293(0.0067)	0.5091(0.0063)	0.2647(0.0197)	0.7147(0.0106)	0.3978(0.0258)	0.8318(0.0075)	0.4193(0.0287)	0.8068(0.0019)	0.3004(0.0084)
AdAP_LP	0.2467(0.0014)	0.5658(0.0028)	0.2785(0.0019)	0.6837(0.0029)	0.4354(0.0031)	0.8292(0.0003)	0.4564(0.0024)	0.8413(0.0028)	0.3690(0.0030)
Table 11: Adversarial robustness against PGD white-box attacks on CIFAR100 dataset
CIFAR100
Methods	class_0	class_1	class_2	class_3
Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean
CE Min.	0.0256(0.0001)	0.5465(0.0085)	0.0256(0.0001)	0.6112(0.0025)	0.0255(0.0000)	0.8932(0.0025)	0.0255(0.0000)	0.7359(0.0072)
AP Max.	0.0255(0.0000)	0.5480(0.0060)	0.0258(0.0002)	0.6348(0.0199)	0.0259(0.0003)	0.8993(0.0022)	0.0255(0.0000)	0.7392(0.0079)
0.0255(0.0000)	0.8255(0.0078)
PGD	0.1124(0.0021)	0.2128(0.0060)	0.1845(0.0020)	0.3414(0.0071)	0.3261(0.0066)	0.6618(0.0036)	0.1141(0.0066)	0.2466(0.0120)
TRADES	0.1177(0.0010)	0.2560(0.0020)	0.1942(0.0028)	0.3888(0.0069)	0.3289(0.0044)	0.6937(0.0106)	0.1663(0.0053)	0.3976(0.0119)
MART	0.1187(0.0027)	0.2188(0.0029)	0.1882(0.0037)	0.3417(0.0080)	0.3258(0.0074)	0.6651(0.0034)	0.1352(0.0144)	0.2389(0.0092)
AdAP_LN	0.1273(0.0028)	0.3131(0.0049)	0.1912(0.0020)	0.4073(0.0036)	0.3658(0.0054)	0.6736(0.0035)	0.1930(0.0011)	0.4403(0.0042)
AdAP_LPN	0.1376(0.0005)	0.2946(0.0014)	0.2144(0.0069)	0.4469(0.0092)	0.3916(0.0096)	0.7342(0.0111)	0.2085(0.0045)	0.4979(0.0019)
class_8
Methods	class_5	class_6	class_7	class_8
Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean
CE Min.	0.0255(0.0000)	0.5990(0.0065)	0.0255(0.0000)	0.7690(0.0039)	0.0255(0.0000)	0.6743(0.0037)	0.0255(0.0000)	0.6033(0.0102)
AP Max.	0.0299(0.0054)	0.6257(0.0081)	0.0262(0.0005)	0.7806(0.0032)	0.0276(0.0029)	0.6974(0.0069)	0.0255(0.0000)	0.6192(0.0111)
0.0255(0.0000)	0.6192(0.0111)
PGD	0.0804(0.0025)	0.1671(0.0151)	0.1054(0.0018)	0.1960(0.0090)	0.1441(0.0021)	0.3205(0.0057)	0.0763(0.0108)	0.0943(0.0066)
TRADES	0.0963(0.0036)	0.2972(0.0068)	0.1533(0.0031)	0.4387(0.0059)	0.1751(0.0011)	0.3660(0.0008)	0.0662(0.0008)	0.1174(0.0054)
MART	0.1805(0.0028)	0.1439(0.0060)	0.1095(0.0079)	0.2531(0.0100)	0.1477(0.0020)	0.2973(0.0030)	0.0700(0.0100)	0.0969(0.0065)
AdAP_LN	0.1131(0.0040)	0.3900(0.0524)	0.2058(0.0060)	0.5916(0.0130)	0.1884(0.0020)	0.4611(0.0076)	0.0693(0.0119)	0.2031(0.0334)
AdAP_LPN	0.1483(0.0038)	0.4268(0.0164)	0.2459(0.0021)	0.5670(0.0153)	0.2170(0.0037)	0.4234(0.0062)	0.0710(0.0003)	0.1552(0.0018)
0.1986(0.0083)	0.5846(0.0105)
Table 12: Adversarial robustness against APGD white-box attacks on CIFAR10 dataset
CIFAR10
Methods	class_0	class_1	class_2	class_3
Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean
CE Min.	0.0518(0.0000)	0.9037(0.0031)	0.0547(0.0017)	0.9687(0.0009)	0.0518(0.0000)	0.8493(0.0057)	0.0518(0.0000)	0.7235(0.0064)
AP Max.	0.0519(0.0001)	0.9056(0.0049)	0.0596(0.0110)	0.9707(0.0026)	0.0518(0.0000)	0.8602(0.0021)	0.0518(0.0000)	0.7398(0.0123)
PGD	0.4291(0.0038)	0.6109(0.0131)	0.4482(0.0033)	0.8869(0.0024)	0.1380(0.0034)	0.3160(0.0131)	0.1120(0.0145)	0.1730(0.0071)
TRADES	0.4334(0.0031)	0.6708(0.0033)	0.4391(0.0050)	0.9109(0.0011)	0.1782(0.0062)	0.3982(0.0094)	0.1289(0.0004)	0.2685(0.0029)
MART	0.4274(0.0032)	0.6032(0.0066)	0.4465(0.0074)	0.8796(0.0061)	0.1509(0.0105)	0.3068(0.0212)	0.0929(0.0092)	0.1731(0.0143)
AdAP_LN	0.4222(0.0034)	0.6498(0.0033)	0.4220(0.0117)	0.9143(0.0064)	0.1371(0.0074)	0.5538(0.0364)	0.1144(0.0024)	0.3342(0.0187)
AdAP_LPN	0.4529(0.0025)	0.7382(0.0018)	0.5153(0.0031)	0.9145(0.0016)	0.2012(0.0030)	0.5016(0.0047)	0.1413(0.0008)	0.2788(0.0038)
class_5	class_6	class_7	class_8
Methods	Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean
CE Min.	0.0518(0.0000)	0.8347(0.0050)	0.0518(0.0000)	0.9403(0.0048)	0.0520(0.0001)	0.9372(0.0022)	0.0537(0.0019)	0.9559(0.0022)
AP Max.	0.0518(0.0000)	0.8420(0.0040)	0.0519(0.0002)	0.9419(0.0022)	0.0518(0.0000)	0.9416(0.0021)	0.0519(0.0002)	0.9569(0.0014)
PGD	0.2314(0.0003)	0.4524(0.0054)	0.2350(0.0028)	0.4111(0.0031)	0.2270(0.0145)	0.5928(0.0232)	0.4113(0.0034)	0.7390(0.0020)
TRADES	0.2217(0.0017)	0.4068(0.0047)	0.2284(0.0016)	0.3492(0.0029)	0.3627(0.0019)	0.7893(0.0011)	0.3790(0.0057)	0.7961(0.0159)
MART	0.2333(0.0009)	0.4510(0.0019)	0.2334(0.0023)	0.4059(0.0030)	0.2853(0.0094)	0.6667(0.0250)	0.4117(0.0017)	0.7397(0.0070)
AdAP_LN	0.2209(0.0030)	0.5091(0.0063)	0.1555(0.0096)	0.7147(0.0106)	0.3591(0.0060)	0.8318(0.0075)	0.3434(0.0055)	0.8608(0.0019)
AdAP_LPN	0.2436(0.0015)	0.5658(0.0028)	0.2737(0.0014)	0.6837(0.0029)	0.4276(0.0033)	0.8292(0.0003)	0.4496(0.0025)	0.8413(0.0028)