SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 13: Adversarial robustness against APGD white-box attack on CIFAR100 dataset
CIFAR100
Methods	class_0	class_1	class_2	class_3
Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean
CE Min.	0.0255(0.0000)	0.5465(0.0085)	0.0255(0.0000)	0.6112(0.0025)	0.0255(0.0000)	0.8932(0.0025)	0.0255(0.0000)	0.7359(0.0072)
AP Max.	0.0255(0.0000)	0.5480(0.0060)	0.0255(0.0000)	0.6348(0.0199)	0.0255(0.0000)	0.8993(0.0022)	0.0255(0.0000)	0.7392(0.0079)
PGD	0.1103(0.0017)	0.2128(0.0060)	0.1831(0.0020)	0.3414(0.0071)	0.3232(0.0063)	0.6618(0.0036)	0.1100(0.0058)	0.2466(0.0120)
TRADES	0.1171(0.0011)	0.2565(0.0020)	0.1929(0.0028)	0.3888(0.0069)	0.3224(0.0048)	0.6937(0.0106)	0.1632(0.0053)	0.3976(0.0119)
MART	0.1101(0.0046)	0.2188(0.0029)	0.1864(0.0033)	0.3417(0.0080)	0.3229(0.0073)	0.6651(0.0034)	0.1279(0.0112)	0.2389(0.0092)
AdAP_LN	0.1245(0.0024)	0.3131(0.0049)	0.1896(0.0020)	0.4073(0.0036)	0.3566(0.0028)	0.6736(0.0035)	0.1878(0.0011)	0.4403(0.0042)
AdAP_LPN	0.1367(0.0007)	0.2946(0.0014)	0.2128(0.0068)	0.4469(0.0092)	0.3871(0.0093)	0.7342(0.0111)	0.2039(0.0047)	0.4979(0.0019)
class_5	class_6	class_7	class_8
Methods	Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean
CE Min.	0.0255(0.0000)	0.5990(0.0065)	0.0255(0.0000)	0.7690(0.0039)	0.0255(0.0000)	0.6743(0.0037)	0.0255(0.0000)	0.6033(0.0102)
AP Max.	0.0255(0.0000)	0.6257(0.0081)	0.0255(0.0000)	0.7806(0.0032)	0.0264(0.0013)	0.6974(0.0069)	0.0255(0.0000)	0.6192(0.0111)
PGD	0.0777(0.0033)	0.1671(0.0151)	0.1047(0.0019)	0.1960(0.0090)	0.1418(0.0025)	0.3205(0.0057)	0.0632(0.0033)	0.0943(0.0066)
TRADES	0.0943(0.0033)	0.2972(0.0068)	0.1438(0.0036)	0.4387(0.0059)	0.1741(0.0011)	0.3660(0.0008)	0.0666(0.0008)	0.1174(0.0054)
MART	0.0789(0.0020)	0.1439(0.0060)	0.1037(0.0081)	0.2531(0.0100)	0.1458(0.0026)	0.2973(0.0030)	0.0575(0.0014)	0.0969(0.0065)
AdAP_LN	0.1085(0.0027)	0.3900(0.0524)	0.1871(0.0074)	0.5916(0.0130)	0.1840(0.0014)	0.4611(0.0076)	0.0558(0.0009)	0.2031(0.0334)
AdAP_LPN	0.1444(0.0035)	0.4268(0.0164)	0.2421(0.0017)	0.5670(0.0153)	0.2150(0.0041)	0.4234(0.0062)	0.0706(0.0008)	0.1552(0.0018)
0.1943(0.0084)
Table 14: Adversarial robustness against black-box attack on CIFAR10 dataset
CIFAR10
Methods	class_0	class_1	class_2	class_3	class_4
Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean	Robust
AP Max.	0.1038(0.0362)	0.9056(0.0049)	0.2553(0.1270)	0.9707(0.0026)	0.0614(0.0063)	0.8602(0.0021)	0.0634(0.0069)	0.7398(0.0123)	0.0989(0.0330)
PGD	0.5773(0.011)	0.6109(0.0131)	0.8229(0.0031)	0.8869(0.0024)	0.3014(0.0125)	0.3160(0.0131)	0.1629(0.0057)	0.1730(0.0071)	0.2438(0.0023)
TRADES	0.6198(0.0030)	0.6708(0.0033)	0.8430(0.0003)	0.9109(0.0011)	0.3597(0.0074)	0.3982(0.0094)	0.2483(0.0028)	0.2685(0.0029)	0.3244(0.0029)
MART	0.5699(0.0060)	0.6032(0.0066)	0.8147(0.0054)	0.8796(0.0061)	0.2886(0.0177)	0.3068(0.0212)	0.1629(0.0105)	0.1731(0.0143)	0.2455(0.0023)
AdAP_LN	0.6079(0.0034)	0.6498(0.0033)	0.8444(0.0037)	0.9143(0.0064)	0.4365(0.0254)	0.5538(0.0364)	0.2873(0.0152)	0.3342(0.0187)	0.2913(0.0039)
AdAP_LPN	0.6888(0.0018)	0.7382(0.0018)	0.8650(0.0011)	0.9145(0.0016)	0.4503(0.0057)	0.5016(0.0047)	0.2550(0.0029)	0.2788(0.0039)	0.3562(0.0057)
class_5	class_6	class_7	class_8	class_9
Methods	Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean	Robust
AP Max.	0.0723(0.0128)	0.8869(0.0040)	0.0947(0.0245)	0.9149(0.0022)	0.1008(0.0342)	0.9416(0.0021)	0.1153(0.0355)	0.9569(0.0014)	0.1712(0.0578)
PGD	0.4204(0.0042)	0.4524(0.0054)	0.3973(0.0035)	0.4111(0.0031)	0.5522(0.0198)	0.5928(0.0232)	0.6732(0.0005)	0.7390(0.0020)	0.4427(0.0112)
TRADES	0.3816(0.0036)	0.4068(0.0047)	0.3358(0.0028)	0.3492(0.0029)	0.7100(0.0019)	0.7893(0.0011)	0.7034(0.0127)	0.7961(0.0159)	0.6216(0.0012)
MART	0.4199(0.0015)	0.4510(0.0019)	0.3924(0.0025)	0.4059(0.0030)	0.6145(0.0249)	0.6667(0.0250)	0.6727(0.0071)	0.7397(0.0070)	0.4428(0.0050)
AdAP_LN	0.4582(0.0042)	0.5091(0.0063)	0.5564(0.0016)	0.7147(0.0106)	0.7371(0.0049)	0.8318(0.0075)	0.7107(0.0022)	0.8068(0.0019)	0.6322(0.0224)
AdAP_LPN	0.5200(0.0024)	0.5658(0.0028)	0.6277(0.0029)	0.6837(0.0028)	0.7652(0.0003)	0.8292(0.0003)	0.7689(0.0017)	0.8413(0.0029)	0.7338(0.0031)
Table 15: Adversarial robustness against black-box attack on CIFAR100 dataset
CIFAR100
Methods	class_0	class_1	class_2	class_3	class_4
Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean	Robust
AP Max.	0.0324(0.0046)	0.5480(0.0060)	0.0632(0.0267)	0.6348(0.0199)	0.1081(0.0581)	0.8993(0.0022)	0.0411(0.0111)	0.7392(0.0079)	0.1034(0.0547)
PGD	0.1816(0.0051)	0.2128(0.0060)	0.3136(0.0068)	0.3414(0.0071)	0.6056(0.0058)	0.6618(0.0036)	0.2366(0.0119)	0.2466(0.0120)	0.4816(0.0025)
TRADES	0.2184(0.0013)	0.2560(0.0020)	0.3454(0.0031)	0.3888(0.0069)	0.6138(0.0088)	0.6937(0.0106)	0.3538(0.0095)	0.3976(0.0119)	0.4858(0.0030)
MART	0.1869(0.0042)	0.2188(0.0029)	0.3145(0.0079)	0.3417(0.0080)	0.6093(0.0038)	0.6651(0.0034)	0.2260(0.0079)	0.2389(0.0092)	0.4812(0.0055)
AdAP_LN	0.2534(0.0022)	0.3131(0.0049)	0.3547(0.0038)	0.4073(0.0036)	0.6101(0.0032)	0.6736(0.0035)	0.3961(0.0053)	0.4403(0.0042)	0.5139(0.0038)
AdAP_LP	0.2548(0.0015)	0.2946(0.0014)	0.3983(0.0075)	0.4469(0.0092)	0.6703(0.0106)	0.7342(0.0111)	0.4542(0.0021)	0.4979(0.0020)	0.5136(0.0024)
Methods	class_5	class_6	class_7	class_8	class_9
Robust	Clean	Robust	Clean	Robust	Clean	Robust	Clean	Robust
AP Max.	0.0820(0.0376)	0.6257(0.0081)	0.0355(0.0079)	0.7806(0.0032)	0.0593(0.0231)	0.6974(0.0069)	0.0360(0.0064)	0.6192(0.0111)	0.0470(0.0112)
PGD	0.1532(0.0143)	0.1671(0.0151)	0.1692(0.0053)	0.1960(0.0090)	0.2976(0.0059)	0.3205(0.0057)	0.0901(0.0051)	0.0943(0.0066)	0.3403(0.0149)
TRADES	0.2389(0.0062)	0.2972(0.0068)	0.3338(0.0036)	0.4387(0.0059)	0.3290(0.0003)	0.3660(0.0008)	0.1021(0.0024)	0.1174(0.0054)	0.3994(0.0071)
MART	0.1256(0.0055)	0.1439(0.0060)	0.2304(0.0102)	0.2531(0.0100)	0.2770(0.0025)	0.2973(0.0030)	0.0938(0.0059)	0.0969(0.0065)	0.3475(0.0133)
AdAP_LN	0.3113(0.0374)	0.3900(0.0524)	0.4656(0.0114)	0.5916(0.0130)	0.3969(0.0047)	0.4611(0.0076)	0.1338(0.0094)	0.2031(0.0334)	0.4442(0.0179)
AdAP_LP	0.3701(0.0149)	0.4268(0.0164)	0.4929(0.0140)	0.5670(0.0153)	0.3867(0.0058)	0.4234(0.0062)	0.1302(0.0013)	0.1552(0.0018)	0.4948(0.0116)
Figure 6: Visualization of the trade-off between robustness and average precision on CIFAR10 dataset
Figure 7: Visualization of the trade-off between robustness and average precision on CIFAR100 dataset
Figure 8: Visualization of the trade-off between robustness and average precision on BDD100K dataset
Figure 9: Visualization of the trade-off between robustness and average precision on CelebA dataset
B Theoretical Results
B.1 Proof of theorem 1
Note that robust ranking error is defined as:
$$
\mathcal {R} _ {r o b} = \mathbb {E} _ {\mathbf {x} _ {i} \sim \mathcal {D} _ {+}} \mathbb {I} \left\{\exists \delta_ {i}, \delta_ {j} \in \mathbb {B} (0, \epsilon), h \left(\mathbf {x} _ {i} + \delta_ {i}\right) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h \left(\mathbf {x} _ {j} + \delta_ {j}\right) \right\}. \tag {13}
$$
Then we can reformulate it as:
$$
\begin{array}{l} \mathcal {R} _ {r o b} = \Pr \left(\exists \delta_ {i}, \delta_ {j} \in \mathbb {B} (0, \epsilon), h \left(\mathbf {x} _ {i} + \delta_ {i}\right) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h \left(\mathbf {x} _ {j} + \delta_ {j}\right) \mid \mathbf {x} _ {i} \in \mathcal {D} _ {+}\right) \\ = \Pr \left(\exists \delta_ {i}, \delta_ {j} \in \mathbb {B} (0, \epsilon), h (\mathbf {x} _ {i} + \delta_ {i}) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h (\mathbf {x} _ {j} + \delta_ {j}) \mid h (\mathbf {x} _ {i}) > \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h (\mathbf {x} _ {j}), \mathbf {x} _ {i} \in \mathcal {D} _ {+}\right) \\ * \Pr \left(h \left(\mathbf {x} _ {i}\right) > \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h \left(\mathbf {x} _ {j}\right) \mid \mathbf {x} _ {i} \in \mathcal {D} _ {+}\right) \\ + \Pr \left(\exists \delta_ {i}, \delta_ {j} \in \mathbb {B} (0, \epsilon), h (\mathbf {x} _ {i} + \delta_ {i}) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h (\mathbf {x} _ {j} + \delta_ {j}) \mid h (\mathbf {x} _ {i}) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h (\mathbf {x} _ {j}), \mathbf {x} _ {i} \in \mathcal {D} _ {+}\right) \\ * \Pr \big (h (\mathbf {x} _ {i}) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h (\mathbf {x} _ {j}) \mid \mathbf {x} _ {i} \in \mathcal {D} _ {+} \big) \\ = \Pr \left( \right.\left\{ \right.\left( \right.h \left(\mathbf {x} _ {i}\right) > \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h \left(\mathbf {x} _ {j}\right)\left. \right\} \wedge \left\{\exists \delta_ {i}, \delta_ {j} \in \mathbb {B} (0, \epsilon), h \left(\mathbf {x} _ {i} + \delta_ {i}\right) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h \left(\mathbf {x} _ {j} + \delta_ {j}\right)\right\} \mid \mathbf {x} _ {i} \in \mathcal {D} _ {+})\left. \right) \\ + 1 * \Pr \left(h \left(\mathbf {x} _ {i}\right) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h \left(\mathbf {x} _ {j}\right) \mid \mathbf {x} _ {i} \in \mathcal {D} _ {+}\right) \\ = \mathbb {E} _ {\mathbf {x} _ {i} \sim \mathcal {D} _ {+}} \mathbb {I} \left\{h (\mathbf {x} _ {i}) > \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h (\mathbf {x} _ {j}) \right\} \mathbb {I} \left\{\exists \delta_ {i}, \delta_ {j} \in \mathbb {B} (0, \epsilon), h (\mathbf {x} _ {i} + \delta_ {i}) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h (\mathbf {x} _ {j} + \delta_ {j}) \right\} \\ + \mathbb {E} _ {\mathbf {x} _ {i} \sim \mathcal {D} _ {+}} \mathbb {I} \left\{h \left(\mathbf {x} _ {i}\right) \leq \max  _ {\mathbf {x} _ {j} \in \mathcal {D} _ {-}} h \left(\mathbf {x} _ {j}\right) \right\} \\ \end{array}
$$
Notice that the last two terms above are exactly the considered boundary error  $\mathcal{R}_{bdy}$  and misranking error  $\mathcal{R}_{nat}$ . Therefore, we have  $\mathcal{R}_{rob} = \mathcal{R}_{bdy} + \mathcal{R}_{nat}$ .
B.2 Extra algorithms for adversarial AP maximization
Besides the Algorithm 1 shown in Section 4.2, we present the other algorithms for adversarial AP maximization in this part. Algorithms for solving AdAP_PZ in (7) and AdAP_LPN in (10) are similar to the Algorithm 1 and we present them in Algorithm 4 and Algorithm 3.
$$
\min  _ {\mathbf {w}} \max  _ {\| \delta \| \leq \epsilon} P (\mathbf {w}, \delta) = - \frac {1}{n _ {+}} \sum_ {\mathbf {x} _ {i} \in \mathcal {D} _ {+}} \frac {\sum_ {s = 1} ^ {n} \mathbb {I} \left(y _ {s} = 1\right) \ell \left(\mathbf {w} , \mathbf {x} _ {s} + \delta_ {s} , \mathbf {x} _ {i} + \delta_ {i}\right)}{\sum_ {s = 1} ^ {n} \ell \left(\mathbf {w} , \mathbf {x} _ {s} + \delta_ {s} , \mathbf {x} _ {i} + \delta_ {i}\right)}. \tag {14}
$$
Different from above algorithms, the algorithm for solving AdAP_MM in (14) needs to solve inner maximization with AP surrogate loss to generate adversarial perturbations. Because this inner maximization problem includes all the samples in the dataset, we apply the similar compositional optimization technique borrowed from [34] to compute an estimate of  $\nabla_{\delta}P(\mathbf{w},\delta)$  by:
$$
\widehat {\nabla} _ {\delta} P (\mathbf {w}, \delta) = \frac {1}{| \mathcal {B} ^ {+} |} \sum_ {\mathbf {x} _ {i} \in \mathcal {B} ^ {+}} \sum_ {\mathbf {x} _ {j} \in \mathcal {B}} \frac {\left(\mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 1} - \mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 2} \mathbb {I} \left(\mathbf {y} _ {j} = 1\right)\right) \nabla_ {\delta} \ell \left(\mathbf {w} , \mathbf {x} _ {j} + \delta_ {j} , \mathbf {x} _ {i} + \delta_ {i}\right)}{| \mathcal {B} | \left(\mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 2}\right) ^ {2}}. \tag {15}
$$
where  $\mathbf{u}_{\mathbf{x}_i}^{\delta 1}$  and  $\mathbf{u}_{\mathbf{x}_i}^{\delta 2}$  are two estimators for tracking  $\varphi_{\delta 1}(\mathbf{w},\delta_i;\mathbf{x}_i) = \frac{1}{n}\sum_{s = 1}^{n}\mathbb{I}(y_s = 1)\ell (\mathbf{w},\mathbf{x}_s + \delta_s,\mathbf{x}_i + \delta_i),\varphi_{\delta 2}(\mathbf{w},\delta_i;\mathbf{x}_i) = \frac{1}{n}\sum_{s = 1}^{n}\ell (\mathbf{w},\mathbf{x}_s + \delta_s,\mathbf{x}_i + \delta_i)$ . At iteration  $t$ , the two estimators  $\mathbf{u}_{\mathbf{x}_i}^{\delta 1}$  and  $\mathbf{u}_{\mathbf{x}_i}^{\delta 2}$  are updated by
$$
\mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 1} = \left(1 - \gamma_ {1}\right) \mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 1} + \gamma_ {1} \frac {1}{| \mathcal {B} |} \sum_ {1} \ell \left(\mathbf {w} _ {t}, \mathbf {x} _ {s} + \delta_ {s}, \mathbf {x} _ {i} + \delta_ {i}\right) \mathbb {I} \left(y _ {j} = 1\right) \tag {16}
$$
$$
\mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 2} = (1 - \gamma_ {1}) \mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 2} + \gamma_ {1} \frac {1}{| \mathcal {B} |} \sum \ell (\mathbf {w} _ {t}, \mathbf {x} _ {s} + \delta_ {s}, \mathbf {x} _ {i} + \delta_ {i}),
$$
where  $\gamma_1 \in (0, 1)$  is a parameter. And the stochastic gradient  $\widehat{\nabla}_{\mathbf{w}} P(\mathbf{w}, \delta)$  is computed by:
$$
\widehat {\nabla} _ {\mathbf {w}} P (\mathbf {w}, \delta) = \frac {1}{| \mathcal {B} ^ {+} |} \sum_ {\mathbf {x} _ {i} \in \mathcal {B} ^ {+}} \sum_ {\mathbf {x} _ {j} \in \mathcal {B}} \frac {\left(\mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 1} - \mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 2} \mathbb {I} \left(\mathbf {y} _ {j} = 1\right)\right) \nabla_ {\mathbf {w}} \ell \left(\mathbf {w} , \mathbf {x} _ {j} + \delta_ {j} , \mathbf {x} _ {i} + \delta_ {i}\right)}{| \mathcal {B} | \left(\mathbf {u} _ {\mathbf {x} _ {i}} ^ {\delta 2}\right) ^ {2}}. \tag {17}
$$
We present the whole algorithm for solving AdAP_MM in (14) in Algorithm 2.
The objective for AdAP_LZ is defined as
$$
\min  _ {\mathbf {w}} \max  _ {\delta \in \Omega^ {n}} P (\mathbf {w}) + R (\mathbf {w}, \delta , \mathcal {D}) = \min  _ {\mathbf {w}} \left\{P (\mathbf {w}) + \max  _ {\delta \in \Omega^ {n}} R (\mathbf {w}, \delta , \mathcal {D}) \right\} \tag {18}
$$
Since the listwise adversarial regularization includes all the samples in the dataset, the optimization for  $\delta$  is also challenging. We extend the approach for computing an estimator of  $\nabla_{\mathbf{w}}R(\mathbf{w},\delta ,\mathcal{D})$  employed in Section 4.2 to estimate  $\nabla_{\delta}R(\mathbf{w},\delta ,\mathcal{D})$ . To avoid redundancy, we give the estimate directly by
$$
\widehat {\nabla} _ {\delta} R (\mathbf {w}, \delta , \mathcal {D}) = \widehat {\nabla} _ {\delta} g (\delta ; \mathbf {w}) ^ {\top} \left(\frac {1}{u _ {2}}, \frac {- u _ {1} - u _ {2}}{\left(u _ {2}\right) ^ {2}}, \frac {1}{u _ {3}}\right) ^ {\top}, \tag {19}
$$
where  $g(\delta; \mathbf{w})$  has the same definition as  $g(\mathbf{w})$  in Section 4.2,  $\mathbf{u} = [u_1, u_2, u_3]$  denotes a vector for tracking  $[g_1(\mathbf{w}), g_2(\mathbf{w}), g_3(\mathbf{w})]$ . The whole algorithm for solving AdAP_LZ is presented in Algorithm 5.
Algorithm 2 Stochastic Algorithm for solving AdAP_MM in (14)
1: Initialize  $\mathbf{w}$ ,  $\mathbf{u}_{\mathbf{x}}^{\delta 1}$ ,  $\mathbf{u}_{\mathbf{x}}^{\delta 2}$ ,  $\gamma_{1}$
2: for  $t = 1, \dots, T$  do
3: Draw a batch of  $B_{+}$  positive samples denoted by  $\mathcal{B}_{+}$ .
4: Draw a batch of  $B$  samples denoted by  $\mathcal{B}$
5: For  $\mathbf{x}_i\in \mathcal{B}\cup \mathcal{B}_+$  , initialize  $\delta_i\sim \alpha \cdot \mathcal{N}(0,1)$
6: for  $m = 1,\dots ,M$  do
7: For each  $\mathbf{x}_i\in \mathcal{B}_+$ , update  $\mathbf{u}_{\mathbf{x}_i}^{\delta 1}$  and  $\mathbf{u}_{\mathbf{x}_i}^{\delta 2}$  by Equation (16)
8: Compute  $\widehat{\nabla}_{\delta}P(\mathbf{w}_t,\delta)$  by Equation (15)
9: Update  $\delta_{i} = \Pi_{\| \cdot \| \leq \epsilon}(\delta_{i} + \eta_{2}\cdot sign(\widehat{\nabla}_{\delta_{i}}P(\mathbf{w}_{t},\delta)))$  , where  $\Pi_{\Omega}(\cdot)$  is the projection operator.
0: end for
11: For each  $\mathbf{x}_i\in \mathcal{B}_+$  , update  $\mathbf{u}_{\mathbf{x}_i}^{\delta 1}$  and  $\mathbf{u}_{\mathbf{x}_i}^{\delta 2}$  by Equation (16)
12: Compute stochastic gradient estimator  $\nabla_{\mathbf{w}_t} = \widehat{\nabla}_{\mathbf{w}_t}P(\mathbf{w}_t,\delta)$  by Equation (17)
13: Update  $\mathbf{w}_{t + 1}$  by using SGD, momentum-methods or Adam.
14: end for
Algorithm 3 Stochastic Algorithm for solving AdAP_LPN in (10)
1: Initialize  $\mathbf{w},\mathbf{u}_{\mathbf{x}},\mathbf{u},\gamma_{1},\gamma_{2}$
2: for  $t = 1, \dots, T$  do
3: Draw a batch of  $B_{+}$  positive samples denoted by  $\mathcal{B}_{+}$ .
4: Draw a batch of  $B$  samples denoted by  $\mathcal{B}$
5: for  $\mathbf{x}_i\in \mathcal{B}$  do
6: Initialize  $\delta_i\sim \alpha \cdot \mathcal{N}(0,1)$
7: for  $m = 1,\ldots ,M$  do
8: Update  $\delta_{i} = \Pi_{\| \cdot \| \leq \epsilon}(\delta_{i} + \eta_{2}\cdot sign(\nabla_{\delta_{i}}G(\mathbf{w}_{t},\mathbf{x}_{i} + \delta_{i},y_{i})))$  , where  $\Pi_{\Omega}(\cdot)$  is the projection operator.
9: end for
0: end for
11: For each  $\mathbf{x}_i\in \mathcal{B}_+$ , update  $\mathbf{u}_{\mathbf{x}_i}^1$  and  $\mathbf{u}_{\mathbf{x}_i}^2$  by Equation (2)
12: Update  $\mathbf{u}$  by Equation (11) and compute  $\widehat{\nabla}_{\mathbf{w}_t}R(\mathbf{w}_t,\delta ,\mathcal{D})$  by Equation (12)
13: Compute stochastic gradient estimator:
$$
\nabla_ {\mathbf {w} _ {t}} = \widehat {\nabla} _ {\mathbf {w} _ {t}} P (\mathbf {w} _ {t}) + \lambda \bigl (\widehat {\nabla} _ {\mathbf {w} _ {t}} R (\mathbf {w} _ {t}, \delta , \mathcal {D}) + \frac {1}{B} \sum_ {\mathbf {x} _ {i} \in \mathcal {B}} L \bigl (h _ {\mathbf {w} _ {t}} (\mathbf {x} _ {i}), h _ {\mathbf {w} _ {t}} (\mathbf {x} _ {i} + \delta_ {i}) \bigr) \bigr)
$$
14: Update  $\mathbf{w}_{t + 1}$  by using SGD, momentum-methods or Adam.
15: end for
Algorithm 4 Stochastic Algorithm for solving AdAP_PZ in (7)
1: Initialize  $\mathbf{w},\mathbf{u}_{\mathbf{x}}^{1},\mathbf{u}_{\mathbf{x}}^{2},\gamma_{1}$    
2: for  $t = 1,\dots ,T$  do   
3: Draw a batch of  $B_{+}$  positive samples denoted by  $\mathcal{B}_+$    
4: Draw a batch of  $B$  samples denoted by  $\mathcal{B}$    
5: for  $\mathbf{x}_i\in \mathcal{B}$  do   
6: Initialize  $\delta_i\sim \alpha \cdot \mathcal{N}(0,1)$    
7: for  $m = 1,\ldots ,M$  do   
8: Update  $\delta_{i} = \Pi_{||\cdot ||\leq \epsilon}(\delta_{i} + \eta_{2}\cdot sign(\nabla_{\delta_{i}}L(h_{\mathbf{w}_{t}}(\mathbf{x}_{i}),h_{\mathbf{w}_{t}}(\mathbf{x}_{i} + \delta_{i}))))$  where  $\Pi_{\Omega}(\cdot)$  is the projection operator.   
9: end for   
10: end for   
11: For each  $\mathbf{x}_i\in \mathcal{B}_+$  , update  $\mathbf{u}_{\mathbf{x}_i}^1$  and  $\mathbf{u}_{\mathbf{x}_i}^2$  by Equation (2)   
12: Update u by Equation (11) and compute  $\widehat{\nabla}_{\mathbf{w}_t}P(\mathbf{w}_t)$  by Equation (3)   
13: Compute stochastic gradient estimator   
 $\nabla_{\mathbf{w}_t} = \widehat{\nabla}_{\mathbf{w}_t}P(\mathbf{w}_t) + \frac{\lambda}{B}\sum_{\mathbf{x}_i\in \mathcal{B}}L(h_{\mathbf{w}_t}(\mathbf{x}_i),h_{\mathbf{w}_t}(\mathbf{x}_i + \delta_i))$
Algorithm 5 Stochastic Algorithm for solving AdAP_LZ
1: Initialize  $\mathbf{w},\mathbf{u}_{\mathbf{x}}^{1},\mathbf{u}_{\mathbf{x}}^{2},\mathbf{u},\gamma_{1},\gamma_{2}$    
2: for  $t = 1,\dots ,T$  do   
3: Draw a batch of  $B_{+}$  positive samples denoted by  $\mathcal{B}_+$    
4: Draw a batch of  $B$  samples denoted by  $\mathcal{B}$    
5: For  $\mathbf{x}_i\in \mathcal{B}$  initialize  $\delta_i\sim \alpha \cdot \mathcal{N}(0,1)$    
6: for  $m = 1,\ldots ,M$  do   
7: Update u by Equation (11) and compute  $\widehat{\nabla}_{\delta}R(\mathbf{w}_t,\delta ,\mathcal{D})$  by Equation (19)   
8: Update  $\delta_{i} = \Pi_{\| \cdot \| \leq \epsilon}(\delta_{i} + \eta_{2}\cdot sign(\widehat{\nabla}_{\delta_{i}}R(\mathbf{w}_{t},\delta ,\mathcal{D}))$  , where  $\Pi_{\Omega}(\cdot)$  is the projection operator.   
9: end for   
10: For each  $\mathbf{x}_i\in \mathcal{B}_+$  , update  $\mathbf{u}_{\mathbf{x}_i}^1$  and  $\mathbf{u}_{\mathbf{x}_i}^2$  by Equation (2)   
11: Update u by Equation (11) and compute  $\widehat{\nabla}_{\mathbf{w}_t}R(\mathbf{w}_t,\delta ,\mathcal{D})$  by Equation (12)   
12: Compute stochastic gradient estimator  $\nabla_{\mathbf{w}_t} = \widehat{\nabla}_{\mathbf{w}_t}P(\mathbf{w}_t) + \lambda \widehat{\nabla}_{\mathbf{w}_t}R(\mathbf{w}_t;\delta)$    
13: Update  $\mathrm{w}_{t + 1}$  by using SGD, momentum-methods or Adam.   
14: end for