SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 3: The first figure depicts the difference in output of three models under different local sample densities  $\rho_{(y_i,x_i,r)}$  divided into different bins. The second figure shows the local empirical risk  $R_{(y_i,x_i,r)}$  of samples under different sum of loss and gradient norms (Loss+Gradnorm). For Loss+Gradnorm, we first normalize both variables separately and then add them up to eliminate magnitude differences. The third figure represents the local empirical risk of local sample densities in different values. The fourth figure displays the local density under different Loss+Gradnorms. The neighborhood radius  $r$  is taken as 0.4.
We conduct transfer attack experiments on three baselines on the CIFAR10 dataset, three different models are chosen as victims. Combining our perspectives above, we use algorithm 2 to select the anchor of each target class for guiding the addition of adversarial perturbations. We implement our strategy by simply using squared loss to match anchor points, i.e. minimizing  $\left\| f(x_{i}^{adv}) - a_{\mathrm{target}_{i}} \right\|^{2}$  (choosing  $q = 10$  and  $\epsilon = 16$  pixels). For comparison, we also use cross-entropy (CE) loss to calculate adversarial examples in the vanilla way, as well as using squared error without a screening mechanism (i.e., randomly selecting the target anchor in the target class) to exclude the influence of different losses. The results are shown in Table 1, our strategy indeed helps to enhance the transferability of target attacks, which further confirms our viewpoints.
Algorithm 2 Target Anchor Screening
Require: Early-stopping classifier  $f_{\mathbf{w}}$ , screening parameter  $q$  and sample set  $S$ .
1: for  $i = 1 \gets n$  do
2:  $\mathrm{loss}_i = \ell_{sce}(f(x_i),y_i)$  ,gradnorm  $= \| \nabla_{x}\ell_{sce}(f(x_{i}),y_{i})\|$
3: end for
4: For each class  $k \in [K]$ , select the  $q$ -th smallest loss and the gradient norm among the samples in that class as thresholds  $\mathrm{thr}_k^{\mathrm{loss}}$  and  $\mathrm{thr}_k^{\mathrm{gradnorm}}$
5: for  $k = 1 \gets K$  do
6:  $A_{k}:= \left\{i\in \mathcal{I}_{k}:\mathrm{loss}_{i} <   \mathrm{thr}_{k}^{\mathrm{loss}},\mathrm{gradnorm}_{i} <   \mathrm{thr}_{k}^{\mathrm{gradnorm}}\right\} ,$ $a_{k} = \frac{1}{|A_{k}|}\sum_{j\in A_{k}}f_{\boldsymbol{w}}(x_{j})$
Table 1: Targeted transfer success rates. "left/middle/right" represent attacks using regular CE loss, square loss with randomly selected target anchors, and square loss with target anchor screening, respectively. In parentheses, the clean accuracy is indicated.
Attack	Src:Res34(95.44%)→VGG16 →Dense121	Src:VGG16(94.27%)→Res34 →Dense121	Src:Dense121(95.47%)→Res34 →VGG16
MIM	14.32%/12.94%/14.44%	19.81%/19.00%/20.09%	14.80%/13.78%/15.13%	13.41%/12.11%/13.83%	17.13%/14.17%/17.17%
TIM	13.53%/12.50%/13.88%	15.23%/14.56%/15.71%	12.49%/11.78%/12.88%	11.14%/10.17%/11.40%	15.74%/14.61%/15.89%
DIM	15.46%/14.39%/15.97%	18.10%/17.61%/18.96%	14.56%/13.28%/15.12%	12.97%/12.11%/13.47%	18.11%/16.83%/18.33%
3 Training Strategy of ESMA
We present our training strategy in this section, as shown in Figure 4. Our training strategy is carried out in two steps, the first step we pre-train the generator's embedding representations, and the second step is to find easy samples of each target class based on our previous conclusions, and then use these samples to guide the generator to generate perturbations from the source domain to the target domain.
Pre-trained Embeddings Guided by Latent Features To obtain better embeddings that more effectively represents inter-class information, since the latent space of deep learning models often extracts enough class information [39], we treat the output features of the local model as a set of a priori embedding. In order to make the generator embedding have a similar structure to such a priori embedding, we refer to the idea of manifold learning, using a strategy similar to the SNE algorithm [28]. Let  $\mu_{j} = \frac{1}{|\mathcal{I}_{j}|}\sum_{i\in \mathcal{I}_{j}}l_{i}$ , where  $l_{i} = f(x_{i})$ , and then we design the following manifold
matching loss. Generator embeddings of various class were pulled to the manifold that output features in to obtain embeddings with a better structure.
Next, we denote the generator embedding of class  $j$  as  $e_j$ . The four matrices  $M^{S_{euc}}$ ,  $M^{E_{euc}}$ ,  $M^{S_{cos}}$ ,  $M^{E_{cos}}$  satisfy  $M_{i,j}^{S_{euc}} = \|\mu_i - \mu_j\|$ ,  $M_{i,j}^{E_{euc}} = \|e_i - e_j\|$ ,  $M_{i,j}^{S_{cos}} = \frac{\mu_i \cdot \mu_j}{\|\mu_i\|\|\mu_j\|}$ ,  $M_{i,j}^{E_{cos}} = \frac{e_i \cdot e_j}{\|e_i\|\|e_j\|}$  respectively. Let
$$
\overline {{M}} _ {i, j} ^ {S _ {\mathrm {e u c}}} = \frac {\exp \left(M _ {i , j} ^ {S _ {\mathrm {e u c}}}\right)}{\sum_ {k = 1} ^ {K} \exp \left(M _ {i , k} ^ {S _ {\mathrm {e u c}}}\right)}, \overline {{M}} _ {i, j} ^ {E _ {\mathrm {e u c}}} = \frac {\exp \left(M _ {i , j} ^ {E _ {\mathrm {e u c}}}\right)}{\sum_ {k = 1} ^ {K} \exp \left(M _ {i , k} ^ {E _ {\mathrm {e u c}}}\right)},
$$
$$
\overline {{M}} _ {i, j} ^ {S _ {\mathrm {c o s}}} = \frac {\exp \left(M _ {i , j} ^ {S _ {\mathrm {c o s}}}\right)}{\sum_ {k = 1} ^ {K} \exp \left(M _ {i , k} ^ {S _ {\mathrm {c o s}}}\right)}, \overline {{M}} _ {i, j} ^ {E _ {\mathrm {c o s}}} = \frac {\exp \left(M _ {i , j} ^ {E _ {\mathrm {c o s}}}\right)}{\sum_ {k = 1} ^ {K} \exp \left(M _ {i , k} ^ {E _ {\mathrm {c o s}}}\right)},
$$
then our manifold matching loss is as follows:
Figure 4: Training strategy of ESMA.
$$
\begin{array}{l} \mathcal {L} _ {\mathcal {M}} = \sum_ {i, j} \bar {M} _ {i, j} ^ {S _ {\mathrm {e u c}}} \log \frac {\bar {M} _ {i , j} ^ {S _ {\mathrm {e u c}}}}{\bar {M} _ {i , j} ^ {E _ {\mathrm {e u c}}}} + \sum_ {i, j} \bar {M} _ {i, j} ^ {E _ {\mathrm {e u c}}} \log \frac {\bar {M} _ {i , j} ^ {E _ {\mathrm {e u c}}}}{\bar {M} _ {i , j} ^ {S _ {\mathrm {e u c}}}} \\ + \lambda_ {1} \left[ \sum_ {i, j} \overline {{M}} _ {i, j} ^ {S _ {\cos}} \log \frac {\overline {{M}} _ {i , j} ^ {S _ {\cos}}}{\overline {{M}} _ {i , j} ^ {E _ {\cos}}} + \sum_ {i, j} \overline {{M}} _ {i, j} ^ {E _ {\cos}} \log \frac {\overline {{M}} _ {i , j}}{\overline {{M}} _ {i , j}} \right] + \lambda_ {2} \sum_ {i = 1} ^ {K} \left\| e ^ {i} \right\|. \\ \end{array}
$$
The last regular term is to prevent losses from collapsing.  $\lambda_{1}$  and  $\lambda_{2}$  are hyperparameters. The pretrained embedding with our strategy has a larger Euclidean distance and a smaller cosine similarity, which greatly alleviates the previous clustering phenomenon. More detailed analysis and discussion are provided in Appendix B.
Training of Multi-target Adversarial Perturbation Generators After pre-training embedding, we freeze the parameters of the embedding layer, combined with our previous conclusions, we select several easy samples in each class to form target anchor sets  $A_{i}$ , and match them with the output of our generator in feature space, the generator we use is a Unet with Resblocks. We propose the following objective easy sample feature matching loss to train the multi-class adversarial generator.
$$
\mathcal {L} _ {E M} = \sum_ {j = 1} ^ {K} \frac {1}{\sum_ {i \in [ n ]} \mathbb {1} (i \notin \mathcal {I} _ {j})} \sum_ {i \notin \mathcal {I} _ {j}} d (f (a _ {j}), f (\operatorname {c l i p} _ {\epsilon} (\mathbf {W} * \mathcal {G} _ {\theta} (x _ {i}))))
$$
where  $\mathrm{clip}_{\epsilon}(x) = \mathrm{clip}\left(\min \left(x + \epsilon, \max \left(x, x - \epsilon\right)\right)\right)$ ,  $\mathbf{W}$  is a differentiable Gaussian kernel with size  $3 * 3$ , such smoothing operations have been demonstrated to further improve transferability[24]. The measure of the distance between the two features  $d(\cdot, \cdot)$  is Smooth L1 loss, which has a unique optimal solution that is not sensitive to exceptional values [40]. Then, our final training strategy can be represented by algorithm 3.
Algorithm 3 Training Strategy of ESMA
Require: Generator  $\mathcal{G}_{\theta}$  with pre-trained embeddings, anchors  $a_{k}, k \in [K]$  and Total epochs  $N$ .
1: for  $t = 1 \gets N$  do
2: for  $i = 1\gets n$  do
3: target  $_i\sim$  Uniform  $(\{1,\dots ,K\})$
4: if  $y_{i} \neq \text{target}_{i}$  then
5: Random choice an anchor  $a_{j}$  from  $A_{j}$
6: Gradient descent step on  $\mathcal{L}_{EM}$
7: end if
8: end for
9: end for
In this section, we verify the effectiveness of our method through experiments on ILSVRC2012 dataset [41]. To evaluate the effectiveness of different components in our strategy, we conduct ablation experiments in Section 4.3. Other ablation experiments can be found in Appendix C.
Dataset The dataset we used comes from the ILSVRC2012 dataset [41], in which we selected ten classes as the training set, which refer to the ten classes used for TTP training in [24], they are 24, 99, 198, 245, 344, 471, 661, 701, 802, 919. We train generators using images of these ten classes in the training set (1300 images per class), and use the images in the validation set (50 images per class) as the validation dataset for the targeted attack.
Models We use the four networks used in [24] as source models —ResNet-50 [42] (Res50), VGG19-bn [43] (VGG19bn), DenseNet-121 [44] (Dense121), ResNet-152 [42] (Res152). Except the above four models, we also select three models from the Inception series: Inception-v3 [45] (Inc-v3), Inception-v4 [46] (Inc-v4), Inception-ResNet-v2 [46] (IncRes-v2) and a transformer vision model, VIT [47]. As the analysis and fundamental assumptions of this paper are based on the condition of training data from the same distribution, in order to explore the transferability between models trained on training data with different distribution, we conduct additional transfer attack on two
adversarily trained models, Inc-v3-adv [48] and IncRes-v2-ens [49]. Results are shown in E.2. In addition, we also test the adversarial transferability of ESMA on the scenario where the source model is an ensemble of different models, corresponded results are reported in E.1.
Baselines We select many iterative instance-specific attack benchmarks, MIM [12], SI-NIM [13], TIM [14], DIM [16], and advanced iterative instance-specific attacks that are competitive in target setting, Po-TI-Trip [20], Logit [21], Rap-LS [23], FGS $^2$ M [50], DMTI-Logit-SU [51], S $^2$ I-SI-Ti-DIM [52]. Generative adversarial attacks HGN [53] and TTP [24] also included in our comparison, where TTP is the current SOTA generative method. As a generative attack, TTP requires training a class-dependent generator specifically for each class.
Parameter Setting For the parameter settings of different attack methods, we refer to the default settings in [20], total iteration number  $T = 20$ , step size  $\alpha = \epsilon / T$ , where the  $\ell_{\infty}$  perturbation restriction  $\epsilon$  is set to 16. Momentum factors  $\mu$  is set to 1, for the stochastic input diversity in DIM, we set the probability of applying input diversity as 0.7. For TIM, the kernel-length is set to 7, which is more suitable for targeted attacks. For Po-TI-Trip, the weight of triplet loss  $\lambda$  is set to 0.01, while the margin  $\gamma$  is set to 0.007. Referring to [21], the number of iteration steps of Logit is set to 300, and for RAP-LS, we choose 400 iteration steps,  $K_{LS}$  is set to 100, and  $\epsilon_{n}$  is set to 12/255 [23]. Then for TTP, since we used a relatively small training set, we added 10 epochs to the original paper [24] settings to ensure the performance of the model, and the learning rate of Adam optimizer is  $1e - 4$  ( $\beta_{1} = .5$ ,  $\beta_{2} = .999$ ). Finally, for our model, we used the AdamW optimizer to train 300 epochs with a learning rate of  $1e - 4$  (350 for cases where the source model is VGG19bn or Dense121), the value of  $q$  used for sample screening is set to 2.
Evaluation Setting We train the generator on the training set of the selected ten classes, verify the targeted transferality on the validation set, for each target class, we use the 450 images of the remaining classes as the source data and perturb them to the target class. For the iterative instance-specific attacks, we directly attack these instances and test their targeted transfer success rate. All methods (including training) were implemented on a single NVIDIA RTX A5000 GPU.
The results of our experiments, reported in Table 3, ESMA outperforms current SOTA generative attack TTP in targeted transfer success rates. Our approach demonstrates significant advantages in terms of efficiency and effectiveness, with TTP having a parameter count of  $7.84\mathrm{M}$  per model compared to ESMA's  $4.40\mathrm{M}$ . Additionally, ESMA achieves an average training time that is  $78.4\%$  of TTP. The experimental results also strongly supports our viewpoint.
To validate the effectiveness of the two components in our strategy design, we compared the cases with (w/) and without (w/o) pretrained embeddings and target anchor screening. The results are shown in Figure 5. In the case without pre-trained embeddings, the performance is significantly weaker than the case with pre-trained embeddings across different training durations. Moreover, the case with target anchor screening shows a noticeable improvement compared to randomly selecting target anchors.
To verify that ESMA can indeed increase the sample density of the target class for the original samples, we use the samples used for adversarial testing in table 3. First, we calculate the sample density
of the target class for the clean samples  $\rho_{(\mathrm{target},x_i,r)}$  (here  $r$  is set to 600). Then, we apply ESMA to generate adversarial samples  $x_{i}^{adv}$ , and calculate the sample density of the target class for these adversarial samples  $\rho_{(\mathrm{target},x_i^{adv},r)}$ . In both cases, the density calculation results were averaged across all target classes for each sample. We normalize the results by dividing them by the maximum value among all results in both cases. Additionally, we further bin and count the number of test samples in different intervals under different perturbation constraints. The results are shown in Table 2. The
Figure 5: Comparison of targeted attack transfer success rates with (w) pre-trained embeddings and without (w/o) pre-trained embeddings at different training epochs. Src:Res50.
target class sample density of the samples perturbed using ESMA has increased in bins with larger values, which confirms that ESMA can indeed enhance the target class sample density.
Table 2: Comparison of sample density (binned).
|Counts|0-0.1	0.1-0.2	0.2-0.3	0.3-0.4	0.4-0.5	0.5-0.6	0.6-0.7	0.7-0.8	0.8-0.9	0.9-1.0
Case for ε = 8/255:	|Clean|32	35	31	45	39	51	57	51	61	48
|ESMA|27	30	32	37	43	49	62	51	70	49
Case for ε = 16/255:	|Clean|32	35	31	45	39	51	57	51	61	48
|ESMA|25	27	36	30	49	50	59	54	71	49
In this work, we provided a novel perspective on the transferability of targeted adversarial attacks. The study theoretically and experimentally demonstrated that adding perturbations towards HSDR of the target class can further enhance the transferability of targeted attacks. We also proposed a method for identifying samples within HSDR, which avoided the impracticality of density estimation in high-dimensional data. Building upon these insights, we introduced an improved generative multi-target attack strategy ESMA, which surpassed previous generative targeted attacks in both effectiveness and efficiency. We believe that our insights not only provide guidance for targeted attacks but also offer insights for dataset selection. Further discussion on these points would be included in our future work.
[47] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
[48] Florian Tramér, Alexey Kurakin, Nicolas Papernot, Ian J. Goodfellow, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
[49] Alexey Kurakin, Ian J. Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, Alan L. Yuille, Sangxia Huang, Yao Zhao, Yuzhe Zhao, Zhonglin Han, Junjiajia Long, Yerkebulan Berdibekov, Takuya Akiba, Seiya Tokui, and Motoki Abe. Adversarial attacks and defences competition. CoRR, abs/1804.00097, 2018.
[50] Lianli Gao, Qilong Zhang, Xiaosu Zhu, Jingkuan Song, and Heng Tao Shen. Staircase sign method for boosting adversarial attacks. CoRR, abs/2104.09722, 2021.
[51] Zhipeng Wei, Jingjing Chen, Zuxuan Wu, and Yu-Gang Jiang. Enhancing the self-universality for transferable targeted attacks. In CVPR, 2023.
[52] Yuyang Long, Qilong Zhang, Boheng Zeng, Lianli Gao, Xianglong Liu, Jian Zhang, and Jingkuan Song. Frequency domain model augmentation for adversarial attack. In ECCV, 2022.
[53] Xiao Yang, Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Boosting transferability of targeted adversarial examples via hierarchical generative networks. In Shai Avidan, Gabriel J. Brostow, Moustapha Cisse, Giovanni Maria Farinella, and Tal Hassner, editors, ECCV, 2022.
[54] Dylan J. Foster and Alexander Rakhlin.  $\ell_{\infty}$  vector contraction for rademacher complexity. CoRR, abs/1911.06468, 2019.
[55] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning. MIT Press, 2012.
[56] Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning - From Theory to Algorithms. Cambridge University Press, 2014.
[57] Yunwen Lei, Ürün Dogan, Ding-Xuan Zhou, and Marius Kloft. Data-dependent generalization bounds for multi-class classification. IEEE Trans. Inf. Theory, 65(5):2995-3021, 2019.
[58] Yong Liu. Refined learning bounds for kernel and approximate  $k$ -means. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, 2021.
[59] Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, 1963.
Supplementary Material:
First, we need the definition below:
Definition 2  $((j,k,x_0,r)$ -Local Output Rademacher Complexity) Let  $f_{\boldsymbol{w}}^{\mathcal{S}} = (f_{\boldsymbol{w}}^{\mathcal{S}_1},\dots ,f_{\boldsymbol{w}}^{\mathcal{S}_K})\in [0,1]^K$ $\mathcal{F}_{\boldsymbol{w}}^{S_k}:= \{f_{\boldsymbol{w}}^{S_k}:\boldsymbol {w}\in \mathcal{W}\}$ , given  $\mathcal{B}(x_0,r) = \{x:\| x - x_0\| \leq r\}$ ,  $(x_0,y_0)\in S_j$ , then the empirical  $(j,k,x_0,r)$ -local output Rademacher Complexity is defined by:
$$
\mathcal {R} _ {n} ^ {(j, x _ {0}, r)} \left(\mathcal {F} _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}}\right) := \mathbb {E} _ {\boldsymbol {\sigma}} \left[ \sup  _ {f _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}} \in \mathcal {F} _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}}} \left| \sum_ {i \in \mathcal {I} _ {(j, x _ {0}, r)}} \sigma_ {i} f _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}} \left(x _ {i}\right) \right| \right],
$$
where  $\sigma_{i}, i \in \mathcal{I}_{(j,x_{0},r)}$  are i.i.d random variables satisfying  $\mathbb{P}(\sigma_i = 1) = \mathbb{P}(\sigma_i = -1) = \frac{1}{2}$ . Correspondingly, the expected  $(j,k,x_0,r)$ -local output Rademacher complexity is defined as  $\mathcal{R}^{(j,x_0,r)}(\mathcal{F}_{\boldsymbol{w}}^{S_k}) = \mathbb{E}_{S_{(j,x_0,r)}}\left[\mathcal{R}_n^{(j,x_0,r)}(\mathcal{F}_{\boldsymbol{w}}^{S_k}) \mid x_i \in \mathcal{C}_{(j,x_0,r)}\right]$ , where  $S_{(j,x_0,r)} = \{(x,y) \in S_j : x \in \mathcal{B}(x_0,r), (x_0,y_0) \in S\}$ .
We first prove the lemma 1 and use it to prove theorem 1, and we need lemma 2 and 3 before proving the lemma 1.
Lemma 1 Given  $\mathcal{B}(x_0,r)$ ,  $(x_0,y_0) \in S_j$ ,  $S_{(j,x_0,r)}$ , and  $f_{\boldsymbol{w}}^{\mathcal{S}_k} \in \mathcal{F}_{\boldsymbol{w}}^{\mathcal{S}_k}$ , then with probability at least  $1 - \delta$ , the following holds for any  $f_{\boldsymbol{w}}^{\mathcal{S}_k} \in \mathcal{F}_{\boldsymbol{w}}^{\mathcal{S}_k}$  and  $k \in [K]$ :
$$
\begin{array}{l} \left| \mathbb {E} _ {x \sim \mathcal {D} _ {x \mid y}} \left[ f _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}} (x) \mid x \in \mathcal {C} _ {(j, x _ {0}, r)} \right] - \hat {\mathbb {E}} _ {\mathcal {S} _ {(j, x _ {0}, r)}} \left[ f _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}} \right] \right| \\ \leq 4 C ^ {\boldsymbol {w}} \sqrt {\frac {K}{\left| \mathcal {I} _ {(j , x _ {0} , r)} \right|}} \log^ {2} \left(\frac {\sqrt {2 \left| \mathcal {I} _ {(j , x _ {0} , r)} \right|}}{b ^ {\boldsymbol {w}}}\right) + 3 \sqrt {\frac {\log (2 / \delta)}{2 \left| \mathcal {I} _ {(j , x _ {0} , r)} \right|}}, \\ \end{array}
$$
where  $K$  is the number of classes,  $C^{\pmb{w}}$  and  $b^{\pmb{w}}$  are constants.
Lemma 2 ( $l_{\infty}$  Contraction Inequality [54]) Let  $\mathcal{F} \subseteq \{f : \mathcal{X} \to \mathbb{R}^K\}$ , and let  $\phi : \mathbb{R}^K \to \mathbb{R}$  be L-Lipschitz with respect to the  $l_{\infty}$  norm, i.e.  $\| \phi(v) - \phi(v') \|_{\infty} \leq L \cdot \| v - v' \|_{\infty}, \forall v, v' \in \mathbb{R}^K$ . For any  $a > 0$ , there exists a constant  $C > 0$  such that if  $|\phi(f(x))| \vee \| f(x) \|_{\infty} \leq \zeta$ , then
$$
\mathcal {R} _ {n} (\phi \circ \mathcal {F}) \leq C \cdot L \sqrt {K} \max _ {i} \tilde {\mathcal {R}} _ {n} \left(\mathcal {F} _ {i}\right) \log^ {\frac {3}{2} + a} \left(\frac {\zeta n}{\max _ {i} \tilde {\mathcal {R}} _ {n} \left(\mathcal {F} _ {i}\right)}\right),
$$
where  $\mathcal{R}_n(\phi \circ \mathcal{F}) = \mathbb{E}_{\pmb{\sigma}}\left[\sup_{f\in \mathcal{F}}|\sum_{i = 1}^n\sigma_i\phi (f(\mathbf{x}_i))|\right]$ ,  $\tilde{\mathcal{R}}_n(\mathcal{F}_i) = \sup_{S\in \mathcal{X}^n}\mathcal{R}_n(\mathcal{F}_i)$ .
Lemma 3 Given  $\mathcal{B}(x_0,r)$ ,  $(x_0,y_0) \in S_j$ ,  $S_{(j,x_0,r)}$  and  $\mathcal{F}_{\boldsymbol{w}}^{S_k}$ , with probability at least  $1 - \delta$ , holds:
$$
\mathcal {R} ^ {(j, x _ {0}, r)} \left(\mathcal {F} _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}}\right) \leq \mathcal {R} _ {n} ^ {(j, x _ {0}, r)} \left(\mathcal {F} _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}}\right) + \sqrt {\frac {\left| \mathcal {I} _ {(j , x _ {0} , r)} \right| \log (1 / \delta)}{2}}.
$$
Proof of Lemma 3 Let  $S_{(j,x_0,r)}'$  be a dataset that has at most one element different with  $S_{(j,x_0,r)}$ , its elements are  $x_i', i \in \mathcal{I}_{(j,x_0,r)}$ , then
$$
\begin{array}{l} \mathbb {E} _ {\boldsymbol {\sigma}} \left[ \sup  _ {f _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}} \in \mathcal {F} _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}}} \left| \sum_ {i \in \mathcal {I} _ {(j, x _ {0}, r)}} \sigma_ {i} f _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}} (x _ {i}) \right| \right] - \mathbb {E} _ {\boldsymbol {\sigma}} \left[ \sup  _ {f _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}} \in \mathcal {F} _ {\boldsymbol {w}} ^ {\mathcal {S} _ {k}}} \left| \sum_ {i \in \mathcal {I} _ {(j, x _ {0}, r)}} \sigma_ {i} f ^ {\mathcal {S} _ {k}} (x _ {i} ^ {\prime}) \right| \right] \\ \leq \mathbb {E} _ {\boldsymbol {\sigma}} \left[ \sup  _ {f _ {\boldsymbol {w}} ^ {S _ {k}} \in \mathcal {F} _ {\boldsymbol {w}} ^ {S _ {k}}} \left[ \left| \sum_ {i \in \mathcal {I} _ {(j, x _ {0}, r)}} \sigma_ {i} f _ {\boldsymbol {w}} ^ {S _ {k}} (x _ {i}) \right| - \left| \sum_ {i \in \mathcal {I} _ {(j, x _ {0}, r)}} \sigma_ {i} f _ {\boldsymbol {w}} ^ {S _ {k}} \left(x _ {i} ^ {\prime}\right) \right| \right] \right] \\ \leq \mathbb {E} _ {\boldsymbol {\sigma}} \left[ \sup  _ {f _ {\boldsymbol {w}} ^ {S _ {k}} \in \mathcal {F} _ {\boldsymbol {w}} ^ {S _ {k}}} \left| \sum_ {i \in \mathcal {I} _ {(j, x _ {0}, r)}} \sigma_ {i} \left(f _ {\boldsymbol {w}} ^ {S _ {k}} \left(x _ {i}\right) - f _ {\boldsymbol {w}} ^ {S _ {k}} \left(x _ {i} ^ {\prime}\right)\right) \right| \right] \leq 1, \\ \end{array}
$$
the last inequality is because  $S_{(j,x_0,r)}'$  differs from  $S_{(j,x_0,r)}$  with at most one element, and  $f_{\boldsymbol{w}}^{S_k} \in [0,1]$ . Then use McDiarmid's inequality ([55], Theorem D.3), we have