SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lianghe Shi Weiwei Liu*
School of Computer Science, Wuhan University
National Engineering Research Center for Multimedia Software, Wuhan University
Institute of Artificial Intelligence, Wuhan University
Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University
Gradual Domain Adaptation (GDA), in which the learner is provided with additional intermediate domains, has been theoretically and empirically studied in many contexts. Despite its vital role in security-critical scenarios, the adversarial robustness of the GDA model remains unexplored. In this paper, we adopt the effective gradual self-training method and replace vanilla self-training with adversarial self-training (AST). AST first predicts labels on the unlabeled data and then adversarially trains the model on the pseudo-labeled distribution. Intriguingly, we find that gradual AST improves not only adversarial accuracy but also clean accuracy on the target domain. We reveal that this is because adversarial training (AT) performs better than standard training when the pseudo-labels contain a portion of incorrect labels. Accordingly, we first present the generalization error bounds for gradual AST in a multiclass classification setting. We then use the optimal value of the Subset Sum Problem to bridge the standard error on a real distribution and the adversarial error on a pseudo-labeled distribution. The result indicates that AT may obtain a tighter bound than standard training on data with incorrect pseudo-labels. We further present an example of a conditional Gaussian distribution to provide more insights into why gradual AST can improve the clean accuracy for GDA.
The key assumption of classical machine learning—that training and test data come from the same distribution—may not always hold in many real-world applications [15]. A data distribution typically evolves due to changes in conditions: for example, changing weather in vehicle identification [14], sensor aging in sensor measurement [43], the evolution of road conditions in self-driving [6], etc. To address this problem, Unsupervised Domain Adaptation (UDA) has been developed to train a model that performs well on an unlabeled target domain by leveraging labeled data from a similar yet distinct source domain.
Various works in UDA [5, 53, 34] theoretically demonstrate that the generalization error can be controlled by the domain discrepancy. Hence, the domain shift between the source domain and the target domain is expected to be small [15]. However, in some applications, the domain shift is substantial, leading to a sharp drop in the performance of the UDA method [26]. Furthermore, since changes in real-world data are more often gradual than abrupt [16], there are many intermediate domains between the source domain and the target domain. To get a better solution to the gradually shifting data, some recent works have focused on the Gradual Domain Adaptation (GDA) problem, where the learner is additionally provided with unlabeled intermediate domains. The large gap between the source and target domains is then divided up into multiple small shifts between the
*Correspondence to: Weiwei Liu <liuweiwei863@gmail.com>.
intermediate domains. Recently, the celebrated work [26] proposes gradual self-training, which iteratively applies the self-training method to adapt the model along the intermediate domains. Empirically, the gradual self-training method greatly improves the target domain's accuracy compared to the traditional direct domain adaptation. After that, remarkable theoretical [45] and algorithmic [10, 1] advances have been achieved in GDA. However, the adversarial robustness of the GDA model remains unexplored, despite its vital role in security-critical scenarios. Adversarial robustness refers to the invariance of a model to small perturbations of its input [39], while adversarial accuracy refers to a model's prediction accuracy on adversarial examples generated by an attacker. Numerous works [33, 41, 49] have shown that very small perturbations, even those imperceptible to humans, can successfully deceive deep neural network (DNN) models, resulting in erroneous predictions.
One popular and effective method of improving robustness is Adversarial Training (AT), which adds adversarial examples to the training data. Due to the lack of labels in the target domain, it is difficult to directly generate adversarial examples. Accordingly, this paper replaces self-training with Adversarial Self-Training (AST), which first generates pseudo-labels on the unlabeled data and then adversarially trains the model based on these pseudo-labeled data. We conduct comprehensive experiments to validate the effectiveness of gradual AST in Section 3. Compared to the gradual self-training method, the proposed gradual AST method delivers a great improvement in adversarial accuracy, from  $6.00\%$  to  $90.44\%$  on the Rotating MNIST dataset. More interestingly, we find that the clean accuracy of gradual AST on the target domain also increases from  $90.06\%$  to  $97.15\%$ . This is a surprising result, since many prior works [52, 38] demonstrate that AT may hurt generalization; in other words, there is a trade-off between clean accuracy and adversarial accuracy. We then empirically investigate the reason why AST improves clean accuracy in GDA and find that the adversarially trained model has better clean accuracy than the standardly trained model when the generated pseudo-labels contain a proportion of incorrect labels.
In Section 4, we present novel generalization error bounds for the proposed gradual AST. The results show that if we have a model  $f$  with low adversarial margin loss on the source domain and the distribution shifts slightly, gradual AST can generate a new model  $f'$  that is also adversarially robust on the target domain. To explain why gradual AST achieves better clean accuracy than vanilla gradual self-training, we use the optimal value of the Subset Sum Problem [3] to bridge the standard error on a real distribution and the adversarial error on a pseudo-labeled distribution. The result shows that AT may yield a tighter generalization guarantee than standard training on pseudo-labeled training data. We further provide an example of a conditional Gaussian distribution to illustrate why AT outperforms standard training when a proportion of the labels in the training data are incorrect.
We summarize our contributions as below:
- We are the first to apply the AST method in GDA. The proposed gradual AST method can not only improve the adversarial robustness but also the clean accuracy for GDA, which is an appealing and nontrivial result, considering that many prior works demonstrate that adversarial training may hurt generalization.
- We empirically explore the reason for the improvements in clean accuracy and find that adversarial training performs better than standard training when the pseudo-labeled training set contains a proportion of incorrect labels.
- From the theoretical perspective, we provide the generalization error bounds for gradual AST, which explain why the trained model is adversarially robust on the target domain. We provide an error bound and a toy example of Gaussian distribution to provide some intuitions for the improvements in clean accuracy.
2.1 Multiclass Classification Learning Framework
Let  $\mathcal{Z} = \mathcal{X}\times \mathcal{Y}$  be a measurable instance space, where  $\mathcal{X}$  and  $\mathcal{Y}$  denote the input space and label space, respectively. The input space  $\mathcal{X}$  is a subset of a  $d$ -dimensional space,  $\mathcal{X}\subseteq \mathbb{R}^d$ . In this work, we focus on multiclass classification, and the label space  $\mathcal{V}$  is  $\{1,\ldots ,k\}$ , where  $k$  is the number of classes. Following the notations used in [5], a domain is envisioned as a tuple  $\langle P,h_P\rangle$ , consisting of a distribution  $P$  on input space  $\mathcal{X}$  and a labeling function  $h_P:\mathcal{X}\to \mathcal{Y}$ . In practice, the true distribution  $P$  is unknown to the learner, which has access only to the training data  $S$  drawn
independent and identically distributed (i.i.d.) according to the true distribution  $P$ . We use  $\widehat{P}$  to denote the empirical distribution of  $P$  according to the training data  $S$ . Moreover, following the notations used in [35], we consider the class  $\mathcal{F} = \{f : \mathcal{X} \to \mathbb{R}^k\}$  of scoring functions  $f$ . We use  $f_{y}(x)$  to indicate the output of  $f$  associated with the data point  $x$  on the  $y$ -th dimension; the output indicates the confidence of the prediction. The label with the largest score is the predicted label of  $x$ . We use  $\mathcal{H}_{\mathcal{F}} = \{h_{f}(\cdot) = \arg \max_{y \in \mathcal{Y}} f_{y}(\cdot) : f \in \mathcal{F}\}$  to denote the labeling function class induced by  $\mathcal{F}$ . The expected risk and the empirical risk of a classifier  $h \in \mathcal{H}$  with respect to a labeling function  $h'$  on distribution  $P$  are defined as  $R_{P}(h, h') \triangleq \underset{x \sim P}{\mathbb{E}} \mathbb{1}[h(x) \neq h'(x)]$  and  $R_{\widehat{P}}(h, h') \triangleq \underset{x \sim \widehat{P}}{\mathbb{E}} \mathbb{1}[h(x) \neq h'(x)]$ , respectively, where  $\mathbb{1}$  is the indicator function. We use  $R_{P}(h)$  and  $R_{\widehat{P}}(h)$  to abbreviate  $R_{P}(h, h_{P})$  and  $R_{\widehat{P}}(h, h_{P})$  respectively. Furthermore, the 0-1 loss is non-differentiable and cannot be minimized directly. Thus, a margin theory for multiclass classification was developed by [25], that replaces the 0-1 loss with the margin loss. The margin of a scoring function  $f \in \mathcal{F}$  for labeled data  $(x, y)$  is defined as  $\rho_{f}(x, y) \triangleq f_{y}(x) - \max_{y' \neq y} f_{y'}(x)$ . And the expected margin loss of a scoring function  $f \in \mathcal{F}$  with respect to another scoring function  $f' \in \mathcal{F}$  on distribution  $P$  is defined as  $R_{P}^{(\rho)}(f, f') \triangleq \underset{x \sim P}{\mathbb{E}} \Theta_{\rho} \circ \rho_{f}(x, h_{f'}(x))$ , where  $\Theta_{\rho}(m) = \min\{1, \max\{0, 1 - m / \rho\}\}$  is the ramp loss. Similarly, we use  $R_{P}^{(\rho)}(f)$  and  $R_{\widehat{P}}^{(\rho)}(f)$  to abbreviate  $R_{P}^{(\rho)}(f, f_{P})$  and  $R_{\widehat{P}}^{(\rho)}(f, f_{P})$  respectively. Note that  $R_{P}(h_{f}, h_{f'}) \leq R_{P}^{(\rho)}(f, f')$ , since the 0-1 loss is upper bounded by the margin loss.
2.2 Gradual Domain Adaptation
Under the standard GDA settings [26, 45], the learner is sequentially trained on  $T + 1$  domains with gradual shifts. The corresponding data distributions are  $P_0, P_1, \ldots, P_T$ , where  $P_0$  is the distribution of the source domain,  $P_T$  is the distribution of the target domain, and  $P_1, \ldots, P_{T-1}$  are the distributions of the intermediate domains. For simplicity, we assume that the number of data points in each domain is the same: namely, each domain  $t$  has a set of  $n$  data drawn i.i.d. from  $P_t$ , denoted as  $S_t$ . Recently, [26] proposes a gradual self-training method for GDA, which successively applies self-training to each of the intermediate domains. The self-training method adapts a pre-trained model trained on the previous domain to the current domain using unlabeled data drawn from the current distribution. Specifically, given a pre-trained model  $f$  and an unlabeled data set  $S$ , the model first predicts the labels of the unlabeled data, then trains  $f$  with empirical risk minimization (ERM) over these pseudo-labeled data, i.e.,
$$
f^{\prime} = ST(f,S) = \operatorname *{arg  min}_{f^{\prime}\in \mathcal{F}}R_{\widehat{P}}^{(\rho)}(f^{\prime},f) = \operatorname *{arg  min}_{f^{\prime}\in \mathcal{F}}\frac{1}{n}\sum_{x_{i}\in S}\Theta_{\rho}\circ \rho_{f^{\prime}}(x_{i},h_{f}(x_{i})).
$$
In gradual self-training, the model is first pre-trained on the labeled source domain and then successively self-trained on the unlabeled dataset of each of the intermediate domains, i.e.,
$$
f _ {t} = S T \left(f _ {t - 1}, S _ {t}\right), t \in \{1, \dots , T \}.
$$
Since the domain shifts between the consecutive domains are assumed to be small [10, 26, 45], the accuracy of the pseudo-labels is expected to be high at each step. The gradual self-training method aims to output a final trained classifier  $h_{f_T}$  with low expected risk in the target domain.
2.3 Adversarial Self-Training
Given a classifier  $h_f$  and a data point  $(x, y)$ , we can generate the corresponding adversarial example  $(x^{adv}, y)$  by adversarially perturbing  $x$  in a small neighborhood  $B_{\epsilon}(x)$  of  $x$ , as follows:  $x^{adv} = \arg \max_{x' \in B_{\epsilon}(x)} \Theta_{\rho} \circ \rho_{f}(x', y)$ . In this paper, we focus on the  $\ell_q$  adversarial perturbation  $B_{\epsilon}(x) \triangleq \{x' \in \mathcal{X} : \| x' - x \|_q \leq \epsilon, q \geq 1\}$ , which is referred to as  $\ell_q$ -attack in the AT context and has been widely studied in existing works. Given a vector  $x \in \mathbb{R}^d$ , we define the  $\ell_q$ -norm of  $x$  as  $\| x \|_q \triangleq \left( \sum_{i=1}^d |x_i|^q \right)^{1/q}$ , for  $q \in [1, \infty)$ , where  $x_{(i)}$  is the  $i$ -th dimension of  $x$ ; for  $q = \infty$ , we define  $\| x \|_\infty \triangleq \max_{1 \leq i \leq d} |x_i|$ . Similar to the definition of the standard risk  $R_P(h, h')$ , we
define the adversarial risk on distribution  $P$  for any two classifiers  $h, h'$  as follows:  $\widetilde{R}_P(h, h') \triangleq \mathbb{E} \max_{x \sim P\|\delta\|_q \leq \epsilon} \mathbb{1}[h(x + \delta) \neq h'(x)]$ . We also define the expected adversarial margin loss for any two scoring functions  $f, f'$  on distribution  $P$  as follows:
$$
\widetilde {R} _ {P} ^ {(\rho)} \left(f, f ^ {\prime}\right) \triangleq \underset {x \sim P \| \delta \| _ {q} \leq \epsilon} {\mathbb {E}} \max  \Theta_ {\rho} \circ \rho_ {f} (x + \delta , h _ {f ^ {\prime}} (x)). \tag {1}
$$
Although the expected adversarial margin loss is asymmetrical, it satisfies the triangle inequality (see proof in Appendix A.1). The empirical adversarial risk and the empirical adversarial margin loss can be defined similarly. We use  $\widetilde{R}_P^{(\rho)}(f)$  and  $\widetilde{R}_{\widehat{P}}^{(\rho)}(f)$  to abbreviate  $\widetilde{R}_P^{(\rho)}(f,f_P)$  and  $\widetilde{R}_{\widehat{P}}^{(\rho)}(f,f_P)$  respectively. To train a robust model with resistance to the adversarial examples, AT aims to find a classifier  $h_f$  that yields the minimal adversarial margin loss, i.e.,  $\arg \min_{f\in \mathcal{F}}\widetilde{R}_{\widehat{P}}^{(\rho)}(f)$ . However, we cannot adversarially train the model directly on the unlabeled target domain because the conventional AT method requires labeled data. Instead, we adapt the self-training method to the adversarial self-training (AST) method by generating adversarial examples over the pseudo-labels, i.e.,
$$
f ^ {\prime} = A S T (f, S) = \underset {f ^ {\prime} \in \mathcal {F}} {\arg \min} \widetilde {R} _ {\widehat {P}} ^ {(\rho)} (f ^ {\prime}, f) = \underset {f ^ {\prime} \in \mathcal {F}} {\arg \min} \frac {1}{n} \sum_ {x _ {i} \in S} \max _ {\| \delta_ {i} \| _ {q} \leq \epsilon} \Theta_ {\rho} \circ \rho_ {f ^ {\prime}} (x _ {i} + \delta_ {i}, h _ {f} (x _ {i})).
$$
3 Empirical Exploration
In this section, we propose a gradual AST method to improve adversarial robustness. Surprisingly, we find that the gradual AST improves not only adversarial robustness but also clean accuracy. Our code is available at https://github.com/whustone007/AST_GDA.
Datasets. We run experiments on two datasets that are widely used in GDA [26, 45]. Rotating MNIST is a semi-synthetic dataset generated by rotating each MNIST image at an angle between 0 and 60 degrees. The 50,000 training set images are divided into three parts: a source domain of 5000 images (0-5 degrees), 21 intermediate domains of 42,000 images (5-60 degrees), and a set of validation data. The rotating degree gradually increases along the domain sequence. The 10,000 test set images are rotated by 55-60 degrees, representing the target domain. Portraits [18] is a real dataset consisting portraits of American high school students across a century. The model aims to classify gender. The dataset is split into a source domain of the first 2000 images, seven intermediate domains of the next 14,000 images, and a target domain of the next 2000 images.
Methods. The standard gradual self-training method [26] successively adapts the model to the next domain via self-training. To improve the adversarial robustness, we replace the vanilla self-training with AST. We implement multiple gradual AST methods with different starting domains  $\tau \in \{0,\dots ,T + 1\}$  where we begin to use AST, i.e.,
$$
f _ {t} = \left\{ \begin{array}{c c} S T (f _ {t - 1}, S _ {t}), & 1 \leq t \leq \tau - 1 \\ A S T (f _ {t - 1}, S _ {t}), & \tau \leq t \leq T \end{array} \right.
$$
In particular, if  $\tau = 0$ , then the model is adversarially trained throughout; if  $\tau = T + 1$ , then the model is standardly trained throughout. Note that  $T = 21$  in Rotating MNIST and  $T = 7$  in Portraits.
Implementation Details. We implement our methods using PyTorch [37] on two Nvidia GeForce RTX 3090 Ti GPUs. Following [26], we use a 3-layer convolutional network with dropout (0.5) and BatchNorm on the last layer. We use mini-batch stochastic gradient descent (SGD) with momentum 0.9 and weight decay 0.001. The batch size is 32 and the learning rate is 0.001. We train the model for 40 epochs in each domain. We set the radius of the bounded perturbation to be 0.1 [51] for Rotating MNIST and 0.031 for Portraits. Following [33], we use PGD-20 with a single step size of 0.01 as the adversarial attacker.
3.1 Results
The results of the different methods on Rotating MNIST are shown in Figure 1(a). The numbers in the abscissa represent different starting domains  $\tau$  of various methods. From Figure 1(a), we can see that the vanilla gradual self-training method  $(\tau = 22)$  achieves a clean accuracy of  $90.06\%$  and adversarial accuracy of  $6.00\%$  on the target domain. Notably, the gradual AST method that uses AST
throughout  $(\tau = 0)$  improves the adversarial accuracy to  $83.54\%$ $(+77.54\%)$ . More interestingly, we find that the clean accuracy is also slightly improved by  $0.53\%$ . Furthermore, the results of varying the starting domains  $\tau$  indicate that the performance will be further improved if we use the vanilla self-training method for the first few domains and then use AST from an intermediate domain. We utilize the validation set of the target domain to choose the optimal starting time  $\tau$ . Experimentally, the best choice of starting time for Rotating MNIST is  $\tau = 9$ , which achieves clean accuracy of  $96.66\%$ $(+6.6\%)$  and adversarial accuracy of  $88.86\%$ $(+82.86\%)$ . We find that AST not only improves adversarial robustness but also clean accuracy with a large step. The results of the different methods on Portraits are shown in Figure 1(b). We find similar results to those on Rotating MNIST: the vanilla gradual self-training method  $(\tau = 22)$  achieves a clean accuracy of  $82.03\%$  and adversarial accuracy of  $40.23\%$ , while the gradual AST method  $(\tau = 0)$  improves the clean accuracy by  $2.73\%$  and the adversarial accuracy by  $37.40\%$ . We provide more experimental results of different  $\epsilon$ , domain numbers, and neural networks in Appendix B.
3.2 Why Does Gradual AST Improve Clean Accuracy in GDA?
It is widely believed that AT would hurt generalization [52, 38] and decrease clean accuracy; thus it is surprising that the proposed gradual AST method improves the clean accuracy of the trained model in the GDA setting. In this section, we speculate on the reasons for this. Recently, some works [39, 42] have revealed that an adversariably pre-trained model tends to improve the target model's accuracy in transfer learning. We accordingly investigate whether the adversariably trained model has better clean accuracy on the next domain and thus generates more accurate pseudo-labels. However, by comparing the gradual AST methods with  $\tau = 0$  and  $\tau = 1$  on Rotating MNIST, we find that the model standardly trained on the source domain achieves a clean accuracy of  $97.30\%$  on the next domain, while the model adversariably trained on the source domain achieves a clean accuracy of  $96.95\%$  on the next domain. On the other hand, given that the pseudo-labels contain a small portion of incorrect labels, we wonder whether the adversariably trained model has better clean performance than the standardly trained model if trained on such a noisy training set. Since we set a fixed random seed for all gradual AST methods with varying starting domains, given a  $\tau_0 \in \{0, \dots, T\}$ , the gradual AST method with  $\tau = \tau_0$  and that with  $\tau = \tau_0 + 1$  have the same training processes before domain  $\tau_0$  and the same pseudo-label set on domain  $\tau_0$ . The difference between the gradual AST method with  $\tau = \tau_0$  and that with  $\tau = \tau_0 + 1$  is whether we adversariably train the model on the domain  $\tau_0$ . We compare the clean accuracy of these two methods on domain  $\tau_0$  in Figure 1(c). As we can observe from the figure, the adversariably trained model ( $\tau = \tau_0$ ) has higher clean accuracy on domain  $\tau_0$  than the standardly trained model ( $\tau = \tau_0 + 1$ ). Note that there are two exceptions in which AT hurts clean performance when the accuracy of pseudo-labels is very high ( $\tau_0 = 0, 1$ ), which explains why using AST from an intermediate domain is better than using AST throughout. Besides, we find that the adversariably trained model predicts more accurate pseudo-labels in the next domain than the standardly trained model when the current training set contains a proportion of incorrect labels. We then conclude that AT benefits the clean accuracy of GDA from two perspectives:
- For the intermediate domains, AT improves the accuracy of the pseudo-labels in the next domain. So the learner can fine-tune the model on a more accurate dataset.
- For the target domain, when the pseudo-labels contain a proportion of incorrect labels, the adversarially trained model has higher clean accuracy on the target domain than the standardly trained model.
3.3 Sensitivity of Filtration Ratio  $\zeta$
Following the standard experimental strategy [26, 45], for each step of self-training, we filter out a portion of images for which the model's prediction is least confident. In Figure 1, the ratio  $\zeta$  is set to 0.1 as in the work of [26]. We further conduct more experiments with varying  $\zeta$ . With smaller  $\zeta$ , more data will be included for the next domain's training, but the accuracy of the pseudo-labels will decrease. Due to the page limit, we attach the detailed results of methods with different values of  $\tau$  and  $\zeta$  in Appendix B. We find that when  $\zeta$  decreases, the performance of the standard gradual self-training also decreases. However, the gradual AST method prefers smaller  $\zeta$  even though more incorrect pseudo-labels are included in the training set. For example, when we set  $\zeta = 0.05$ , the gradual AST with the optimal starting domain  $\tau = 9$  (chosen using the validation set) achieves clean accuracy of  $97.15\% (+7.09\%)$  and adversarial accuracy of  $90.44\% (+84.44\%)$  on Rotating MNIST.
(a) The results on Rotating MNIST for varying starting domains
(b) The results on Portraits for varying starting domains