D a MSTF : Domain Adversarial Learning Enhanced Meta Self-Training for Domain Adaptation (2023)

Abstract: Self-training emerges as an important research line on domain adaptation. By taking the model’s prediction as the pseudo labels of the unlabeled data, self-training bootstraps the model with pseudo instances in the target domain. However, the prediction errors of pseudo labels (label noise) challenge the performance of self-training. To address this problem, previous approaches only use reliable pseudo instances, i.e., pseudo instances with high prediction confidence, to retrain the model. Although these strategies effectively reduce the label noise, they are prone to miss the hard examples. In this paper, we propose a new self-training framework for domain adaptation, namely Domain adversarial learning enhanced Self-Training Framework (DaMSTF). Firstly, DaMSTF involves meta-learning to estimate the importance of each pseudo instance, so as to simultaneously reduce the label noise and preserve hard examples. Secondly, we design a meta constructor for constructing the meta-validation set, which guarantees the effectiveness of the meta-learning module by improving the quality of the meta-validation set. Thirdly, we find that the meta-learning module suffers from the training guidance vanish- ment and tends to converge to an inferior optimal. To this end, we employ domain adversarial learning as a heuristic neural network initialization method, which can help the meta-learning module converge to a better optimal. Theoretically and experimentally, we demonstrate the effectiveness of the proposed DaMSTF. On the cross-domain sentiment classification task, DaMSTF improves the performance of BERT with an average of nearly 4%.

## introduction domain adaptation, which aims to adapt the model trained on the source domain to the target domain, † contributed equally to this work * corresponding author attracts much attention in natural language processing (nlp) applications @xcite @xcite . since domain adaptation involves labeled data from the source domain and unlabeled data from the target domain, it can be regarded as a semi-supervised learning problem. from this perspective, self-training, a classical semi-supervised learning approach, emerges a prospective research direction on domain adaptation @xcite . self-training consists of a series of loops over the pseudo labeling phase and model retraining phase. in the pseudo labeling phase, self-training takes the model's prediction as the pseudo labels for the unlabeled data from the target domain. based on these pseudo-labeled instances, self-training retrains the current model in the model retraining phase. the trained model can be adapted to the target domain by repeating these two phases. due to the prediction errors, there exists label noise in pseudo instances, which challenges self-training approaches @xcite . previous self-training approaches usually involve a data selection process to reduce the label noise, i.e., preserving the reliable pseudo instances and discarding the remaining ones. in general, higher prediction confidence implies higher prediction correctness, so existing self-training approaches prefer the pseudo instances with high prediction confidence @xcite . however, fitting the model on these easy pseudo instances cannot effectively improve the model, as the model is already confident about its prediction. on the contrary, pseudo instances with low prediction confidence can provide more information for improving the model, but contain more label noise at the same time. to simultaneously reduce the label noise and preserve hard examples, we propose to involve in meta-learning to reweight pseudo instances. within a learning-to-learn schema, the meta-learning mod-ule learns to estimate the importance of every pseudo instance, and then, allocates different instance weights to different pseudo instances. ideally, hard and correct pseudo instances will be assigned larger weights, while easy or error pseudo instances will be assigned smaller weights. to achieve this, the process in the meta-learning module is formulated as a bi-level hyperparameters optimization problem @xcite , where instance weights are taken as the hyperparameters and determined by a series of meta-training steps and meta-validation steps. in the meta-training step, the model is virtually updated on the metatraining set with respect to the current instance weights. in the meta validation step, we validate the virtually updated model with an unbiased meta validation set, and optimize the instance weights with the training guidance back-propagated from the validation performance. according to the analysis in @xcite , a high-quality meta validation set, which is clean and unbiased to the test set, is important for the effectiveness of the meta-learning algorithm. to this end, we propose a meta constructor oriented to the domain adaptation scenario. at each self-training iteration, the meta constructor selects out the most reliable pseudo instances and inserts them into the meta validation set. since the instances in the meta validation set are all from the target domain and vary along with the self-training iterations, the data distribution in the constructed meta validation set approximates the one in the target domain. thus, the meta constructor reduces the bias of the meta validation set. on the other hand, selecting the most reliable pseudo instances can reduce the label noise, making the meta validation set cleaner. another challenge for the meta-learning module is the training guidance vanishment, referring to the gradient vanishment on hyperparameters. with a theoretical analysis, we attribute this problem to the gradient vanishment on the meta validation set. to this end, we introduce a domain adversarial learning module to perturb the model's parameters, thereby increasing the model's gradients on the meta validation set. in damstf, we also interpret the domain adversarial learning module as a heuristic neural network initialization method. before the model retraining phase, the domain adversarial learning module first initializes the model's parameters by aligning the model's feature space. for domain adaptation, the global optimal refers to the state where the model's parameters are agnostic to the domain information but discriminative to the task information. thus, the training process in the domain adversarial learning module makes the model's parameters closer to the global optimal, serving as a heuristic neural network initialization. our contributions can be summarized as follows: ## problem formulation we denote the set that involves all instances in the source domain as d s , and denote the set that contains all instances in the target domain as d t . from d s , we can obtain a labeled dataset for training, i.e., d @xmath0, y i )} n @xmath1. in text classification tasks, the input x i is a text from the input space x , the corresponding label y i is a c-dimensional one-hot label vector, i.e., y @xmath2, 1} c , where c is the number of classes. based on d s , we learn a hypothesis, h : @xmath3, 1} c . since d s comes from d s (i.e., d s ⊆ d s ), the learned hypothesis h usually performs well on d s . when we transfer the hypothesis h from d s to d t , h may perform poorly due to the domain shift. the goal of domain adaptation is to adapt the hypothesis h to d t . in general, unlabeled text in the target domain is available @xcite . we denote the unlabeled target domain dataset as d u @xmath4, where x @xmath5. in some cases, we can even access an in-domain dataset, i.e., a small set of labeled data in the target algorithm 1 damstf require: labeled source dataset ds, unlabeled target dataset d u t , in-domain dataset d l t 1: pretrain θ on ds, dm ← d l t 2: while the termination criteria is not met do 3: compute pseudo label ŷt on d u t 4: @xmath6: sort the d p t with respect to h in ascending order, and denote the first k data as de, the remaining data as d tr t 6: @xmath7: domainadversarial(ds ∪ d u t , θf , ϑ) 8: metalearning(ds ∪ d tr t , θ, w) 9: end while 10: function metalearning(d, θ, w) 11: for training batch b in d do 12: for @xmath8: compute θ(w t ) via eq. (3) 14: compute weight w t+1 via eq. (6) 15: end for 16: w * ← w t m , update θ with eq. (7) 17: end for 18: return θ, w 19: end function 20: function domainadversarial(d, θf , ϑ) 21: ## theoretical analysis this section first introduces the training guidance vanishment problem and then explains the effectiveness of damstf in achieving domain adaptation. the proofs are detailed in appendix. a and appendix. b. ## experiments we provide the experiment settings in § 5.1 and compare damstf with previous domain adaptation approaches in § 5.2. in § 5.3, we analyze the effectiveness of the meta constructor and the domain adversarial learning module with an ablation study. § 5.4 validate that exposing more unlabeled data to damstf can improve the domain adaptation performance (theorem 3). appendix e provides extra experiments of the domain adversarial learning module in preventing the training guidance vanishment problem, and the meta-learning module in highlighting the hard and correct pseudo instances. ## conclusion this paper proposes an improved self-training framework for domain adaptation, named damstf. damstf extends the basic framework for selftraining approaches by involving a meta-learning module, which alleviates the label noise problem in self-training. to guarantee the effectiveness of the meta-learning module, we propose a meta constructor to improve the quality of the meta validation set, and propose a domain adversarial module to prevent the training guidance vanishment. also, the domain adversarial learning module can align the feature space along with the self-training iterations. extensive experiments on two popular models, bigcn and bert, verify the effectiveness of damstf. the ablation studies demonstrate that the meta-learning module, the meta constructor, and the domain adversarial module are indispensable for the effectiveness of the damstf. the limitation, ethical considerations, and social impacts of this paper are in appendix f and g.