Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models (2020)

Abstract: Recent work has shown the importance of adaptation of broad-coverage contextualised embedding models on the domain of the target task of interest. Current self-supervised adaptation methods are simplistic, as the training signal comes from a small percentage of randomly masked-out tokens. In this paper, we show that careful masking strategies can bridge the knowledge gap of masked language models (MLMs) about the domains more effectively by allocating self-supervision where it is needed. Furthermore, we propose an effective training strategy by adversarially masking out those tokens which are harder to reconstruct by the underlying MLM. The adversarial objective leads to a challenging combinatorial optimisation problem over subsets of tokens, which we tackle efficiently through relaxation to a variational lowerbound and dynamic programming. On six unsupervised domain adaptation tasks involving named entity recognition, our method strongly outperforms the random masking strategy and achieves up to +1.64 F1 score improvements.

## introduction contextualised word embedding models are becoming the foundation of state-of-the-art nlp systems @xcite @xcite @xcite @xcite . these models are pretrained on large amounts of raw text using self-supervision to reduce the labeled data requirement of target tasks of interest by providing useful feature representations @xcite . recent work has shown the importance of further training of pre-trained masked language models (mlms) on the target domain text, as the benefits of their contextualised representations can deteriorate substantially in the presence of domain mismatch @xcite @xcite . this is particularly crucial in unsupervised domain adaptation (uda), where there is no labeled data in the target domain @xcite and the knowledge from source domain labeled data is transferred to the target domain via a common representation space. however, current self-supervised adaptation methods are simplistic, as the training signal comes from a small percentage of randomly masked-out tokens. thus, it remains to investigate whether there exist more effective self-supervision strategies to bridge the knowledge gap of mlms about the domains to yield higher-quality adapted models. a key principle of uda is to learn a common embedding space of both domains which enables transferring a learned model on source task to target task. it is typically done by further pretraining the mlm on a combination of both source and target data. selecting relevant training examples has been shown to be effective in preventing the negative transfer and boosting the performance of adapted models @xcite . therefore, we hypothesise that the computational effort of the further pretraining should concentrate more on learning words which are specific to the target domain or undergo semantic/syntactic shifts between the domains. in this paper, we show that the adapted model can benefit from careful masking strategy and propose an adversarial objective to select subsets for which the current underlying mlm is less confident. this objective raises a challenging combinatorial optimisation problem which we tackle by optimising its variational lower bound. we propose a training algorithm which alternates between tightening the variational lower bound and learning the parameters of the underlying mlm. this involves proposing an efficient dynamic programming (dp) algorithm to sample from the distribution over the space of masking subsets, and an effective method based on gumbel softmax to differentiate through the subset sampling algorithm. we evaluate our adversarial strategy against the random masking and other heuristic strategies including pos-based and uncertainty-based selection on uda problem of six ner span prediction tasks. these tasks involve adapting ner systems from the news domain to financial, twitter, and biomedical domains. given the same computational budget for further self-supervising the mlm, the experimental results show that our adversarial approach is more effective than the other approaches, achieving improvements up to +1.64 points in fscore and +2.23 in token accuracy compared to the random masking strategy. this domain-then-task-tuning procedure resembles the pretrain-then-finetuning paradigm of mlm where the domain tuning shares the same training objective with the pretraining. in domain tuning step, off-the-shelf mlm is further pretrained on an equal mixture of randomly masked-out source and target domain data. ## adversarially trained masked lms given a finite computational budget, we argue that it should be spent wisely on new tokens or those having semantic/syntactic shifts between the two domains. our observation is that such tokens would pose more challenging puzzles to the mlm, i.e. the model is less confident when predicting them. therefore, we propose to strategically select subsets for which the current underlying mlm b θ is less confident about its predictions: henceforth, we assume that the size of the masked set k for a given sentence x x x is fixed. for example in bert @xcite , k is taken to be 15% × |x x x| where |x x x| denotes the length of the sentence. we denote all possible subsets of indices in a sentence with a fixed size by s k . ## experiments we evaluate our proposed masking strategy in uda for named entity span prediction tasks coming from three different domains. ## conclusion we present an adversarial objective for further pretraining mlm in uda problem. the intuition behind the objective is that the adaptation effort should focus on a subset of tokens which are chal-lenging to the mlm. we establish a variational lower bound of the objective function and propose an effective sampling algorithm using dynamic programming and gumbel softmax trick. comparing to other masking strategies, our proposed adversarial masking approach has achieve substantially better performance on uda problem of named entity span prediction for several domains.