UDAPTER - Efficient Domain Adaptation Using Adapters (2023)

Abstract: We propose two methods to make unsupervised domain adaptation (UDA) more parameter efficient using adapters – small bottleneck layers interspersed with every layer of the large-scale pre-trained language model (PLM). The first method deconstructs UDA into a two-step process: first by adding a domain adapter to learn domain-invariant information and then by adding a task adapter that uses domain-invariant information to learn task representations in the source domain. The second method jointly learns a supervised classifier while reducing the divergence measure. Compared to strong baselines, our simple methods perform well in natural language inference (MNLI) and the cross-domain sentiment classification task. We even outperform unsupervised domain adaptation methods such as DANN and DSN in sentiment classification, and we are within 0.85% F1 for natural language inference task, by fine-tuning only a fraction of the full model parameters. We release our code at this URL.

## introduction fine-tuning pretrained language models (plm) is the predominant method for improving nlp tasks such as sentiment analysis, natural language inference, and other language understanding tasks @xcite . however, fine-tuning forces us to modify all the parameters of the model and store one copy of the model for one task. given the large size of current plms, this can be expensive. furthermore, fine-tuning needs large-scale data to be effective and is unstable when using different seeds @xcite . a new approach to alleviate this is parameterefficient fine-tuning -freezing the plm parameters * the first two authors contributed equally. and fine-tuning only a small fraction of the parameters. fine-tuning with adapters @xcite is one of these methods in which small additional layers are tuned within each plm layer. fine-tuning with adapters has many advantages: performance comparable to full fine-tuning @xcite , and robustness to different seeds and adversarial examples @xcite . unsupervised domain adaptation (uda) aims to adapt models to new domains and considers situations where labeled data are available only in the source domain and unlabeled data are available in the target domain. uda methods in general have two components: the first reduces the divergence between the source and target domains, and the second reduces the loss corresponding to a particular task @xcite . however, they fine-tune a large number of parameters and are susceptible to catastrophic forgetting. adapters @xcite can help solve these problems. however, the benefits of using adapters finetuning for domain adaptation have been mostly overlooked. how well can adapter fine-tuning perform across different domains and can we make domain adaptation more efficient? in this work, we answer these questions and propose models to perform domain adaptation using adapters. adapters are known to perform well in lowresource scenarios where a small amount of supervised data is available in a new domain or language @xcite . in this work, using the principles of uda, we propose to make domain adaptation more effective using unsupervised data from the target domain. we introduce two methods that we collectively call the u nsupervised d omain a daptation method using adapters (udapter). the first method is a two-step process: first, we learn domain adapters -where we use a divergence measure to bring two probabilistic distributions closer together. this helps us to learn representations that are independent of the domain from which they come. second, we use the domain-invariant information learned as input to another task adapter that learns to perform an nlp task using labeled data from the source domain. we combine the two adapters by stacking them. the second method adds a single adapter without stacking, where we simultaneously reduce the divergence between domains and learn the task in the source domain. domain adversarial neural networks (dann) and domain separation networks (dsn) are the most common methods for unsupervised domain adaptation in nlp @xcite . we compare our proposed methods with these strong baselines that fine-tune all model parameters, on amazon @xcite and the mnli dataset @xcite consisting of five domains each. udapter performs better than all baselines. it achieves competitive performance compared to uda methods by fine-tuning only a fraction of the parameters. in an era where large resources are spent to further pretrain language models on large amounts of unsupervised data to achieve domain adaptation @xcite , it is necessary to provide cheaper, faster solutions. ## method setup. we consider an nlp task (sentiment analysis) consisting of data x and labels y (positive, negative). there exist two different distributions, called the source domain d s and the target domain d t over x × y. unsupervised domain adaptation (uda) consists of a model c that receives labeled input samples x s : (x s , y s ) ns @xmath0: (x t ) nt @xmath1. the goal of uda is to learn a model c such that we perform well in the nlp task for the target domain d t . the popular method in uda is to learn representations that are invariant in the input domain and still have sufficient power to perform well in the source domain @xcite . then according to the theory of domain divergence @xcite shows that the error in the target domain is bounded by the error in the source domain and the divergence. the unsupervised domain adaptation method thus consists of two components: the reduction of the divergence measure and a classifier for the source domain. a new classifier must be learned for every pair of source-target domains, and the method fine-tunes a large number of parameters. udapter makes unsupervised domain adaptation more parameter efficient (cf. § 2.1, § 2.2) using adapters. we follow the framework proposed by @xcite where small bottleneck layers are added to the transformer layers, fine-tuning only the adapter parameters while keeping the other parameters frozen, and propose the following. ## literature review parameter efficient fine-tuning methods. adapters @xcite are task-specific modules added to frozen transformer layers, with only the adapter parameters updated. their plug-and-play characteristics and the avoidance of catastrophic forgetting have resulted in their use for nlp tasks: machine translation @xcite , named entity recognition @xcite , etc. recently, @xcite have shown that they are efficient in scenarios where there is minimal supervised data. however, they neither test their performance under domain shift nor propose methods to improve adapter fine-tuning. closely related to our method is the work of ngo trung et al. (2021), who learns a shared-private representation per layer, similar to dsn @xcite . their method requires balancing multiple loss functions, compared to our simpler two-step domain adaptation method. the stacking of adapters has been followed before by @xcite for cross-lingual tasks: learning a language adapter first and stacking a task adapter. however, one language adapter is learned per language, assumes large amounts of unsupervised data to be available in all the languages, and requires supervised data to be available to learn a task, which is not applicable for domain adaptation. compared to other methods, we make domain adaptation more efficient using principles of unsupervised domain adaptation. ## discussion this work shows that domain adaptation in nlp can be made more efficient using adapters. we use adapters fine-tuning @xcite proposed before and stacking of adapters that have been proposed before for a cross-lingual setting @xcite for the unsupervised domain adaptation. the approach we have discussed will make domain adaptation more practical for real-world use cases, making adaptation faster and cheaper. however, in this work, we have used bert-base-uncased for all of our methods. using other backbone transformer models is part of our future work. we deal only with a classification and natural language inference task. adapters have previously been used for machine translation @xcite and other generation tasks @xcite . we need to explore our domain adaptation methods for other generation tasks. in this work, we reduce the marginal distribution of the two distributions. previous works such as @xcite show that reducing only the marginal distribution is not sufficient and aligning the label distributions is necessary. however, nlp works do not consider this and would require further investigation by the community. ## conclusion in this work, we propose udapter, to make unsupervised domain adaptation more parameterefficient. our methods outperform other strong baselines, and we show that we can perform better than just training a task adapter on supervised data. we perform competitively to other uda methods at a fraction of the parameters and outperform them when there is limited data -a more practical scenario. future work should explore other parameter-efficient methods such as prefix-tuning (li and liang, 2021) for domain adaptation. nlp should also consider other avenues, such as continuous adaptation to new domains and adaptation to new domains when there are no data available.