Revisiting Pretraining with Adapters (2021)

Abstract: Pretrained language models have served as the backbone for many state-of-the-art NLP results. These models are large and expensive to train. Recent work suggests that continued pretraining on task-specific data is worth the effort as pretraining leads to improved performance on downstream tasks. We explore alternatives to full-scale task-specific pretraining of language models through the use of adapter modules, a parameter-efficient approach to transfer learning. We find that adapter-based pretraining is able to achieve comparable results to task-specific pretraining while using a fraction of the overall trainable parameters. We further explore direct use of adapters without pretraining and find that the direct fine-tuning performs mostly on par with pretrained adapter models, contradicting previously proposed benefits of continual pretraining in full pretraining fine-tuning strategies. Lastly, we perform an ablation study on task-adaptive pretraining to investigate how different hyperparameter settings can change the effectiveness of the pretraining.

## introduction pretrained language models (plm) are predominant in tackling current natural language processing (nlp) tasks. most plms based on the transformer architecture @xcite are first trained on massive text corpora with the selfsupervised objective to learn word representations @xcite , and then are fine-tuned for a specific target task. the pretraining and fine-tuning of plms achieves state-ofthe-art (sota) performance in many nlp tasks. inspired by the benefits of pretraining, there have been studies demonstrate the effects of continued pretraining on the domain of a target task or the target task dataset @xcite @xcite . @xcite adapt plms on the target task by further pretraining roberta @xcite on the target text corpus before it is fine-tuned for the corresponding task and showed that this task adaptation consistently improves the performance for text classification tasks. however, this full process of pretraining and then fine-tuning can be parameter inefficient for recent plms that have millions or billions of parameters @xcite . this parameter inefficiency becomes even worse when one continues pre-training all the parameters of plms on the task-specific corpus. furthermore, recent plms need more than 100s of mb to store all the weights @xcite , making it difficult to download and share the pre-trained models on the fly. recently, adapters have been proposed as an alternative approach to decrease the substantial number of parameters of plms in the fine-tuning stage @xcite . finetuning with adapters mostly matches the performance of those with the full fine-tuning strategy on many nlp tasks including glue benchmark @xcite and reduces the size of the model from 100s of mb to the order of mb @xcite . as such, a natural question arises from the successes of the adapter approach: can the adapter alone adapt plms to the target task when it is used in the second phase of the pretraining stage and thus lead to the improvement of the performance on the corresponding task? in this paper, we explore task-adaptive pretraining, termed tapt @xcite , with adapters to address this question and overcome the limitations of the conventional full pretraining and fine-tuning. we only train the adapter modules in the second phase of pretraining as well as the fine-tuning stage to achieve both parameter efficiency and the benefits of continual pretraining and compare those with the adapter-based model without pretraining. surprisingly, we find that directly fine-tuning adapters performs mostly on par with the pre-trained adapter model and outperforms the full tapt, contradicting the previously proposed benefits of continual pretraining in the full pretraining fine-tuning scheme. as directly fine-tuning adapters skips the second phase of pretraining and the training steps of adapters are faster than those of the full model, it substantially reduces the training time. we further investigate different hyperparameter settings that affect the effectiveness of pretraining. ## pretraining and adapters pre-trained language model we use roberta @xcite , a transformer-based language model that is pre-trained on a massive text corpus, following @xcite . roberta is an extension of bert @xcite with optimized hyperparameters and a modification of the pretraining objective, which excludes next sentence prediction and only uses the randomly masked tokens in the input sentence. to evaluate the performance of roberta on a certain task, a classification layer is appended on top of the language model after the pretraining and all the parameters in roberta are trained in a supervised way using the label of the dataset. in this paper, training word representations using roberta on a masked language modeling task will be referred to as pretraining. further, taking this pretrained model and adding a classification layer with additional updates to the language model parameters will be referred to as fine-tuning. task-adaptive pretraining (tapt) although roberta achieves strong performance by simply fine-tuning the plms on a target task, there can be a distributional mismatch between the pretraining and target corpora. to address this issue, pretraining on the target task or the domain of the target task can be usefully employed to adapt the language models to the target task and it further improves the performance of the plms. such methods can be referred to as domain-adaptive pretraining (dapt) or task adaptive-pretraining (tapt) @xcite . in this paper, we limit the scope of our works to tapt as domain text corpus is not always available for each task, whereas tapt can be easily applied by directly using the dataset of the target task while its performance often matches with dapt @xcite . in tapt, the second phase of pretraining is per-figure 1 : the adapter achitecture in the transformer layer @xcite formed with roberta using the unlabeled text corpus of the target task, and then it is fine-tuned on the target task. adapter adapter modules have been employed as a feature extractor in computer vision @xcite and have been recently adopted in the nlp literature as an alternative approach to fully fine-tuning plms. adapters are sets of new weights that are typically embedded in each transformer layer of plms and consist of feed-forward layers with normalizations, residual connections, and projection layers. the architectures of adapters vary with respect to the different configuration settings. we use the configuration proposed by @xcite in figure 1 , which turned out to be effective on diverse nlp tasks, and add the adapter layer to each transformer layer. pfeiffer et al., 2020c use two types of adapter: language-specific adapters and taskspecific adapters for cross-lingual transfer. these two types of adapter modules have similar architecture as in figure 1 . however, the language adapters involve invertible adapters after the embedding layer to capture token-level language representation when those are trained via masked language modeling in the pretraining stage, whereas the task adapters are simply embedded in each transformer layer and trained in the fine-tuning stage to learn the task representation. following pfeiffer et al., 2020c, we employ language adapter modules with invertible adapter layers to perform pretraining adapters on the unlabeled target dataset. however, we perform fine-tuning pre-trained parameters of the language adapter modules for evaluation to align with @xcite ) and low-resource (chemprot @xcite , acl-arc @xcite , scierc @xcite , hyperpartisan @xcite settings. tapt, whereas pfeiffer et al., 2020c employ both the language and the task adapters by stacking task adapters on top of the language adapters. ## experiments we now propose an adapter-based approach that is a parameter efficient variant of task-adaptive pretraining (tapt) and measure the margin of the performance between the pre-trained adapter model and the adapter model without pretraining. for pretraining adapters, we added the adapter module in each transformer layer of roberta using adaptertransformer @xcite 1 and continued pretraining all the weights in adapter layers on target text corpus while keeping the original parameters in roberta fixed. after finishing the second phase of pretraining, we performed fine-tuning of roberta by training the weights in the adapters and the final classification layers while keeping all of the parameters in roberta frozen. ## conclusion our work demonstrates that adapters provide a competitive alternative to large-scale task-adaptive pretraining for nlp classification tasks. we show that it is possible to achieve similar performance to tapt with pretraining training just 1.32% of the parameters through pretraining with adapters. however, the most computationally efficient option is to skip pretraining and only perform fine-tuning with adapters. we found that skipping pretraining altogether and just fine-tuning with adapters outperforms or performs mostly on par with tapt and the adapter model with pretraining across our tasks while substantially reducing the training time. xiang zhang, junbo zhao, and yann lecun. 2015. character-level convolutional networks for text classification. in advances in neural information processing systems, volume 28, pages 649-657. curran associates, inc.