Gradual Fine-Tuning for Low-Resource Domain Adaptation (2021)

Abstract: Fine-tuning is known to improve NLP models by adapting an initial model trained on more plentiful but less domain-salient examples to data in a target domain. Such domain adaptation is typically done using one stage of fine-tuning. We demonstrate that gradually fine-tuning in a multi-step process can yield substantial further gains and can be applied without modifying the model or learning objective.

## introduction domain adaptation is a technique for practical applications in which one wants to learn a model for a task in a particular domain with too few instances of in-domain data to directly learn a model. common approaches for domain adaptation make use of fine-tuning @xcite @xcite , in which a model is pretrained on a large amount of out-of-domain but task-relevant data and then refined toward the target domain by subsequently training on in-domain data. this fine-tuning procedure is often performed in one stage: the pretrained model is trained on the in-domain data until convergence @xcite . we propose a gradual finetuning approach, in which a model is iteratively trained to convergence on data whose distribution progressively approaches that of the in-domain data. intuitively, the model is eased toward the target domain rather than abruptly shifting to it. inspired by the general approach of curriculum learning of training a model on a trajectory from easier instances to more difficult instances @xcite , we train a model on a sequence of datasets, each of which would be increasingly difficult to learn on its own due to its size. each dataset in the sequence interpolates between the data in the previous iteration and the target domain data. we hypothesize that just as in curriculum learning where first learning from easier instances helps models subsequently learn from more difficult instances, the interpolation process yields datasets with distributions that are increasingly useful for helping models learn during subsequent stages of the gradual fine-tuning procedure. we begin by training the model on data that contains a mix of out-of-domain and in-domain instances, then increase the concentration of in-domain data in each fine-tuning stage. in this way, at each stage of finetuning we increase the similarity between the current domain and the target domain, which enables the model to potentially better fit the distribution of the target domain. the approach is illustrated in figure 1 . ## method domain adaptation via fine-tuning aims to improve performance on a target domain by using information learned from general domain data when training on data in the target domain. in other words, it is expected that training on general domain data would provide better initialization for subsequently training on target domain data than random initialization would provide @xcite . ## conclusion we have proposed a gradual fine-tuning technique that iteratively steers the distribution of augmented training data toward that of a target domain. gradual fine-tuning can be straightforwardly applied to an existing codebase without changing the model architecture or learning objective. through experiments on dialogue state tracking and event extraction tasks, we have demonstrated that gradual finetuning outperforms standard one-stage fine-tuning for domain adaptation.