Adaptor: Objective-Centric Adaptation Framework for Language Models (2022)

Abstract: This paper introduces Adaptor library, which transposes traditional model-centric approach composed of pre-training + fine-tuning steps to objective-centric approach, composing the training process by applications of selected objectives. We survey research directions that can benefit from enhanced objective-centric experimentation in multitask training, custom objectives development, dynamic training curricula, or domain adaptation. Adaptor aims to ease reproducibility of these research directions in practice. Finally, we demonstrate the practical applicability of Adaptor in selected unsupervised domain adaptation scenarios.

## introduction recent development in natural language processing (nlp) heavily benefits from a high level of maturity of open-source frameworks, such as fairseq @xcite or huggingface transformers @xcite . thanks to the standardized interfaces, these libraries allow for immediate experimentation with the most recent research results, practically fostering the speed of further progress in the area. while their use is seamless for countless conventional use-cases of transformer models and fine-tuning to a specific end-task @xcite , divergence from this framework requires feasible, but elaborate and complex customizations, increasing the risk of logical errors and complicating the reproducibility of experiments. a characteristic group of problems requiring significant changes to the standard pipeline are multi-step and multi-task adaptations. this paper introduces the adaptor library, which aims to simplify the more complex training processes that their training objectives can easier describe. adaptor challenges the conventional modelcentric framework, where data and task selection are constrained by the requirements of the selected language model architecture. instead, it introduces an objective-centric training pipeline, with objective as the central abstraction of the process. the adaptor framework aims to help nlp researchers and practicioners engage in projects that include any of the following: this paper is structured as follows: section 2 provides an overview of recent work demonstrating the potential of multi-objective training in domain and task adaptation. section 2.4 also describes other software frameworks applicable for similar use cases. section 3 describes the design of adaptor, showing the users how to confidently integrate novel objectives and schedules. in section 4, we describe and implement a set of non-trivial, yet promising domain adaptation experiments using adaptor and collect their results. as adaptor remains under active development, we close in section 5 with an outline of the upcoming features. we welcome contributions of novel objectives and schedules. ## background this section provides an overview of recent work that demonstrates the potential of multi-objective training and schedules that motivated the design of adaptor. our overview consists of a nonexhaustive list of applications that adaptor aims to make more accessible for practical use and in future research. ## adaptor design this section describes the structure and functions of the adaptor framework. we introduce its primary components bottom-up. figure 3 depicts the relations of these components and compares user interaction with the traditional model-centric pipeline. ## experiments we use adaptor in a set of domain adaptation experiments for a machine translation use-case, aiming to answer the following research question: how well can unsupervised objective(s) substitute labeled parallel data. in our methodology, we permute the easily-configurable parts of adaptor's training configuration foot_0 and compare the results of the resulting model to a baseline adaptation scenario. we experiment with an architecture identical to the base model of @xcite , with a configuration of @xcite . data. we train the model on english-to-czech translations on different domains of opus (tiedemann, 2012) chosen for their significant distinctiveness: we use wikimedia as a large-scale, supervised domain (denoted as in-domain, i.e. id), opensubtitles as an adapted domain (ad) and bible for the evaluation of a model's robustness on out-of-domain (ood) samples. pre-training vs. fine-tuning. we simulate two basic scenarios: training the model from a random initialization and fine-tuning the existing translation model with no control over its pre-training data. in the latter cases, we perform fine-tuning from the checkpoint of @xcite . schedules. we implement and experiment with two objective schedules: i) sequential schedule, sampling and differentiating the model sequentially by each objective until convergence by evaluation loss, or for a maximum of 100,000 updates. ii) parallel schedule, concurrently sampling training batches uniformly from every given objective. using gradient accumulation, we differentiate the model based on all given objectives. we perform updates until the convergence of all objectives, or for a maximum of 50,000 updates for each objective. objectives selection. we implement and experiment with the following adaptor objectives: using these components, we construct the following experiments: ## conclusion and future work this paper introduces the adaptor library, which provides objective-centric training framework wellsuitable for multi-task and multi-domain training scenarios, and the development of novel objectives and sampling schedules. we find that even in the conventional single-objective training routines, adaptor can reduce volumes of custom implemen-tation and increases readability and reproducibility. having used adaptor already for several production use cases, we are happy to share it with the nlp community. our future work aims to further enhance adaptor's user comfort with existing and novel unsupervised objectives, dynamic schedules, and demonstrations on novel use cases.