Modular and Parameter-Efficient Fine-Tuning for NLP Models (2022)

Abstract: State-of-the-art language models in NLP perform best when fine-tuned even on small datasets, but due to their increasing size, fine-tuning and downstream usage have become extremely compute-intensive. Being able to efficiently and effectively fine-tune the largest pre-trained models is thus key in order to reap the benefits of the latest advances in NLP. In this tutorial, we provide a comprehensive overview of parameter-efficient fine-tuning methods. We highlight their similarities and differences by presenting them in a unified view. We explore the benefits and usage scenarios of a neglected property of such parameter-efficient models—modularity—such as composition of modules to deal with previously unseen data conditions. We finally highlight how both properties——parameter efficiency and modularity——can be useful in the real-world setting of adapting pre-trained models to under-represented languages and domains with scarce annotated data for several downstream applications.

## motivation and objectives the emergence of large pre-trained language models @xcite has led to a watershed moment in nlp, accelerating progress and improving performance across a wide range of nlp benchmarks. these models have quickly superseded previous baseline models and are now a core part of every nlp researcher and practitioner's toolkit. while pre-training such models has always been prohibitively expensive, recent pre-trained models have been getting so large @xcite ) that even their fine-tuning and downstream usage are extremely challenging. in practice, the largest models perform best, even when fine-tuned on small datasets @xcite . therefore, being able to efficiently and effectively fine-tune the largest pre-trained models is key in order to reap the benefits of the latest advances in nlp. this is a major challenge that threatens to further exacerbate the inequality between resource-rich and resourceconstrained research and production environments. recent work has highlighted the benefit of parameter-efficient methods to fine-tune such large pre-trained models. these parameter-efficient finetuning methods include soft prompt methods that preprend a small set of trainable continuous parameters to the input or intermediate layers @xcite @xcite , low-rank methods that train a small number of parameters in a low-dimensional subspace using random projections @xcite , and adapter methods that insert trainable transformations at different layers @xcite . other methods only tune a subset of the model's parameters @xcite . an alternative set of methods relies on identifying performant sparse subnetworks, which can be updated in isolation @xcite @xcite . these methods reduce not only the number of parameters during fine-tuning but also have been shown to be more robust than standard fine-tuning and to outperform it in low-resource conditions @xcite @xcite . in the first part of this tutorial, we will give a comprehensive overview of such parameterefficient fine-tuning methods. we will highlight the similarities and differences of a wide array of these methods by presenting them in a unified view, which expands on recent work @xcite highlighting the connections between adapters and prefix tuning. based on this common view, we will be able to clearly show the respective benefits and trade-offs of a diverse set of parameter-efficient fine-tuning methods. a commonality of parameter-efficient methods-illustrated clearly in this framework-is that they learn a modification vector that is added to the pretrained model parameters, which are kept fixed. this property opens the door to modularity, which we view as a neglected benefit of the parameterefficient usage of pre-trained models. in the second part of the tutorial, we will explore the benefits and usage scenarios of such modular approaches. we will demonstrate how modular 'expert' modules can be learned for specific data settings @xcite @xcite . moreover, they can provide further benefits when combined and adapting to previously unseen settings @xcite . we will additionally discuss how modular approaches can be used to augment models with new capabilities or knowledge, such as memory for lifelong learning @xcite , numerical reasoning @xcite , and factual or linguistic knowledge @xcite . a key benefit of modularity is that it enables the storage and composition of modules to deal with previously unseen data conditions @xcite . we will highlight this benefit based on prior work @xcite @xcite and explore applications that it may enable in the future. finally, as an nlp 'history lesson', we will revisit modular approaches that preceded pre-trained models @xcite and highlight how they may be relevant for recent approaches. overall, we will encourage attendees to think of pre-trained models not as monoliths but as building blocks than can be augmented for specific purposes and data settings. tying both previous parts together, the third part of the tutorial will focus on applications: we will demonstrate how the properties explored so farparameter efficiency and modularity-can be useful in practical settings. specifically, we will focus on the important real-world setting of adapting pre-trained models to under-represented languages and domains with scarce annotated data for several downstream applications, e.g., crosslingual transfer @xcite and nmt @xcite @xcite . we will highlight approaches that enable learning language-specific components using previously presented techniques such as adapters @xcite @xcite or sparse subnetworks @xcite . we will specifically discuss challenges and possible solutions when using such methods to adapt pre-trained models to extremely low-resource scenarios, such as test time adaptation @xcite , parameter generation @xcite @xcite , domain adaptation @xcite , and usage of alternative data sources @xcite . ## tutorial outline in what follows, we provide finer-grained descriptions of the main topics covered in the tutorial, along with tentative time allocation: ## diversity the third part of the tutorial focuses on how the described methods can be applied to improve models especially for low-resource and under-represented languages. this aligns with a long-term aim and promise of multilingual nlp to bring language technology to virtually any language of the world. we aim to make scripts available that demonstrate how the discussed methods can be applied in this setting. we hope this will help to diversify the audience, especially in the emerging regions such as africa and central and south america, and make the tutorial accessible to both beginners and advanced researchers. ## ethics statement the methodology introduced in the tutorial potentially inherits standard undesirable biases stemming from pretraining language models on large (and unverified) multilingual text collections. during the tutorial, we will ensure to remind nlp researchers and practitioners to bear in mind these biases, and apply appropriate data filtering and debiasing techniques before deploying any text encoders and relevant methodology to real-world language technology applications. ## acknowledgments ivan vulić is supported by a personal royal society university research fellowship, and his work has also been supported by a huawei research donation awarded to the language technology lab.