Composable Sparse Fine-Tuning for Cross-Lingual Transfer (2022)

Abstract: Fine-tuning the entire set of parameters of a large pretrained model has become the mainstream approach for transfer learning. To increase its efficiency and prevent catastrophic forgetting and interference, techniques like adapters and sparse fine-tuning have been developed. Adapters are modular, as they can be combined to adapt a model towards different facets of knowledge (e.g., dedicated language and/or task adapters). Sparse fine-tuning is expressive, as it controls the behavior of all model components. In this work, we introduce a new fine-tuning method with both these desirable properties. In particular, we learn sparse, real-valued masks based on a simple variant of the Lottery Ticket Hypothesis. Task-specific masks are obtained from annotated data in a source language, and language-specific masks from masked language modeling in a target language. Both these masks can then be composed with the pretrained model. Unlike adapter-based fine-tuning, this method neither increases the number of parameters at inference time nor alters the original model architecture. Most importantly, it outperforms adapters in zero-shot cross-lingual transfer by a large margin in a series of multilingual benchmarks, including Universal Dependencies, MasakhaNER, and AmericasNLI. Based on an in-depth analysis, we additionally find that sparsity is crucial to prevent both 1) interference between the fine-tunings to be composed and 2) overfitting. We release the code and models at https://github.com/cambridgeltl/composable-sft .

## introduction fine-tuning of pretrained models @xcite , inter alia) is arguably the dominant paradigm in nlp at present. originally, "fine-tuning" involved supervised learning of all the parameters of a model pretrained on unlabeled texts. however, given the size of transformer-based architectures, this approach is often time-and resource-inefficient, and may result in catastrophic forgetting and interference @xcite during multiple adaptations. to overcome these limitations, two main alternatives have emerged: 1) through adapters, new parameters can be added to a pretrained model in the form of extra intermediate layers @xcite and fine-tuned while keeping all the pretrained parameters fixed; 2) sparse fine-tuning (sft) of a small subset of pretrained model parameters @xcite @xcite . adapters have proven especially useful in multilingual nlp @xcite @xcite @xcite because they exhibit a surprising degree of modularity. this ability to disentangle and recombine orthogonal facets of knowledge in original ways @xcite allows for separately learning a task adapter from labeled data in a source language and dedicated language adapters from unlabeled data in the source language and target languages. by stacking these components, it is possible to perform zero-shot cross-lingual transfer. compared to sequentially fine-tuning the full model on both the task and target language, this yields superior performance and efficiency @xcite . notably, achieving coverage over n t tasks in n l target languages with the sequential approach requires n t n l models to be trained, whereas the modularity of adapters reduces this to n t + n l . meanwhile, the advantage of sfts over adapters is their expressivity: rather than a non-linear transformation of the output of transformer layers (e.g., using a shallow mlp as with adapters), they can operate directly on a pretrained model's embedding and attention layers. it therefore seems natural to search for a parameter-efficient fine-tuning method that is both modular and expressive. to this end, we propose lottery ticket sparse fine-tuning (lt-sft), a simple and generalpurpose adaptation technique inspired by the lottery ticket hypothesis (lth; @xcite , which was originally conceived for pruning large neural networks. in particular, after fine-tuning a pretrained model for a specific task or language, we select the subset of parameters that change the most. then, we rewind the model to its pretrained initialization (without setting any value to zero, contrary to the original lth algorithm). by re-tuning again only the selected subset of parameters, we obtain a sparse fine-tuning in the form of a vector of differences with respect to the pretrained model. multiple sfts can be composed by simply summing them with the pretrained model. we provide a graphical representation of our method in figure 1 . we benchmark lt-sft on a series of multilingual datasets, including universal dependencies @xcite for part-of-speech tagging and dependency parsing, masakhaner @xcite for named entity recognition, and americas-nli @xcite for natural language inference. we evaluate it in a zero-shot cross-lingual transfer setting on 35 typologically and geographically diverse languages that include both languages seen and unseen during masked language modeling of the pretrained model. the results in all transfer tasks indicate that lt-sft consistently achieves substantial gains over the current state-of-the-art adapter-based method for cross-lingual transfer, mad-x @xcite . in addition to its superior performance, modularity, and expressivity, lt-sft offers a series of additional advantages over adapters: 1) the number of parameters remains constant, which prevents the decrease in inference speed observed when adapter layers are added; 2) the neural architecture remains identical to the pretrained model, which makes code development model-independent rather than requiring special modifications for each possible architecture @xcite . finally, 3) we empirically demonstrate that the peak in performance for lt-sft is consistently found with the same percentage of tunable parameters, whereas the best reduction factor for mad-x is task-dependent. this makes our method more robust to the choice of hyper-parameters. in addition, we find that a high level of sparsity in language and task fine-tunings is beneficial to performance, as this makes overlaps less likely and poses a lower risk of creating interference between the knowledge they contain. moreover, it makes fine-tunings less prone to overfitting due to their constrained capacity. thus, sparsity is a fundamental ingredient for achieving modularity and composability. these properties in turn allow for systematic generalization to new combinations of tasks and languages in a zero-shot fashion. ## background to establish a broader context for our research, we first provide a succinct overview of current methods for efficient fine-tuning, such as adapters and sft. we then recapitulate the lottery ticket hypothesis, upon which our newly proposed method is built. ## experimental setup to evaluate our new method extensively, we benchmark its zero-shot cross-lingual performance on four distinct tasks: part-of-speech tagging (pos), dependency parsing (dp), named entity recognition (ner), and natural language inference (nli). ## results and discussion we report the average test performance of zeroshot cross-lingual transfer for the best reduction factor (or equivalent k) in we remark that lt-sft's zero-shot performance also exceeds translation-based baselines on the americasnli task, achieving an average accuracy of 51.4%, compared with the 48.7% of the 'translate-train' baseline of @xcite . in figure 2 , we provide a more detailed overview of average cross-lingual model performance across a range of different reduction factors. the results for the lt-sft and rand-sft methods gener-ally improve or stay steady as the number of trainable task parameters increases. on the contrary, there is not such a trend for mad-x, as lower reduction factors may degrade its results. this makes it easier to choose a good setting for this hyper-parameter when using sft. moreover, it is worth stressing again that, contrary to mad-x, this hyper-parameter does not affect inference time. bitfit performs much worse than the other methods which perform language adaptation across all tasks. bearing in mind the strong trend towards increasing performance with increasing k for the other sft methods, it seems likely that bitfit, with two orders of magnitude fewer trainable parameters, lacks the capacity to learn effective task el ro ru th tr xlm-r base, full ft 71.1/54.3 78.3/63.7 74.1/57.8 67.1/55.7 67.5/51.1 xlm-r large, full ft @xcite and language sfts. for additional results at the level of individual languages and an analysis of the efficacy of language adaptation for high-versus low-resource target languages, we refer the reader to appendix c. ## related work within the framework of the lottery ticket hypothesis, a series of improvements have been suggested to make the original algorithm to find winning tickets @xcite more stable: after fine-tuning, @xcite rewind the parameters to their values after a few iterations rather than their values before training, whereas @xcite also rewind the learning rate. in addition, @xcite found that 1) different criteria can be used to select weights as an alternative to the magnitude of their change; 2) different rewinding methods are also effective, such as restoring the original sign, but not the value. in future work, we will investigate whether these variants also benefit our method for cross-lingual transfer, where the lth is used for adaptation rather than pruning. whereas the lth was originally conceived in the vision domain for convolutional architectures, it is also effective for pruning models trained on nlp tasks @xcite , such as neural machine translation, and based on transformer architectures @xcite . recently, @xcite adapted the lth specifically to prune pretrained models after fine-tuning. to the best of our knowledge, @xcite is the only instance where winning tickets were composed in previous work. in their experiment, a set of task-specific masks were linearly combined at inference time, in order to generalize to new tasks in a continuous learning setting. ## conclusion and future work we have presented a new method to fine-tune pretrained models that is both modular (like adapters) and expressive (like sparse fine-tuning). this method is based on a variant of the algorithm to find winning tickets under the framework of the lottery ticket hypothesis. we infer a sparse vector of differences with respect to the original model for each individual language (by modeling unlabeled text) and each individual task (with supervised learning). the adaptations for a language and a task can then be composed with the pretrained model to enable zero-shot cross-lingual transfer. comparing our method with the state-of-the-art baseline in several multilingual tasks, the results have indicated substantial gains across the board in both languages seen and unseen during pretraining (which includes many truly low-resource languages). in future work, our method offers several potential extensions. in addition to the variants to the lottery ticket algorithm surveyed in §6, given the importance of sparsity for modularity ( §5.2), we plan to experiment with additional algorithms previously applied to pruning that can identify and fine-tune a subset of the model parameters, such as diffpruning @xcite and childtuning @xcite . finally, given its simplicity and generality, our method is suited for many other applications of transfer learning in addition to cross-lingual transfer, such as multimodal learning, debiasing, and domain adaptation. the code and models are available online at https: //github.com/cambridgeltl/composable-sft.