FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing (2024)

Abstract: Standard fine-tuning of language models typically performs well on \textit{in-distribution data} , but suffers with generalization to \textit{distribution shifts} . In this work, we aim to improve the generalization of adapter-based cross-lingual task transfer where such cross-language distribution shifts are imminent. We investigate scheduled unfreezing algorithms –originally proposed to mitigate catastrophic forgetting in transfer learning – for fine-tuning task adapters. Our experiments show that scheduled unfreezing methods close the gap to full fine-tuning and achieve stronger cross-lingual transfer performance, suggesting that these methods can go beyond just mitigating catastrophic forgetting. Next, aiming to understand these empirical findings, we investigate the learning dynamics of scheduled unfreezing using Fisher Information. Our experiments reveal that scheduled unfreezing induces different learning dynamics compared to standard fine-tuning, and provide evidence that the dynamics of Fisher Information during training correlate with cross-lingual generalization performance. We additionally propose a general scheduled unfreezing algorithm that achieves an average of 2 points improvement over four datasets compared to standard fine-tuning and provides empirical evidence for a theory-based justification of the heuristic unfreezing schedule for task adapter training.

## introduction in the standard cross-lingual task transfer setup, a typical and often valid assumption is that only english data is available for fine-tuning and validation of a pretrained multilingual model, due to resource constraints in many languages @xcite . however, models trained in this setup also need to generalize well to text inputs provided in other languages: a requirement that can be seen as 1 https://github.com/ukplab/ naacl2024-fun an extreme but natural case of distribution shifts generalization @xcite . parameter-efficient fine-tuning methods such as adapters @xcite @xcite with separate language and task components are often used to achieve effective cross-lingual transfer, especially to low-resource languages @xcite @xcite . these adapters insert a small number of trainable parameters into a frozen pretrained multilingual language model (e.g., mbert, @xcite to achieve positive transfer while avoiding catastrophic forgetting @xcite of previously learnt knowledge after adapting to new tasks. 2 in other words, adapters enable catastrophic forgetting free learning, referred to as cf-free in this paper. while efficient, adapter methods often incur a cross-lingual performance gap when compared to full model fine-tuning. gradual unfreezing (gu) is a technique which unfreezes layers of deep neural network models from top to bottom during training @xcite . gu was previously proposed for general transfer learning of in-distribution (id) data in monolingual contexts in nlp, and has been predominantly applied to full fine-tuning. more recently, 'linear-probing-then-fine-tuning' (lpft, @xcite was proposed for transfer learning of both id and distribution-shifted evaluation data using full fine-tuning in computer vision. lpft first trains the classification layer only, and then the full model. the main notion connecting these methods is training different layers of a neural network by unfreezing layers at different times (i.e., with a schedule). designed to mitigate cf, these 'scheduled unfreezing' methods have shown promising transfer learning results. however, it is unclear whether scheduled unfreezing can do more than just mitigate cf, and benefit cf-free methods and cross-lingual transfer (which is a different type of distribution shift than previously studied). in this work, we begin by asking the following question: do scheduled unfreezing methods improve cross-lingual transfer and close the gap to full fine-tuning in the cf-free setting? we use scheduled unfreezing to train task adapters following the standard adapter-based cross-lingual transfer setup of @xcite . we find that scheduled unfreezing enhances generalization, bridging the gap to full fine-tuning, confirming our hypothesis that, since cross-lingual transfer can be seen as a form of distribution shifts, methods such as gu are effective, even in the cf-free setting. our results suggest that there indeed is more to the original scheduled unfreezing training than just mitigating catastrophic forgetting @xcite . we further analyze the learning dynamics during training, with a particular focus on gu, using the trace of the fisher information matrix (kleinman et al., 2023, denoted by tr(f ) henceforth). our experiments reveal 1) that scheduled unfreezing changes the dynamics of tr(f ) during training, 2) that tr(f ) is a potential proxy for studying crosslingual generalization. based on our analysis, we then propose an automatic scheduled unfreezing algorithm based on maximizing the tr(f ) (termed fisher unfreezing or fun), to generalize from previous heuristicbased methods. fun achieves comparable results to heuristic-based methods and provides empirical evidence that gu may implicitly maximize tr(f ) during training in our experimental setting. contributions. in sum, our contributions are as follows. 1) to the best of our knowledge, we are the first to demonstrate that scheduled unfreezing in adapter training for cross-lingual transfer closes the performance gap to full model fine-tuning. 2) we present a generalized scheduled unfreezing framework that encompasses several existing methods, allowing easy extensions to new algorithms. 3) we demonstrate that fisher information is an effective tool for studying generalization in cross-lingual transfer and find that the dynamics of fisher information correlate with cross-lingual transfer results. 4) we propose a tr(f )-based scheduled unfreezing method (fun), which achieves comparable performance to heuristic methods and indicates that gu achieves good generalization in adapter training. fisher information and generalization. fisher information @xcite is an established concept in optimization theory and practice @xcite , e.g., to measure parameter importance @xcite or for pruning (singh and alistarh, 2020). @xcite study learning dynamics foot_1 in neural network training with fisher information. @xcite show that fisher information correlates with generalization in computer vision and non-transformer architectures. @xcite propose to regularize fisher information during the training of a neural network for better in-distribution generalization in computer vision. @xcite ; @xcite create sparse masks using fisher information for better parameter-efficient tuning. concurrently, @xcite uses fisher information selecting layers for fine-tuning transformers. we study fisher information in adapters and for cross-lingual transfer, which has the potential to guide the understanding of new methods in this area. the base model) to efficiently adapt the large base model to a specific language @xcite or task @xcite . the prior adapter-based fine-tuning process typically spans two stages for cross-lingual task transfer. first, different language adapters are trained (often separately, per language) with the base model frozen, using the masked language modelling (mlm) objective in a target language. second, task adapters and a task-specific output head are randomly initialized and inserted into the base model along with now-trained source language (i.e., typically english) adapters. in this stage, only the task adapters and a task-specific output head are trainable, while all other parameters are kept fixed. at inference time, the source language adapters are replaced by the language adapter of the target language, while the task adapter is retained, to achieve zero-shot cross-lingual transfer. in this work, we base our main experiments on the state-of-the-art cross-lingual adapter framework: mad-x (pfeiffer et al., 2020). see figure 6 in appendix a for architecture details. ## conclusion in this work, we first investigated whether scheduled unfreezing algorithms can help with generalization in the zero-shot cross-lingual transfer setting, and close the gap between parameter-efficient task-adapter training and full fine-tuning. our experiments showed that scheduled unfreezing was indeed successful in closing this gap. next, we investigated the training dynamics of scheduled unfreezing using the trace of the fisher information matrix (tr(f )). our experiments revealed a link between scheduled unfreezing, tr(f ) and generalization performance. finally, we proposed a general scheduled unfreezing algorithm (tr(f )-based scheduled unfreezing, fun) that achieves performance comparable to existing heuristic variants, across multiple models and datasets. we hope to look into utilizing scheduled unfreezing to improve the cross-lingual generalization capabilities of large language models in the future.