Parameter-Efficient Finetuning for Robust Continual Multilingual Learning (2023)

Abstract: We introduce and study the problem of Continual Multilingual Learning (CML) where a previously trained multilingual model is periodically updated using new data arriving in stages. If the new data is present only in a subset of languages, we find that the resulting model shows improved performance only on the languages included in the latest update (and a few closely related languages) while its performance on all the remaining languages degrade significantly. We address this challenge by proposing LAFT-URIEL, a parameter-efficient finetuning strategy which aims to increase the number of languages on which the model improves after an update, while reducing the magnitude of loss in performance for the remaining languages. LAFT-URIEL uses linguistic knowledge to balance overfitting and knowledge sharing across languages, allowing for an additional 25% of task languages to see an improvement in performance after an update, while also reducing the average magnitude of losses on the remaining languages by 78% relative.

## introduction a learning-based nlp model may need to be periodically updated for a variety of reasons, e.g., to incorporate newly available training data, adapt to data shifts, etc. continual learning @xcite and online learning @xcite are paradigms where a model is sequentially trained on packets of new training data, without having access to old training data. in such settings, the goal is to ensure that the model is able to incorporate incremental knowledge from the new data without forgetting the knowledge obtained from prior training. as multilingual nlp grows in prominence, the underlying models used for multilingual tasks are increasingly being developed as a single deep neu-ral network trained on data from all supported languages @xcite @xcite . having a shared multilingual model instead of one model per language allows one to reduce the number of models to train and maintain for the downstream task, improve performance on lower-resource languages due to crosslingual sharing of knowledge, and improve inference on code-mixed inputs. just like monolingual models, multilingual nlp models also need to be regularly updated, thereby making them suitable for application of continual learning strategies. however, continual learning of multilingual models involves additional challenges due to involvement of multiple languages. for instance, during any update, the new training data may cover only a small subset of the languages, which may negatively impact the performance on languages not represented in this new data. this scenario is often true for multilingual models deployed in production settings. in spite of its importance and real-world significance, the problem of continual multilingual learning (cml) has not been much explored. we fill this gap in this paper. in the cml setting, a single-task multilingual model needs to be continually updated with additional training data from a subset of the supported languages arriving in stages, while keeping the model capacity fixed and without relying on any data from previous training steps. given a shared multilingual model, the goal of updating it on new data for the same task would be to (1) improve the model performance across most, if not all, languages and (2) ensure that none of the languages incur a significant loss in performance. the second scenario may occur if the new training data is highly skewed towards a subset of languages, making it easier for the model to overfit on the language specificities of the new data while forgetting the same for languages not represented in this update. in our study, we find that balancing the two goals is we observe that the additional training results in improved performance on the new data languages (hindi) and a few closely related languages (due to positive transfer), while negatively affecting the remaining languages. this paper proposes strategies to maximize positive transfer while minimizing negative impact on the remaining languages during cml. in contrast, cml builds on top of an existing multilingual model while keeping model capacity fixed. we start with the intuition that constraining the number of trainable parameters in the network would help control for losses due to languagespecific forgetting. we operationalize this through different parameter-efficient finetuning strategies, namely adapters @xcite and composable sparse-finetuning @xcite . we find that such methods provide a middleground by allowing limited cross-lingual sharing of knowledge while reducing model's tendency to overspecialize on the languages of the new data. with this initial promise, we develop laft-uriel, a novel finetuning strategy which uses adapters and uriel language similarity metrics @xcite to balance the trade-off between encouraging positive cross-lingual transfer and discouraging language-specific forgetting. our contributions are as follows: we present the cml setup in figure 1 . ## problem setup we consider a setting where a trained, task-specific multilingual model, which we will call as the deployed model, is further finetuned on new finetuning data for the same task, to give us the updated model. to ensure the best possible multilingual performance, we will assume that the deployed model has been previously trained on data from all supported languages for the task (say n l in number). in an update, the deployed model will be finetuned on new task-specific data, to give us the updated model. in the real world, as one may have no control over how the new data is distributed across languages, we will assume the worst case scenario for our setup (i.e., maximum skew) where the new data is only present in one of the n l languages. we divide the entire setup into two stages: the t th continuation stage can be written as: to compare different finetuning strategies, we focus on the @xmath0. for a task in n l languages, there can be n l different continuation stages to transition from the deployed model at @xmath1. we consider all such cases. training data for each finetuning stage is created by partitioning the full training data into equal parts, independently across all languages. on a fixed test set, we expect the language-wise performance of the deployed model and the updated model to differ due to multiple reasons: (1) additional task-learning using new task data (2) catastrophic forgetting of language-specific knowledge (3) positive or negative cross-lingual transfer. ideally, we would want the performance delta to be positive or neutral across each language. hence, the goal is to devise strategies that encourage language-agnostic task learning and positive ## baseline finetuning strategy and metrics the baseline finetuning strategy in our setup would be finetuning while keeping all the parameters of the model trainable during a continuation stage. we call this the full finetuning (fft) baseline. we anticipate that continued finetuning on new task data (which is skewed towards a particular language) would cause non-uniform changes in language-wise performance on a fixed test set. in particular, we expect performance gains on the language seen during a continuation stage and losses across some subset of the remaining languages. to compare different finetuning strategies, we measure the percentage change in language-wise performance after the continuation stage foot_1 . for the updated model to be fit for deployment (e.g., in a production setting), it is necessary to ensure that the performance drop on any language is not too high. also, given the shared multilingual model, an ideal strategy should be able to spread the gains in performance across most if not all languages. to this end, we construct the following metrics to compare different strategies in our setup: avgpercentloss: average magnitude of percentage loss after continuation. calculated by averaging the absolute percentage change in performance over all languages which suffered a loss in performance. for an ideal model, this should be 0. numimprovedlangs: average number of languages with a positive change in performance after a continuation stage. for an ideal model, this should be the count of all supported languages for a given task, n l . since there can be n l different continuation stages (where the new data is only present in one of n l languages for the task), we report the average of the above two metrics across all such transitions. additional constraints: after a continuation stage, we would at the very least expect that, (1) the sum of gains are higher than the magnitude of the sum of losses and (2) the magnitude of maximum gain are higher than the magnitude of the maximum loss in performance. a finetuning strategy which is unable to obey these constraints can simply be declared as unfit for our setup. we therefore compute an average of sum(gains)/abs(sum(losses)) (sumratio) and max(gains)/abs(max(losses)) (maxratio) across the different continuation stages and check whether the two values @xmath2. ## parameter-efficient finetuning for continual multilingual learning we propose the use of parameter-efficient finetuning methods to build improved finetuning strategies in our setup. the benefits of using these methods would be two-fold. firstly, such methods should allow one to constrain the changes being made in the model, which should help in controlling the losses due to forgetting. secondly, these methods can be used to decompose task learning into languagespecific and language-agnostic parts. this property can be used to update the model in a languageagnostic fashion which should help in spreading gains across languages. the following subsection will give an overview of the methods we intend to use to build improved finetuning strategies for the task. ## comparison of finetuning strategies in this section, we use the metrics defined in §3 to compare the four finetuning strategies (fft, sft, laft, laft-uriel) on the four tasks described in §2. it is important to note that the absolute language-wise performances of the deployed models in sft, laft and laft-uriel cases are at par or slightly greater than the same for fft (see appendix d). since our metrics are computed over changes in performance, the above fact ensures that the comparison is fair (or slightly favourable towards fft). we ask the following research questions: foot_2 1. how does the behaviour of our proposed strategy differ from that of the naïve baseline ( §5.1), 2. do parameter-efficient finetuning methods improve spread of gains after while constraining the losses in our setup? ( §5.2 and §5.3), 3. how does our proposed strategy perform when there are multiple continuation stages? ( §5.4) ## conclusion in this paper we introduce and study the problem of continual multilingual learning (cml) where a multilingual model is continually updated using new data from a subset of the languages at a time. we observe that unconstrained updates to the model can lead to drastic losses for a subset of the languages, especially those not covered during an update. we propose laft-uriel, a parameterefficient finetuning stragegy which uses linguistic information to effectively balance overfitting and knowledge sharing across different languages, resulting in 25% increase in the proportion of task languages whose performances improve during an update while achieving 78% relative decrease in average magnitude of losses on the remaining languages.