Leitner-Guided Memory Replay for Cross-lingual Continual Learning (2024)

Abstract: Cross-lingual continual learning aims to continuously fine-tune a downstream model on emerging data from new languages. One major challenge in cross-lingual continual learning is catastrophic forgetting: a stability-plasticity dilemma, where performance on previously seen languages decreases as the model learns to transfer to new languages. Experience replay, which revisits data from a fixed-size memory of old languages while training on new ones, is among the most successful approaches for solving this dilemma. Faced with the challenge of dynamically storing the memory with high-quality examples while complying with its fixed size limitations, we consider Leitner queuing, a human-inspired spaced-repetition technique, to determine what should be replayed at each phase of learning. Via a controlled set of quantitative and qualitative analyses across different memory strategies, we show that, just like humans, carefully picking informative examples to be prioritized in cross-lingual memory replay helps tame the stability-plasticity dilemma. Compared to vanilla and strong memory replay baselines, our Leitner-guided approach significantly and consistently decreases forgetting while maintaining accuracy across natural language understanding tasks, language orders, and languages.

## introduction cross-lingual continual learning is a machine learning paradigm aimed at continually adapting a downstream model to datastreams drawn from different languages @xcite . naive approaches to cross-lingual continual learning involve training a new model from scratch each time a new language is available or training jointly over all languages, which can be inefficient and even inaccessible. faced with an overwhelming stream of languages, modelers turn to continual learning techniques to adapt models such that maximal learning from data is achieved when available data is temporally limited. the consequence of a finite data buffer limitation on a potentially infinite data source is catastrophic forgetting @xcite . catastrophic forgetting exemplifies the stability-plasticity dilemma @xcite @xcite : it is inherently hard to preserve the previously acquired knowledge (stability) while learning novel information (plasticity). various continual learning approaches have proposed to mitigate catastrophic forgetting by either restricting entire sets of parameters from changing @xcite @xcite , designing language-specific model components @xcite , or replaying a fixed buffer memory from previously seen languages @xcite . @xcite show that memory-based approaches are more robust than other approaches in taming the stability-plasticity dilemma. moreover, they are more scalable than other approaches, such as model expansion, which grows in complexity as a function of the underlying downstream architecture. experience replay (er) @xcite ) is a cognitively inspired memory-based approach that reinforces previously seen experiences similar to the process of memory consolidation in biological systems @xcite . as more languages are incorporated into the datastream, fitting examples from new languages into a fixed-size memory buffer becomes more challenging. this invites a critical question: how can we dynamically come up with informative memory examples to keep for each language? in this paper, we propose a human-inspired approach for learning what to replay at each phase of cross-lingual continual learning. we hypothesize figure 1 : an overview of leitner-guided memory replay for multi-phase cross-lingual continual learning: on top of a cross-lingual datastream, we build a skill rating system to continually guide the memory population and update. skill ratings are scores from 1 to 5 obtained from leitner queues; a higher score reflects greater learnability. at the end of each phase, the skill ratings on the main data items from the phase language are used to choose what goes in the memory, and the skill ratings of data items already in the memory are re-evaluated to determine if they can remain. our approach is language-agnostic in the sense that it uses leitner queues to determine which examples to keep in the memory at the end of each phase irrespective of the language. ## methodology in this section, we start by describing our er approach adapted to cross-lingual continual learning ( §2.1). then, we explain the mechanism for determining skill ratings based on leitner queues ( §2.2). after that, we explain how we use this leitnerbased skill rating system to guide memory storage and update in cross-lingual er ( §2.3). ## experimental setup we start by presenting the different baselines and model variants used to compare between different experimental scenarios ( §3.1). we then describe the benchmark datasets and their base models ( §3.2) along with the multilingual datastreams ( §3.4) that we focus on in this evaluation. more implementation details, such as the hyperparameters, number of parameters used, and runtime for different models can be found in appendix a. ## results & analysis in this section, we provide an extensive analysis to demonstrate the effectiveness of our leitnerguided cross-lingual experience replay approach. we present both a summary of the test performance based on the best epoch given dev data split performance and over each epoch throughout different training stages ( §4.1). then, we present a more finegrained analysis, shedding light on the language orders and languages for which our leitner-based skill rating system is particularly helpful ( §4.2). last but not least, we present a qualitative analysis of different categories of skill ratings and what makes ruling out hard examples useful ( §4.3). ## related work continual learning work inspired by human-like learning falls into several categories. notable categories include spaced-repetition @xcite @xcite @xcite , mechanisms of sleep @xcite @xcite , and reactivation of memories @xcite . leitner queues, one of the most famous spaced repetition techniques, started garnering attention for machine learning recently. however, most of the work is focused on scheduling when to review data in non-continual learning setups. @xcite show the sample efficiency of a human-inspired memory model to determine when to review each item as a function of the difficulty of the item and the strength of the network. @xcite propose a monte carlo tree search approach for memory replay. more work @xcite demonstrates the effectiveness of leitner queues at determining spurious data and confident labels for self-training applications. our work, by contrast, is the first to test for the effectiveness of leitner queue-based skill ratings in mitigating forgetting in cross-lingual continual learning. some studies, such as @xcite , have looked into analyzing the forgetting problem from the perspective of knowledge transfer and context sensitivity from pre-training to fine-tuning. finegrained analyses such as @xcite look at different types of general knowledge in large language models that are saliently forgotten and at the role that model scale plays in forgetting. to the best of our knowledge, no previous work has analyzed forgetting in the cross-lingual continual learning context. ## conclusion in this paper, we formulate a human-inspired experience replay approach specifically for crosslingual continual learning. we propose a leitnerbased skill rating system to dynamically populate and update the memory with high-quality items. our approach can deal with the stability-plasticity dilemma better than random selection, especially for complex tasks and consistently over languages and language orders. the implications of this analysis include a recipe for how to incorporate aspects of human learning in the design of memory replay in cross-lingual continual learning.