Learn it or Leave it: Module Composition and Pruning for Continual Learning (2024)

Abstract: In real-world environments, continual learning is essential for machine learning models, as they need to acquire new knowledge incrementally without forgetting what they have already learned. While pretrained language models have shown impressive capabilities on various static tasks, applying them to continual learning poses significant challenges, including avoiding catastrophic forgetting, facilitating knowledge transfer, and maintaining parameter efficiency. In this paper, we introduce MoCL-P, a novel lightweight continual learning method that addresses these challenges simultaneously. Unlike traditional approaches that continuously expand parameters for newly arriving tasks, MoCL-P integrates task representation-guided module composition with adaptive pruning, effectively balancing knowledge integration and computational overhead. Our evaluation across three continual learning benchmarks with up to 176 tasks shows that MoCL-P achieves state-of-the-art performance and improves parameter efficiency by up to three times, demonstrating its potential for practical applications where resource requirements are constrained.

## introduction continual learning (cl) is a learning paradigm aiming at incrementally acquiring and integrating new knowledge over time without forgetting existing knowledge. this capability is essential for machine learning models to stay effective as they encounter dynamic and evolving real-world environments. while pretrained language models (plms) have demonstrated remarkable capabilities on various static tasks, adapting them for continual task learning remains challenging. in particular, there are three notable challenges for continual learning. (1) avoiding catastrophic forgetting: the newly learned information should not disrupt and degrade previously acquired knowl-edge @xcite . (2) facilitating knowledge transfer: the knowledge from past tasks should be reused for efficient learning of new tasks. (3) maintaining parameter efficiency: the language models need to stay lightweight and effective even if the continual learning sequence scales to hundreds of tasks. to mitigate catastrophic forgetting, a line of prior works adopt the idea of parameter isolation @xcite @xcite , which allocates isolated parameters dedicated for each task to avoid inter-task interference. while parameter isolation typically does not allow knowledge transfer across tasks @xcite , there are attempts to address both challenges of catastrophic forgetting and knowledge transfer at the same time, e.g., by progressively concatenating @xcite or composing task-specific modules @xcite . despite their effectiveness in terms of task performance, parameter isolation methods do not scale well with the number of tasks. when the number of tasks in a continual learning sequence is growing into the hundreds, the progressive expansion of task-specific parameters leads to parameter inefficiency and significantly increases computational and storage costs. in this paper, we address all three continual learning challenges simultaneously and introduce mocl-p, a lightweight continual learning approach that leverages task representation-guided module composition and adaptive pruning. first, to avoid catastrophic forgetting, mocl-p continually adds task-specific modules to plms for learning new tasks while keeping the modules frozen once the training on the respective tasks is finished. in addition, to enable knowledge transfer across tasks, mocl-p allows the model to reuse existing knowledge via module composition. finally, to keep the language model lightweight, mocl-p adopts an adaptive pruning strategy by removing modules with redundant information and retaining only the most salient modules throughout the continual learning process. in our evaluation on three popular datasets as continual learning benchmarks with up to 176 tasks in the learning sequence, mocl-p stands out by not only showing state-of-the-art performance but also outperforming prior algorithms in parameter efficiency by up to three times across benchmarks. to the best of our knowledge, this is the first paper that tackles the three challenges of continual learning simultaneously: mocl-p avoids catastrophic forgetting, allows knowledge transfer and ensures parameter efficiency. thus, mocl-p proposes a sustainable way for continual learning, allowing models to remain lightweight and effective as they evolve with accumulating tasks. the code base for mocl is available online. 1 2 related work ## problem definition continual learning focuses on addressing a series of tasks which arrive in a sequential order. the primary goal is to optimize the model's average performance across all tasks after learning them sequentially. formally, the sequence of tasks is denoted as {t 1 , . . . , t n }. each task contains a set of input samples {(x i n , y i n )}. for the text classification tasks we study in this work, x i n is the input text, y i n is the ground-truth label, and @xmath0, . . . , n } is the task identity. in this work, we focus on rehearsal-free continual learning, i.e., data from earlier tasks is not available when training later tasks. therefore, our model does not suffer from the memory or privacy issues associated with rehearsal-based methods. we assume the task labels are provided during both training and testing, i.e., task-incremental continual learning @xcite . however, mocl-p can be adapted for class-incremental learning, where the task labels are not given during testing, with minor modifications following @xcite . we leave the exploration of other continual learning settings for future work. ## method in this section, we describe mocl-p, our proposed cl approach for language models, as illustrated in figure 1 , which tackles catastrophic forgetting and enhances knowledge transfer with superior parameter efficiency at the same time. ## experimental setup in this section, we describe datasets, training details and baselines for our experiments. ## results and analysis in this section, we present and analyze our experimental results. ## conclusion in this paper, we introduce mocl-p, a novel continual learning approach that addresses the core challenges of catastrophic forgetting, knowledge transfer, and parameter efficiency in continual learning. we utilize learnable task representations for module composition and adaptive pruning, maintaining a lightweight model while achieving state-of-the-art performance across various benchmarks. notably, mocl-p scales effectively to long continual learning sequences, handling up to 176 tasks without compromising performance. these experimental results showcase mocl-p's potential to enhance practical machine learning applications by effectively managing computational costs, thus providing a scalable and efficient solution for real-world scenarios where minimum resource requirements are crucial.