SAPT : A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models (2024)

Abstract: The continual learning (CL) ability is vital for deploying large language models (LLMs) in the dynamic world. Existing methods devise the learning module to acquire task-specific knowledge with parameter-efficient tuning (PET) block and the selection module to pick out the corresponding one for the testing input, aiming at handling the challenges of catastrophic forgetting and knowledge transfer in CL. However, these methods tend to address only one of the challenges, ignoring the potential of aligning the two modules to effectively address catastrophic forgetting and knowledge transfer simultaneously. To this end, we propose a novel Shared Attention Framework (SAPT), to align the PET learning and selection via the Shared Attentive Learning & Selection module. Extensive Experiments on two CL benchmarks demonstrate the superiority of SAPT. Moreover, SAPT consistently demonstrates its superiority when we scale it to different model sizes (from 770M to 13B), different model architectures (T5 and LLaMA-2) and unseen tasks.

## introduction endowing the continual learning (cl) ability for large language models (llms) @xcite @xcite to learn different tasks sequentially is crucial for their deployment in the real-world, which allows them to dynamically adapt to novel tasks and acquire additional knowledge @xcite @xcite . however, this scenario presents two significant challenges: (1) catastrophic forgetting (cf), referring to the loss of previously acquired knowledge when learning new tasks @xcite , and (2) knowledge transfer (kt), involving the efficient utilization of knowledge from past tasks to facilitate the learning of new ones @xcite . due to the heavy burden on computation resources, recent attempts study the cl of llms based on parameter-efficient tuning (pet) methods @xcite . inspired by the parameter isolation cl methods @xcite , existing methods can be conceptualized as two pivotal components working in the pipeline fashion. as shown in figure 1 (dashed lines), when a new dialogue generation task arrives, a private pet block is allocated by the learning module to acquire task-specific knowledge and then saved to the pet pool for the following selection module to pick it out when a test sample is coming. however, the designs of each module in current works exhibit certain limitations in effectively dealing with kt and cf challenges. on one hand, the design of learning module is supposed to function to facilitate kt among different tasks. unfortunately, for existing works, the learning of pet block is either performed seperately within each single task @xcite , or kept orthogonal to each other to minimize interference @xcite . such isolation cuts off the potential transfer of acquired knowledge stored in the previous pet blocks and hinders them to assist the current acquisition of new knowledge. on the other hand, the selection module plays the pivotal roles in mitigating cf because only when it is capable of automatically selecting the pet block to which the current input belongs can the llm backbone successfully accomplish the current task. however, it would make llms vulnerable to cf by simply implementing such selection process via the summation @xcite or concatenation @xcite of all existing pet blocks or selecting them from a fixed pet pool @xcite . more importantly, they ignore the opportunity of aligning the two modules to address challenges of cf and kt simultaneously. the intuition is that (illustrated by solid lines in figure 1 ), in order to facilitate kt in the learning of the new task, the learning module should rely on task correlations to leverage the most relevant knowledge in previous pet blocks. and such attentive process, expressed as shared attention in our study, could be naturally repeated by the selection module to resist cf through the combination of the corresponding pet blocks belonging to each testing input. as a result, the end-to-end alignment of these two modules is established via such shared attention. to this end, we propose a novel shared attention framework for parameter-efficient continual learning (sapt) of large language models. in sapt, the shared attentive learning & selection module (sals) is devised, where each training sample is navigated to utilize the optimal combinations of existing pet blocks for completing the current task. this is achieved through an attention weight obtained via instance-level shared attention operation. then inputs in the testing time are capable of following the same shared attention operation to reach the attention weight and pick out the appropriate pet blocks accordingly. however, continually updating the sals leads to the optimal attentive combination only for the newest task, resulting in the forgetting for that of previous ones. thus, we introduce attentive reflection module (arm) to help sals recall what the shared attention operation of inputs from previous tasks should be originally performed with pseudo samples. and the success of arm offers a new perspective for the utilization of generated pseudo samples instead of just blindly mixing them with samples of new tasks for multi-task training. we conduct extensive experiments to evaluate sapt on superni @xcite and long sequence @xcite benchmarks. state-of-the-art performance is achieved by sapt compared with recent pet-based cl methods. moreover, sapt also exhibits superior performance when we scale it to different model sizes (from 770m to 13b), different model architectures, including t5 @xcite (encoder-decoder) and llama-2 @xcite (decoderonly) and previously unseen tasks. the main contributions of this work are summarized as follows: ## problem definition and setup continual learning seeks to address challenges within ongoing sequences. formally, a sequence of tasks {t 1 , . . . , t t } arrive in a streaming fashion. each task t @xmath0, y i t nt @xmath1. at any time step t, the model not only needs to adapt to the t-th task, but also keep performances on all previous tasks. in this study, we delve into the more challenging and practical settings, addressing: (1) diverse task types: unlike previous approaches that merely focus on classification problems @xcite , the model would encounter a sequence of tasks encompassing various types, such as dialogue generation, information extraction, etc. ## conclusion in this paper, we propose sapt, a novel framework for the parameter-efficient continual learning 11648 of llms. in sapt, we ingeniously align the two key processes of parameter-efficient block learning and selection through the shared attention, allowing it to effectively alleviate catastrophic forgetting and promote knowledge transfer simultaneously. more importantly, sapt works under the practical settings where no task-ids are provided for the inputs to select their corresponding parameters. experimental results also demonstrate the applicability of sapt across different parameter-efficient tuning methods, models of varying scales and architectures, highlighting its universality.