Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System (2022)

Abstract: Catastrophic forgetting is a challenge for model deployment in industrial real-time systems, which requires the model to quickly master a new task without forgetting the old one. Continual learning aims to solve this problem; however, it usually updates all the model parameters, resulting in extensive training times and the inability to deploy quickly. To address this challenge, we propose a parameter-efficient continual learning framework, in which efficient parameters are selected through an offline parameter selection strategy and then trained using an online regularization method. In our framework, only a few parameters need to be updated, which not only alleviates catastrophic forgetting, but also allows the model to be saved with the changed parameters instead of all parameters. Extensive experiments are conducted to examine the effectiveness of our proposal. We believe this paper will provide useful insights and experiences on developing deep learning-based online real-time systems.

## introduction in industry, many text-related applications have enjoyed a superior performance boost from the emerging of pre-trained language models, such as word2vec @xcite , @xcite , gpt @xcite , and bert @xcite @xcite . however, when a fine-tuned model needs to be updated to master a new task swiftly, it usually loses the ability to handle previous tasks. this phenomenon is known as catastrophic forgetting @xcite , and it poses a significant issue in industrial settings. continual learning aims to incrementally expand acquired knowledge for future learning @xcite , and mitigate the impact of catastrophic forgetting in the meantime. existing continual learning methods usually use data replay * * corresponding author. @xcite , parameter isolation @xcite , and regularization @xcite to make models adapt to new tasks without catastrophic forgetting. however, these approaches lack research on implementing continual learning in industrial scenarios, where endowing models with continual learning capabilities meets numerous practical constraints. for time constraints, when new data or tasks arrive, the model should be launched in minutes or even seconds, which is common in time-sensitive scenarios, e.g., blocking certain rumors content. for space constraints, the strong demand for tracing tasks makes it necessary to save every model once it is changed. so for the current large-scale pre-trained models, storage becomes an industrial challenge with the increase of new tasks. to solve these industrial challenges, we propose a parameter-efficient continual learning framework based on an offline parameter selection strategy. the framework consists of two parts, i.e., offline calculation and online training. in the offline calculation part, all the parameters that are important to the old task are selected to be fixed, while the remaining parameters are employed to learn the new task. since in a real industrial scenario, the arrival of a new task will have an interval of hours or even days, we can make full use of this interval to advance the selection of parameters. during the online training phase, the model is parameterefficiently trained on a new task within a small set of parameters and further combines multiple regularization-based methods @xcite to overcome catastrophic forgetting. to alleviate storage costs, we only save the modified parameters for each snapshot. extensive experiments demonstrate that our framework can maintain the old task performance while learning a new task quickly. our implementa-tion is based on uer-py pre-training toolkit 1 @xcite . the main contributions of this paper can be summarized as follows: ## methodology we introduce a parameter-efficient continual learning framework, as shown in figure 1 . our frame- mask fisher information matrix calculation selection parameter inheritance bert parameters after selection ewc loss lwf loss ce loss regularization-based methods ## conclusion this paper introduces a parameter-efficient continual learning framework, which is designed for realtime incremental learning system. in offline stage, the framework identifies the parameters that are less important to the old tasks. by updating these parameters in online training stage, the model is able to learn new tasks in short time without forgetting the old ones. furthermore, we surprisingly find that decent results can be achieved by only training a small subset of parameters (e.g. 0.1%). this observation enables us to largely decrease the storage of the snapshot, which is important for the system requiring frequent update.