Domain Incremental Lifelong Learning in an Open World (2023)

Abstract: Lifelong learning (LL) is an important ability for NLP models to learn new tasks continuously. Architecture-based approaches are reported to be effective implementations for LL models. However, it is non-trivial to extend previous approaches to domain incremental LL scenarios since they either require access to task identities in the testing phase or cannot handle samples from unseen tasks. In this paper, we propose Diana: a dynamic architecture-based lifelong learning model that tries to learn a sequence of tasks with a prompt-enhanced language model. Four types of hierarchically organized prompts are used in Diana to capture knowledge from different granularities. Specifically, we dedicate task-level prompts to capture task-specific knowledge to retain high LL performances and maintain instance-level prompts to learn knowledge shared across input samples to improve the model’s generalization performance. Moreover, we dedicate separate prompts to explicitly model unseen tasks and introduce a set of prompt key vectors to facilitate knowledge sharing between tasks. Extensive experiments demonstrate that Diana outperforms state-of-the-art LL models, especially in handling unseen tasks.

## introduction an essential ability of humans is to learn new tasks continuously in their lifetime since our surrounding world is ever involving @xcite . humans need to learn inputs from unseen new tasks everyday. however, neural network based nlp models tend to rapidly lose previously acquired knowledge when trained on new tasks. this phenomenon is referred to as catastrophic forgetting task 𝑇 task 𝑇 task 𝑇 general prompt format prompt task prompt meta prompt ꞏꞏꞏ 𝑃 𝐹 𝑃 𝐹 𝑃 𝑇 ꞏꞏꞏ 𝑃 𝑇 𝑃 𝑇 ꞏꞏꞏ 𝑃 ꞏꞏꞏ 𝑃 𝑃 𝑃 𝑃 figure 1 : an overview of diana. a pre-trained language model is used to learn tasks in different formats with hierarchically organized prompts. @xcite , and it's important to equip nlp models with the lifelong learning (ll) ability to alleviate this issue in advanced ai applications. an effective method to build ll models is the architecture-based approach @xcite @xcite , in which task-specific components are used to isolate knowledge for each separate task @xcite . recently, to leverage the power of pre-trained language model (plm), some architecture-based ll models convert nlp tasks into a unified language modeling (lm) format @xcite and learn these tasks using a plm. separate prompts @xcite or adapters @xcite are allocated for different tasks to avoid the catastrophic forgetting issue. however, despite the reported effectiveness, most above models are designed for the task incremental learning scenario, in which we assume task ids for testing samples are available @xcite . this setting limits the application of ll models because practical applications usually follow a more general domain incremental learning scenario @xcite , i.e., we cannot access the task ids of most input samples. there are generally two approaches to building ll models for domain incremental learning. one is to predict the task id of each testing sample @xcite , and activate specified components based on the prediction (figure 2a ). this scheme achieves high ll performances if the predicted id is correct @xcite . however, these models cannot handle samples from unseen tasks since there are no components designated for these samples and thus no task ids to be predicted. this hinders the application of ll models because we often encounter samples from unseen tasks in practical situations @xcite . another approach to building domain incremental ll models is to organize model components at the instance-level, i.e., a pool of fine-grained components are dynamically combined in the forward pass for each input instance (figure 2b ). this approach avoids the trouble of explicitly determining task ids. however, it usually yields low ll performance because there are no dedicated components for each task to capture task-specific knowledge @xcite . in this study, we combine the advantages of the above two approaches and propose diana: a dynamic architecture-based lifelong learning model. we convert different nlp tasks into a unified lm format and propose to learn these tasks using a prompt-enhanced plm (figure 1 ). specifically, diana maintains four types of prompts to capture task knowledge from different granularities: 1. a general prompt p g is used for all tasks; 2. the format prompts p f are shared between tasks in a similar format; 3. a task prompt p t is assigned for each incoming task; 4. a pool of meta prompts p m are dynamically combined for each input instance. these four types of prompts present a hierarchical structure with a decreasing knowledge granularity, i.e., p g captures global knowledge between all tasks, while p m captures local knowledge that is shared between instances. diana can better generalize to unseen tasks while achieving high ll performances since its components are organized at both task and instance level. moreover, we also maintain key vectors for p t and p m to better share task knowledge, and allocate separate task prompts to explicitly model samples for unseen tasks. extensive experiments on benchmark nlp tasks indicate that diana outperforms state-of-the-art (sota) baselines, especially in handling unseen tasks. our main contributions are: ## related work lifelong learning aims at incrementally acquiring new knowledge without catastrophically forgetting previously learned ones. generally, three categories of ll methods are proposed: 1. rehearsal-based methods @xcite @xcite @xcite preserve past knowledge by replaying data from learned tasks; 2. regularization-based methods @xcite @xcite @xcite ) consolidate model parameters that are important to previous tasks by introducing additional regularization terms; 3. architecture-based methods @xcite @xcite add task-specific parameters to an existing base model for each task to prevent forgetting. experiment settings of ll methods can be generally classified into three scenarios based on whether the task id is provided for testing samples and whether it must be inferred (van de ven and tolias, 2019), i.e., task-incremental learning @xcite , domain-incremental learning @xcite , and class-incremental learning @xcite . in this work, we focus on the domain-incremental learning setting, where task id is not provided for each testing sample. one line of methods in this category attempt to detect the task id for each input sample @xcite . however, these methods fail to generalize to unseen tasks @xcite . another line of methods try to build a dynamic architecture for each input sample, for example, maintaining a pool of prompts that can be dynamically combined @xcite . however, these methods yield sub-optimal performance since no task-specific parameters are used. our model diana is the first attempt to take advantage of the two aforementioned types of methods. pre-trained lm is becoming the de facto standard component for nlp models. to encourage knowledge sharing, existing approaches attempt to cast all nlp tasks into a unified text-to-text format @xcite and learn these tasks by finetuning a plm. a similar work compared to ours is proqa @xcite , in which different qa tasks are unified and a set of structured prompts are used. however, proqa only considers two qa tasks and is limited to the task incremental learning scenario, while our model is designed to tackle more general nlp tasks in a more general domain incremental learning scenario. ## conclusion we propose diana, a novel ll model for the domain incremental learning scenario. diana converts different nlp tasks into a unified sequence generation format and uses a prompt-enhanced plm to learn these tasks. we introduce four types of hierarchically organized prompts in diana to capture knowledge in different granularities. these prompts are shared between different scopes of samples and are dynamically combined based on a set of key vectors. the space of key vectors is learned with several distance-based regularization terms. dedicated components are also allocated in diana to model samples from unseen tasks. experiments and empirical analysis on two sets of tasks show that diana outperforms sota ll models, especially in handling samples from unseen tasks.