Continual Sequence Generation with Adaptive Compositional Modules (2022)

Abstract: Continual learning is essential for real-world deployment when there is a need to quickly adapt the model to new tasks without forgetting knowledge of old tasks. Existing work on continual sequence generation either always reuses existing parameters to learn new tasks, which is vulnerable to catastrophic forgetting on dissimilar tasks, or blindly adds new parameters for every new task, which could prevent knowledge sharing between similar tasks. To get the best of both worlds, in this work, we propose continual sequence generation with adaptive compositional modules to adaptively add modules in transformer architectures and compose both old and new modules for new tasks. We also incorporate pseudo experience replay to facilitate knowledge transfer in those shared modules. Experiment results on various sequences of generation tasks show that our framework can adaptively add modules or reuse modules based on task similarity, outperforming state-of-the-art baselines in terms of both performance and parameter efficiency. We make our code public at https://github.com/GT-SALT/Adaptive-Compositional-Modules .

## introduction current state-of-the-art language generation models can achieve great performance on a wide range of sequence generation tasks @xcite with a static data distribution. however, real-world scenarios are often changing which requires the model to learn with dynamic data distributions. in such cases of data distributions shift, current generation models often suffer from catastrophic forgetting @xcite : models completely and abruptly forget previously learned information upon learning new information. continual learning (cl) @xcite has been introduced to improve model's ability to learn tasks in a stream by mitigating forgetting and facilitating knowledge transfer @xcite , however, continual sequence generation is relatively under-investigated. comparing to continual learning on text classification and question answering @xcite @xcite , continual sequence generation is more challenging, since the output is no longer discrete labels but sequential text data in different styles/domains. based on how to retain old knowledge while learning new tasks, current continual sequence generation methods can be categorized into two types. the first one continually learns new tasks on old parameters (fig 1 a ), with approaches like experience replay @xcite and regularization @xcite to maintain old knowledge. however, since all tasks share the same parameters, some degree of interference between tasks is unavoidable. another line of work continually inserts new task-specific modules (adapters proposed by @xcite into every transformer layer for every new task while freezing pretrained mod-els and modules used by old tasks @xcite , which might prevent knowledge transfer between tasks and introduce possible parameter redundancy. in this work, we aim to get the best of both worlds: how to encourage the models to reuse modules from previous tasks as much as possible and to only add new modules if needed? to this end, we propose continual sequence generation with adaptive compositional modules, as shown in fig 1 c . specifically, we introduce a two-stage process for every new coming task: a decision stage and a training stage. during decision stage, we decide which modules to reuse and whether we need to add a new module. during training stage, the model architecture is determined and fixed. we augment new task's training process with pseudo experience replay @xcite to further mitigate forgetting and facilitate knowledge transfer in those shared layers. our model architecture is adaptive, as it can automatically add new modules for dissimilar tasks and reuse modules for similar tasks, thus making it robust to different scenarios of continual learning. furthermore, it is compositional because for every new task, our new architecture is composed of reused modules from old tasks and newly added modules, which allows knowledge reuse and transfer. to evaluate the above adaptive compositional framework, we experiment with four representative sequence generation tasks following prior work @xcite : natural language generation, sql query generation, summarization and task-oriented dialogue arriving in a stream. different from prior work that only tests their methods on very short task sequences or long task sequences with similar tasks only, we validate our approach on longer sequences containing diverse tasks with different levels of similarity. we believe this is a suitable scenario to validate both the model's ability to mitigate forgetting and its ability to facilitate knowledge transfer. in summary, this work makes two key contributions: (1) we propose continual sequence generation with adaptive compositional modules, to maximize knowledge transfer via module-reusing while adaptively adding new modules to mitigate task-interference and catastrophic forgetting. (2) experiments with longer and more task sequences show that our approach outperformed baselines with higher parameter efficiency. avoiding privacy concerns, this work also follows a line of work that doesn't store real examples for experience replay, such as generating examples by gan @xcite , synthesizing examples @xcite by model-inversion @xcite , and using unlabeled data in the learning environment @xcite . in language domain, lamol @xcite trains the language model to solve current tasks and generate current training examples simultaneously, then this model can generate "pseudo" old examples for replay before any new tasks. we adopt this pseudo experience replay along to alleviate the forgetting in the shared modules of our approach. ## two-stage methods motivated by prior continual sequence generation work @xcite that uses adapter @xcite to insert new adapter module into every transformer layer for each new coming task, we propose to strategically decide whether we can reuse some adapter modules from old tasks before training on each new coming task, in a twostage manner: decision stage and training stage, where the former determines the architecture for new tasks and the later trains the model. ## results and analysis to evaluate the overall performance on all tasks, we use the mean of all tasks' performance score following @xcite ; @xcite ; @xcite . for each scenario (similar tasks and dissimilar tasks), we report the average of mean scores on all sequences as an overall metric. be-yond these, we also provide (i) evaluation results using geometric mean and (ii) final performance of each task in appendix a. ## conclusion this work examined continual sequence generation with adaptive compositional modules, where we proposed hidden state mixing to adaptively compose old and new modules for new tasks and utilized pseudo experience replay to facilitate knowledge transfer. experiments conducted on various sequence generation tasks demonstrated that our method achieves better performances with higher parameter efficiency over previous state-of-the-art baselines, both on similar task sequences and dissimilar task sequences. our work is also subject to a few limitations such as the introduced extra training time. in the future, we plan to investigate how to further speed up the decision stage more efficiently and generalize the current framework to more diverse nlp tasks such as text classification and machine translation. note that the original design of adapter+cl @xcite uses perplexity to distinguish which task each testing example belongs to. in this work, we ignore that part and assume that the task-id of each testing example is given during inference for all baselines and our approach to ensure fair comparison.