Compositional Language Continual Learning

1 INTRODUCTION . Continual Learning is a key element of human intelligence that enables us to accumulate knowledge from a never ending stream of data . From machine learning perspective , there is no guarantee that information accessed at a current task to be revisited later in future tasks . This leads to what is known as Catastrophic Forgetting ( McCloskey & Cohen , 1989 ; McClelland et al. , 1995 ) ; significant drop in previously obtained knowledge of an AI system as it learns new information and gets less/no exposure to old information . Several approaches have been proposed to bridge the gap between machine and human continual learning skills with catastrophic forgetting being the central problem . Existing continual learning methods have focused mostly on classification tasks ( e.g . ( Rebuffi et al. , 2017 ; Lopez-Paz & Ranzato , 2017 ; Shin et al. , 2017 ; Li & Hoiem , 2016 ; Shmelkov et al. , 2017 ; Triki et al. , 2017 ; Li & Hoiem , 2016 ; Triki et al. , 2017 ; Rusu et al. , 2016 ; Lee et al. , 2017 ; Elhoseiny et al. , 2018 ; Kirkpatrick et al. , 2017c ; Zenke et al. , 2017 ; Chaudhry et al. , 2018 ) ) . In this paper , we propose a new scenario of continual learning which handles sequence-to-sequence tasks common in language learning . Continual language learning is an open question which has not been studied extensively in machine learning and NLP domains . It may facilitate a variety of applications in NLP systems . For example , it enables a robot to keep on learning new tasks via natural language instruction , a conversational agent to adapt to new conversation topics quickly , and a neural machine translation system to expand its vocabulary continually . Humans learn language by leveraging systematic compositionality , the algebraic capacity to understand and produce large amount of novel combinations from known components ( Chomsky , 1957 ; Montague , 1970 ) . Compositional generalization is critical in human cognition ( Minsky , 1986 ; Lake et al. , 2017 ) . It also helps humans acquire language from a small amount of data , and expand vocabulary sequentially ( Biemiller , 2001 ) . In contrast to humans ’ such ability , state-of-the-art continual learning approaches do not achieve the expected generalization . Table 1 and Figure 3 show the performance of state-of-the-art approaches ( Kirkpatrick et al. , 2017a ; Aljundi et al. , 2018 ) when tested ∗Corresponding author : yuanpeng16 @ gmail.com †Work partially done while visiting Baidu Research in instruction learning and machine translation tasks . This highlights the lack of generalization of these approaches , designed after classification tasks , on sequence generation language tasks and the importance of studying the design of continual learning methods for language learning . Modeling continual language learning with improved compositional understanding is at the heart of this paper . More concretely , we address the challenge of open and growing vocabulary problem with continual learning . It requires optimizing over two objectives . First , previously learned knowledge should be transferred and combined with new knowledge . Second , the learned model should resist catastrophic forgetting ( Kirkpatrick et al. , 2017b ) , where a model adapted to a new distribution no longer works on the original one . To achieve these objectives , we use compositionality to separate semantics and syntax of an input sentence , so that we can convert label prediction algorithm to sequence to sequence algorithm for continual learning . The contributions of this paper can be summarized as follows . • We propose a new scenario of continual learning which handles sequence-to-sequence tasks common in language learning . • We propose an approach to use label prediction continual learning algorithm for sequenceto-sequence continual learning by leveraging compositionality . To our knowledge , this is the first work for applying compositionality to continual learning of sequence-to-sequence tasks , targeting at both knowledge transfer to later stages and catastrophic forgetting prevention on previous stages . • Experiments show that the proposed method has significant improvement over multiple state-of-the-art baselines in both knowledge transfer and catastrophic forgetting prevention with almost 85 % accuracy up to 100 stages on language instruction task . It also shows significant improvement in a machine translation task . 2 RELATED WORK . Our work is closely related to compositionality and continual learning or lifelong learning . Here , we briefly review some related work in these areas . Compositionality Compositional generalization is critical in human cognition ( Minsky ( 1986 ) ; Lake et al . ( 2017 ) ) , and it helps humans acquire language from a small amount of data , and expand vocabulary sequentially ( Biemiller ( 2001 ) ) . Therefore , researchers have been studying how to enable human-level compositionality in neural networks for systematic behaviour ( Wong & Wang , 2007 ; Brakel & Frank , 2009 ) , counting ability ( Rodriguez & Wiles , 1998 ; Weiss et al. , 2018 ) and sensitivity to hierarchical structure ( Linzen et al. , 2016 ) . Recently , people proposed multiple related tasks ( Lake & Baroni , 2018 ; Loula et al. , 2018 ; Lake et al. , 2019 ) and methods ( Lake & Baroni , 2018 ; Loula et al. , 2018 ; Kliegl & Xu , 2018 ) with different kinds of RNN models and attention mechanisms . Though these methods enable generalization when the training and test sentences have small difference , it has been an open problem ( Yang et al. , 2019 ) to reach human-level compositionality generalization . More recently , Li et al . ( 2019 ) proposed an entropy regularization method that achieves high performance on several NLP tasks . In this paper , we study compositionality from continual learning angle . By leveraging the compositional learning approach , we propose the con- tinual learning algorithm by encoding compositionality into DNN . To our knowledge , our work is the first to apply compositionality to continual learning in DNN . Continual learning Continual learning or lifelong learning involves multiple stages . Each stage has a set of classes and corresponding data , and the training can only access the data in the current stage . Based on the way for overcoming catastrophic forgetting , continual learning work may be categorized into data-based and model-based approaches . In data-based approaches , some methods store previous data either with replay buffer ( Rebuffi et al. , 2017 ; Lopez-Paz & Ranzato , 2017 ) or generative model ( Shin et al. , 2017 ) ; other approaches ( Li & Hoiem , 2016 ; Shmelkov et al. , 2017 ; Triki et al. , 2017 ) , employ the new task data to estimate and preserve the model behavior on previous tasks , mostly via a knowledge distillation loss as proposed in Learning without Forgetting ( Li & Hoiem , 2016 ) . These approaches are typically applied to a sequence of tasks with different output spaces . To reduce the effect of distribution difference between tasks , ( Triki et al. , 2017 ) propose to incorporate a shallow auto-encoder to further control the changes to the learned features , while ( Aljundi et al. , 2017 ) train a model for every task ( an expert ) and use auto-encoders to help determine the most related expert at test time given an example input . In model-based approaches , some methods dynamically increase model size for the growing information ( Rusu et al. , 2016 ; Xu & Zhu , 2018 ) ; other methods ( Fernando et al. , 2017 ; Lee et al. , 2017 ; Kirkpatrick et al. , 2017c ; Zenke et al. , 2017 ) focus on the parameters of the network . The key idea is to define an importance weight ωk for each parameter θk in the network indicating the importance of this parameter to the previous tasks . When training a new task , network parameters with high importance are discouraged from being changed . In Elastic Weight Consolidation , ( Kirkpatrick et al. , 2017c ) estimate the importance weights Ω based on the inverse of the Fisher Information matrix . ( Zenke et al. , 2017 ) propose Synaptic Intelligence , an online continual model where Ω is defined by the contribution of each parameter to the change in the loss , and weights are accumulated for each parameter during training . Memory Aware Synapses ( Aljundi et al. , 2018 ) measures Ω by the effect of a change in the parameter to the function learned by the network , rather than to the loss . This allows to estimate the importance weights not only in an online fashion but also without the need for labels . Finally , Incremental Moment Matching ( Lee et al. , 2017 ) is a scheme to merge models trained for different tasks . Modelbased methods seem particularly well suited for our setup , given that we work with an embedding instead of disjoint output spaces . In this paper , we propose a method with minimal increase of model structure in each stage , and we leverage compositionality with explainable mechanisms that align with human learning . 3 CONTINUAL LEARNING WITH COMPOSITIONALITY . 3.1 PROBLEM DEFINITION . Conventional continual learning algorithms are designed after fixed size input and label classification output . However in many tasks , such as language , both input and output are sequences and bridging this gap between continual learning and sequence-to-sequence models is at the heart of our work . We facilitate more accurate continual sequence-to-sequence artificial learner by proposing an approach that can leverage Label Prediction Continual Learning ( LP-CL ) compositionally into Sequence-toSequence Continual Learning ( S2S-CL ) . LP-CL : Label Prediction Continual Learning In LP-CL , we consider a word to label mapping problem , with input word x and corresponding output label y . In initial learning stage , y takes one of K classes : y ∈ Vinit = { c1 , c2 , . . . , cK } . In continual learning stage , y takes a new class : y ∈ Vcont = { cK+1 } . In test , y takes all previous classes : y ∈ Vinit ∪ Vcont . For example , in language instruction task , input x is a primitive word , and output Y is the corresponding action symbol ; in word-level machine translation , input x is an English content word , and output Y is the corresponding French word . In initial training stage , we have multiple input word and output symbol pairs . In continual learning stage , we have a new input and output word pair . We train a model in initial training stage , and do not use the initial training data any longer in the rest of training stages . We then switch to the data in continual learning stage , and continually update the model . In test stage , we evaluate whether model can predict labels from both initial and continual learning stages . We denote label prediction continual learning model ( LP-CL ) as P ( y|x ; θ ) . S2S-CL : Sequence to Sequence Continual Learning For sequence to sequence continual learning ( S2S-CL ) , we consider sequential input X = x1 , x2 , . . . , xn and output Y = y1 , y2 , . . . , ym . Each output label yi , i ∈ { 1 , . . . , m } is from the corresponding label set in label prediction problem . We want to make a model P ( Y |X ) for sequence to sequence continual learning . Our goal is to facilitate better Sequence to Sequence Continual Learning ( S2S-CL ) capability quantified as P ( Y |X ) by leveraging access and joint-learning with Label Prediction Continual Learning ( LP-CL ) model , P ( y|x ; θ ) .

Summary: The paper proposes a continual learning algorithm for label prediction to deal with sequence-to-sequence continual learning problems. The proposed method is designed to leverage compositionality.  The key idea of the proposed method is to enable the network to represent syntactic and semantic knowledge separately. This allows the neural network to leverage compositionality for knowledge transfer while alleviating catastrophic forgetting.