Compositional Language Continual Learning
Authors: Yuanpeng Li
Liang Zhao
Kenneth Church
Mohamed Elhoseiny
Venue:  (2020)
Keywords: Compositionality
Continual Learning
Lifelong Learning
Sequence to Sequence Modeling

Abstract: Motivated by the human's ability to continually learn and gain knowledge over time, several research efforts have been pushing the limits of machines to constantly learn while alleviating catastrophic forgetting. Most of the existing methods have been focusing on continual learning of label prediction tasks, which have fixed input and output sizes. In this paper, we propose a new scenario of continual learning which handles sequence-to-sequence tasks common in language learning. We further propose an approach to use label prediction continual learning algorithm for sequence-to-sequence continual learning by leveraging compositionality. Experimental results show that the proposed method has significant improvement over state-of-the-art methods. It enables knowledge transfer and prevents catastrophic forgetting, resulting in more than 85% accuracy up to 100 stages, compared with less than 50% accuracy for baselines in instruction learning task. It also shows significant improvement in machine translation task. This is the first work to combine continual learning and compositionality for language learning, and we hope this work will make machines more helpful in various tasks.

Published as a conference paper at ICLR 2020
COMPOSITIONAL LANGUAGE CONTINUAL LEARNING
Yuanpeng Li Liang Zhao, Kenneth Church Baidu Research
Mohamed Elhoseiny KAUST, Stanford University
ABSTRACT
Motivated by the human's ability to continually learn and gain knowledge over time, several research efforts have been pushing the limits of machines to con- stantly learn while alleviating catastrophic forgetting (Kirkpatrick et al. (017b)) Most of the existing methods have been focusing continual learning of label prediction tasks, which have fixed input and output sizes. In this paper, we pro- pose new scenario of continual learning which handles sequence-to-seqeeme tasks common in language learning. We further propose an approach to use la- bel prediction continual learning algorithm for sequence-to-ssqqueece continual learning by leveraging compositionality (Chomsky 1957). Experimental results show that the proposed method has significant improvement over ofate-of-the-art methods. It enables knowledge transfer and prevents catastrophic forgetting, re- sulting in more than 85% accuracy up to 100 stages, compared with less than 50% accuracy for baselines in instruction learning task. It also shows signifi- cant improvement in machine translation task. This is the first work to combine continual learning and compositionality for language learning, and we hope this work will make machines more helpful in various tasks. The code is available at: https://qithub: com http://...ooomlllll/aaaaaa
INTRODUCTION
Continual Learning is key element of human intelligence that enables to accumulate knowledge from a never ending stream of data. From machine learning perspective, there is no guarantee that information accessed current task to be revisited later in future tasks. This leads what is known as Catastrophic Forgetting (McCloskey & Cohen 1989 | McClelland et al.) 1993); significant drop in previously obtained knowledge of an Al system as learns new information and gets less/no exposure to old information. Several approaches have been proposed to bridge the gap between machine and human continual learning skills with catastrophic forgetting being the central problem. Existing continual learning methods have focused mostly on classification tasks (e.g. (Rebuffi et al.)
2017
Lazzz-Pazz & Ranzato 2017 Shin ct al. 2017 [i & Hoiem 2016 Shmelkov et al. 2017 | Triki
ct al. 2017] & Hoiem| 2016 Triki et ad.][211] Rusu et al. 2016 Lee et al. 2017 Elhoseiny al.
2018
Kirkpatrick et al. 2017c Zenke et al. 2011 7017] Chaudhry et al. 2018)).
In this paper, we propose new scenario of continual learning which handles
tasks common in language learning. Continual language learning is an open question which has not been studied extensively in machine learning and NLP domains. It may facilitate a variety of applications in NI.P systems. For example, it enables a robot to keep on learning new tasks via natural language instruction, conversational agent to adapt to new conversation topics quickly, and a neural machine translation system to expand its vocabulary continually.
Humans learn language by leveraging systematic compositionality, the algebraic capacity to under- stand and produce large amount of novel combinations from known components (Chomsky) 1957] Montague 1970). Compositional generalization is critical in human cognition (Minsky) 1986 Lake et al.] (017). D. also helps humans acquire language from small amount of data, and expand vocab- ulary sequentially (Biemiller) 2001). In contrast to humans' such ability, state-of-the-arr continual learning approaches do not achieve the expected generalization. Table and Figure 3 show the per- formance of state-of-the-art approaches (Kirkpatrick et al. 2017a Aljundi et al. 2018) when tested
"Corresponding author: yuanpengl 60 (@gmmail com Work partially done while visiting Baidu Research

---

Published as a conference paper at ICLR 2020
in instruction learning and machine translation tasks. This highlights the lack of generalization of these approaches, designed after classification tasks, on sequence generation language tasks and the importance of studying the design of continual learning methods for language learning.
Transfer
Forget
Long-forge forgef
Method Staage
10
100
1
10
100
1
10
100
Standard
2.3
0.2
0.0
30.8
0.9
0.0
30.8
1 11.2
7.9
Compositional
98.8
15.0
0.0
99.3
71.7
0.7
99.3
85.5
47.4
EWC
2.8
0.2
0.0
35.0
1.0
0.2
35.0
11.5
11.1
MAS Proposed 99.9
0.6
0.2
0.0
20.0
0.8
0.1
20.0
10.8
9.8
99.8
90.7
100.0
99.9
89.5
100.0
100.0
86.0
Table I: Mean of evaluation accuracy (%) on instruction learning tasks (Section for details). Base- lines include Compositional (Li et al. 2019), EWC (Kirkpatrick et al. [2077). and MAS (Aljundi) et al. 2018). Please refer to Table 3 in Appendix for more results and standard deviations.
Modeling continual language learning with improved compositional understanding is at the heart of this paper. More concretely, we address the challenge of open and growing vocabulary problem with continual learning. It requires optimizing over two objectives. First, previously learned knowledge should be transferred and combined with new knowledge. Second, the learned model should resist catastrophic forgetting (Kirkpatrick al. 2017b), where a model adapted new distribution no longer works on the original one. To achieve these objectives, we use compositionality to separate semantics and syntax of an input sentence, so that we can convert label prediction algorithm to sequence to sequence algorithm for continual learning.
The contributions of this paper can be summarized as follows.
We propose new scenario of continual learning which handles sequence-to to-seueneeeeeeeeee tasks common in language learning.
We propose approach use label prediction continual learning algorithm for sequence- to-sequence continual learning by leveraging compositionality. To our knowledge, this is the first work for applying compositionality continual learning of soquence-to-sqqenne tasks, targeting at both knowledge transfer to later stages and catastrophic forgetting pre- vention on previous stages.
Experiments show that the proposed method has significant improvement over multiple state-of-the-art baselines in both knowledge transfer and catastrophic forgetting prevention with almost 85% accuracy up to 100 stages on language instruction task. It also shows significant improvement in machine translation task.
2 RELATED WORK
Our work is closely related to compositionality and continual learning or lifelong learning. Here, we briefly review some related work in these areas.
Compositionality Compositional generalization is critical in human cognition (Minsky (186);; Lake et al. (2017). and it helps humans acquire language from small amount of data, and ex- pand vocabulary sequentially (Biemiller (200T)). Therefore, researchers have been studying how to enable human-level compositionality neural networks for systematic behaviour (Wong & Wang Brakel & Frank 2009), counting ability (Rodriguez & Wiles 1998] WWiss et al.) 2018) and
2007
sensitivity to hierarchical structure (Linzen al. | (0166). Recently, people proposed multiple related
tasks (Lake & Baroni, 2018 Loula et al.) 20188 Lake et al.) 2019) and methods (Lake & Baroni,
Loula et al.] 20188; Kliegl & Xu 2018) with different kinds RNN models and attention mechanisms. Though these methods enable generalization when the training and test sentences have small difference, it has been an open problem (Yang et al. 2019) to reach human-level compo- sitionality gemeralization. More recently, Li al. (2019) proposed entropy regularization method that achieves high performance on several NLP tasks. In this paper, we study compositionality from continual learning angle. By leveraging the compositional learning approach, we propose the
2018
2

---

Published as a conference paper at ICLR 2020
tinual learning algorithm by encoding compositionality into DNN. To our knowledge, work is the first to apply compositionality to continual learning in DNN.
Continual learning Continual learning or lifelong learning involves multiple stages. Each stage has a set of classes and corresponding data, and the training can only access the data in the current stage. Based on the way for overcoming catastrophic forgetting, continual learning work may be categorized into data-based and model-based approaches. In data-based approaches, some methods store previous data either with replay buffer (Rebufli et al., 2017; Lopez-Paz & Ranzato 2017 7) or generative model (Shin et al. 2017); other approaches (Li & Hoiem 2016 Shmelkov et al. 2017
Triki et al. 2017), employ the new task data to estimate and preserve the model behavior on previous tasks, mostly via knowledge distillation loss proposed in Learning without Forgetting (Li &
Hoiem 2016). These approaches are typically applied to a sequence of tasks with different output spaces. To reduce the effect of distribution difference between tasks, (Triki al.) 2017) propose to incorporate a shallow auto-encoder to further control the changes to the learned features, while AAljundi al. 2017) train a model for every task (an expert) and use auto-encoders help determine the most related expert at test time given an example input. In model-based approaches, some
methods dynamically increase model size for the growing information (Rusu et al., 2016 Xu &
Zhu 2018); other methods (Fernando et al. [207 et al.) (017) Kirkpatrick et al. 2017c Zenke
et al. 2017)) focus on the parameters of the network. The key idea is to define an importance weight We for each parameter 0₂ in the network indicating the importance of this parameter to the previous tasks. When training a new task, network parameters with high importance are discouraged from being changed. In Elastic Weight Consolidation, (Kirkpatrick et al. 2017c) estimate the importance weights & based on the inverse of the Fisher Information matrix. Zenke et al. 2017) propose Synaptic Intelligence, parameter to the change in the loss, and weights are accumulated for each parameter during training. Memory Aware Synapses (Aljundi et al. (0188) measures f2 by the effect of change in the parameter to the function learned by the network, rather than to the loss. This allows to estimate the importance weights not only in an online fashion but also without the need for labels. Finally, Incremental Moment Matching (Lee et al., (017) is a scheme to merge models trained for different tasks. Model- based methods seem particularly well suited for our setup, given that we work with an embedding instead of disjoint output spaces. In this paper, we propose method with minimal increase of model structure in each stage, and we leverage compositionality with explainable mechanisms that align with human learning.
online continual model where 52 is defined by the contribution of each
CONTINUAL LEARNING WITH COMPOSITIONALITY
3.1 PROBLEM DEFINITION
Conventional continual learning algorithms are designed after fixed size input and label classification output. However in many tasks, such as language. both input and output are sequences and bridging this gap between continual learning and sequence-to- to-suue-n--e models is at the heart of our work. We facilitate more accurate continual sequence-tar requence-to-sequeme to-sequence artificial learner by proposing an approach that can leverage Label Prediction Continual Learning (LP-CL) compositionally into Sequence-to- Sequence Continual Learning (S25-CL).
LP-CL: Label Prediction Continual Learning In LP-CL, consider a word to label mapping problem, with input word x and corresponding output label y. In initial learning stage, y takes one of K classes: y E Vinit {c1, C2,......... C.. In continual learning stage, y takes a new class: E Vamm = {ck+1). }. In test, y takes all previous classes: y E Viai U Veom- For example, in language instruction task, input x is a primitive word, and output Y the corresponding action symbol: in word-level machine translation, input x is an English content word, and output Y is the corresponding French word. In initial training stage, we have multiple input word and output symbol pairs. In continual learning stage, we have new input and output word pair. We train model in initial training stage, and do not use the initial training data any longer in the rest of training stages. We then switch to the data in continual learning stage, and continually update the model. In test stage, we evaluate whether model can predict labels from both initial and continual learning stages. We denote label prediction continual learning model (LP-CLL) as P(yla;;).
3

---

Published as a conference paper at ICLR 2020
S2S-CL: Sequence to Sequence Continual Learning For sequence sequence continual learn- ing (S2S-CL,) we consider sequential input X = I1. 22..... 2n and output = V/1 V2. 3/m Each output label yı, i {1..... m is from the corresponding label set in label prediction problem. We want to make a model X) for sequence to sequence continual learning.
Our goal is to facilitate better Sequence Sequence Continual Learning (S2S-CL) capability quan- tified as P) by leveraging access and joint-learning with Label Prediction Continual Learning (LP-CL) model, P(y|2;;) 0).
3.2 USE LP-CL ALGORITHM FOR S2S-CL WITH COMPOSITIONALITY
The core idea of this work is compositionality to separate semantics and syntax, that can convert label prediction algorithm to sequence to sequence algorithm for continual learning. In Kirkpatrick al. (2017a), continual learning can be probabilistically defined follows.
log P() log P(Dr(v)) log P(v)-D...rr log P(Dr)
Here, log P(Drlt) is the negative of loss function in task T' and log P(e(PI---r-)) is regularization related to parameters learned during tasks 1... T - 1. this work, we have two parts of parameters 0, o for semantics 0 and syntax processing. With compositionality (Li et al. 2019), make and @ conditionally independent given DI..T-1-
log P( D) = log P(Dr) v)) + log P(A, o(D1...r-1) - log P(Dr) log P(D,(())))) + log ((- log
log P(Dr)
We assume syntax d does not change after the initial stage, so we realize regularization log P(o(DI..-T-)) by freezing during learning in task T. We use label prediction continual learn- ing algorithm for regularization log PI) Please see Figure I for illustration.
Semantics: p
Initial train
Continual train
0o
01
Input: X
Output: Y
LP-CL
LP-CL
LP-CL
$0
$1
h1
Syntax: f
leam
freeze
freeze
Figure 1: Flowcharts. We compositionality to separate semanties and syntax (left). We use label prediction continual learning algorithm for (semantics), and freeze & (syntax) during continual train (right).
We derive the proposed approach based on the above ideas. To use label prediction algorithm in se- quence sequence problem, we need to extract label prediction problem from sequence to sequence problem. Language is generally composed of semantics p and syntax f, so that we decompose an input sequence to them with compositionality.
In language instruction task, for example, input X is word sequence, and output Y is label sequence. We consider X has two types of information: which labels are present (XP), and how the labels should be ordered (Xx). Y is constructed by the output label types (YP), and output label order (Y*5) (Eq. []. We then use chain rule (Eq. (2). With compositionality. y functionally depends only X². and given Yr, YP depends only XP (Fqq. 3). For intuitive example, in language instruction, order of output actions depends only on input function words (syntax), and given the order, each output action (semantics) depends only on the corresponding input primitive. In machine translation, output order depends only on input part-of-speech information (syntax), and given the order, each output word label (semantics) depends only on the corresponding input word.
P(Y = P(Y Y'IX','' xx,'x""") =PV/I//PY"
(2)
(3)
4

---

Published as a conference paper at ICLR 2020
We use LP continual learning for S2S continual learning (our goal) by decomposing output sequence to labels. We assume that the labels ......, Li are conditionally independent given output syntax Y/ and imput semantic information X² (Eq, 4 (we design model in this way). We then use total probability 5). We further design that z depends only on v and X" (attention mechanism), and with label prediction component, 2z depends only on o ()..6).
P(Y = ( II POJ X²)
(4)
J=1
= P(Y
Pla 19)
(5)
P(Y II P(utl,
(6)
P(aflu/,XX' XP) is an operation to apply attention map of on a sequence of value vectors XP, so that it does not have parameters. Let 0 be the parameter for label prediction module P(vf) >=; 0), and be the parameter for attention map generator I
X(Y = P(Y/ )) I Pa Pan XP)
Since we assume continual learning stage does not contain new syntax patterns, we can freeze during continual learning stage. 0 is the parameter for label prediction module. Therefore, use label prediction continual learning model (LP-CL) to enable compositional sequence to sequence continual learning (S2S-CL) as detail in the next subsection.
3.3 DISENTANGLE SEMANTIC AND SYNTACTIC REPRESENTATIONS
Our S2S-CL approach is inspired by the idea of decomposing syntactic and semantic representations with compositionality (Li al. 2019). Note that it is not a continual learning approach but shows how compositionality can be encoded in sequence- to to-sequence models. The method disentangles syntactic and semantic representtts by using two representations. One generates attention maps, and the other maps attended word to action. It reduces entropy of the representations.
Suppose there are input and output y. x contains a sequence of words, where each input word from input vocabulary of size U. y contains sequence of output symbols, where each out- put symbol is from an output vocabulary of size V. Both vocabularies contain an end-of-sentence symbol which appears the end of x and y, respectively. The model output is prediction for y- Suppose both input words T1, Ti,..., In and output symbols y1...., Ym are in one-hot representation.
[
[y , Vml
To disentangle information, an input sentence converted to semantic representation and syn- tactic representation f. Specifically, each word is encoded with two embeddings.
(xb) E R
fi Emb, f (zx) E R
Then, they concatenated form two representations for the entire input sequence, i.e.,
p= (p......... .P.] R**x"
= lf. fa] E R
Entropy regularization is introduced to achieve disentanglement by regularizing the La norm of the representations Cregularize = L2(p) + L2()), and then adding noise to the representations.
p' = p ac, - * Ep ~ 1).
f + acy E 1 x ,y N00
fl is fed to sequence-to-see to-seuence module for decoding. At each step j. the decoder generates by E gn, and attention map aj is obtained with Softmax. With the attention map, weighted average
We make loose usage of syntactic and semantic. In natural instruction learning, syntactic refers func- tional and semantic refers to primitive.
5

---

Published as a conference paper at ICLR 2020
Dy noised semantic representations p' is computed. Then it is fed to fully connected one-layer network fpedieet to get score ly. and a Softmax is used to compute the output distribution Øy- The decoding ends if arg max Dj is end-of-sentenee symbol.
0j Softmax (bf)
B/ p'a, E R I = (1²) GRY
Softmax (1,)
The cross entropy of and ý is used as prediction loss firediits and the final loss L is the combination of prediction loss and entropy regularization loss. À is regularization weight.
Ppediict
CrossEntropy (uj, 9z),
C = Eprodict + Aergulariz
3.4 LABEL. PREDICTION ALGORITHM FOR CONTINUAL, LANGUAGE LEARNING
For language problems, it is natural use non non-parametrie algorithm label prediction continual learning algorithm, because input words and output actions are usually associated with embeddings. In each stage, since the original method uses two embeddings E, k (r E {p. f)) for word and one embedding W for an action, we append the new semantic Cp. syntactic ey and action w embeddings (Figure 2 We freeze the old embedding parameters and only learn the newly added ones in the stage.
One-hot input
Output nodes
W
E
Up
kr
Kv
Vo
x (Uo + 1)
JUMP
(Vo 1) x Ko
jump
Figure 2: Illustration for the first continual learning stage. Up and Vo are initial vocabulary sizes for input and output, respectively. Left is input word embedding (we only show one of two input word embeddings for simplicity). Middle is model architecture. Right output action embedding. Parameters and data for the input word and output action embeddings of previous stage are in blue (filled boxes, solid lines), and for the new stage are in red (unfilled boxes, dashed lines). Other parts of the network are in black (unfilled boxes, solid lines).
EXPERIMENTS
We evaluate the proposed method in continual learning task with multiple stages. The first stage is standard process in which we train a model with combinations of multiple words in various sentence structures. In each continual stage, add new input word and corresponding new output symbol. The training dataset contains only one sample, whose input is a sentence with the new word, and output a sequence with the new symbol. For each stage, can only use the data in that stage, and have no access to data in previous or future stages.
We have two objectives in continual learning. We want previously learned knowledge to be trans- ferred and combined with new knowledge (transfer learning), and updated model to work on previous data (catastrophic forgetting prevention). We evaluate transfer learning by testing whether the model works on data where the new word appears with old ones (Transfer). We evaluate catas- trophic forgetting prevention by testing whether the model works on data that only contain words up to the last stage (Forget). We are also interested in preventing long-term catastrophic forgetting, because more difficult than preventing short-term one. Thus, we test whether the new model works the evaluation dataset in the initial stage (Long-forget).
Sosennceett-Squenee contin- ual learning validate approach since, to knowledge, this is the first work for continual learning natural language instructions and machine translation. We applied standard sequence- to-sequence model (Standard) to continual setting, and also the compositional generalization
Baselines. We designed baseline methods for compositionality

---

Published as a conference paper at ICLR 2020
method (Compositional) (Li al. 2019). We also compare with state-of-the-ard continual learn- ing baselines. To fit in the experimental setting, we focus on those that do not use replay buffer, and require minimum model structure extension, that we added EWC (Kirkpatrick et al. 2017a) and MAS (Aljundi et al., 2018) as comparable baselines due to their popularity and competitive performance in label prediction setting. The detailed implementation of the baseline and proposed methods can be found in Appendix B
Metric. We use accuracy as metric for both instruction learning and machine translation experi- ments. A prediction is correct if and only if it is completely identical to the ground truth. We run all experiments for five times with different random seeds.
Instruction learning.
Machine translation.
100
100
Accuracy (%)
75
Standard
75
Compositional
50
EWC
Accuracy
50
Proposed
25
20
40
60
80
100
20
40
60
80
100
Stage
Stage
(a) Transfer.
(d) Transfer.
100
100
80
Accuracy (%6)
Accuracy (%)
60
50
40
25
20
20
40
60
80
100
20
40
60
80
100
Stage
Stage
(b) Forget.
(e) Forget.
100
100
0
A
80
Accuracy (9)
60
60
40
40
20
20
20
40
100
80
100
Stage
Stage
(c) Long-forget..
(f) Long-forget
Figure 3: Mean of evaluation accuracy (%) for all methods (best viewed in color). Baselines include Compositional (Li et al., 2019), EWC (Kirkpatrick et al., (017a), and MAS (Aljundi et al. 2018) The proposed method is significantly better than all baselines. Please refer to Figure and Figure in Appendix for details.
Instruction Learning We first experiment on instruction learning task using SCAN dataset (Lake
& Baroni 2017). The task summarized in Table in Appendix. The details of dataset generation in Appendix A The results are in Figure (left) and Table (more details in Table in Ap-

---

Published as a conference paper at ICLR 2020
pendix). The proposed method has significantly better results than the baselines. It maintains high accuracy up to 100 stages for both transferring knowledge from previous stages to future stages, and catastrophic forgetting prevention. On the other hand, baseline methods drop performance over time. Methods without compositionality (EWC, MAS) reduces quickly, maybe because they are entangled, all parame-
not designed for transferring knowledge, and since the representations ters are quickly changed, causing catastrophic forgetting. Compositional method is better, but still drops, maybe because the parameters for syntax are changed over time. This experiment shows the advantage of the proposed method over baselines.
Machine Translation We also investigated whether the proposed approach works for other tinual language learning problems. As an example, we conduct a proof-of-concepl experiment for machine translation. We modified the English-French translation task in (Lake & Baroni 2018). In each continual learning stage, we add an additional English-Frenn ENGLISH", "je suis FRENCH"). Neither English word French word appears in previous stages. This pair is used as training data in the stage, but test data contains other patterns. Appendix A provides more details on dataset and model configuration. The result shown in Figure 3 (right) and Table performance than baselines. For Transfer and Forget evaluation, the baseline methods drop quickly. However, for Long-forget evaluation, they keep positive accuracy over time. This means the base- line methods have the ability to learn knowledge and remember for long time, but they are not as strong the proposed method. This experiment shows that the proposed approach has promise to be applied to real-world tasks.
word pair, in the format ("I am
4 in Appendix. It shows that the proposed approach has stable and significantly higher
DISCUSSIONS
5.1 ATTENTION MAP VISCALIZATION
We hope to use compositionality for continual learning, so we want to find whether the model works
in the expected mechanism. We visualize activations of attention maps on the evaluation data in the first continual stage (Figure
opposite
<B0S>>
opposite
<BOS>
opposite
left <BOS>
jump
right
after
jump
right
left
bwice
and
turn
left
walk
right
twice
aftor
run
0.8
LTURN
RTURN
LTURN
RUN
JUMP
LOOK
RTURN
RTURN
LTURN
RTURN
WALK
JUMP
LTURN
RTURN
RTURN
LOOK
RTURN
JUMP
LTURN
WALK
<BOS>
<EOS>
<EOS>
0.0
(a) Transfer.
(b) Forget.
(c) Long-forget
Figure 4: Visualization of attention maps. The horizontal and vertical dimensions are the input and output position sequences respectively. The figures show that the model identifies the appropriate input to output position mapping. This indicates that the proposed method successfully leverages compositionality in continual learning.
The visualization shows that, for each output action, the attention on the corresponding input word. Also, for the output end-of-sentence symbol, it is the input end-of-sentence symbol. It is consistent with the original work, and the way humans apply compositionality. This indicates that the proposed method may be applicable to other tasks where humans use compositionality.
8

---

Published as a conference paper at ICLR 2020
5.2 EMBEDDING VISUALIZATION
We visualize how the new embedding parameters fit in the space with predefined dimensions, and accommodate with previously learned parameters. The visualization of attention maps explains the syntactic information, and we are also interested in semantic information.
We use 1-SNE (Maaten Hinton 2008 to project high dimensional embeddings to two dimen- sional space for visualization. Our analysis focuses on semantic embedding, because it reflects how new information is encoded in the model. Since action embedding shares much information with semantic embedding, and syntactic embedding is not supposed to contain new information because grammar does not change over stages, we leave them in Appendix
200
200
100
100
0-
0-
-100
-100
0-25
1-25
26-50
-200
-20
-200
-100
0
100
200
-200
-100
100
200
(a) Stage 1-25.
(b) Stage 26-50.
200
200
100
100
0
-100
-100
5
4
0-50
0.75
51-75
76
76-100
-20
200
-200
-100
0
100
200
-200
100
100
200
(c) Stage 51-75.
(d) Stage 76-100.
Figure 5: Embedding visualization for semantic embeddings. We see two phases. In (1-50), embed- dings explore outside space. In (51-100), embeddings squeeze into the explored space.
Figure 5 shows two phases in the continual learning experiment. The first phase is from the first stage to around stage 50, where the new embeddings explore outside space. The second phase the rest of the stages, where the embeddings squeeze into the explored space, maybe because exploring becomes expensive with the dense population under regularization. This may be explanation for the performance decrease in the later stages of instruction learning experiment.
6 CONCLUSION
In this paper, we propose an approach to use label prediction continual learning algorithm for seuueence-t-ssquenee continual learning problem by leveraging compositionality. To our knowl- edge, this is the first work to combine continual learning and compositionality for sequence-to- sequence learning. Experiments show that the proposed method has significantly better results than baseline methods, and it maintains almost more than 85% accuracy for both transfer learning and catastrophic forgetting prevention up to 100 stages. The results demonstrate that language composi- tionality helps continual learning of natural language instruction both efficiently and effectively. We hope this work will advance the communication between humans and machines, and make machines more helpful in various tasks.

---

Published as a conference paper at ICLR 2020
REFERENCES
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, al. Tensorflow: system for large- scale machine learning. In 12th {USENIX) Symposium on Operating Systems Design and Imple- mentation ((OSDI) 16), pp. 265-283, 2016.
Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gate: Lifelong learning with network of experts. In CVPR, 2017.
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, 2018.
Andrew Biemiller. Teaching vocabulary: Early, direct, and sequential. American Educator, 2001.
Philémon Brakel and Stefan Frank. Strong systematicity in sentence processing by simple recurrent networks. In 31th Anmual Conference of the Cognitive Science Society (COGSCI-2009), pp- 1599-1604. Cognitive Science Society, 2009.
Arslan Chaudhry, Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00420, 2018.
Noam Chomsky. Syntactic structures. Walter de Gruyter, 1957.
Mohamed Elhoseiny, Francesca Babiloni, Rahaf Aljundi, Mareus Rohrbach, Manohar Paluri, and Tinne Tuytelaars. Exploring the challenges towards lifelong fact learning. In Asian Conference on Computer Vision, pp. 66-84. Springer, 2018.
Chrisantha Fernando, Dylan Banarse, Charles Blundell. Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.0873: 2017.
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska- Barwinka, and al. Overcoming catastrophic forgetting in neural networks. PNAS, 2017a.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barsskskka,ikkaa, al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, pp. 201611835, 2017c
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- -arrskiskaa, al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3222 2017b.
Markus Kliegl and Wei Xu. More systematic than claimed: Insights the scan tasks. ICLR Workshop, 2018
Brenden Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of sesuueenc--t--squennee recurrent networks. In International Conference on Machine Learning, pp. 2879-2888, 2018.
Brenden M Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of souuuecce-t--squenee recurrent networks. arXiv preprint arXiv:1711.003500 2017.
Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel Gershman. Building machines that learn and think like people. Behavioral and Brain Sciences, 40, 2017.
Brenden M Lake, Tal Linzen, and Marco Baroni. Human few-shot learning of compositional in- structions. arXiv preprint 1901.04900.04 2019.
Sang- Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung- Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In Advances Neural Information Processing Systems, pp. 4652-4662. 2017.
10

---

Published as a conference paper at ICLR 2020
Yuanpeng Li. Liang Zhao, Jianyu Wang, and Joel Hestness. Compositional generalization for prim- itive substitutions. EMNLP, pp. 4284 4293, 2019.
Zhizhong Li and Derek Hoiem. Learning without forgetting. In European Conference on Computer Vision, pp. 614-629. Springer, 2016.
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. Assessing the ability Istms to learn syntax- sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521- 535, 2016.
David Lopez-Paz and MarcAurelio Ranzato. Gradient episodic memory for continual learning. In NIPS, 2017.
Joao Loula, Marco Baroni, and Brenden M Lake. Rearranging the familiar: Testing compositional generalization in recurrent networks. arXiv preprint arXiv:1807.07544 2018.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine
learning research, 9(Nov):2579 2605, 2008
James L McClelland, Bruce L McNaughton, and Randall C O'reill. Why there complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and motivation, 24:109- 165, 1989.
Marvin Minsky. Society of mind. Simon and Schuster, 1986.
Richard Montague. Universal grammar. Theoria, 36(3):373-398 1970.
S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert. icarl: Incremental classifier and repre- sentation learning. In CVPR, 2017.
Paul Rodriguez and Janet Wiles. Recurrent neural networks can learn to implement symbol-sensitive counting. In Advances in Neural Information Processing Systems, pp. 87-93, 1998.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671. 046666600467,, 2016.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pp. 2990-2999, 2017.
Konstantin Shmelkov, Cordelia Schmid, and Karteck Alahari. Incremental learning of object detec- tors without catastrophic forgetting. In The IEEE International Conference on Computer Vision (ICCV), 2017.
Amal Rannen Triki, Rahaf Aljundi. Mathew B Blaschko, and Tinne Tuytelaars. Encoder based lifelong learning. arXiv preprint arXiv:1004.01920, 2017.
Gail Weiss, Yoav Goldberg, and Eran Yahav. On the practical computational power of finite preci- sion mns for language recognition. arXiv preprint 805.0080 2018.
Francis CK Wong and William SY Wang. Generalisation towards combinatorial productivity in lan- guage acquisition by simple recurrent networks. In 2007 International Conference on Integration of Knowledge Intensive Multi-Agent Systems, pp. 139-144 144. IEEE, 2007.
Ju Xu and Zhanxing Zhu. Reinforced continual learning. In Advances in Neural Information Pro- cessing Systems, pp. 899-908, 2018.
Guangyu Robert Yang. Madhura R Joglekar, Francis Song. William Newsome, and Xiao-Jing Wang. Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience, pp. 1, 2019.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pp. 3987- 3995. PMLR, 06-11 Aug 2017.
11

---

Published as a conference paper at ICLR 2020
A DATASET PREPARATION
We extend the grammar in SCAN dataset (Lake & Baroni 2017) to generate data. For the initial stage, we generate data with three action words (walk, look, run). We then randomly and uniformly divide the data to two disjoint sets, We use one set for initial stage training data (6,601 samples), and reserve the other set initial dataset to evaluate catastrophic forgetting continual stages (Forget, 6,602 samples). The reserved data is also used to evaluate long-term catastrophic forgetting (Long- forget). In each continual stage, we generate data that contain new action word (jump for the first continual stage). The data contain one-word sentence (jump: JUMP), and we use it as training data (1 sample). We use other data in the generated data as evaluation dataset (Transfer, 7,706 samples for the first continual stage). We then add Transfer to Forget for the next stage. We set limitation of 100,000 samples for each evaluation dataset. When it has more samples than that, we select the amount uniformly random after generating all data once. Machine translation dataset is generated similarly from the translation dataset in SCAN dataset (Lake & Baroni, (017). We use the original training data the initial training data, and the original test data the initial test data.
Stage Initial
Train
Transfer
Forget
Long-forget
walk twice
run and look look before run walk after run
1
jump
jump twice
walk
2
skip
look and skip
look twice
3
crawl
crawl after skip
skip after jump
run thrice
Table Example imputs for continual learning setup (initial and first three continual learning stages). Initial stage training data contain action words of "walk", "run" and "look". Bold words are intro- duced in current continual stage. Italic words are introduced in previous continual stages.
B IMPLEMENTATION
We use bidirectional LSTM as encoder, and unidirectional LSTM with attention as decoder for sequence sequence architecture. The first and last states of encoder concatenated initial state of decoder. The state size is h = 32 for encoder, and 2/ 64 for decoder. We also use kp = 64, ky 8, a 0.1. For EWC (Kirkpatrick al. 2017a) and MAS (Aljundi et al. 22018), use 10 for parameter regularization weight. In initial stage, batch size is 512, and we run traiinn 5,000 steps. In each continual stage, batch size is as each continual stage only contains one sample, and we run training 1,000 steps. We have 100 continual stages. We implement all methods with TensorFlow (Abadi et al. 2016).
C DETAILED EXPERIMENT RESULTS
More results and standard deviation for continual learning can found in Table 3 for instruction learning and Table for machine translation experiments.
D EMBEDDING VISUALIZATION
Syntax embeddings in Figure O show that the embeddings in continual learning stages do not explore outside space, but live in the space explored in the initial stage.
show two phases for exploration, similar with semantic embeddings. This fits expectation, because the compositionality and training encourage semantic embeddings and action embeddings to be similar.
Action embeddings in Figure 7 show
12

---

Published as a conference paper at ICLR 2020
Method SStage
2
3
10
100
Standard
2.3 3.0
1,8 +
1.0 +
1.2
0.2 0.1
0.0
Compositional
98.8 1.5
98.1 + 1.7
97.5
- 1.8
15.0 =29,9
0,0 0.0
EWC
2.8
3.7
1.4 +
1.2
0,4
- 0.3
0.2 = 0.2
0.0 0.0
MAS
0.6 0.7
1,4
1
1.6
0.3
1 0.4
0.2
0.1
0.0 + 0.0
Proposed
99.9
= 0.1
99.9
+
0.2
99.8
- 0.2
99.8 = 0.3
90.7 #14.5
(a) Transfer learning evaluation (Tramsfer).
Method Staage
1
2
3
10
100
Standard
30.8 16.4
10.0 +
4.3
3.7
0.9 +
0.0 0.0
Compositional
99.3 1.2
98.7 = 1.3
98.2
- 1.5
71.7 #17.6
0.7 0.2
EWC
35.0 =25.9
11.0 = 9.8
5.5 4.9
1.0 1.0
0.0 0.0
MAS
20.0 113.3
8.1 = 6.6
4.6 4.0
0.8 + 0.7
0.0 0.0
Proposed
100.0 + 0.0
100.0 = 0.1
99.9
- 0.1
99.9 + 0.2
89.5 119.
(b) Catastrophic forgetting evaluation (Forget).
Method SStage
2
3
10
100
Standard
30.8 1:1.44
20.9 117.5
13.5 12.33
11.2 7.2
7.9 4.6
Compositional
99.3 + 1.2
99.1 4: 1.1
98.9 + 1.1
85.5 118.1
47.4 428.4
EWC
35.0 +25.9
23.1 +20.0
17.7 115.9
11..5
11.1 #11.6
MAS
#13.3
17.2 #13.7
14.7 +12.5
10.8 = 8.5
9.8 + 7.9
Proposed
100.0 + 0.0
100.0 + 0.0
100.0 + 0.0
100.0 + 0.0
86.0 +22.0
(c) Long-term catastrophic forgetting evaluation (Long-forget).
Table 3: Evaluation accuracy (%) for proposed and baseline methods in instruction learning ex-
periment. Baselines include Compositional (Li et al. 2019), EWC (Kirkpatrick et al. 2017a). and MAS (Aljundi et al. 2018).
Method SStage
1
2
3
10
100
Standard
30.0 117.0
22.5
122.9
10.0 114.6
0.0
Compositional
40.0 +28.9
40.0
114.6
42.5
5.0 11000
0.0 0.0
EWC
52.5 114.6
45.0
+15.0
42.9
7.5
0.0 0.0
MAS
42.5 115.0
25.0
117.7
42.03
5.0 110.0
0.0 4: 0.0
Proposed
62.5 0.0
62.5 + 0.0
62.5 + 0.0
62.5 + 0.0
62.5 + 0.0
(a) Transfer learning evaluation (Transfer).
Method SStage
3
10
100
Standard
30.0 117.0
32.5 111.5
38.3 115.2
18.8
6.8
1.8
Compositional
40.0 228
37.5 422.4
37.5
20.2 =12.
1.8 + 0.9
EWC
52.5 144.6
52.5
111.6
40.8
29.3 8.5
3.0 1.0
MAS
42.5 115.0
41.3
420.0
33.3
17.7 6.0
2.0 - 1.0
Proposed
62.5 + 0.0
62.5 + 0.0
62.5 + 0.0
62.5 - 0.0
62.4 + 0.1
(b) Catastrophic forgetting evaluation (Forget).
Method \Stagee
3
10
100
Standard
37.5 7.9
37.5
42.5 110.0
27.5
9.4
30.0
112.7
Compositional
35.0 +26.7
40.0 +21.5
40.0 5.0
27.5
#24.2
17.5 +15.0
EWC
50.0 +13.7
52.5 +16.6
50.0
42.5
+20.3
37.5 +15.8
MAS
55.0 + 6.1
42.5
+17.0
57.5
45.0 + 6.1
47.5 +184
Proposed
62.5 0.0
62.5 4 0.0
62.5 + 0.0
62.5 - 0.0
62.5 + 0.0
(c) Long-term catastrophic forgetting evaluation (Long-forget).
Table 4: Evaluation accuracy (%) for proposed and baseline methods in machine translation ex- periment. Baselines include Compositional (Li et al. 2019), EWC (Kirkpatrick et al. 2017a). and MAS (Aljundi et al. 2018).
13

---

Published as a conference paper at ICLR 2020
15
15
10
10
44
-
-
-10
0
-10
0-25
1-25
26-50
15
155
-10
5
0
10
-10
10
(a) Stage 1-25.
(b) Stage 26-50.
10
10:
5
0
0
-5
44
-5
-10
0-50
-10
0-75
51-75
76-100
-15
-10
-5
10
-10
-5
10
(c) Stage 51-75.
(d) Stage 100.
Figure 6: Embedding visualization for syntax embeddings.
14

---

Published as a conference paper at ICLR 2020
200
200
4
100
100
0
A
0
4
b
4
-100
4
-100
A
&
0-25
1-25
Aa
26-50
-200
200
-200
-1000
100
200
200
-100
100
200
(a) Stage 1-25.
(b) Stage 26-50.
200
200
100
100
&
0
4
0
4
4
-100
-100
0-50
'0-95
51-75
76-100
-200
-200
-200
-100
100
200
200
-100
100
200
(c) Stage 51-75.
(d) Stage 76-100.
Figure 7: Embedding visualization for action embeddings.
15


Reviews:
[{"review_id": "ryeyDTniFH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an approach for continual learning for applications in sequence-to-sequence continual learning (S2S-CL). More specifically, the paper addresses the problem of growing vocabulary. The overall approach is intuitive yet quite effective. The method assumes a split between the syntax (f) and semantics (p) of the sequence -- in other words, each token is associated with two labels. Furthermore, the syntax is assumed to be shared across all the training sequences and the sequences that are not encountered during the initial training. A sequence model (LSTM) is used for learning the syntax f over the initial training and is then frozen for all the downstream tasks (i.e. continual learning). The sequence model predicts the correspondence between the output token and input tokens. Another network (a 1-layer  MLP) is used to predict the semantic label of the selected token in the target domain. Barring some notational confusion, I believe the method is very reasonable and should work well. They perform experiments on 2 datasets (Instruction learning & machine translation) in 3 continuous learning paradigms with varying difficulties and demonstrate that the proposed method significantly outperforms the baselines by an impressive margin.\n\nI am leaning towards accepting this paper since I believe the proposed approach can be a promising direction for continual learning in S2S settings and the empirical results are convincing.\n\nHowever, I have some concerns that might require clarification or additional experiments:\n    1. The novelty of the proposed method is somewhat limited in my opinion. It seems it is a very simple adaptation of Li et al. 19 and the only addition to that work is the usage of a very simple label prediction scheme by fixing everything else. However, this might be okay since the experiments are quite compelling and the method is applied to a different setting.\n\n    2. Notations and language in section 3.2-3.4 are hard to follow:\n        - At the bottom of page 4, I believe some of the equations are not correct. For example, the first term of line 2 should be P(Y^f | X^f, X^p) rather than Y^p?\n        - In the second equation of page 6, the second term should be v_j = p’ \\cdot a_j\n        - In Figure 1, k_r, E_r, and W are never defined (although their meaning can be inferred).\n        - In section 3.4, it reads that new word embedding is appended to both semantic and syntactic embedding matrices but this doesn’t make sense because the syntactic network is already fixed so it shouldn’t be able to handle new symbols; therefore, I believe only new row is added to the semantic embedding.\n\n    3. I believe that f_predict here is parameterized by \\theta and \\theta is also frozen during the continual learning phase which contradicts the claim at the end of section 3.2 that only \\phi is frozen. Otherwise, it’s hard to see why f_predict does not suffer from catastrophic forgetting. From the provided source code, it seems that it is indeed the case that only the new embedding is being updated. In other words, the only thing happening at this stage is that the newly added embedding are optimized to adapt to the frozen f_predict.\n\n    4. Due to point 3, I have some doubts about how scalable this approach is if the other embedding is not allowed to co-adapt. However, perhaps this problem could be alleviated if f_predict has extremely high capacity at initialization? More on this in point 6.\n\n    5. Assuming all my assessment above is correct, it seems that the performance of the proposed method should *not* decrease at all so I would like to see more analysis on the decreasing performance in the instruction learning task (Figure 2, left column). Is this be due to the fact that f_predict or the other embedding is not allowed to update and the newly added embedding is mapped close to existing embedding? In that case, can increasing the model capacity solve this problem? I understand the paper makes argument about regularization but I believe this warrants thorough study for gauging the significance of this approach in more realistic settings.\n\n    6. I would like to see a discussion of the short-comings of the approach and possible ways to overcome them. For example, freezing the syntactic network seems limiting in machine translation settings if a new language, say Italian, is added. Intuitively, knowing how to translate English-French should help translating English to Italian but fixing the syntax prevents this. Another example is that prior knowledge about the syntax needs to be known about the language for labeling the f and p and this can be expensive and cannot handle words that have more than 1 usages (e.g. run can be used as a noun but also as a verb).\n\nI am willing to increase my score to accept if the revised manuscript can address the majority of, if not all of the concerns listed above.\n\n========================================================================================================\nMinor comments that did not affect my decision:\n    - These sections could greatly benefit from an overall flow chart of how everything fits together\n    - It says the experiments are repeated with 5 different random seeds so why not add error bars to the plots?\n    - What characteristics of the 2 tasks cause the baselines to behave so differently?\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "rke6W0h6YH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "*Summary \n\nThe paper proposes a continual learning algorithm for label prediction to deal with sequence-to-sequence continual learning problems. The proposed method is designed to leverage compositionality.  The key idea of the proposed method is to enable the network to represent syntactic and semantic knowledge separately. This allows the neural network to leverage compositionality for knowledge transfer while alleviating catastrophic forgetting.\nThe experiments showed that their method performed significantly better results than baseline methods.  The method was tested on two different datasets, e.g., instruction Learning and machine translation.\n\n\n*Decision and supporting arguments\n\nI think this paper has enough quality to be be accepted as a conference paper.\nThe main reasons of my decision are two-folds.\nFirst, the proposal is quite insightful. The separation of semantics and syntax of an input sentence for using compositionality is an excellent idea.\nSecond, the proposed method improved the performance on two dataset significantly. This supports the usefulness of the idea.\n\n\n*Additional feedback\n\nMy concern is about evaluation. Table 1 shows the significant difference between the proposed method and the baseline methods. It looks to nice. But, this suggests that the datasets might be too artificial for this evaluation. To my understanding, both of the datasets are artificial to some extent. Hopefully, the method should be evaluated on the more realistic dataset.  \n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "S1xahryRFr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is about continual learning on NLP applications like natural language instruction learning or machine translation. The authors propose to exploit \"compositionality\" to separate semantics and syntax so as to facilitate the problem of interest.\n\nIn summary, the current manuscript is clearly not ready for publication. The writing is not good, as I cannot see clearly the backbone of the paper. Honestly, I got very confused by the presented contents. What’s the problem/motivation? No preliminary background knowledge? What’s the classic or naïve method to solve the problem of interest? What’s the main advantage/novelty of the presented method compared to that classic/naïve method?\n\nPlease see the detailed comments below.\n\nIn the Abstract, you mentioned \"One of the key skills … ability to produce novel composition’’. Do you imply your method can continually learn new compositions? If so, how does that reflect in the technical parts and the experiments?\n\nThe paragraph before Section 3 might be redundant.\n\nMany typos exist. Such as the word \"iuput’’ in the 1st paragraph of Page 4.\n\nWhat’s the problem of S2S-CL? Increasing input number n and output number m?\n\nWhat does the word \"COMPOSITIONALITY’’ mean in Section 3.2? Also, what’s the relationship between the last two equations of Page 4?\n\nHow do you defend the simplifications adopted in the first Equation of Page 5?\n\nThe notation \"{0,1}^{Uxn}’’ usually represents a binary matrix of size Uxn. It is not suitable to use them to represent a matrix containing one-hot columns.\n\nAt the beginning of Page 6. Why entropy regularization can be introduced via L2 norm on the embedding matrixes p and f? Also why that L2 norm regularizations can `achieve disentanglement’? Please provide the detailed proof or the reference. \n\nWhat are the detailed settings of the demonstrated experiments?\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses the task of continual learning in NLP for seq2seq style tasks. The key idea of the proposed method is to enable the network to represent syntactic and semantic knowledge separately, which allows the neural network to leverage compositionality for knowledge transfer and also solves the problem of catastrophic forgetting. The paper has been improved substantially after the reviewers' comments and also obtains good results on benchmark tasks. The only concern is that the evaluation is on artificial datasets. In future, the authors should try to include more evaluation on real datasets (however, this is also limited by availability of such datasets). As of now, I'm recommending an Acceptance.
Decision: Accept (Poster)