Paper:
Motivated by the human's ability to continually learn and gain knowledge over time, several research efforts have been pushing the limits of machines to constantly learn while alleviating catastrophic forgetting (Kirkpatrick et al., 2017b). Most of the existing methods have been focusing on continual learning of label prediction tasks, which have fixed input and output sizes. In this paper, we propose a new scenario of continual learning which handles sequence-to-sequence tasks common in language learning. We further propose an approach to use label prediction continual learning algorithm for sequence-to-sequence continual learning by leveraging compositionality (Chomsky, 1957). Experimental results show that the proposed method has significant improvement over state of the art methods. It enables knowledge transfer and prevents catastrophic forgetting, resulting in more than 85% accuracy up to 100 stages, compared with less than 50% accuracy for baselines in instruction learning task. It also shows significant improvement in a machine translation task. This is the first work to combine continual learning and compositionality for language learning, and we hope this work will make machines more helpful in various tasks.

Review:
This paper proposes an approach for continual learning for applications in sequence-to-sequence continual learning (S2S-CL). More specifically, the paper addresses the problem of growing vocabulary. The overall approach is intuitive yet quite effective. The method assumes a split between the syntax (f) and semantics (p) of the sequence -- in other words, each token is associated with two labels. Furthermore, the syntax is assumed to be shared across all the training sequences and the sequences that are not encountered during the initial training. A sequence model (LSTM) is used for learning the syntax f over the initial training and is then frozen for all the downstream tasks (i.e. continual learning). The sequence model predicts the correspondence between the output token and input tokens. Another network (a 1-layer  MLP) is used to predict the semantic label of the selected token in the target domain. Barring some notational confusion, I believe the method is very reasonable and should work well. They perform experiments on 2 datasets (Instruction learning & machine translation) in 3 continuous learning paradigms with varying difficulties and demonstrate that the proposed method significantly outperforms the baselines by an impressive margin.
I am leaning towards accepting this paper since I believe the proposed approach can be a promising direction for continual learning in S2S settings and the empirical results are convincing.
However, I have some concerns that might require clarification or additional experiments:
    1. The novelty of the proposed method is somewhat limited in my opinion. It seems it is a very simple adaptation of Li et al. 19 and the only addition to that work is the usage of a very simple label prediction scheme by fixing everything else. However, this might be okay since the experiments are quite compelling and the method is applied to a different setting.
    2. Notations and language in section 3.2-3.4 are hard to follow:
        - At the bottom of page 4, I believe some of the equations are not correct. For example, the first term of line 2 should be P(Y^f | X^f, X^p) rather than Y^p?
        - In the second equation of page 6, the second term should be v_j = p’ \cdot a_j
        - In Figure 1, k_r, E_r, and W are never defined (although their meaning can be inferred).
        - In section 3.4, it reads that new word embedding is appended to both semantic and syntactic embedding matrices but this doesn’t make sense because the syntactic network is already fixed so it shouldn’t be able to handle new symbols; therefore, I believe only new row is added to the semantic embedding.
    3. I believe that f_predict here is parameterized by \theta and \theta is also frozen during the continual learning phase which contradicts the claim at the end of section 3.2 that only \phi is frozen. Otherwise, it’s hard to see why f_predict does not suffer from catastrophic forgetting. From the provided source code, it seems that it is indeed the case that only the new embedding is being updated. In other words, the only thing happening at this stage is that the newly added embedding are optimized to adapt to the frozen f_predict.
    4. Due to point 3, I have some doubts about how scalable this approach is if the other embedding is not allowed to co-adapt. However, perhaps this problem could be alleviated if f_predict has extremely high capacity at initialization? More on this in point 6.
    5. Assuming all my assessment above is correct, it seems that the performance of the proposed method should *not* decrease at all so I would like to see more analysis on the decreasing performance in the instruction learning task (Figure 2, left column). Is this be due to the fact that f_predict or the other embedding is not allowed to update and the newly added embedding is mapped close to existing embedding? In that case, can increasing the model capacity solve this problem? I understand the paper makes argument about regularization but I believe this warrants thorough study for gauging the significance of this approach in more realistic settings.
    6. I would like to see a discussion of the short-comings of the approach and possible ways to overcome them. For example, freezing the syntactic network seems limiting in machine translation settings if a new language, say Italian, is added. Intuitively, knowing how to translate English-French should help translating English to Italian but fixing the syntax prevents this. Another example is that prior knowledge about the syntax needs to be known about the language for labeling the f and p and this can be expensive and cannot handle words that have more than 1 usages (e.g. run can be used as a noun but also as a verb).
I am willing to increase my score to accept if the revised manuscript can address the majority of, if not all of the concerns listed above.
========================================================================================================
Minor comments that did not affect my decision:
    - These sections could greatly benefit from an overall flow chart of how everything fits together
    - It says the experiments are repeated with 5 different random seeds so why not add error bars to the plots?
    - What characteristics of the 2 tasks cause the baselines to behave so differently?