Paper:
Recent studies on catastrophic forgetting during sequential learning typically focus on fixing the accuracy of the predictions for a previously learned task. In this paper we argue that the outputs of neural networks are subject to rapid changes when learning a new data distribution, and networks that appear to "forget" everything still contain useful representation towards previous tasks. Instead of enforcing the output accuracy to stay the same, we propose to reduce the effect of catastrophic forgetting on the representation level, as the output layer can be quickly recovered later with a small number of examples. Towards this goal, we propose an experimental setup that measures the amount of representational forgetting, and develop a novel meta-learning algorithm to overcome this issue. The proposed meta-learner produces weight updates of a sequential learning network, mimicking a multi-task teacher network's representation. We show that our meta-learner can improve its learned representations on new tasks, while maintaining a good representation for old tasks.

Review:
Summary: This paper introduces a variation on measuring catastrophic forgetting in sequential learning at the representation level and attempts to resolve forgetting issue with the help of a meta-learner that predicts weight updates for previous tasks while it receives supervision from a multi-task learner teacher. The new method is evaluated on sequences of two tasks while task 1 data remains available at all times to the teacher.
Pros:
(+): This paper is very well-written and very well-motivated. 
(+): Tackling continual learning from a meta-learning approach is novel and not yet well-explored. 
(+): Literature review is done precisely well.
Cons that significantly affected my score and resulted in rejecting the paper are two-fold. 
First, based on my understanding from the paper, it appears that this work has a significant contradictory assumption with a regular continual learning setup and that is to provide access to the entire dataset from an old task while we learn a new task. This changes the problem from continual/sequential/lifelong learning to multi-task learning. All the prior work that were beautifully reviewed in section 1 and 2 obey this assumption where access to previous tasks’ data is either impossible (ex. [1,3,4,5,6,7,8] in the below list ) or is very limited (ex. [2]). 
Second, is the experimental setting. The experiments are accurately described and performed but authors have only considered sequence of 2 tasks which is far from being considered as a continual learning setting. I would like to ask the authors to explain how this method can be extended to multiple tasks and how much of the past data they should provide while training? Another drawback in the experiments is about the baselines. Despite addressing the most recent papers in section 2, authors have only made comparison against two relatively old approaches (EWC by Kirkpatrickthat et al from 2016 as well as LwF by Li & Hoiem presented at ECCV 2016, I believe the authors have cited the journal version of the work published in 2018 but the work is actually from ECCV 2016). Although these methods are still included as baselines in the literature, more recent approaches which have outperformed these need to be provided as well. I have provided a list of papers which achieved superior performance to the current baselines below which is arranged chronologically and is indeed not limited to this list as it is not realistic to list all prior work since 2016 in here. 
I would be happy to change my score if authors can address the above concerns about considering distinguishing multi-task learning from continual learning and providing a realistic evaluation setup with more than 2 tasks and comparison with current state of the art methods.
[1] Zenke, Friedemann, Ben Poole, and Surya Ganguli. "Continual learning through synaptic intelligence." Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017.
[2] Lopez-Paz, David, and Marc'Aurelio Ranzato. "Gradient episodic memory for continual learning." Advances in Neural Information Processing Systems. 2017.
[3] Shin, Hanul, et al. "Continual learning with deep generative replay." Advances in Neural Information Processing Systems. 2017.
[4] Nguyen, Cuong V., et al. "Variational continual learning." arXiv preprint arXiv:1710.10628 (2017).
[5] Serrà, J., Surís, D., Miron, M. & Karatzoglou, A.. (2018). Overcoming Catastrophic Forgetting with Hard Attention to the Task. Proceedings of the 35th International Conference on Machine Learning, in PMLR 80:4548-4557
[6] Schwarz, Jonathan, et al. "Progress & compress: A scalable framework for continual learning." arXiv preprint arXiv:1805.06370 (2018).  
[7] Mallya, Arun, and Svetlana Lazebnik. "Packnet: Adding multiple tasks to a single network by iterative pruning." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[8] Ebrahimi, Sayna, et al. "Uncertainty-guided Continual Learning with Bayesian Neural Networks." arXiv preprint arXiv:1906.02425 (2019).
[9] Aljundi, Rahaf, et al. "Online continual learning with no task boundaries." arXiv preprint arXiv:1903.08671 (2019).
------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------------------
POST-REBUTTAL review:
I disagree with the authors claiming that this work is continual learning (sequential learning + avoiding forgetting).
Despite introducing 9 recent continual learning work to authors in my initial review, they added 2 meta-learning baselines (MAML,REP), keeping 2 naive and old CL baselines is not acceptable. I reply to authors comment below regarding the baselines:
[Authors' reply:] Lopez-Paz, David, and Marc'Aurelio Ranzato. "Gradient episodic memory for continual learning." and Shin, Hanul, et al. "Continual learning with deep generative replay." and Nguyen, Cuong V., et al. "Variational continual learning” and Aljundi, Rahaf, et al. "Online continual learning with no task boundaries." We argue that these paper have a different setting compared to ours since they require a buffer whereas our method has no storage of past data/gradient. Having past data storage can usually improve the performance and our method can potentially also get a boost. Having a data buffer can also cost a lot of memory storage depending on input/weight dimension. Therefore, we argue it won’t be a fair setting to compare with methods with data buffers.
[Reviewer's reply:] GEM (Lopez-Paz et al., 2017) and its faster version (A-GEM) (Chaudhry, et al. 2018) and other memory based methods  such as MER (Riemer et al. 2018), ER-RES (Chaudhry et al. 2019), they use memory sizes of at most 6MB to store samples but they only do a **single epoch** through the data. So if it is not fair, it would be for those methods given the computational expenses of this paper. VCL (Nguyen et al. 2018) in its vanilla version does not use coreset memory if that is still your concern.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[Authors' reply:] Zenke, Friedemann, Ben Poole, and Surya Ganguli. "Continual learning through synaptic intelligence." has very similar performance to EWC in their paper. We have cited this work already.
[Reviewer's reply:] This method is an online version of EWC which is faster despite the on-par performance. So it has its own advantage.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[Authors' reply:] Serrà, J., Surís, D., Miron, M. & Karatzoglou, A.. (2018). Overcoming Catastrophic Forgetting with Hard Attention to the Task and other pruning based papers. Thanks for pointing out. We have cited and will compare to them in the future. One thing to note is that, in these works, the model needs to know which task ID it is currently dealing with, and thus can turn on the pruning procedure for the next session. This can potentially be a limitation for dynamic incoming tasks.
[Reviewer's reply:] Comparing with HAT paper (Serrà et al. 2018) is really easy using their provided code and is one of the strongest baselines in continual leaning literature. They do NOT do any pruning. Their approach simply learns an attention mask which regularizes weights and prevent changes on them without using any memory. Regarding the task number, this is indeed not an issue for your approach with 2 tasks.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[Authors' reply:] Ebrahimi, Sayna, et al. "Uncertainty-guided Continual Learning with Bayesian Neural Networks." has not been published in a conference venue, and it may be too early to compare with it. 
[Reviewer's reply:] I agree that this work is not published and hence can't be asked for comparison but I encourage authors to read it.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Authors are neglecting one important difference between meta-learning and continual learning: in MAML and REP, it is assumed that we have access to ALL tasks distributions from which we sample from in the beginning (look at page 3, algorithm 1, line 3). This is in contrast to continual learning where one cannot even assume how many tasks will be given. Moreover, the computational expense of this work which causes performing more than 2 tasks to be a future work is also not acceptable when there are significantly cheaper and are able to do a lot more than 2. (In all the references I mentioned, the length of the sequence in experiments is at least 5.)
While this work might be interesting to meta-learning community, I think it is far from being introduced as a method that prevents catastrophic forgetting and hence be included in the CL literature. Therefore, I intend to keep my score as reject.