Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Summary:

The authors do an evaluation of the application of weight-importance continual learning methods to recurrent neural networks (RNNs). They draw out the tradeoff between complexity of precessing and just remembering (working memory) in terms of the applicability of these weight importance methods. They also provide some theoretical interpretation based on stying linear RNNs.

Overall, I vote for accepting this paper because the work is well-motivated, thorough, and provides useful insights. My major concerns are listed below. 

Strengths:

+ The paper is very well written, and the motivation, methods and inferences are quite clearly described. The main question the authors are considering is very clear.
+ The results around use of existing continual learning methods to RNNs is timely and relevant.
+ The insight into the tradeoff between complexity of processing and working memory requirements and its effect on the ability of the network to continually learn is very interesting. Similarly the fact that hypernetwork based approaches work better than other approaches most of the time is useful.
+ The analysis of the above tradeoff using a linear RNN is also interesting since it provides a nice intuition for why the tradeoff exists.

Weaknesses:

- The motivation and conclusions from the ssMNIST task is not very evident and the tasks doesn't seem to make a clear point. 
- Readability of some parts of the paper depend very heavily on the supplement, and the paper itself doesn't stand by itself. For example, the linear RNN analysis, description of some tasks (I had to read the supplement to actually understand the permuted copy and the pattern manipulation tasks). As an aside, I hope the authors submit an extended version to an appropriate venue, because I think many of the results and discussions relegated to the supplement seem quite interesting and relevant for the community (e.g. the task-conditional processing).
- An analysis of why hypernetworks perform better would be interesting. So would have been some proposals for methods designed specifically for CL with RNNs.

Minor:

The main text doesn't clearly mention that vanilla RNNs are used in section 5.1
review_title: An interesting and timely analysis of CL for RNNs

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper provides a systematic evaluation of the performance of different CL methods on RNN. The study suggests that high working memory requirements increase difficulty of learning new tasks, while the average length of input sequence is not strictly related to the difficulty of learning new tasks. The author proposes to overcome this problem by using a hypernetwork-based CL approach, which shows promising results in the experiments.

Strength:
* The paper provides extensive study to compare different continual learning methods.
* The conclusion is well supported by analysis of intrinsic dimension and performance on different tasks.
* The paper is well written and easy to follow

Weaknesses
* It would be interesting to see results on more realistic tasks, like sentence classification.
* The conclusion on working memory requirement didn’t consider the possibility of knowledge sharing between tasks. For example, two complicated tasks may share a common sub-network that is essential for solving both tasks. Such that, in an ideal situation, the model doesn't need to allocate large amounts of extra resources to learn the second task. It would be interesting to see how different CL methods can reuse knowledge learnt from previous tasks.

review_title: Extensive study and convincing results

final_score: rating: 7: Good paper, accept
review_content: Pros: Most work on continual learning addresses only feedforward networks. This paper
provides apparently the first systematic discussion and comparison of CL methods for RNNs
Thereby it provides an important service to the community. The material is presented thoroughly in the Suppl. 

Cons:
Provides on the other hand less algorithmic innovation. In particular, it focuses on methods related to (Oswald et al., 2020), a paper that was accidentally omitted from the reference list, but apparently is this ICLR 2020 paper that contains related material:
von Oswald, J., Henning, C., Sacramento, J., & Grewe, B. F. (2019). Continual learning with hypernetworks. arXiv preprint arXiv:1906.00695. ICLR 2020.

I am uncertain about the generalizability of results that were demonstrated for the chosen benchmark tasks. In particular, the conceptually important distinction between challenges arising from working memory load and sequence length is tested by variations of the copy task with padded inputs, where relevant and irrelevant input bits are distinguished in a very simple way that is hardly met by real-world scenarios. 

There may also be differences arising from different types of RNNs, and it is not clear to me to what extent one can make conclusions about all of them by testing on just one type.

I tend to vote for accept.

review_title: This review on continual learning for RNNs provides a very valuable service to the community.

Meta-review: I agree with the reviewers, and I find the careful analysis of CL approaches relying on regularization for RNN useful and insightful. I do feel that a lot of the interesting content is still in the appendix (from a quick skim and looking at the plots in the appendix) but I think something like this can potentially be unavoidable. 

I do like the separation between sequence length and memory requirements. I think making observations about different types of recurrent architectures is hard, but I think the paper does a good job to raise some interesting questions. 

A note that I would make (that I haven't seen raised through a quick look in the paper) is that is not clear how the Fisher Information Matrix should be computed in case of a recurrent model (which is a problem in general). E.g. a typical thing is to compute it as for a feed-forward model (using the gradients coming from BPTT) which is feasible computationally, but actually that is problematic as you first sum gradients before taking their outer-product rather than summing the outer-products corresponding of the different terms in the gradient. I'm wondering if that plays a role here as well.

Overall I think the paper does careful analysis and ablation studies and raises some interesting observation of how one should approach CL algorithms for RNN models.


Decision: Accept (Poster)