Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:

This work focuses on improving continual learning framework by introducing neural importance determined by activation value. In doing so, the authors introduce neuron importance as weight factor in minimizing catastrophic forgetting via regularization term. They also investigate continual learning by changing the order of the tasks. 


Strengths:
- introduces the neuron importance in regularization terms for minimizing catastrophic forgetting  
- the paper is well written 
- sound quantitative evaluation and performance analysis  using several data sets

Weaknesses:
- lack novelty and limited methodological contributions 
- it is unclear the need for introducing neuron importance in regularization techniques for catastrophic forgetting, since the activation value of neurons is also derived from the network weights
- the gains are marginal for some data sets


Questions:
- Since neural networks (neuron activity) is too sensitive with change in representation (or input) and therefore, does not guarantee stable results on a particular historical task. This work focuses on determining neuron importance by activation value, however does this activation value correlates to task performance?
- Since neural activity/importance is calculated based on the network weights, introducing neural importance as well as weight regularization is redundant? Please clarify. 

Additional comments:
- Unclear, what is f_n_k in equation 2?

Additional references (On Page 1, para 2 or 3):
1. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences  2017. 
2. Gupta, P., Chaudhary, Y., Runkler, T., Schütze, H. Neural Topic Modeling with Continual Lifelong Learning. In ICML 2020.
review_title: Interesting work but lacks sufficient methodological contributions

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper describes an approach to regularization-based continual learning that looks to preserve parameters based on node importance rather than weight importance. The paper categorises existing techniques into these groups (most prior work is based on weight importance), argues that node importance is a better measure, and then performs evaluation on numerous benchmarks and with different variations (importance calculation, task ordering, etc).

This is an important and promising research direction, but unfortunately I think the paper has a number of problems in its current form which preclude publication at this stage.

First, the approach seems to derive extensively from Jung et al (2020), and it is unclear whether there is any algorithmic or technical innovation that is added here. From Section 2.1, the only potential improvement I could see is that the average activation is scaled by the standard deviation, which is quite a trivial change. Reinitialization of weights is also discussed, but this is present in previous work (e.g. Ahn et al (2019)) as well. Figure 3 doesn’t show much of a difference in the distribution of weight importances (beyond the obvious scaling), and most crucially, the experiments do not compare to Jung et al, which makes it very difficult to gauge whether there is any novelty at all.

Second, the approach compares against SI/MAS/EWC as the canonical weight-importance-based approaches, but these are quite old, and a number of more recent techniques also fall into this category, such as VCL [1] (without coreset) and BGD [2] - which estimate the posterior distribution over weights at each timestep, hence determining which ones are important. The experiments would be more convincing if they demonstrated improvement over these more recent approaches.

Last, the writing is quite unclear in parts, and I’d recommend a thorough proofread for spelling/grammar and clarity. A few things I spotted are listed below, but there are many others.

Other questions and comments:
- Clarification for page 1: EWC computes weight importance after each task, not “after network training”
- Not clear why Fig 1b and 1c are necessarily different categories - the only way they are different is how the node importance is calculated. Is this fundamentally a different case, and why?
- One claim in page 3: “If average activation value is used as neuron importance, method will prefer to keep the weights of earlier layers” is stated as a problem - why is this so? This seems reasonable, as lower layers (edge filters, etc) are more likely to be relevant to many tasks, and the higher layers that model more complex features are more likely to need greater plasticity to adapt to new tasks.
- At the start of page 2, it says that with weight-importance based methods, “it is impossible to reinitialize weights at each training of a new task, which decreases the plasticity of the network.” We usually don’t want to reinit weights for each task, so it’d be good to provide more context upfront for why we might want to do this (this comes later in section 2.1.1).

Some typos and writing issues:
- Page 1, not sure what “utilize the weights of a given network to the hit” means.
- Page 2, “One of our key observation…” should be observations; “We propose comprehensive evaluation …” should be “... a comprehensive…”.
- Page 3: “As exampled in …”; “not only networks have to…”; “we can let the model starts…”, etc.

[1] Nguyen, Cuong V., et al. "Variational continual learning." arXiv preprint arXiv:1710.10628 (2017).
[2] Zeno, Chen, et al. "Task agnostic continual learning using online variational bayes." arXiv preprint arXiv:1803.10123 (2018).


=================
Post-discussion:

After reading the authors' response and the changes to the paper, I must unfortunately stick with my current score. I applaud the authors for taking my feedback on board, and certainly many changes have been made to improve the paper, but my main concern of novelty regrettably still remains. The paper offers a very simple improvement over Jung et al (effectively scaling the importance measure), which I believe is quite incremental in this setting. While I believe simple advances can often have broad impact (eg. dropout, batchnorm, etc), in this case it is not clear that the proposed change offers any benefits outside of the very specific area of importance-based continual learning.

review_title: Interesting direction, but unclear if there's any meaningful contribution over previous work (Jung et al, 2020)

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes a method that tackles the problem of catastrophic forgetting in continual neural networks by assigning importance to neuron activations while tasks are executed in sequence. Similar to previous research (Jung et al., 2020), the proposed method measures neuron importance using average activation values divided by corresponding standard deviation. This strategy is accompanied by weight re-initialization to guarantee that new tasks are fully learned. The method is tested in benchmark datasets for continual learning. 

As positive aspects of the paper I would remark: 
-	The paper is clearly placed within the existing literature in continual learning, as part of regularization-based approaches. 
-	The experimental evaluation tests several options of tasks sequences to prevent results to be dependent on the task order. This is an important aspect to demonstrate the validity of the results. 

Weaknesses of the paper are: 
-	The main weakness in my opinion is that the proposed method represents a simple extension of previous work (Jung et al., 2020). This compromises the novelty of the proposed approach. Furthermore, the experimental results do not denote a remarkable advantage of the proposed method compared to existing approaches. 
-	I am missing some insights on how the experimental results are connected to the motivation of the proposed method. For example, how adding a sense of the standard deviation of neurons importance help to prevent keeping weights of earlier layers in the experimental datasets?  
-	The experimental setup is limited to a few tasks, which undermines the results presented in the sense that continual learning systems may be composed of several tasks (hundreds, thousands) received in sequence. 

Some remaining comments/questions: 
-	There are some minor typos/formatting problems along the paper. E.g. caption in Figure 3 does not mention from which dataset these results come from. The presentation of plots of standard deviations alongside mean accuracies is unnatural – why not to use error bars so the differences are clearer?
-	How adding a sense of the standard deviation of neurons importance help to prevent keeping weights of earlier layers in the experimental datasets? 

review_title: Review for Continual learning with neural activation importance

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper introduces a regularization approach for stable continual learning of sequential tasks. The proposed method computes neuron importance based on the activation values of nodes with their respective standard deviation. It further suggests a weight re-initialization scheme to achieve better performance. Experimental results on several continual learning scenarios show that the presented approach is comparable to other existing competitors.

The paper first gives analyses by categorizing existing continual learning strategies based on regularization and tackles an important issue that arises when the order of incoming tasks changes. From the analyses, the paper proposes an approach to defining new neuron importance, as shown in equation (2). However, the presented strategy to find neuron importance is simple and incremental. Moreover, no rigorous analyses of the proposed scheme exist on how it is derived, what are benefits, etc. Thus, I am not convinced that it is particularly useful compared to its strong competitors sharing a similar motivation and strategy. For example, I am not sure why the proposed method can be robust to the order of tasks.

Since the proposed method deals with an order-robust approach, it is compared with other approaches sharing the same goal (e.g., Yoon et al., Scalable and Order-robust Continual Learning with Additive Parameter Decomposition, ICLR 2020). A comparison with existing similar approaches would make this work stronger.

Experimental results do not show the proposed method is promising. The proposed method does not outperform existing works and even performs poorer than some competitors for some experiments. From the results, I am not convinced about the benefits of the proposal. Figure 13 seems not notable output.

review_title: The paper is rather incremental and does not show its potential sufficiently.

Meta-review: Although this paper proposes an intriguing method for using neuron-importance-based regularization to reduce catastrophic forgetting in continual learning, the method is substantially based upon Jung et al (2020), reducing its novelty. Additionally, the experimental evaluation was unconvincing that the proposed method is an improvement over current methods and precisely how the proposed method differs from Jung et al.  The authors are encouraged to revise the paper to incorporate the reviewers suggestions and many of the points the authors raised in their rebuttals, which the reviewers felt were not adequately addressed in the current version of the paper (as mentioned in private discussions among the reviewers).
Decision: Reject