Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a new approach for continual learning that actively forgets the old knowledge when learning the new tasks. The approach uses Bayesian continual learning with synaptic expansion-convergence for active forgetting. The paper evaluates the new method on several continual learning benchmarks, including CIFAR10 regression, visual classification, and Atari reinforcement tasks.
main_review: The synaptic expansion-convergence idea for active forgetting proposed in this paper seems novel for continual learning, although it is quite related to the power EP method by Minka. My main concern with the paper is its clarity, especially in the main technical part of the method. In particular:
1. The description of the proposed method in Section 3.2 is not very clear. For example, where is \theta_e in Eq (10)? How is \theta_e constructed (e.g. by expanding the network architecture or other methods)? How are the weights merged to obtain the original architecture? I find that the illustration in Figure 2 is not understandable. More details should be added here.
2. The paper should give more details on how to derive Eq (9).
3. Since the new prior has been changed by Eq (6), how does that affect the computation of the Fisher Information matrix in Eq (10)? More details should be added here, especially when there are more than 2 tasks.
4. Additionally, more details should be given on how the method can be applied with MAS, SI, and RWALK.
5. The CIFAR10 regression task in Section 4.1 is not realistic. Why would we want to convert a classification task into such a regression task? It may be better to consider a real regression task. Furthermore, the paper reported average accuracy for this regression task but did not give any explanation on how this average accuracy is computed.

===

The authors' rebuttal has adequately addressed my concerns. So I increase my score to 6.
limitations_and_societal_impact: No potential negative societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper investigates how to avoid negative transfer in Bayesian continual learning setting.
The main contribution is the proposition of an active forgetting mechanisms inspired by biological neural networks for mitigating negative transfer in CL.
The experiments on CL benchmarks have been conducted to validate the performances of the proposed method.

main_review: Originality:
  This paper proposes a novel approach -- Active Forgetting with synaptic Expansion-Convergence (AFEC), for reducing the interferes of knowledge from past learning. The method can be a plug-and-play to a CL model for boosting its performance.
  Relationship to previous works has been explained and relevant literature has been cited appropriately.

Quality:
  The work is relevant to audience at Neurips and provides reasonable details and code for reproducibility.
  The authors have conducted experiments on several CL benchmarks to evaluate the effectiveness of the proposed method, and demonstrated SOTA performance.

Clarity:
  The paper is overall clearly written and the method is adequately described.
It provides reasonable details and code for reproducibility.

Significance:
  The improvement of the proposed method is significant on several CL tasks including regression, classification, and RL.

[After Rebuttal]
After reading other reviewers' comments and the authors responses, I'd like to keep my rating 6 because the authors have addressed an important question in CL and most reviewers' concerns in their responses. They also provide enough details/codes for readers to reproduce the results.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposed a continual learning method that attempts to promote better forward transfer among similar tasks. Towards this, the authors attempt to suppress the negative knowledge, the knowledge that impedes learning a new task, from the previous task while learning a new task. This is done by forgetting such knowledge from the previous task without increasing the forgetting catastrophically. The authors claim that such “active forgetting” is biologically inspired. Similar to the Elastic Weight Consolidation (EWC), the authors derive the learning objective, by applying the laplace approximation to the posterior of a new task. The only difference is that the posterior of the previous task is replaced by a weighted product distribution, where a weight of ‘\beta’ is introduced to capture active forgetting. This change allowed them to derive a new regularization penalty whereby a separate model is trained for a new task and then subsequently merged with that of the previous one. It is in this merging that the negative knowledge of the previous task is eliminated. Experimentally, the authors show strong performance on regression, classification and RL tasks. The authors further show that their method can be added to other regularization- and memory-based methods to improve their performance. 

main_review: Positives:

- Overall the paper is well-written and intuitively easy to understand. 
- Experiments are conducted thoroughly across various domains. 

Negatives:

- Details of the method: It seems that some implementation details of the method are missing and, in fact, difficult to discern from the main learning objective (Eq. 10). Do the authors first train the separate network to obtain the optimal parameters for the new task $\theta_e^*$, then compute $F_e$ for this optimal model, and finally use Eq. 10 to merge the knowledge from the two tasks/ models? If not, then how is the $\theta_e^*$ and $F_e$ are obtained? If yes, then would there be an additional $L_B(\theta)$ term in objective? 

- Progress and Compress (Schwarz et al.): It seems that conceptually the method is very similar to the progress and compress (P&C) where a new “active” column/ network is learned for a new task, that is later distilled to the unified model of the previous task. The authors of the present work seem to do the same thing but, of course, the mechanism of merging is different: whereas in P&C knowledge distillation is used, AFEC (present work) uses an additional EWC-type loss. I would want the authors to compare against P&C both theoretically and experimentally. 

- Short term measure of negative knowledge: It seems that AFEC takes a short term view of negative knowledge (i.e.) the knowledge that is negative for only the current task. I can imagine scenarios where an orthogonal current task would force the network to remove most of the knowledge from previous task(s), because it was deemed negative for the current task, even though it would have been beneficial for the future tasks. What is the take of the authors on that?

--------------------------------------------
Post-rebuttal:

I quite likes how objectively the authors responded to the review. While the two-step training for each new task has its drawbacks, I overall quite like the paper. The authors response addressed some of my concerns. Therefore, I am increasing my score to 6.
limitations_and_societal_impact: N/A
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a method for continual learning that aims to mitigate the problem of catastrophic forgetting by actively forgetting knowledge from previous tasks. The authors use a Bayesian continual learning framework to design their method. AFEC relies on identifying a set of expanded network parameters which are specific to the new task. The method incorporates a term in the loss function that considers the merge between these parameters and the network parameters being updated for the current task, therefore encouraging active forgetting. Catastrophic forgetting is also controlled by including a term that merges between parameters for old tasks and network parameters for the current task (i.e. the EWC term). Experimental results are reported for datasets on regression, classification and reinforcement learning tasks. These results show some advantages of the method in particular in the forward transfer setting. 
main_review: Strengths of this paper are: 
- The paper is very well-written, well-structured, and easy to follow. 
- The method explores a new avenue which has been rather ignored in previous continual learning research, in terms of promoting forgetting when appropriate. I believe this is the main strength and novelty of the proposed approach. 
- The contribution is technically sound, and the claims are reasonably supported by the experimental results in a variety of tasks (regression, classification, reinforcement learning)

Weaknesses of this paper are: 
- In the experimental results reported in Section 4.2, it is not clear whether the 5 runs involved 5 different task orders or just 5 different random seeds. This is important as the order of tasks has a direct influence in forward and backward transfer – i.e. learning easier tasks first and then hard ones may give different results than learning hard tasks first and then easier ones. 
- Although it is explicitly mentioned that the set of expanded parameters does not need to be stored for later use, the computational cost of solving Eq. 10 is not mentioned. In particular, does this increase with the size of the network, for example for a rather large number of tasks?
- Although I appreciate the analogy with biological active forgetting, a clear difference I see of the proposed approach vs. a biological system is the number of tasks. How would AFEC work well for a rather large number of tasks (e.g. hundreds, thousands)? I would believe the trade-off between actively forgetting and retaining knowledge of old tasks would be increasingly challenging with an increasing number of tasks. 
- The use of some language could be improved. For example, the word “significantly” is used in several parts of the paper, although no tests for statistical significance are reported. There is also a typo at the end of the first paragraph of Section 2 (“which toke advantages… ”). 

limitations_and_societal_impact: I would suggest the authors to explicitly mention/point to limitations in terms of complexity of tasks (which implies a larger set of parameters) and number of tasks (which increases the challenge for balancing active forgetting and knowledge retention of old tasks).
review_title: null

Meta-review: This work touches on an important problem in Continual Learning that has not been as thoroughly investigated as catastrophic forgetting, namely graceful forgetting (or an active and controlled mode of forgetting) that would allow forward transfer. 
While the need of graceful forgetting has been discussed in the past, there are not many methods that manage to do so in a successful way, and this work proposes a practical way of doing just that. 

Given the reviews, and the active participation of the authors in the discussion (including new experiments) I believe the current manuscript is of interest to the community and matches the expectations from the conference. 
Decision: Accept (Poster)