Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: Motivated by biological principles, the authors propose to combine a generative classifier and a meta-learned discriminative embedding network to address meta-continual learning. The results show that the method is more robust against forgetting.
main_review: Originality: 
The idea is moderately novel, since essentialy replaces the prototype classifier used in some methods (eg. iCArL) by a Bayesian version, i.e. a generative classifier. The method is evaluated in the standard benchmarks. 

Quality:
The submission seems sound.The claims are supported by several experiments. 

Clarity:
The paper is generally well written and clear. The figures as well. However, the authors overemphasize the connections with human learning.

Significance:
The results are positive and state-of-the-art, and the discriminative-generative framework is useful. Another advantage is its computational efficiency and scalability.

Comments:
- Results from other families of meta-learning should be included.
- It is not bd clear the actual connection to biological the authors mention in the motivation, and the mechanism of concept learning.

limitations_and_societal_impact: The authors mention on passing positive impact, but not limitations nor potential negative impact.
Note that the claim that lower energy consumption makes the model more "planet-friendly" is usually not true. Actually, it is often the opposite (e.g. rebound effect).
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a probabilistic generative classifier that is immune to catastrophic forgetting. The proposed lightweight generative classifier achieves high accuracy for new unseen continual learning problems.
main_review: [Strengths] 

The writing of the article is very good.
The method have achieved relatively high classification accuracy. The method of experimental comparison is comprehensive.

[Weaknesses]

In line 140-141, why the generative model is completely immune to catastrophic forgetting, needs a deeper explanation.] 
limitations_and_societal_impact: The authors adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a probabilistic generative approach using meta-learning for continual learning. It is inspired by cognitive and neuroscience about meta-continual learning to address catastrophic forgetting. It presents the problems occurring in the discriminative approaches and suggests how to handle them by using generative classifiers. Using new update scheme for discriminative
classifiers, it shows that the catastrophic forgetting problem is reduced.
It also improves the existing prototypical learning by Bayesian approach.




main_review: Strong Points:
1- The paper is well organized and easy to follow. It provides sufficient information for each component used in the model and sare code for cross-checking the results.

2-  It shows the use of meta-learning in continual learning and presents the advantage of the generative model over the discriminative model. It proposes a way to improve the performance of the discriminative model by a new updated scheme that reduces the catastrophic forgetting problem.

3-  The proposed model is implemented in the fully incremental
setting. It includes a solid ablation analysis for each component.

Weaknesses:
1-  There are several existing approaches for Meta-Continual Learning [a,b].  It is unclear to me the exact differences between the existing and the proposed method except for the generative approach.  Would you please provide a precise difference and justify how it handles the shortcoming compared to previous approaches.

2- Experimental results are missing for the CIFR-100 dataset.

3- Can you apply the same meta-learning trick with discriminative approaches? Then it will be helpful to see the contribution to the generative approach.

4- It should include some non-meta learning approaches for comparison like [c].

5-  This approach can be implemented for online continual learning? If yes, then how?

[a]- Meta-Learning Representations for Continual Learning, NeurIPS 2019.

[b]- La-MAML: Look-ahead Meta-Learning for Continual Learning, NeurIPS 2020.

[c]- Online Class-Incremental Continual Learning with Adversarial Shapley Value, AAAI 2021.





limitations_and_societal_impact: Please see the weaknesses in the main review section.  Include the block diagram for the proposed approach in the main paper.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors propose the generative classifier for meta continual learning. This approach is an extension of a well-known method for few-shot learning called prototypical networks. The proposed method is evaluated on two challenging continual learning datasets (Omniglot and MiniImagenet) and compared to other methodsin this field as well as adequate baselines. 
The main contributions of the paper can be summarized as:
- The introduction of the new method for continual learning using the generative classifier. 
- Clearly explained and visualized with toy example the difference between catastrophic forgetting in case of generative and discriminative classifiers.
- Precisely formulated and justified statements about immunity to catastrophic forgetting.
- Insightful empirical analysis of the proposed method (mostly covered in the appendix). 

main_review: Originality: The paper presents an approach to meta continual learning using a generative classifier. The proposed method is closely related to the method introduced for few-shot learning called Prototypical Networks. Authors points out that Prototypical Networks is a special case of their method. This suggests that the proposed method should be traeted as a valuable extension of the existing algorithm rather than a novel approach. 
The proposed method is immune to catastrophic forgetting i.e., at the end of continual training, its performance is equal to the performance of the same model in standard i.i.d setting, which is remarkable. However, this is achieved explicitly by two design choices: static embedding network, which is frozen after the meta training phase, and parameter isolation enforced by the generative classifier. While this approach offers immunity to catastrophic forgetting, it simultaneously forbids the network to continuously adapt to upcoming data and reuse previously learned knowledge with transfer learning. This raises the question of whether the approach is a continual learning method. 

Quality: The paper is a complete piece of work that starts with introducing the general idea backed up by the toy example demonstrating the advantage of the proposed solution. Next section presents the mathematical details to conclude the paper with the section of empirical results validating the proposed method. The experimental section is thorough and well written and the proposed baselines are sensible. However, I have a few questions which were not answered in this section. I will be more than happy to discuss them with the authors. The statements presented in the paper are precise, and the argumentation in favor of these statements is convincing. The only exception is the claim that the authors make in the introduction section: “Encouraged by generative classifiers, we propose a new update scheme for discriminative classifiers to reduce the catastrophic forgetting problem.” If I understand correctly, the authors refer here to the PGLR baseline in which only the part of the classifying layer is updated according to the present task. I find it challenging to understand the novelty of this method since all the classifiers working in the regime of Task Incremental scenario works the same way. I would appreciate it if the authors elaborate on this topic.

Additional questions to the authors:
The model’s effectiveness is linked to the quality of the pretrained embedding network, and the paper lacks the thorough discussion of this part. I enlist the following questions, which may help to clarify this part.
In all experiments the meta train split forms a larger part of the whole dataset. How the size of the meta-train dataset impacts the overall performance of the method? Does the perforamcen plateaus at some point or does it constantly improve with the bigger meta train dataset?
Regarding results in Table 2 (appnedix). Why the use of OML backbone results in worse results? This backbone is almost 40 x bigger than the backbone used in the main experiments, yet it achieves significantly worse results. Is it a general tendency that the bigger the backbone the worse the results? 
I miss the details about learning procedure of the pretrained backbone. In Table 1 (main paper) the difference between pretrained backbone and meta learning on ImageNet is relatively small while in case of Omniglot is substantially bigger. What might be the reason for this?
 
The last question is rather technical. In Fig.4 (main paper) X axis ends on 600th class while the test set should be composed of 723 classes because the meta train and meta validation split has 900 classes and the whole dataset is composed of 1623. What is the reason for this?


Clarity: This paper is well written and has a correct structure. The code enclosed in supplementary files is well-structured and easy to follow. Additionally, the details for experimental results are clearly presented. 

Significance: Authors presents the method clearly and evaluates is effectiveness on challenging benchmarks on which it achieves state-of-the-art results compared to other methods in this field. However, as mentioned earlier, I find it inappropriate to classify this method as a continual learning method due to its abovementioned limitations. Nevertheless, the proposed solution to use the generative approach for classification in continual learning is a valuable insight. It could serve as a building block for other methods in this field. 
limitations_and_societal_impact: The authors correctly identified the main limitations of this method, however, did not provide any discussion on how these limitations can be circumvented. Additionally, the authors did not provide any discussion about how this method can be further developed and what are the possible future research paths. The other issue is the evaluation procedure for the method. To assign class for any datasample during the test the method finds maximum probability over all already known class distributions. This leads to the linear growth of the method runtime and memory requirements  with respect to the classes. I would be interested to see if the authors has any potential ideas how to reduce this requirements?
review_title: null

Meta-review: This paper addresses catastrophic forgetting in continual learning using inspirations from cognitive and neuroscience. The drawbacks of discriminative models are first demonstrated, and then a probabilistic generative method is presented, leveraging prototypical learning by Bayesian approach.  Experimental results confirm the effectiveness of the new method.

All reviewers and I find the paper interesting and well written.  The new method is efficient and effective in practice, and the intuition is well illustrated.  The rebuttal well addressed some concerns, and the authors are encouraged to incorporate them into the final version.
Decision: Accept (Poster)