Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The setting of this paper is few-shot class incremental learning (FSCIL). The key idea is using neural collapse to do FSCIL. This paper use a set of pre-defined classifier prototypes as ETF for the whole label space, and then aligns all classes, including base session's classes and novel classes, to the pre-defined classifier prototypes. What's more, a novel loss function is proposed to drive the features into the corresponding prototypes. Experiments show that the proposed method achieves state-of-the-art performances on three standard benchmarks. 

strength_and_weaknesses: Strength:
(1) I think the paper is well organized, clearly motivated, and contributes novelty.
(2) Using Neural Collapse for FSCIL is inspiring, and the proposed loss function to drive the output features to the corresponding prototypes is novel.

Weaknesses:
(1) Unfair comparison. From Table.5 and Appendix: IMPLEMENTATION DETAILS, this method, and competitors use different network architectures. And even this method use model weights pre-trained on ImageNet on the CUB benchmark. However, all other methods (FACT, LIMIT, etc.) train the model from scratch. 
(2) This method uses memory to save old class prototype features. However, other methods (FACT, LIMIT, etc.) do not save any data. What's more, the feature extractor is fixed after the base phase, saving features is the same as saving raw data.

clarity,_quality,_novelty_and_reproducibility: This paper is reproducible and the idea to use neural collapse seems novel. 
summary_of_the_review: This paper uses neural collapse for FSCIL and obtain good results. Nonetheless, there is a concern about unfair comparison.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This study involves a few-shot class incremental learning method, inspired by neural collapse inspired feature-classifier alignment. By using fixed prototype vectors to replaced the learned vectors, which have no constraints, it reduces the confusion and collapse caused by incremental classes with few-shot training samples. Experiments on mini-ImageNet and CIFAR-100 validate the effectiveness of the proposed approach. Overall, this is a piece of interesting work, considering an important yet unsolved problem in the machine learning and computer vision areas.
strength_and_weaknesses: Strength:

The idea is novel and interesting. The proposed FSCIL method assigns neural collapse inspired feature-classifier alignment as a target and trains a model towards the same optimization in each session to avoid class confusion and conflict. The performance is compare to, if not outperforms, the recently published methods.

Weakness:

The formulation of neural collapse is not that strict. In my view, the ``neural collapse” appears more like a class confusion in a feature space. The neural collapse, as illustrated in Fig. 1, is also built on the assumption that the feature space is fixed. While as is known, the feature space would change significantly during the model training (either base training or finetuning) procedure. I would like the authors to clarify this point during rebuttal.

The authors claimed that the proposed approach achieved an average 3.5\% accuracy improvement over the STOA. But I note in the first session (session 0) in Table 1, the proposed approach has a 3.42\% accuracy higher than the compared method. While in the last session, it solely outperforms by 3.83\%. This means that the performance retain is only slightly better. In other words, the proposed approach benefit from a stronger baseline (the first session). 
clarity,_quality,_novelty_and_reproducibility: The clarity of the ``neural collapse” formulation could be improved. The originality is good. Using fixed prototypes to equally divide the feature space is a novel idea. There is no code provided, such that the re-producibility of the submission is difficult to be evaluated.
summary_of_the_review:  While I approve the novel idea of this submission, I am not convinced by the formulation of neural collapse. The performance retain across training session is an important metric, which is missed in the experiments.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposed a new loss function for incremental learning, inspired by the inductive bias of neural collapse. The basic idea is to preallocate a sufficiently large Equiangular Tight Frame (ETF),  so that features from different classes (old and new) will be maximally separated from each other. At the same time, use remembered features from the old classes to fix the representations of the old classes.

The authors empirically show the effectiveness of the proposed method.
strength_and_weaknesses: Strength:

The proposed method is an innovative application of Neural Collapse to incremental learning. The method seems to be effective in the experiments shown.




clarity,_quality,_novelty_and_reproducibility: Overall, the paper is well written. The proposed solution is simple but novel.
summary_of_the_review: Overall, the authors proposed a simple yet effective solution to catastrophic forgetting in incremental learning. I hope that the authors can publish the code for the sake of reproducibility.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Inspired by Neural collapse, this paper proposed to ﬁx a learnable classiﬁer as a geometric structure for few-shot incremental learning. 
The pre-assigned classifier could avoid optimization conﬂict among sessions in the training stage. They also provided a detailed theoretical analysis to show their method could hold the neural collapse optimality.  Experiments on the three datasets achieve the best performance compared with previous methods.
strength_and_weaknesses: Strength: 

The proposed framework that combines neural collapse inspired framework in the few-shot class incremental learning is interesting and inspiring. I also appreciate the extensive experiments (ablation studies and extensions) conducted by the authors.

(1) The proposed neural collapse inspired framework on few-shot incremental learning seems novel.  The pre-assigned classifier could avoid optimization conﬂict in each session in the training stage. Its effectiveness was successfully proved by experiments and theoretical analysis. This is the biggest contribution of this paper.   Although adding an ETF classifier is not so novel, its effectiveness was shown by the experiments. 

(2) The experiments in the supplementary material are comprehensive and detailed. 

Weakness:

Major concerns: 

(1) The writing needs to be improved (not for the problem of English expression).  e.g.,  The last sentence of the first paragraph. What’s the importance and difficulty of the FSCIL?  The first sentence of the first contribution is completely uninformative and does not show their contribution at all. 

(2) In the second paragraph, "Besides, as shown in Figure 1 (a), there will be a misalignment between the adjusted classiﬁer and the ﬁxed features of old classes." How did you know the misalignment between the adjusted classiﬁer and the ﬁxed features of old classes?  and in Figure 1 (a), how did you know the old features and new classes are close?

(3) Apart from the fact that the motivation of neural collapse inspired framework is novel, the novelty of this paper still seems incremental. 
They only use Yang et al. method for the few-shot incremental learning.  Even the written in sec3.2 is similar to Yang et al. Could you please explain the technical innovation of your approach versus the Yang et al. approach other than in FSCIL?

(4) Since this paper pre-assigned the classifier at the beginning of all classes, even the tested classes,   but in real life, we cannot know how many classes will be tested during the test stage. So how does this method work when there is no way to know the number of classes to test?

(5) It would be better to separate the comparisons between memory-based and non-memory-based methods in Table 1,2.






 

clarity,_quality,_novelty_and_reproducibility: See the Strength and Weaknesses.
summary_of_the_review: I believe this paper is slightly below the acceptance threshold.


The paper presents a novel framework that inspired neural collapse for the FSCIL.  and quantitative experiments show that the proposed method outperforms baseline methods.   But, the overall novelty is incremental. (See the weakness.) 

But if the authors can explain well the weakness I raised, I will consider raising my score.


============== Post Rebuttal Update

Sorry for the late response. I thank the authors for their diligent efforts in the rebuttal. It was nice to see that the authors could give more detailed intuitions to improve their work and make relevant changes to their paper. Also, based on the other reviewer's comments and the responded of the authors, I have increased my score based on the author's responses.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper investigates the recent observation for neural collapse and how it can be combined with few-shot class incremental learning. The idea is to align a set of prototypes during neural collapse with prototypes required for few-shot learning, and like that benefits experimentally. The idea of neural collapse is obviously not new, however, the application of it to few-shot learning is new, and results are strong.
justification_for_why_not_higher_score: Combination of two fields, on how an interesting theoretical observation applies to a practical learning setting, is a reason why the paper should be in the spotlight.
justification_for_why_not_lower_score: The core theoretical idea is not new.
Decision: Accept: notable-top-25%