Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a brand-new few-shot class-incremental learning method called soft-subNetworks (SoftNet). The two main problems in few-shot class-incremental learning contains catastrophic forgetting and overfitting. The former one asks the new session training not to interfering the former session and the latter one asks only updating a few parameters irrelevant to previous tasks. Inspired by Regularized Lottery Ticket Hypothesis, the paper points out that the subnet can perform on-par or better than the whole network. Taking usage of this hypothesis, the authors split the network into a major subnetwork and a minor one. In the base classes session, soft-subnetwork parameters and weight score are learned. In the incremental learning session, minor parameters of the subnetwork are updated. The SoftNet surpasses the state-of-the-art baselines over datasets.
strength_and_weaknesses: Strengths:
1.	Borrowing idea from regularized lottery ticket hypothesis to incremental learning is a novel and ingenious idea for its both maintaining performance in subnetwork and solving overfitting.
2.	The paper is well written for its clear logic, figures, equations, and organization.
3.	The outstanding performance of SoftNet is convincing.
Weaknesses:
1.	The choice of top-c weights is handcraft without theory support. Just comparing results on overall accuracy is not enough. The authors can refer to the works in subnetwork to refine their analysis.
2.	The random choice of parameter follows uniform distribution, but condensation argues that there are always some weights much important than the others. The authors should explain why they use uniform distribution or the future improvement in this prior.
3.	Is there any possibility of extending SoftNet to the other kind of incremental learning? Or is the SoftNet limited to only class-incremental learning?

clarity,_quality,_novelty_and_reproducibility: see strength
summary_of_the_review: The paper outstands for its novel and the ingenious subnetwork training strategy. Also, the surpassing experiment results on benchmarks convince its idea. However, the theoretical analysis is quite week so that the readers cannot get the clear knowledge of the mechanism of subnetwork. The theoretical analysis should include how the split of major and minor help the forgetting and overfitting. Overall, the advantage outweighs its disadvantage so it can be recommended to the community.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Inspired by Regularized Lottery Ticket Hypothesis, which hypothesizes that smooth subnetworks exist within a dense network, this paper propose Soft-SubNetworks (SoftNet), an incremental learning strategy that preserves the learned class knowledge and learns the newer ones. The SoftNet jointly learns the model weights and adaptive soft masks to minimize catastrophic forgetting and to avoid overfitting novel few samples in Few Shot Class-Incremental Learning. Experiments are conducted on the CIFAR-100 and miniImageNet datasets under various settings.
strength_and_weaknesses: Strengths:

S1) Few-shot Class Incremental Learning is an important and active topic in the community, which is addressed in this work.

S2) The paper is generally well written and easy to follow.

S3) The idea of jointly learns the model weights and adaptive soft masks to minimize catastrophic forgetting and to avoid overfitting novel few samples is interesting and makes sense. 

S4) The method significantly surpasses many State-Of-The-Art methods under various variants on the benchmarks.

Weakness:

W1) Some of the details are not fully discussed. For example in section 3.3, why the update scheme of $\theta$ can effectively regularize the weights of the subnetworks for incremental learning.

W2) Though the effectiveness of SoftNet is verified via extensive experiments. These experiments are conducted only on two datasets (CIFAR-100 and miniImageNet), which seem not adequate enough to show the robustness of the propsed method. Results on other popular benchmark datasets such as CUB200 may add to the convincingness of the SoftNet.
clarity,_quality,_novelty_and_reproducibility: The work is well written and easy to follow. The idea is quite interesting and the methodology is in general clarified. The authors did not mention if they would release the source project on public.
summary_of_the_review: The paper tackles an important problem, proposing an incremental learning strategy that preserves the learned class knowledge and learns the newer ones.  Experiments are comprehensive and seem to show improved performance in two common benchmarks. 

I have carefully reviewed all the comments and responses. Though a reviewer show concerns about the inadequate comparison with SOTAs, I still consider that this paper provides an interesting and effective idea. I would like to keep my original rating.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a Regularized Lottery Ticket Hypothesis inspired network to deal with the well-known catastrophic forgetting problem. By only updating partial weights in the network, the proposed method claims to have a good performance on both the classes in previous and current sessions.
strength_and_weaknesses: Strengths: 

1). The paper is well present. In particular, Fig 1 and Algorithm 1 are clear and informative. 

2). The method is simple and easy to follow.

Weaknesses:

1). From the comparison experiments, it is hard to distinguish whether the proposed method has a good performance since there are plenty of works for few-shot class-incremental learning that the paper fails to have a comparison with. 

- Few-shot incremental learning with continually evolved classifiers. (CVPR ‘21)
- Few-shot class incremental learning by sampling multi-phase tasks. (TPAMI)
- Subspace regularizers for few-shot class incremental learning. (ICLR ‘22)
- Metafscil: A meta-learning approach for few-shot class incremental learning. (CVPR ‘22)
- Constrained few-shot class-incremental learning. (CVPR ‘22)
- Few-shot class incremental learning via entropy-regularized data-free replay. (ECCV ‘22)
- Few-shot class-incremental learning from an open-set perspective. (ECCV ‘22)

As an ICLR 23 submission, most of the baseline methods in this paper are from 2 years ago. An important baseline in FSCIL, CEC (Zhang et al., CVPR 2021), is even not compared with. I want to note that the above-mentioned methods seem to outperform the proposed method in this paper by a large margin. 

2). The proposed method seems to have a better result than F2M in the final session, but with a much better results in the session 1, which may indicate that the proposed method cannot prevent the network from forgetting, which is opposite from what is claimed in the introduction.

3). The proposed method sounds not that new to me. I found the authors fail to give a reference for AANet (Adaptive Aggregation Networks for Class-Incremental Learning, CVPR ‘21). However, from my own perspective, the proposed method shares a similar motivation with AANet since both want to alleviate the catastrophic forgetting by fixing some parameters inside the network. Though CIL and FSCIL are different benchmarks, AANet seems to perform better and have a capability of preventing catastrophic forgetting. The authors should discuss the comparison with AANet in details. 


-------After rebuttal-----

The authors offer detailed response and clarification. I am happy to increase the score, but I sitll have the concern about the similar motivation to AANet. 

clarity,_quality,_novelty_and_reproducibility: The clarity of this work is good, but the novelty and experimental performance are limited, which impedes the quality of this paper.  Details are stated in the Weaknesses section. 
summary_of_the_review: Despite good paper writing, the novelty and the experimental performance are the largest issues of this paper. Most of the baseline methods are from 2 years ago. An important baseline in FSCIL, CEC (Zhang et al., CVPR 2021), is even not compared with. 

See weaknesses for details.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a few-shot incremental learning approach, referred to as Soft-SubNetworks (SoftNet). The approach is inspired by 
the Regularized Lottery Ticket Hypothesis, which states that competitive smooth (non-binary) subnetworks exist within a dense network, 


Strengths
------------

- Reviewer 5Go7, Reviewer ZoLy: Borrowing idea from regularized lottery ticket hypothesis to incremental learning is a novel and ingenious idea for its both maintaining performance in subnetwork and solving overfitting 
- Reviewer 5Go7, Reviewer ZoLy, Reviewer 5XQs: The paper is well written its clear logic, figures, equations, and organization. 
- Reviewer 5Go7, Reviewer ZoLy: The outstanding performance of SoftNet is convincing
- Reviewer 5XQs: The method is simple and easy to follow.


Weaknesses
---------------

- Reviewer ZoLy, Reviewer 5Go7: Some of the details are not fully discussed. For example in section 3.3, why the update scheme or can effectively regularize the weights of the subnetworks for incremental learning.
- Reviewer ZoLy: Though the effectiveness of SoftNet is verified via extensive experiments. These experiments are conducted only on two datasets (CIFAR-100 and miniImageNet), which seem not adequate enough to show the robustness of the propsed method. Results on other popular benchmark datasets, such as CUB200, may add to the convincingness of the SoftNet.
-Lack of related work coverage and discussion  (e.g., below by   Reviewer 5XQs)
Few-shot incremental learning with continually evolved classifiers. (CVPR ‘21)
Few-shot class incremental learning by sampling multi-phase tasks. (TPAMI)
Subspace regularizers for few-shot class incremental learning. (ICLR ‘22)
Metafscil: A meta-learning approach for few-shot class incremental learning. (CVPR ‘22)
Constrained few-shot class-incremental learning. (CVPR ‘22)
Few-shot class incremental learning via entropy-regularized data-free replay. (ECCV ‘22)
Few-shot class-incremental learning from an open-set perspective. (ECCV ‘22)


In conclusion, two reviewers recommended acceptance, and one recommended weak rejection (less negative after rebuttal). . On balance, the AC finds the paper could be accepted after the improvement in the rebuttal. However, the authors should attend to the main points in the review when preparing a final version. ,

justification_for_why_not_higher_score: Two reviewers recommended acceptance, and one recommended weak rejection (less negative after rebuttal). Unfortunately, we did not reach a consensus. The paper can benefit from implementing the feedback from the reviewers especially related work coverage. The authors partially address these concerns in the rebuttal. 
justification_for_why_not_lower_score: Two reviewers recommended acceptances, and one recommended week rejection. Despite that, there is no consensus, the reviewers agree on the significance of the results. On balance, the paper will be a valuable addition to the ICLR program.
Decision: Accept: poster