Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a zero-shot incremental learning approach that doesn't store past examples or metadata for experience reply. It synthesizes past experiences from the network parameters through a memory recovery paradigm.  The proposed method doesn't rely on external memory or parallel networks and achieves very competitive results on Task-IL and Class-IL tasks. 
main_review: Strengths
- It is a cool idea to build a continual learning method that doesn't grow the model size over time or rely on external memory.  The proposed method distills information from the past networks to form a synthetic set of past concepts. Although the example synthesis method is largely based on the Zero-Shot Knowledge Distillation (Nayak et al. 2019), embedding this technique into a continual learning framework is very reasonable and novel. 
- The paper is well written and the structure is clear.  The clarity of `Modeling the Network Output Space` can be improved though as it's unclear how exactly to synthesize images from the past weights. 

Cons
- "our method needs no fixed-size memory" - This might be inaccurate as the model still needs to store the network from previous tasks (or the teacher network). It doesn't need external memory to store past examples or meta data. 
- Missing ablation study on example synthesis.  Is image synthesis necessary? Can you only synthesize features?  How fast is the synthesis procedure? It seems the computation is still an issue since this approach requires synthesizing images first and then training on the new tasks.  It might make sense to compare with existing approaches on computation, training time, etc. 
- Missing ablation study on the hyperparameters. How are the temperature, number of synthetic samples,  lambda is chosen? 
- Missing baselines in FS-IL and ZS-IL.  Table 1 reports the results in FS and ZS settings but didn't refer to previous methods for comparison. This Github link maintains recent works in this line. https://github.com/zhoudw-zdw/Awesome-Few-Shot-Class-Incremental-Learning
- Missing comparison with the synthesis approaches with metadata.  How well does the method perform in terms of image realism? For example, ACE uses the mean and std of the past data to synthesize images. (https://openaccess.thecvf.com/content_ICCV_2019/html/Wu_ACE_Adapting_to_Changing_Environments_for_Semantic_Segmentation_ICCV_2019_paper.html)
- It might be better to try the approach on more benchmarks like CIFAR-100,  CUB, iNaturalist, etc.






summary_of_the_review: This paper provides a very interesting and novel idea for continual learning. However, the empirical evaluation and ablation study seem to be weak. 

=== Post-rebuttal Comments===
The authors addressed most of my concerns in the feedback.  I kept my score and leaned towards acceptance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In order to achieve exemplar-free incremental learning, the authors introduce a memory recovery paradigm to synthesize past exemplars. They also propose to evaluate incremental learning based on the zero-shot learning setting. Extensive experiment results are provided to show the effectiveness of the proposed method. 
main_review: ### Strengths

&nbsp;

- This paper is well-organized and easy to follow.
- The proposed method is technically sound. 
- Extensive experiment results are provided. 

&nbsp;

### Weaknesses

&nbsp;

- ***Replaying synthesised samples for old classes is not a novel idea.*** It has been proposed in [A]. In my view, the proposed memory recovery paradigm is very similar to the “data-free generative replay” strategy in [A].

&nbsp;

- ***The authors only provide experiment results on small-scale datasets, e.g., CIFAR-10 and Tiny-ImageNet.*** However, we usually use larger datasets to evaluate incremental learning methods. For example, iCaRL (Rebuffi et al., 2017), BiC (Wu et al., 2019), Mnemonics (Liu et al., 2020) run experiments on CIFAR-100 and ImageNet-1k in their original papers. I don’t see why the authors don’t follow the previous work and choose small-scale datasets.

&nbsp;

- ***Updating synthesised samples requires additional computational costs.*** The authors should compare the training time of their method with related work. If their method requires much additional training time, it is not practical. 

&nbsp;

- ***The authors reduced the margin, which might practically violate the length limit.*** For example, the margin in Section 6 is significantly different from the ICLR template. 

&nbsp;

- ***There are some minor formatting issues in this paper,*** for example, in the last paragraph on Page 4. The authors should carefully check the paper and make sure there is no such issue in the revision. 

&nbsp;

- ***It would be better to provide a reproducibility statement in the paper.*** The authors provide a GitHub link, but I cannot see the code using that link. 

&nbsp;

### Reference
*[A] Choi, Yoojin, Mostafa El-Khamy, and Jungwon Lee. "Dual-Teacher Class-Incremental Learning With Data-Free Generative Replay." CVPR 2021 Workshops.*
summary_of_the_review: Overall, I think this paper proposes an interesting method and provides extensive experimental results. So I tend to accept this paper. However, similar ideas have appeared in the existing work, such as [A]. Many details of this paper are not polished. So the overall score is “borderline accept.”

[A] Choi, Yoojin, Mostafa El-Khamy, and Jungwon Lee. "Dual-Teacher Class-Incremental Learning With Data-Free Generative Replay." CVPR 2021 Workshops.

&nbsp;

### === Post-rebuttal Comments===
The authors addressed most of my concerns in the feedback. They also provided the additional experimental results I asked for. I recommend acceptance. 

The authors should include the additional results and corresponding analyses in the next revision.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to generate pseudo data using the gradient-based method for class- and task-incremental learning. Experimental results show the effectiveness of the proposed method.
main_review: - The proposed method takes an advantage of generative model-based approach without an explicit generative model.

- Experimental results looks strong.

- Discussion on memory replay methods is incomplete or misleading. Recent works (roughly, those published in 2019 or later) proposed to solve the bias problem, while the statement in this paper sounds the solution is not investigated. There are some early works addressing the bias problem but missing in this work: [Castro et al.], [Javed et al.], [Lee et al.].

- The proposed method is not really zero-shot "learning"; In zero-shot learning, your model has never seen zero-shot classes but inferred them from auxiliary information.  It seems they argue so because their method has no extra memory or generative model, but I believe zero-shot learning is not the term for this case, thus misleading. The experimenal setting of LwF (though later works added extra memory to keep previously seen data in their experiments for fair comparison) or the usage of unlabeled data in [Lee et al.] (again, though they also added extra memory for previously seen data for fair comparison) also do not require an extra memory or generative model, but they have never claimed that their method is zero-shot (incremental) learning. I found that the zero-shot knowledge distillation work is cited, but their work is about zero-shot "knowledge distillation," not zero-shot learning. I found that this paper did not cite any zero-shot learning works; I recommend [Xian et al.] to read.

- More ablation study is required to convince the experimenal results: for example, the choice of the prior distribution for the recommender vector (even one-hot + Gaussian noise + normalization could be a simple baseline), the learning schedule for the memory recovery step (by the way, you should also report how long this step takes, otherwise your method has an unfair advantage on the training time than other compared methods), the size of the synthesized dataset, and so on.

- In Table 2, I recommend to put the name of each types, either in the caption or by adding a column; otherwise it is hard to figure out how they are grouped, unless you fully memorize the related work section and the abbreviation of all prior works.

[Castro et al.] End-to-end incremental learning. In ECCV, 2018.\
[Javed et al.] Revisiting Distillation and Incremental Classifier Learning. In ACCV, 2018.\
[Lee et al.] Overcoming Catastrophic Forgetting With Unlabeled Data in the Wild. In ICCV, 2019.\
[Xian et al.] Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly. TPAMI, 2018.

## Post rebuttal

As the authors addressed most of my concerns, I change my recommendation to 6: marginally above the acceptance threshold.

summary_of_the_review: The method is simple yet effective, which is good. However, "zero-shot" sounds misleading, and ablation study is not enough.
review_title: null

Meta-review: This paper presents a zero-shot incremental learning approach that does not store past samples for experience replay. The idea is novel and well motivated, and the paper is well written. Reviewers' comments were mainly about missing baselines, missing ablation studies, and  clarifications about the proposed method. In the revised paper, the authors provided more justifications and added new experimental results on large benchmark datasets as well as ablation studies. After discussion, all the reviewers are positive about this submission. 

Thus, I recommend to accept this paper. I encourage the authors to take the review feedback into account in the final version.
Decision: Accept (Spotlight)