Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a method for class-incremental learning, i.e. learning a classification task while during training the model has no access to already used data and new samples can contain new classes. Dividing the classification network in feature extractor and classification head, the authors start with the substitution of the classification head by a 1-nearest-neighbour decision landscape (Voronoi diagram) and derive refined methods from there.  
The model is tested on the three established datasets CIFAR-100, TinyImageNet and ImageNet-Subset. 
Specifically, the authors develop four components that enhance their baseline model: parametrised normalisation, a divide-and-conquer strategy for iterative construction of decision landscapes, augmentation consensus and integration and multilayer compatibility. 
The authors test different combinations of these components to test their respective effects as well as conducting comparison to several benchmark models. The method outperforms all benchmark models in terms of common performance measures on the three datasets by far. 
strength_and_weaknesses: **Positive:**
- the paper is structured well and the ideas are presented in a intuitive way
- Illustrations support the main text well 
- Extensive citation of the literature
- Code will be made available 
- The method shows improvement compared to benchmark methods for CIL

**Negative:**

- (Major) The experimental results show strong two-digit improvement compared to the best baseline (cf. Tab. 1). I missed a section where the authors provide an intuition and reasoning why the method works so much better than everything that has been done before. There is neither a discussion nor a conclusion in “4 Discussion and Conclusion”, more a summary. 

- (Minor) It is easy to loose track of the different versions and reading flow suffers from looking up in the text what the different iVoro-x models actually are. 

**Questions:**
- It seems the baseline method already shows major improvements. You write *“We suspect that this is because the features generated by the frozen feature extractor can be satisfactorily separable by linear bisectors (Fig. 2). As we can see, the features for other methods are all dramatically changing during the phases, but those for iVoro are all fixed, making incremental VD construction possible. “*  Can you provide a line of argumentation why this could be beyond the above sentence? 
- 2.1 - C_t once is the dataset at time step t and once is the set of classes at phase t. If those are different letters I could not distinguish them. The sentence, that C_i are pairwise disjoint irritates because this is not clear. 
clarity,_quality,_novelty_and_reproducibility: **Clarity:**
The paper is structured well and the methods are introduced in a clear way. However, the paper suffers in my opinion from the nomenclature of the method itself. 

**Quality:**
Literature is quoted in a generous way and experiments are extensive. There is additional experiments and results in the appendix. 

**Reproducibility:**
The code will be made available, so the results should be reproducible. 


summary_of_the_review: The work at hand is well structured and written. The presented methods are to the best of my knowledge novel in the context of CIL. While the experiments and results stress superiority to other methods, the paper does not discuss its findings sufficiently. An attempt to explain why the method outperforms benchmark models to such a great extent would strengthen the work.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper constructs a Voronoi Diagram in an incremental manner to perform Data-free Class-Incremental Learning (CIL). The proposed method is shown to be a flexible, scalable and robust with theoretical insights and experiments, and it promotes the performance of CIL. 
strength_and_weaknesses: Strength:
- The paper is clearly written and easy to understand. 
- The paper deals with class incremental learning in a new perspective. 
- The paper investigates fundamental problem in machine learning.

Weakness:
- I am not quite sure how the method will perform when the distribution of class labels is (highly) skewed. 
- The proposed method and theory will work when separation between classes in the feature space is sufficiently clear. The authors mention this as a challenge in the introduction, but I am not quite sure how they alleviated this problem.  
clarity,_quality,_novelty_and_reproducibility: The text is clear and this work can be reproduced if one has sufficient knowledge in basic machine learning. 
summary_of_the_review: I am not an expert in this field so I may be wrong, but based on the papers of my expertise that I have read before, I think this is a good paper in general.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies class incremental learning and proposes several approaches based on Voronoi Diagrams called iVoro. In particular, the paper contributes four approaches: iVoro, iVoro-D, iVoro-AC/AI, and iVoro-L. The introduced approaches are validated on three datasets (CIFAR-100, Tiny ImageNet and ImageNet-Subset) reporting large improvements over previous non exemplar class incremental methods.
strength_and_weaknesses: Strengths:
Nice idea of using Voronoi Diagrams in class incremental learning
Rather extensive validation

Weaknesses (details see section below):
Clarity of presentation could be improved
Might be hard to reproduce the models from the current writing of the methodology section

clarity,_quality,_novelty_and_reproducibility: Clarity: 
The introduction is rather verbose and could be shortened and improved. It does a good job motivating the need of class incremental learning however it misses some important information, e.g.:

•	In general, the structure of the introduction is a bit confusing. Prior to getting to the contributions part, there is little information about the introduced iVoro models – adding a diagram/Figure of an over iVoro system would make it easier to understand. The details about the models are presented as a part of the contributions’ enumeration – which makes the contribution summary rather lengthy. In general, it might be worth to consider presenting one iVoro system in the introduction as the introduced approach and outline other three in the experimental section as baselines or ablations.

•	Overall, the reviewer finds the contributions rather unclear – consider rewriting the enumerate part of the intro.

•	It might be worth to add a note about the anticipated impact of the introduced model. The authors highlight the strong performance that is nice but adding a few comments on what these results/models change for the research community would strengthen the paper.

•	The authors study “… the challenging data-free CIL problem under the strictest memory and privacy constraints – no stored exemplars and fixed model capacity.” – form the current text it is unclear to the reviewer why this aspect of CIL is worth studying. Adding some motivation would improve the manuscript.

•	The captions of the figures should be self-explanatory and contain all the details required to understand the figures.

•	In Fig. 2 all the scenarios start with different placing of points in the plane – see the left most figure in all scenarios. It would be easier to do across methods comparison if all would start with the same point placings.

•	Methodology section is also hard to follow, it is math heavy and contrails little intuitions on why particular design decisions have been made. Rewriting the methodology part with a focus on presentation simplicity and reproducibility would strengthen the paper.

Quality:
Putting aside the presentation clarity part, the quality of the paper looks good overall. The paper has fair number of experiments and ablations. The reported results show large gains over prior art. Adding standard deviations to reported values would strengthen the quality of the presentation. 

Novelty:
The reviewer has not seen before the use of Voronoi Diagram in the context of DNN for class incremental learning, thus there seem to be some edge of novelty. Note, that the reviewer is not actively working on class incremental learning thus might be unaware of some prior art.

Reproducibility:
The authors motion they will release the code on github. However, based on current presentation of the methodology section it might be hard to code up the methods and reproduce the reported results. The reviewer would encourage the authors to rework the methodology section focusing on reproducibility ease.


summary_of_the_review: The introduced ideas are interesting and show good performance in the task of class incremental learning. The paper is well validated and contains significant number of ablations. Overall, the reviewer would lean towards acceptance, assuming that the authors could improve the presentation clarity with a focus on the introduction and methodology sections.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary:
This paper presents an algorithm for class-incremental learning utilizing Voronoi diagrams.

Strengths:
- The paper is well written with nice illustrations and a great state of the art section
- Novel perspective on class incremental learning
- Convincing experiments

Weaknesses:
- A discussion section with strengths and weaknesses would make the paper even stronger
justification_for_why_not_higher_score: While all reviewers are positive and recommend acceptance, they are not strongly confident, and most of them give weak accepts -- I do not feel comfortable giving the paper a higher score.
justification_for_why_not_lower_score: All reviewers recommend accept and I have no reason to go against their recommendation.
Decision: Accept: poster