Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: This paper proposes a new procedure for continual supervised learning from a non-IID data stream that assumes ability to maintain some of the stream examples in a buffer and use the buffer to improve updates of a prediction model. The proposed procedure is called Continual Prototype Evolution (CoPE), which controls evolution of prototypes to prevent catastrophic forgetting. 
Strenghts:
- the paper presents very detailed experimental results
Weaknesses:
- the paper is not easy to read
- the underlying assumptions about the data stream are not clearly defined. Thus, it remains unclear what problem the proposed algorithm is trying to solve.
- the proposed algorithm is a collection of heuristics that are not clearly justified. There is no attempt to provide a theoretical insight about the behavior of the algorithm.
- the experiments were performed for one particular synthetic setup on one benchmark data set and the proposed algorithm is compared to a very limited class of baselines. Thus, even the empirical evaluation is not very insightful.
- there is no discussion about the computational cost 
Overall rating:
This paper has too many weaknesses and is not ready for publication

review_title: A new heuristic for online learning from non-IID data

final_score: rating: 7: Good paper, accept
review_content: This paper covers an interesting topic of continual learning of the stream of data. One limitation of the existing classification algorithms is their close-set assumption. In close-set methods, a predefined set of classes are considered and a model is trained on the available data from these classes, based on the assumption that test data will be driven from a similar distribution as the training data. However, most of the real-world problems are open-set problems. Open-set models should be able to learn continuously in an online manner with minimum or zero supervision. In other words, they should be able to learn new classes or update the existing classes based on the received new data on-the-fly, without forgetting the previously learned knowledge.  
Pros: 
In this paper, the authors provide an incremental learning approach that prevents catastrophic forgetting. 
Their approach can work on both balanced and unbalanced data.  
cons: 
The authors need to improve the presentation of the manuscript by providing more explanation. It could be confusing for readers who are not familiar with the topic. 
It would be very helpful to publically share the code. 
I highly recommend adding the confusion matrix or F1 score in addition to the accuracies. 

review_title: Recommendation to accept 

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Summary:

This article introduces a learner-evaluator framework that incorporates the different variations of problems related to incremental learning. It also proposes a method called Continual Prototype Evolution for dealing with the most general version of the problem, incremental learning on data streams, in which the learning task is not specified. The paper presents an extensive amount of experiments indicating that in this scenario, the proposed method improves significantly on existing approaches in terms of accuracy and memory efficiency. The article is well organized, easy to read and understand.

Positive aspects:

- It presents an adequate coverage of the literature on continual learning in section 3 and an interesting framework organizing the area in section 2.
- The method does not need information about the task being learned, being more applicable to real-world scenarios.
- An extensive experimental evaluation of the proposed method and comparisons with related methods is provided.
- Ablation studies indicate the most important aspects of the proposed model. 

Points to discuss/improve:

- Momentum parameter: It took me a while to understand why the authors use a “high momentum”. While in gradient descent the momentum creates a tendency to keep the parameter changing in the previous directions of motion, here the momentum is supposed to make it change more slowly. There is another way of formulating Eq. 1, in which alpha works as a learning rate. Defining alpha = (1-alpha), a slow learning rate instead of high momentum, we can have Eq. 1 as: p^c = p^c + alpha(p^c - \bar{p}^c). This is easier to understand in my opinion, but authors can choose to disregard this if they prefer the current form.

- The method assumes that one prototype for each class is enough. However, in certain problems, a class might need to be represented by different prototypes, indicating the different ways of being from the same class. How the model would handle these situations?

- The conclusion does not discuss adequately the main findings of the article, probably due to lack of space.

- Page 15 is completely black in most PDF viewers I tried. Only Chrome was able to display it correctly.

Conclusion:

I believe this article should be accepted as a see it presents interesting findings in the area of incremental learning on non-stationary data streams.
review_title: An interesting framework and model for learning on non-stationary data streams

Meta-review: This paper presents an interesting idea for task-free incremental learning on the data stream. The reviewers have extensive discussions after reading all the reviews and the author's rebuttal. There are concerns raised about the presentation of the method and the justification for some parts of the model design choices. The reviewers believe that after addressing these weaknesses the work can be made stronger and may be accepted in a competitive venue.  
Decision: Reject