Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper presents an "Unsupervised progressive learning" (UPL) problem, where a model is exposed to data in an non-iid manner, and each training example is presented once. Simple to continual learning, but a little more explicit in the connections to the way biological agents learn. They present a model that uses clustering and long-term memory (buffered) and compare on a few UPL tasks with additional supervision signal (classification) or unsupervised (clustering).

The paper is interesting. I appreciate the straightforward outline of the problem and connection to biological learning and some of the motivations for the model (e.g., long term buffered memory of centroids).

The model itself though isn't very clear in a few regards that I think are rather important.

1) for the architecture itself, are the cluster centroids propagated to the next layer, or is the same input used at the next layer? If so, I'm not sure what is hierarchal here, it's just ordered in terms of "patch dimension".

2) How large does the LTM get during training? Is there a maximum size? I didn't see any restrictions on the size of the LTM, so it seems this buffer could get more and more precise and eventually resemble just a normal buffer over remembered examples (with some small permutations depending on alpha.

3) During supervision the model has access to the number of classes?

Other questions:
Instead of a separate test-set, one could evaluate online, collecting the classification accuracies across learning before the learning step in the centroids, since each example is seem exactly once.

My main concerns though are on the size of the LTM and if this is really doing anything other than KNN with K=1 over a buffered memory of examples. I need to have some evidence that the LTM is doing something *general* w.r.t. those cluster centroids.
review_title: Review: Unsupervised Progressive Learning and the STAM Architecture

final_score: rating: 2: Strong rejection
review_content: In this paper, it is proposed to have a new problem setting of "unsupervised feature learning", which as claimed, is supposed to be different from standard continual learning settings. In this proposed setting, a data point can be seen only once by a model for training. I think, this is too strict a condition, as it is reasonable for a model to keep a data point in memory for a short while so as to use it for training across multiple epochs. In the experimental setup, neural baselines are trained only for a "single" epoch, in consideration of the proposed problem setting, which doesn't make sense for all practical purposes as neural models need a decent number of epochs for training. Furthermore, the final classifier used after learning the unsupervised representations is k-NN which is again unrealistic in the context of continual learning literature, especially in the context of neural baselines. All of this is justified for accommodating the  proposed problem setting which is itself not well motivated. 

The proposed model for the introduced problem setting is more of an engineering approach, relying upon some basic techniques such as clustering, novelty detection. There is no clear motivation for the learning algorithm, it is not clear how the model is optimized wholistically. It is more of a heuristic driven approach. The model is claimed to be brain-inspired; for instance there is a component in the proposed model which has a "hierarchy of increasing receptive fields", which is nothing but CNN-like neural net.  

For experiments, I have the following suggestions.
(1) In Fig. 3, x-label should be changed to something more appropriate, number of new classes introduced.
(2) Results are good for MNIST dataset primarily.
(3) Any other evaluation metrics besides accuracy? Accuracy can be misleading for multi-class scenario. 
(4) Report accuracies separately for new classes introduced and the old classes.
(5) It needs to be clarified exactly what "class boundary information" the baseline methods have access to.
(6) Only 10 new classes introduced from the 5 phases, what about the datasets with a large number of classes?
review_title: Inappropriate evaluation setup - more engineering than science

final_score: rating: 7: Good paper, accept
review_content: The authors propose and formulate a new problem setting UPL which addresses a number of limitations of current methods. I think this problem formulation is very important and relevant and the authors provide a fairly solid foundation about how to address this problem. 

There are a number of strengths to this work. I found the non-deep learning approach they took refreshing and their experimentation demonstrates the effectiveness of it in this setting. I found the evaluation relatively convincing. More specifically, I find the classification and clustering results good. Again, the classification results are difficult to interpret as you do use the labels (albeit not for learning the representation), but it becomes more like semi-supervised learning at this point. On the other hand, the clustering results do not have this problem and are more impressive. I appreciate the code being included in the supplementary material.

I have a number of questions/concerns about the work.
	1) How effective is this hierarchical receptive field processing approach? My feelings are that it may be insufficient for larger/more complex data. Evaluation involving these would be much welcomed. Further, how does it perform for non-image data types?
	2) The focus on classification is a little out of place, I can see why it's useful, but as a main section of the paper, preceding the purely unsupervised setting, it can be confusing to readers. It would be clearer in my opinion if the focus was on the unsupervised setup, and then the semi-supervised setup as an additional evaluation (i.e., not preceding the purely unsupervised approach as it does now). This is a criticism of the structure of the paper (and perhaps the main message), more than the work itself. 
	3) It seems like a limitation of this approach is that clustering algorithms that can be used on the representation are limited to discrete distance measures. Can the authors speculate on potential future limitations of this for the setting?
	4) "the small batch size required by UPL" I do not follow this - can you further explain?
	5) Further, while it may be unfair on CURL to compare directly, but as it is a very similar approach,  I do think it would be interesting to see it how it performs, while acknowledging the difficulties and reasons why. If not a full set of experiments, "We have experimented with CURL but we found that its performance collapses in the UPL setting" leaves me wondering what exactly the collapse is.

Overall, I think this is worthy of acceptance, acknowledging the fact  that it is early work in this new area.
review_title: Lays the foundation for a needed new problem setting for unsupervised progressing learning that acceptably addresses limitations of similar settings..

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors propose an approach (architecture + algorithms) to unsupervised progressive learning in a non-stationary environment (the number of classes grows gradually) by keeping centroids at several hierarchies, using a combination of techniques from online clustering, via computing and updating centroids, with novelty detection, and dropping (forgetting those deemed outliers).  A variety of experiments are performed on several image datasets (MNIST, EMNIST, SVHN, CIFAR-10) with comparisons to other adapted methods. They evaluate performance in a supervised setting where they describe how they learn centroid to label(s) mappings.



The experiments are somewhat promising,  and I liked the hierarchical aspect of the centroids.

But I have concerns about
applicability of the approach to practice: The approach has some of
the shortcomings of nearest neighbors: efficiency specially at
classification/test time (as the number of classes goes up to many
1000s, and required number of centroids per class goes up), choice of
distance, and so on.     What do you do if too many centroids?  space and
speed of finding closest centroids to a patch?  How is fast nearest
neighbor done?  Also, different classes require different complexities
(in terms of distance to use, manifold learning), and one parameter
for novelty detection and another for when to add a centroid to LTM,
may not be sufficient...

Regarding evaluation: Why the particular elaborate learning/classification scheme chosen (that computes
 a mapping from centroids to labels)?    the research or
motivation behind it? Why not a plain classifier, such as a simple nn,  svm, etc... given the features (centroids)
available? Perhaps because there is no explicit feature representation.    A quick discussion would be good.


The paper was overall clearly written, and the supplements provide much useful detail on the experiments.

Some  detailed comments:

section I, hierarchy: ruo_l x ruo_l, the subscript 'l' is not
explained  before introduced (later as layer). It would be good to
quickly give examples of the patch dimensions.

section 2.II:

what is t in C_l(t) (in footnote, they say they drop time index, so t
is probably time)

Does the online update take place only if (patch is) not deemed
outlier? The text seems to imply that, but it's not clear.  How do you
(re)use centroids that have been moved to long-term memory? It's not
clear from the text but it appears all centroids are used to compute
the nearest centroid (both in LTM and STM, ie C_l is the union).




review_title: The authors propose an approach (architecture + algorithms) to unsupervised progressive learning in a non-stationary environment (the number of classes grows gradually) by keeping centroids at several hierarchies, using a combination of techniques from online clustering, via computing and updating centroids, with novelty detection, and dropping (forgetting those deemed outliers). 

final_score: rating: 5: Marginally below acceptance threshold
review_content: ##########################################################################
Summary:

The paper introduces a non-parametric approach, STAM, for unsupervised progressive learning (UPL), a variant of continual unsupervised learning with a single-stream requirement. STAM is developed for visual tasks. It comprises several components: (1) online clustering of hierarchical visual features (2) novelty detection (3) dual-memory for prototypical features. Experiments show STAM performs better than GEM, MAS in specific scenarios. 

##########################################################################

Reasons for score: 

Currently, I vote for rejecting the paper. The paper is well written, easy to follow. My main concerns are (1) to what extent UPL is a real and practical problem, compared with unsupervised continual learning; (2) the comparisons with GEM, and MAS do not look fair; (3) each technical component is not new.

##########################################################################

Pros: 

The paper is well written, with enough details (and code) to reproduce the results;

The proposed method is technically sound;

Discussions on related work are comprehensive;

The experiments are thorough. Ablations are performed to justify the various design considerations.

##########################################################################

Cons:

I’m not sure if UPL is a real and practical problem. To me, the single-stream requirement looks somewhat unnecessary and is unfair to all parametric models (including deeply learned models), especially when the number of training iterations is small (e.g. on epoch on MNIST). On the other hand, as STAM relies on prototypical features from online clustering, essentially no parameters need to be learned. Thus, the comparisons between STAM and MAS/GEM presented in the paper are unfair as I believe the MAS/GEM models were highly undertrained in this case. What if STAM/MAS/GEM are all trained for two epochs, or trained on a larger dataset for one epoch? I guess the performance of MAS/GEM will be improved but I’m not sure if the performance of STAM can be improved.
 
STAM is only evaluated on small datasets (e.g. MNIST, SVHN, CIFAR10). I’m not sure if it is expressive enough to model larger datasets (e.g. CIFAR 100) with a reasonable memory footprint (e.g. comparable with a deeply learned model)



review_title: I'm not sure if the evaluations are fair

Meta-review: After reading the reviews, rebuttal, and looking through the paper I do feel that UPL setting is one that we need to consider. However is not clear to me that proposed approach matches the conditions described by the authors. In particular the scalability constraints seem important. I do feel that for the UPL setting makes sense particularly in the large case scenario of many examples and classes and how the system behaves under strict computational budgets for learning and inference. And I'm wondering whether in that limit parametric models would actually becomes relevant again, and whether there is a "burn-in" that one has to pay to use parametric models. 
That said I don't think current approaches (CURL, AGEM etc.) will do well even in that setting, partially because they were not necessarily thought for that. 
So in summary, I find the problem interesting, probably more so than the solution and particularly in an large scale setting.

However I think for the paper to have the impact it needs, and be ready for acceptance it needs a bit more. I think looking at a larger scale setting, and relying on that to motivate the problem will considerably help with its impact.  Also is not clear to me how the proposed solution scales (non-parametric approaches don't always do well in large scale settings), which I think is needed for it to be convincing. 
Decision: Reject