Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: The paper presents a new problem formulation in the broader area of lifelong learning.
Specifically, the Unsupervised Progressive Learning (UPL) problem, requires a learner to consume a stream of data, where each data point is associated with a class, but only very few labels are provided. Similar to continuous learning the learner can only access current and not access previous data in the stream. The paper clearly describes how this setup is different to commonly other learning setups studied, including continual learning. 

To solve this problem the paper proposes a deep-learning-free approach based on clustering and novelty detection.

Strength:
-	I think the problem formulation is very realistic and interesting and I am not aware of it being explored previously
-	The newly proposed architecture STAM is also interesting.
-	The paper evaluates STAM on UPL on three datasets, MNIST, EMNIST, SVHN
-	The paper ablates several aspects of the model evaluates the effect of hyper parameters.

Weaknesses:
1.	The paper misses to include baselines, both, w.r.t. the proposed method, as well as for the learning problem. 
1.1.	The paper argues that deep learning cannot work, I am wondering, why don’t include standard baselines, e.g. auto-encoder based with a prototype stored whenever a labeled example is seen. (For single pass continual learning, see e.g. [B], although that work is supervised)
1.2.	Hebbian learning inspired algorithms might similarly form a valuable baseline
2.	The UPL problem makes sense to me, however, the experimental setup could be improved. Specifically, the paper uses the entire dataset to find hyperparameters. This is highly concerning as this basically means the entire dataset has been seen multiple times before the final pass over the dataset (the one with the best hyperparameters) is performed. This is basically a contradiction to the setup of UPL, which assumes all data is seen only once.
3.	Related Work:
3.1.	[A] seems to study a similar setup and proposes an approach which should also be applicable in this work.







Conclusion:
The paper’s contribution is a new learning setup (UPL) and an approach (STAM). While it remains unclear if STAM can generalize to realistic images, the idea of UPL is very interesting and speaks for accepting the paper, although there are several weaknesses the authors should address.
It would also be great if the authors plan to release the exact experimental setup (i.e. which images are sampled in which order and which ones are labeled) so future work can compare to this work.



Reference:
[A] Continuous Online Sequence Learning with an Unsupervised Neural Network Model
Yuwei Cui, Subutai Ahmad and Jeff Hawkins; Neural Computation, 2016
[B] Gradient episodic memory for continual learning, D Lopez-Paz, MA Ranzato ; NeurIPS, 2017


==== Post rebuttal comment:
Thanks for the clarifications and additional experiments;
I do think the the paper should be accepted and increased my score.

Regarding 2) I was thinking more of using a small part of the dataset to do hyper parameter selection, see e.g. 
Efficient Lifelong Learning with A-GEM
A Chaudhry, MA Ranzato, M Rohrbach, M Elhoseiny
International Conference on Learning Representations (ICLR)



review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: This paper sets up a new problem based on a continuous stream of potentially partially labelled data, which the authors call the Unsupervised Progressive Learning problem. The paper also introduces a new model designed to approach this problem, called the STAM architecture, with many concepts applied in a novel way. The STAM architecture is tested on example problems from the UPL problem.

I find this paper to approach a problem that is not well studied in the literature, and it is well-written and easy to read. I am not aware of previous works that tackle this specific problem setup. Many previous works seem to slowly be converging to tackling this kind of problem, but this is the first work that directly tackles this realistic problem. It would be good if the authors made the benchmark (/how they are generated) public.

The STAM architecture is an interesting way of tackling image classification, and it is refreshing to see a technique not dependent on neural networks. The experiments in the paper are extremely detailed, with good figures and ablation studies.

I am recommending this paper be accepted. But I have one main question about this work: what is the memory cost of the STAM architecture? Many 1000s of LTM centroids are stored, what is the memory cost of this? Is this memory cost greater than the cost of just storing all the labelled inputs that have been seen? I would imagine that just replaying these stored labelled inputs (or just training a NN on these stored inputs) would provide extremely high accuracies for the datasets considered in this paper.

I understand that STAM has potential beyond just replaying memory for potentially larger datasets; the authors also make the point that they do not believe that the brain works by replaying memory. However, I would then like to see one (or both) of the following tests: a run with much fewer LTM centroids (with the corresponding memory cost detailed explicitly), and/or a dataset where it is clear that the memory cost incurred by the STAM architecture is a better use of memory than just storing previous data.

I would also argue that the brain *does* replay memory in some manner in order to learn. But that is a debate for another time!

By fixing the LTM centroids that are learnt on previous data, the method is essentially freezing previous knowledge and then learning new knowledge by increasing model capacity. It would be interesting (as future work / other papers) to see an adopted version of eg Progressive Neural Networks and how that compares on the same benchmarks.

I would also be interested to see, as future work, the uncertainty estimates of this new architecture and its robustness to adversarial examples.

Finally, a couple of minor points:
- Figure 4 is small and hard to read, particularly the left column.
- The authors find (in Appendix E) that increasing the \Delta hyperparameter leads to a growing number of LTM centroids. I wonder if that is still the case if \theta is also increased along with \Delta?
- Typos: last line of Section 3.2: 'lowst'; a few places where \citet{} and \citep{} are incorrect.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper proposes a method called Self-Taught Associative Memory (STAM) for Unsupervised Progressive Learning (UPL) , i.e., learning salient representation from streams of mostly unlabeled data with occasional class labels, where the number of class increases over time. The motivation of this paper is quite interesting in that the authors try to mimic how animals learn. The surrounding environments of animals are considered to be unlabeled, and animals gradually learn to distinguish between objects without explicit information. The model shed light on the problem of catastrophic forgetting by introducing dual-memory organization. To be specific, Short-Term Memory contains a set of centroids associated with the unlabeled data, whereas Long-Term Memory stores the prototypical centroids, which are frequently seen patterns. In addition, the model utilizes novelty detection technique to introduce new centroids to each layer of the model, and it prepares the newly created centroids to be associated with new classes. 
Overall, the paper reads well and it is self-explanatory with clear notations and hyperparameters. Each step of the architecture is well-formulated mathematically and the necessity of the step in the model is explained clearly. The problem proposed, Unsupervised Progressive Learning (UPL) problem, is novel.
On the other hand, there are shortcomings of the paper, one being the model evaluation on dataset such as MNIST, SVHN, and EMNIST. The dimensional size of the dataset and the number of classes are small which is not convincing to evaluate the performance of the model. In addition, each layer of the model is independent from other layers (no connections between units at different layers), keeping the updated representation of patches to itself. This fact would hinder the model from understanding complex representation. Furthermore, there is a concern on the use of L2 distance metric for the similarity between the patch and patterns (centroids). L2 is not meaningful in high dimensional spaces.
This paper is a good start in tackling the Unsupervised Progressive Learning problem, but some weaknesses are present in the nature of the model architecture as mentioned above. However, the approach taken appears to be ad hoc. It is difficult to imagine the method can work for more complex situations. Even for the small datasets used in the paper, the learning require 10’s of thousands of training example. This does not mimic animal learning at all. Also, the accuracy of the model on EMNIST with 47 classes is around 65% which makes me doubt the application of the model on real world.
-	Shed some light on the problem of catastrophic forgetting and continual learning without supervision
-	The model architecture is well defined, but has some weaknesses in the methodology
-	Interesting motivation of trying to mimic how animals learn in an environment

review_title: Official Blind Review #1

Decision: Reject