Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors approach the solution to the problem of meta-learning as simply running a recurrent network through various tasks and back-propagating to train the RNN to adapt to new tasks (as in some previous works). The main advance of this work is proposing an architecture - a Deep Convolutional LSTM (with few architectural details) and showing that it works very well. The simplicity of this advance is a plus. The authors should change initial writing and clarify what they mean by distributed memory (see below in more detail) but otherwise the paper is easy to read. A good number of experiments is conducted, though it would be good if it was tested in harder tasks.
main_review: There are the basic cons of this work: The first one should be easy to fix: The description from the start with very mysterious - talking about distributed memory, makes a reader wonder what kind of new paradigm have they devised, just to later find out it is just a different RNN architecture. Taking about this as “memory” is misleading because while the RNN activations do store information, often times one considers weights of RNN to be the memory, and learning being the adaptation of the weights - which is not what they are meta-learning (the weights update has a fixed algorithm - back-propagation). The authors should make it clear from the beginning what they mean by memory (=activations of an RNN) and what they mean by distributed (=across layers in deep RNN). Once the reader knows that, the paper is well written and easy to read.

The second issue is that the tasks they use are have quite a short time spans - I am reluctant to call this meta-learning. Nevertheless, these types of problems have been studied before, and this paper makes a nice contribution with a number of experiments. However it is not clear how well would this learning to adapt the activations work in problems with much larger time scales.

Details: 

- I would say in the abstract what the basic idea is: e.g. something like “We device a variant of deep convolutional LSTM architecture, the feature of which is a large capacity state of activations, and use standard end-to-end training by back-propagating through sequence of tasks, …”

I would even ideally change a title - distributed memory can mean many things, like adapting weights of a networks and neither I feel like this should even be referred to as meta-learning (due to shot time scale nature of the problems).

- This kind of architecture is indeed quite capable - very similar one was used in RL: “An Investigation of model-free planning”, Guez at al., - you could add the reference. 

- 134-142 - May be add here some of you previous relevant references of works that do this (train RNN end-to-end) - end of the second last sentence.

- 202: You mention pre-training but what it is is only mentioned in the sentence after the next one.

- 243: In 240 you said that task has 5 samples, but here you are changing to 5-20 - does it have more examples or the same 5 are repeated more. If the former, shouldn’t the performance of the model, Figure 3, theoretically increase since you see more examples of a class?

- 247: Meaning that every query example is started with the same state? (That after training set of tasks)

- Figure 4: I don’t understand what is on the x axis. I thought we have a sequence of tasks, and then test the model on the query set containing samples from all the subtasks.

- Section 4.5: How do you do the training here? Do you sample examples from training set at random and treat them as a long sequence, back-propagating every n steps, or how is this done? What is the architecture of CL-ResNet-20 say? Resnet followed by convLSTM followed by LSTM or something else?

I would be happy to raise my score if the issues are addressed and there are no major other issues I have missed.
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper, the authors propose a novel network architecture for online few-shot learning. They propose to use an LTSM framework and store the previous information in the hidden states of different layers. They also evaluate their methods on other tasks, such as online continual learning, online few-shot semantic segmentation, and standard supervised learning. 
main_review: - ***This paper is poorly organized and very hard to read.*** I have been working on few-shot learning for several years and have read lots of papers in this field. However, after reading the abstract and introduction, I totally don’t understand what they want to do. For example, many concepts occur before they are defined. In Line 2, the authors mention “the persistent state”. What is it? It is not defined. In Line 134, the authors also use “h”. I cannot find the definition for h before this paragraph. Is that a value, a vector, or a matrix? 

- ***The motivation for using memory-based methods is unclear.*** I think the authors try to motivate their method in Lines 30-36. However, I have lots of questions about this motivation, e.g., (1) what are the definitions of generality and flexibility in few-shot learning, and (2) why using a straightforward loss formulation and standard optimization techniques is better? After reading the introduction, I still don’t know what is the main challenge in this paper and how the authors address it.

- ***The claim “their method outperforms gradient-based, prototype-based, and other memory-based meta-learning strategies” seems incorrect.*** Most of the meta-learning methods, such as MAML and ProtoNets, evaluate the methods in the standard few-shot classification setting. However, the authors evaluate their methods in the online few-shot learning setting. Besides, the comparing methods (LSTM, NTM, APL, OPN, and CPM) don’t contain a gradient-based method. 

- ***The experiments are conducted on very small datasets.*** The authors use CIFAR-FS and Omniglot, which are two small datasets. In standard few-shot classification, we usually use miniImageNet and tieredImageNet. In [18], they use RoamingImageNet, which is based on tieredImageNet. So it is not reasonable to run experiments on such small datasets. 

- ***The method section is too simple.*** The authors only use one page for the method section. Many details about the implementation are unclear. 

limitations_and_societal_impact: The authors have included the limitations and potential negative societal impact.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes to adapt Meta RNNs (memory-based meta-learning) by instantiating multiple layers of LSTMs where some of them are convolutional.
The authors refer to this as distributed memory due to each layer having its own LSTM-based memory.
They demonstrate good performance in few-shot learning and continual learning.
main_review: ## Originality and Significance

The authors suggest that instead of using a single LSTM in MetaRNNs (memory-based meta-learning) multiple layers should be added and some should be convolutional.
While this seems like a fairly incremental change, they demonstrate convincingly that this helps in few-shot learning and continual learning.
In particular, the improvement / similar performance over ANML and OML is interesting as no inner gradients are required.

## MetaRNNs (memory-based meta-learning) not correctly cited

The proposed architecture is very similar to Meta RNNs (supervised [1], RL [2,3]) that the authors do not cite. They only discuss other forms of external memory in the related work section. The author's contribution of adding multiple layers of LSTMs and making some of them convolutional should be highlighted more clearly. This contribution tends to get lost in the paper over discussions that also apply to previous work.

[1] Hochreiter, S., Younger, A. S., & Conwell, P. R. (2001). Learning to learn using gradient descent. International Conference on Artificial Neural Networks.

[2] Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., & Abbeel, P. (2016). RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning. ArXiv Preprint ArXiv:1611.02779.

[3] Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016). Learning to reinforcement learn. ArXiv Preprint ArXiv:1611.05763.

## Connections to other recent work on distributed memory / fast weights should be added

Recent work [4] also introduced a form of distributed memory for meta-learning online adaptation strategies / learning algorithms. In that case, each weight in a neural network is replaced by an LSTM, which is a different but related architectural choice compared to the proposed work. The connection should be discussed and cited.

[4] Kirsch, L., & Schmidhuber, J. (2020). Meta Learning Backpropagation And Improving It. ArXiv Preprint ArXiv:2012.14905.

## Conclusion

Overall I think it is a good paper with interesting results on meta-learning with distributed memory. Unfortunately, it is missing some crucial relationships to previous work. I would like to give the paper a good score based on appropriate adjustments in the rebuttal.
limitations_and_societal_impact: Limitations and potential negative societal impact is adequately discussed.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper the authors propose a distributed memory system to tackle the online few-shot setting. The system is based on layers of conv-LSTMs that take as input an image and the labels at previous steps to generate activations and store information. 

Overall I think that the technical contribution is marginal. Both the system structure and the problem formulation make the method impractical and difficult to scale to more complex settings. Experiments are limited to small datasets and do not show the potential of the proposed solution. Limitations are not appropriately discussed and a better framing w.r.t. more recent online few-shot learning work is missing. In the current form the paper is not ready to be accepted. I am open to change my mind and increase my score if the authors provide a strong rebuttal and a satisfactory answer to the concerns I have outlined below.
main_review: 1) Clarification on the importance of having a memory. I have checked algorithm 1 in the appendix and the details of how task are sampled and aggregated in appendix A.1. The authors wrote: "we sample 10-shot 5-way online few-shot tasks by sampling 5 classes and 10 samples per class (a total of 50 images), which are fed to the model at a rate of one image per time step". My understanding is that those 50 images are shuffled and presented in a random order. I am wondering if there are simpler ways to manage this setting, for instance by just accumulating the classes seen so far into bins and then pass the histogram instead of the one-hot label. This could be done by simply summing the one-hot vectors seen so far (and optionally normalize by the total number of images in the sequence). One could have a standard neural network that makes a prediction by using the image at time t and the accumulated histogram. If I am correct in my assumptions, then this seems to be a trivial baseline to try in order to justify the use of a memory. In other words, why we need a complex memory mechanism if the only information that matters is the current image and the labels accumulated so far in the sequence? The authors should clarify this point and provide empirical evidences that this trivial baseline is not as competitive as the proposed method.

2) Problem setting. The particular formulation used in this work has been used previously, for instance in [1], and it is based on the key assumption that labels are delayed by a given factor. However, this formulation is just one of the possible ways meta-learning can be applied to the online setting, and it may suffer several shortcomings when used in practice. For instance, in many real-world settings the feedback can be delayed by a variable factor, it can change between train and evaluation stages, and even be overwritten by inputs belonging to different classes. Recent work has investigated online few-shot learning in some of those challenging settings but the authors did not mention it or discuss it. I suggest including these papers [2,3,4] with an explanation of the advantages/disadvantages of the formulation used by the authors.

3) Computational cost. A satisfactory discussion of computational costs and memory issues has not been provided. The system seems quite expensive in terms of computational resources. The authors pointed out in lines 154-157 some stability issues that they assume are due to: "the same network being repeated 50 times". If backpropagation is performed from the last step to the first one, it will require the storage of all the activations along the way. This is a serious limitation, the model may not scale well to large images and deeper backbones due to this bottleneck. A discussion of all these issues is expected.

4) Datasets. The datasets used for the evaluation of the proposed method are Omniglot and CIFAR-FS. I think that Omniglot is starting to be outdated and has been replaced by more sophisticated benchmarks in recent publications (e.g. Meta-Dataset [5], SlimImageNet [2]). CIFAR-FS has also some serious limitations (limited diversity, small resolution, etc) when compared to recent benchmarks. The authors are encouraged to move towards more recent benchmarks if they want to fully showcase the potential of the proposed solution.


References
----------

[1] Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016). One-shot learning with memory-augmented neural networks. arXiv preprint arXiv:1605.06065.

[2] Antoniou, A., Patacchiola, M., Ochal, M., & Storkey, A. (2020). Defining benchmarks for continual few-shot learning. arXiv preprint arXiv:2004.11967.

[3] Caccia, M., Rodriguez, P., Ostapenko, O., Normandin, F., Lin, M., Caccia, L., ... & Charlin, L. (2020). Online fast adaptation and knowledge accumulation: a new approach to continual learning. arXiv preprint arXiv:2003.05856.

[4] Ren, M., Iuzzolino, M. L., Mozer, M. C., & Zemel, R. S. (2020). Wandering within a world: Online contextualized few-shot learning. arXiv preprint arXiv:2007.04546.

[5] Triantafillou, E., Zhu, T., Dumoulin, V., Lamblin, P., Evci, U., Xu, K., ... & Larochelle, H. (2019). Meta-dataset: A dataset of datasets for learning to learn from few examples. arXiv preprint arXiv:1903.03096.
limitations_and_societal_impact: Yes
review_title: null

Meta-review: The paper operates in a simple approach to meta-learning, of running a recurrent network through tasks and back-propagating to learn the adaptation. They find that simple deep-convolutional lstm networks (with details) outperforms many of the previous architectures studies before on standard benchmarks and interestingly also works in classic supervised setting at state of the art level (as a single system). 

On the positive side, the method is simple and the experiments are done carefully. The major drawback is that it is not being tested on harder problems and more modern online learning settings. Pushing the system this would we provide a major improvement to the paper. Despite this, the currently results are still sufficiently interesting. The are few points in terms of writing the paper that should be improved and I hope the authors will address that for the final version. 
Decision: Accept (Poster)