Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes to meta-learn the backpropagations by representing weights of a network by small RNNs. Different RNNs share the weights, and the weights can be thought of as representing the states of RNN. The approach can thus implement back-propagation solely in the forward dynamics of an RNN. 
main_review: I like the general idea of the paper but have a few concerns regarding the approach and its motivation - 

1. The paper uses evolutionary algorithms to train their model. But evolutionary algorithms can also be used to directly learn the weights of a neural network, and it has been shown to achieve nice results.[1] What's the advantage of using VSML as an alternative to back-propagation?

2. For meta-learning can the authors show results over Omniglot dataset, which is considered a more standard dataset for meta-learning.

3. Since VSML is computationally expensive, what's the advantage of using it instead of back-propagation? 

[1] Designing neural networks through neuroevolution. Stanley et al.
limitations_and_societal_impact: No, the authors haven't addressed it. Though I don't feel that the paper would have any negative societal impact.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The authors in this paper propose to learn learning algorithms (LAs) like backpropagation using an RNN/LSTM to replace all the weights of a network. Subsequently, they use LA on a different dataset to show that the LA has generalized.
main_review: The authors model each weight with an LSTM's state. The forward and backward passes of the NN are coded into the LSTM's state updates. The work is novel and is an attempt to find algorithms that can replace backpropagation for training NN. The authors show that the learned backpropagation can learn on other datasets. They also show that the learning is faster and can learn with lesser examples.
limitations_and_societal_impact: The paper is hard to read in many places. For, e.g., in eq-3, the sudden use of $s_{cai}$ without defining the term kept me guessing. Also, it is unclear how the error passing is learned using gradient descent for the backward pass emulating backpropagation. 

I would recommend adding a step-by-step algorithm to the paper to make things more straightforward. Including the gradient descent update for both forward and backward weights would be helpful. 
Also, given the paper's claim of learning the meta-algorithm, I think the authors should also include the results of using this method on conv networks. 

A single ablation using a small conv network trained on Cifar using LA learned over FC layers would significantly improve the impact of the paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes variable shared meta learning,  and implements the network forward and backward computation with RNN.  The proposed method can also be used for learning general learning algorithms. Experiments on MNIST and Fashion MNIST dataset demonstrate the effectiveness of the proposed method. 

main_review: Originality:

The proposed method is novel compared to existing works. The authors clearly describe the difference w.r.t to previous contributions and well-positioned with related works. 

Quality:

This submission is technically sound.  The main claims are supported by the experimental results. The methods are carefully designed and demonstrated its advantage. 

Clarity:

This paper is clearly written and well-structured.

Significance:

1. The experimental section only use simple datasets, e.g., MNIST. I would like to see more real dataset, e.g., CIFAR10 or CIFAR100. This would improve the applicability on the real tasks. 

2.  RNN is hard to train in practice, how to ensure the stability of the proposed methods in real datasets since the RNNs are interactived with each other in the proposed method.

3.  I wonder if the proposed method can be applied on other network architectures, e.g., convolutional network. This will broaden the applicability of the proposed method. 

4. I would like to see what is the performance on larger-scale network structure, currently only experiment on small-scale networks. 
limitations_and_societal_impact: The authors sufficiently discuss the limitations and societal impact of the work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes VSML RNN, an architecture for RNNs that is suitable for meta-learning problems in which the RNN activations represent task-specific quantities and RNN weights represent task-independent quantities. The authors use the insight that RNNs for meta learning (meta RNNs as called in the paper) perform better when the meta-parameters are constrained relative to the task-specific parameters. VSML RNN is essentially an RNN with a particular weight sharing and sparsity structure. The authors demonstrate that VSML RNN can implement *within its hidden state* backpropagation on a smaller model, where the parameters of the smaller model are represented in the VSML RNN's hidden state. Empirically, the authors show that the learning algorithm implemented by a VSML RNN can outperform baselines, and can learn faster than gradient descent. The authors also demonstrate that the learning algorithm is transferable to unseen problems.
main_review: **Originality**
The proposed approach appears novel. Unlike prior work on meta RNNs, the authors recognize that overparameterization of RNNs can lead to learning algorithms implemented by the RNN to overfit to the problems on which it is trained. The proposed method is explicitly designed to avoid such overfitting. Unlike prior work on learning learning rules, VSML RNN does not impose a particular architecture on  the task-specific learning algorithm.

**Quality**
The theoretical claims in the paper appear well supported. However, the experimental results are limited, particularly when it comes to comparisons with other meta RNN approaches. The paper would be strengthened by comparing VSML RNN to meta RNNs other than a standard meta RNN- for example, adding memory mechanisms to the RNN. Moreover, interpreting the learning algorithm implemented by VSML RNN would be helpful. The current analysis views the output of the model over the course of training, but finding a way to interpret the internal structure of VSML RNN activations would provide insight into the nature of the learned learning algorithm.

Furthermore, it is stated in the introduction that the results may be relevant in the field of biologically plausible learning algorithms. It would help if the authors could state how VSML RNN could be relevant here. Specifically, (how) can VSML RNN be used to discover biologically plausible learning rules in a way that existing approaches cannot? It seems that VSML RNN may be less useful for this compared to other approaches because it imposes fewer constraints on the form of the learning algorithm implemented by the RNN.


**Clarity**
VSML RNN is a difficult concept to understand- this makes clarity of the paper important. Unfortunately, there are some areas of the paper that are unclear or could be better explained. Specifically:
1) The authors state in section 3 that "A simple conceptual way of understanding VSML is taking a standard recurrent or feed-forward architecture and replacing all weights with small RNNs." This is hard to interpret since a weight is a parameter and a small RNN is a function- they do not seem comparable.
2) The distinction between the view of VSML RNN as an RNN (where s is a hidden state) and the view of it as implementing a learning algorithm on a neural network (where s could be a parameter) is not very clear. This is partly because section 3 tries to introduce both viewpoints: the equations look like typical RNN equations and suggest the first view, while Figure 1 encourages the second view. It may be helpful to structure the section to introduce the two viewpoints separately and then explain the mapping between them.
3) Visually illustrating the indexing of s would be helpful. Initially, it is not clear why the indexing in Equation 3 for example is constructed the way it is.
4) Similarly, visually illustrating how stacking VSML RNNs works would be helpful. Figure 1 is not very clear- the distinctions and equivalences between VSML RNN states, parameters, messages and graph edges is not visually apparent.


**Significance**
If the experimental results can be improved, VSML RNNs may become the state-of-the-art approach in the field of meta RNNs. The learning algorithms produced by VSML RNNs may also provide more insight into how to develop more general, better performing learning algorithms. Finally, VSML RNNs further unify meta RNNs and learning algorithms on neural networks, which may lead to more flexible meta-learning procedures.
limitations_and_societal_impact: The limitations and impact of this work are well-discussed in section 7.
review_title: null

Meta-review: The reviewers were in agreement that the method developed were novel and well evaluated, particularly with the additional details and experimental results that came up in the discussion. These results should be incorporated in the final version of the paper. Overall this paper should be of significant interest to the NeurIPS community.
Decision: Accept (Poster)