Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors consider the behaviour of networks of artificial recurrent neural networks (RNNs), specifically the stability of RNNs of RNNs. This is achieved using nonlinear control theory, specifically to consider the contraction properties of the networks (where a contracting system is one in which it's long-term behaviour is exponentially independent of initial conditions). 


strengths_and_weaknesses: The authors identify that understanding networks of neural networks is significant both for using all available neural data in neuroscience as well as for new deep learning architecture frameworks. They use nonlinear control theory to analyze the contraction of networks of RNNs. I believe that this, in itself is novel and is further improved on by them deriving several original results. They also review several recent results in machine learning theory, such as the sufficient conditions for stability of nonlinear systems and give clarification to their applicability. The authors implement a form of RNNs of RNNs which are expected to satisfy appropriate contractivity, given their theoretical exposition. They find strong performance when compared to other baselines. 

Overall, I did not find signficant weaknesses. The manuscript was well structured, on an original topic, and the authors were thorough in their investigation. 
questions: Is it possible to discern the role of different sub-networks in forming learned representations? What is the effect of ablating sub-networks during training?  

How well does the results on stability and contractivity apply to other network architecture? Could a similar theoretical analysis be applied to other recurrent models e.g. neural cellula automata? 
limitations: It wasn't immediately clear to me how well their results generalise to other network architectures. I also note this as a question to the authors above.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors analyze a model of interconnected recurrent neural networks, trained to perform various tasks. They emphasize the importance of network stability to obtain good performance, and analyze it using  tools from contraction analysis. Specifically, they derive conditions under which a network of RNNs can remain stable. These, in turn, are used to parametrize the network for a training protocol that preserves stability during training.
This setup is used to test two different variants of a network of networks on the sequential MNIST and CIFAR tasks. Performance is higher than several similar models. Finally, the effect of hyper-parameters such as sparseness is numerically explored. 

strengths_and_weaknesses: As recording from multiple brain areas simultaneously becomes more feasible in experiments, it is timely to study how networks of networks can collaborate to perform more complex and higher-level computations. This paper brings to the front the importance of making sure that certain properties of RNNs are retained when they are combined and assembled into a larger network. 
The topic is thus timely and important.
Stability of trained networks has been recognized as important for performance (e.g., Sussillo & Abbott 2009), but mainly studied in the reservoir computing framework - where there is feedback to a single network. The current work extends this framework to the scenario of multiple networks.

The main weaknesses are the difficulty to understand the main objective of the paper, and a lack of reference to existing literature.
The motivation in the abstract is to better understand multi-area RNNs as models in neuroscience. It isn’t clear why benchmarks serve this purpose. On the other hand, if this is a proposal for a new ML architecture, it seems that performance isn’t better than state of the art.
Furthermore, the improved performance in the modular case isn’t accompanied by any analysis of the resulting networks. Do different modules handle different parts of the computation? Is the optimal number of modules related to the task? If so, how? These questions are much more relevant to the neuroscience motivation, but are not explored at all.

questions: 1.	Benchmarks. I am not an expert in benchmarks of these form, but a quick search resulted in what seems like an RNN with a similar number of parameters and better performance (https://paperswithcode.com/paper/parallelizing-legendre-memory-unit-training/review/?hl=27784). Again - this is simply the first result I found.
2.	L289-290 “stability and modularity unexplored”. For instance: Aljadeff, Stern, Sharpee PRL 2015.
3.	L74-75. The linear coupling between nonlinear networks is not an intuitive choice from a neuroscience perspective. It is claimed that the nonlinear case will also be considered, but I didn’t find it.
4.	Equation 2 : is it x_i or x_j ?
5.	You show that modular networks learn better when initialized sparsely - this is an interesting result (perhaps related to the advantage of sparse connectivity in reservoir networks). Unfortunately, no intuition or analysis of why this is the case is provided.


limitations: Yes
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper tries to address the stability issue of combining multiple stable RNNs. The motivation is that the stability of a single RNN is well studied, but the stability of this kind “RNNs of RNNs” is little studied, particularly, as compared to the recent rising research direction in neuroscience. The “RNNs of RNNs” is defined in eq.(2), which considers the simplest case: linear combination.  The authors provided some theoretical justification to prove that the global stability of “RNNs of RNNs” can be achieved. The authors tested one instance network: stability constrained network on sequential MNIST and Cifar10, showed that good performance can be achieved as compared to baselines.
strengths_and_weaknesses: Strengths:
1. The core idea of studying RNN contracting stability is interesting as well as the connection to neuroscience.
2. Some theoretical findings would be helpful in understanding the “network of network” stability.

Weaknesses:
1. There are some unclear points, which makes this paper become less and less understandable in the later stage.
2. The experimental section is weak. Experiments on harder tasks such as language modelling, text classification, would make this section much stronger. 

questions: 1. What is the "stability" of RNN meaning? it should be given and explained in a better way at the beginning of the paper. This causes confusion in the later stage.
2. The term "RNNs of RNNs" is confusing, in eq (2), the connection between RNNs is not recurrent/recursive, but linear connection. 
3. Given the linear combination of RNNs, what is the difference to ensemble RNNs?
4. How the empirical results support the main claims? Good accuracy presents stability? 

5.There are many methods give better performance on mentioned datasets, just give a few examples:   
[1]Chang et al, Dilated Recurrent Neural Networks, NeuriPS 2017. 

[2] Tim Cooijmans et al, Recurrent Batch Normalization, ICLR 2017. 

[3] Wang et al, State-Regularized Recurrent Neural Networks, ICML 2019. 

[4] Trinh et al, Learning longer-term dependencies in rnns with auxiliary losses, ICML 2018. 

limitations: yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: With the increasing ability of experimental neuroscientists to obtain simultaneous recordings of brain activity from multiple areas of the brain, it is important to develop computational theory for RNNs that can model such situations. In this context, the submission is focused on the question of the stability of networks of multiple RNNs that are connected and in particular the question of contractive stability. The paper presents a network of networks model that involves multiple continuous time rate-based vanilla RNNs whose state updates depend on the states of the other RNNs.  A parameterization of negative feedback between the multiple RNNs is presented which allows for optimization using gradient-based techniques and it is proved that this ensures contractive stability of the combined system (assuming each individual RNN is contracting). In addition, multiple sufficient conditions are presented to ensure contractive stability of the individual RNNs. 

Two particular instantiations of the networks of networks model based on two of the stability results are empirically evaluated on the seqMNIST, permuted seqMNIST and seqCIFAR sequence classification tasks to determine if the stability constraints impede expressivity. One of the variants outperforms other recent stable RNNs on both permuted SeqMNIST and seqCIFAR and outperforms an LSTM baseline on all three. In addition, experiments are performed to study how the performance increases as more networks are added and the sparsity is adjusted.


strengths_and_weaknesses: -Originality: While the proposed network of RNNs appears to fall within the class of multidimensional RNNs and multidimensional state space models (though see Question #1 below for clarification on this point) and the idea of using negative feedback to ensure stability of combined systems is well-known in the nonlinear systems literature, this appears to be the first work to study in detail how to parameterize these stability conditions in a way that is straightforward to optimize using deep learning techniques. In addition, the conditions derived to ensure contractive stability of a single RNN appear to be novel. It seems however that the paper would benefit from a more detailed related works section to help the reader have a better survey of where this work fits into the literature.

-Quality: The theoretical results appear to be sound. The empirical results seem to support the basic claim that some of the theoretical stability conditions can be used in a way that does not severely inhibit the expressivity of the network. However, I have some lingering questions related to these results that could be clarified. See Questions section below. In addition, the major motivation for the approach in the paper is for modeling neuroscience data recorded from multiple brain areas, so some kind of modeling experiment related to this (even if synthetic) would strengthen the submission.

-Clarity: The paper is in general well-written and clear. See Questions sections below for a few detailed suggestions for improvement.

-Significance: This paper will be of interest to the computational neuroscience community as well as the general machine learning community interested in the stability of RNNs.  This paper seems to have connections to multidimensional RNNs and a more thorough discussion of this connection could broaden the reach to people interested in modeling multidimensional data.  The theoretical stability results could help guide future development of these methods. In addition, in my experience seqCIFAR is still a challenging task for most methods and good performance on this task tends to translate to good performance on other tasks requiring the modeling of long-range dependencies. So the fact that these combined networks perform better than many of the baseline methods is quite interesting.

questions: 1. There seems to be a close connection with multidimensional RNNS (Graves 2007) and multidimensional state space models (https://en.wikipedia.org/wiki/Multidimensional_system), where the update equations for each of the multiple states of the system depend on the other states. Perhaps a more detailed discussion of this connection would be useful for the readers and potentially broaden the reach of this work? There could be relevant ideas in the multidimensional systems theory literature as well. 

2. Lines 74-75 say that later on extensions to nonlinear couplings will be discussed. This sounded interesting, but unless I am missing something, the only mention of this I saw was the last sentence of line 107, which I am not sure I would consider a discussion.  I would recommend either rewording lines 74-75 where you are not overpromising or expanding the discussion (or referring to its location more explicitly if I am missing it somewhere else).

3. The freezing of the weights of the Sparse Combo Net has a connection to Echo State Networks which might also be included in a related works section.

4. Why are the results for the SVD Combo net not presented in the results Table 1? It seems strange to specifically discuss this network and then not present the results. Some of the results (though I do not think all) appear to be in the text and appendix, but it seems including this in the table would reduce confusion.

5. Why were only the two variants determined by Theorem 1 and 5  (Sparse combo net and SVD combo net) considered instead of other variants based on the other theorems as well? Unless I missed this discussion in the appendix, a more detailed discussion of this choice might be helpful.

6. Section 4.2.1 discusses adjusting the network size. Increasing the number of modules increases the number of parameters, so one natural question would be how does the performance of 2 size 32 modules compare to the performance of 1 size 64 module, and so on...? It is somewhat unclear to me what this would look like for the Sparse combo net since many of the parameters are not trained, but it seems natural for the SVD combo net. 

Minor points:

-Line 194-195 says "only trained the connections between subnetworks (Figure 2A)" but I think this should say Figure 2B (the sparse combo net)

-Line 195 says "we trained all parameters of the model (Figure 2B)" but I think this should be Figure 2C (the SVD Combo net)

-There are few places where being a bit more explicit about the shapes of some of the variables might help. For example, in Corollary 1 in the appendix, I think you would benefit from more explicitly stating that L is a matrix made up of matrices (perhaps explicitly providing the general dimensions) where each $L_{ij}$ is a matrix. Similarly, explicitly saying u(t) is made up of $u_i(t)$ could be helpful since in the main text u(t) is only defined for eq. 1 (a single RNN). Alternatively, you could maybe define $\tilde{u}(t)$ analogously to $\tilde{x}$. 
limitations: The authors discuss limitations in an extended discussion in the appendix.
review_title: null

Meta-review: metareview: In this paper the authors propose a so-called "RNN of RNNs"  architecture where an assembly of contractively stable RNNs can still work collaboratively under contractive stability.  The authors draw the motivation from neuroscience where multiple brain areas can work simultaneously for complex behaviors. Conditions are investigated under which multiple interacting RNNs with feedback connections will preserve contractive stability.  Experiments are carried out on seqMNIST, permuted seqMNIST and seqCIFAR datasets. It is shown that the proposed network of RNNs can give decent results. Notably, SOTA performance is claimed within the class of provably stable recurrent architectures.  All reviewers consider the work theoretically solid and timely to the community.  Authors' rebuttal clears most of the concerns.  The paper can be accepted.  One lingering concern which, however, still stands after the rebuttal and discussion is its connection and impact to neuroscience where the motivation is drawn.  Following the authors' narrative, the work would be more convincing if the proposed "RNN of RNNs" can show effectiveness directly on neuroscience datasets.  The authors should revise the paper accordingly to reflect that.  
confidence: Certain
Decision: Accept