RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks
Authors: Leo Kozachkov
Michaela M Ennis
Jean-Jacques Slotine
Venue: NeurIPS 2022 Accept (2022)
Keywords: Neuroscience
Recurrent Neural Networks
Control Theory
Machine Learning
Dynamical Systems

Abstract: Recurrent neural networks (RNNs) are widely used throughout neuroscience as models of local neural activity. Many properties of single RNNs are well characterized theoretically, but experimental neuroscience has moved in the direction of studying multiple interacting areas, and RNN theory needs to be likewise extended. We take a constructive approach towards this problem, leveraging tools from nonlinear control theory and machine learning to characterize when combinations of stable RNNs will themselves be stable. Importantly, we derive conditions which allow for massive feedback connections between interacting RNNs. We parameterize these conditions for easy optimization using gradient-based techniques, and show that stability-constrained "networks of networks" can perform well on challenging sequential-processing benchmark tasks. Altogether, our results provide a principled approach towards understanding distributed, modular function in the brain.

RNNs of RNNs:
Recursive Construction of Stable Assemblies of Recurrent Neural Networks
Leo Kozachkov' Michaela Ennis? and Jean-Jacques Slotine
Depaarteent of Brain and Cognitive Sciences, Massachusetts Institute of Technology Division of Medical Sciences, Harvard University
DDpartment of Mechanical Engineering, Massachusetts Institute of Technology
'Google AI
'Equal contribution (leokoz8, mennis, jjs @mit.
Abstract
Recurrent neural networks (RNNs) are widely used throughout neuroscience models of local neural activity. Many properties of single RNNs are well charac- terized theoretically, but experimental neuroscience has moved in the direction of studying multiple interacting areas, and RNN theory needs to be likewise extended. We take constructive approach towards this problem, leveraging tools from non- linear control theory and machine learning to characterize when combinations stable RNNs will themselves be stable. Importantly, derive conditions which al- low for massive feedback connections between interacting RNNs. We parameterize these conditions for easy optimization using gradient-ban baased techniques, and show that stability-constrannee 'networks networks' can perform well on challenging benchmark tasks. Altogether, our results provide principled approach towards understanding distributed, modular function in the brain.
Introduction
The combination and reuse of primitive "modules" has enabled a great deal of progress in computer science, engineering, and biology. Modularity particularly apparent the structure of brain, as different parts are specialized for different functions [Kandel et al., 2000]. Accordingly, most experimental studies throughout the history of neuroscience have focused on a single brain area in association with single behavior [Abbott and Svoboda, 2020]. Similarly, RNN models of brain function have mostly been limited to single RNN modeling a single area. However, neuroscience entering age where recording from many different brain areas simultancously during complex behaviors is possible. As experimental neuroscience has shifted towards multi-area recordings, computational techniques for analyzing, modeling, and interpreting these multi-area recordings have blossomed [Michaels et al., 2020, Mashour et al., 2020, Abbott and Svoboda, 2020, Perich et al., 2021, Semedo al., 2019, Yang and Molano-Mazón, 2021, Machado al., 2022]. Despite this, RNN theory has lagged behind.
The theoretical question of RNN stability is crucial for understanding information propagation and manipulation [Vogt et al., 2020, Engelken et al., 2020, Kozachkov et al., 2022a]. The conditions under which single, autonomous RNNs are chaotic stable well-studied. in particular when the RNN weights are randomly chosen and the number of neurons tends to infinity [Sompolinsky et al., 1988, Engelken et al., 2020]. However, there is very little work addressing the theoretical
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

question of stability in 'networks of networks'. Two facts make this question challenging. Firstly, connecting two stable systems does not, in general, lead a stable overall system. This is true even for linear systems. Secondly, there is massive amount of feedback between brain areas, so one cannot reasonably assume near-decomposabiilty [Simon, 1962, Abbott and Svoboda, 2020].
Given that the brain seems to dynamically reorganize and adapt interareal connectivity to meet task demands and environmental constraints [Miller and Cohen, 2001, Sych al., 2022], this question of how stability is maintained is of the utmost importance. Here we take a bottom-up approach, more specifically asking "what stability properties of the individual modules lend themselves to rapid reorganization?"
1.1 Contraction Analysis
We focus special type of stability, known contractive stability [Lohmiller and Slotine, 1998]. Loosely, a contracting system is a dynamical system that forgets its initial conditions exponentially quickly. Contractive stability is strong form of dynamical stability which implies many other forms of stability, such as certain types of input-to-state stability [Sontag, 2010]. See Section A.2 for a mathematical primer contraction analysis.
Contraction analysis has found wide application in nonlinear control theory [Manchester and Slotine, 2017], synchronization [Pham and Slotine, 2007], and robotics [Chung and Slotine, 2009], but has only recently begun to find application in neuroscience and machine learning [Boffi et al., 2020, Wensing and Slotine, 2020, Kozachkov et al., 2020, Revay and Manchester, 2020, Jafarpour et al., 2021, Kozachkov et al., 2022a, Centorrino et al., 2022, Burghi et al., 2022, Kozachkov et al., 2022b]. Contraction analysis is useful for neuroscience because it is directly applicable to systems with external inputs. It also allows for modular cubbiltty-preserving combination properties to be derived (Figure 1). The resulting contracting combinations can involve individual systems with different dynamics, as long as those dynamics are contracting [Slotine and Lohmiller, 2001, Slotine, 2003].
Moreover, modular stability and specifically contractive stability have relevance to evolutionary biology [Simon, 1962, Slotine and Lohmiller, 2001]. In particular, it is thought that the majority of traits that have developed over the last 400+ million years are the result of evolutionary forces acting on regulatory elements that combine core components, rather than mutations in the core components themselves. This mechanism of action makes meaningful variation in population phenotypes much more feasible to achieve, and is appropriately titled "facilitated variation" " [Gerhart and Kirschner, 2007]. In addition to the biological evidence for facilitated variation, computational models have demonstrated that this approach produces populations which are better able to generalize to new environments [Parter et al., 2008], an ability that will be critical to further develop in deep learning systems. However, the tractability of these evolutionary processes hinges on some mechanism for ensuring stability of combinations, Because contraction analysis tools allow complicated contracting systems to be built up recursively from simpler elements, this form of stability would be well suited for biological systems [Slotine and Liu, 2012]. Our work with the Sparse Combo Net in Section 4 has direct parallels to facilitated variation, in that we train this combination network architecture only through training connections between contracting subnetworks.
Ultimately, our contributions are three-fold:
A novel parameterization for feedback combination of contracting RNNs that enables direct optimization using standard deep learning libraries.
Novel contraction conditions for continuous-ting nonlinear RNNs, to use in conjunction with the combination condition. We also identify flaws in stability proofs from prior literature.
Experiments demonstrating that our 'network of networks' sets a new state of the art for stability-constrained RNNs on benchmark sequential processing tasks.
2 Network of Networks Model
In this paper we analyze rate-based neural networks. Unlike spiking neural networks, these models are continuous and smooth. We consider the following RNN introduced by Wilson and Cowan [1972], which may be viewed as an approximation to a more hophysically-detaii spiking network:
π - Wo(x) + u(t)
(1)

---

Xb
Xa
Feedback
Hierarchical
Feedback
Hierarchical
Figure 1: Contractive stability implies a modularity principle. Because contraction analysis tools allow complicated contracting systems to be built up recursively from simpler elements, this form of stability well suited for understanding biological systems. Contracting combinations can be made between systems with very different dynamics, as long those dynamics contracting.
Here > is the conssctnttnnt of the network [Dayan and Abbott, 2005], and the vector x E IR contains the voltages of all neurons in the network. The voltages converted into firing-rates through static nonlinearity 6. We only consider monotonic activation functions with bounded slope: in other words, 0 S ø' S ? (unless otherwise noted). We do not restrain the sign of the nonlinearity. Common example nonlinearities ((x) that satisfy these constraints are hyperbolic tangent and ReLU. The matrix W _=*= contains the synaptic weights of the RNN. is this matrix that ultimately determines the stability of (1), and will be main target for our analysis. Finally, u(!) the potentially time-varying external input into the network, capturing both explicit input into the RNN from the external world, as well as unmodeled dynamics from other brain areas.
Note that (1) is equivalent to another commonly used class of RNNs where the term Wx + appears inside the nonlinearity. See Section A.1 or [Miller and Fumarola, 2012] for details. Our mathematical results apply equally well both types of RNNs.
In order to extend (1) into a model of multiple interacting neural populations, we introduce the index which runs from 1 to p. where p the total number of RNNs in the collection of RNNs. For now we will assume linear interactions between RNNs, because this the simplest case. The linearity assumption can also be motivated by the fact that RNNs have been found to be well-approximated by linear systems many neuroscience contexts [Sussillo and Barak, 2013, Langdon and Engel, 2022]. This leads to the following equation for the 'network of networks':
Tx; = -x (xx; + Σ Lyj + uz(()
Vi 1......p
where the matrix Lij captures the interaction between RNN and RNN j. INNN has N; neurons and RNN j has Nj neurons, then Lij is an N; Nj matrix.
We can now formalize the question posed in the introduction. Namely, "what types of connections between stable RNNs automatically preserve stability?" becomes "what restrictions on L-j must be met order to ensure overall stability of the network of networks?". We will now derive two combination 'primitives', negative feedback and hierarchical, which allow for recursive connection of contracting modules.
2.1 Generalized Negative Feedback Between RNNs Preserves Stability
We set aside for a moment the problem of determining when (1) is contracting, For now, assume that we have collection
contracting RNNs interacting through equation (2). Recall that contraction is defined with respect to metric, way of measuring distances between trajectories in state space. Thus, the ith RNN is contracting with respect to some metric M,. We assume for simplicity that this metric is constant, which means that M, is simply symmetric, positive definite matrix. In the case where every RNN receives feedback from every other, we can preserve stability by ensuring these connections negative feedback. In the simplest case where all RNN modules contracting in

---

the identity metric, the negative feedback may be written as:
Ly Lif
This is a well known result from the contraction analysis literature [Slotine, 2003]. Our first novel contribution is a generalization and parameterization of this negative feedback which allows for direct optimization using gradient-basee -based techniques. In particular, if each Lij is parameterized as follows:
Lا = Bi M, M.M,, Vi.j
(3)
for arbitrary matrix Bij. then the overall network of networks retains the assumed contraction properties of the RNN subnetworks. We provide detailed proof in Section C.I, but the basic idea relies on ensuring skew-symmetry of Lı in the appropriate metric. This can be achieved via the constraint:
M,L.j = -L,M,,
Plugging (3) into the above expression verifies that it is indeed satisfied. Because contraction analysis relies on analyzing the symmetric part of Jacobian matrices, the skew-symmetry of Lij 'cancels out' when computing the symmetric part, and leaves the stability of the subnetwork RNNs untouched. In the remainder of the paper, in the experimental sections, we will use feedback combinations constrained by (3). However, it is possible to significantly generalize this condition (see C.1 for details).
Recursive Properties of Contracting Combinations The feedback combination (3), taken to- gether with hierarchical combinations, may be used as combination primitives for recursively con- structing complicated networks networks while automatically maintaining stability. The recursion comes from the fact that single contracting system, which may in turn be combined with other contracting systems, ad infini- tum. Note that while the feedback (3) requires linear interareal connections, hierarchical interareal connections may be nonlinear [Lohmiller and Slotine, 1998].
modular system is shown be contracting it may be treated
Various Ways to Achieve Local Contraction
In this section we return to the question of achieving contraction in the subnetwork RNNs. Recall that we wish to find restrictions on W, such that the subnetwork RNN, described by (1), is contracting. Here we derive five such novel conditions (see Section C for detailed proofs). As we will discuss, not all contraction conditions are equally useful for example some conditions are casier to optimize or have higher model capacity than others. In this section we also point out some flaws in existing stability proof' in the RNN literature, and suggest some pathways towards correcting them.
Theorem (Absolute Value Restricted Weights). Let W denote the matrix formed by taking the element-wii wise absolute value of W. If there exists a positive, diagonal P such that:
P(g - 1) + (g W - 1) -0
with g being the maximum slope of Φ. then (1) contracting metric P. If,, S 0, then [|ss: may be set to zero to reduce conservatism.
It is easy to find matrices that satisfy Theorem 1, and given matrix the condition is as easy to check as linear stability is. Moreover, the condition guarantees we can obtain a metric that the system is known to contract in (see Section 4.1 for details). It is less straightforward to enforee this condition during training, however we found that subnetworks constrained by Theorem I can achieve high performance in practice by simply fixing W and only optimizing the connections berween subnetworks (Section 4.2). As there are fewer parameters to optimize, this training technique faster.
Theorem 2 (Symmetric Weights). If W = W and gW x I, and o' > O, then (1) is contracting.
It has been known since the early 1990s that if (1) is autonomous (i.e the input u is constant) and has symmetric weights with eigenvalues less than 1/g. then there exists a unique fixed point that the network converges to from any initial condition [Matsuoka, 1992]. Theorem 2 generalizes this statement to say that if (1) has symmetric weights with eigenvalues less than 1/g, contracting. This includes previous results as a special case, because an autonomous contracting system has unique fixed point which the network converges to from any initial condition.

---

Theorem 3 (Product of Diagonal and Orthogonal Weights), If there exists positive diagonal matrices P1 and P2, as well as Q Q" 0 such that
W - P
then (I) contracting in metric M (PIQP:)
In contrast to the first two contraction conditions, Theorem 3 is very easy to optimize. To meet the constraint that the P matrices positive, can parameterize their diagonal elements as Pa = di + e, for some small positive constant e, and optimize d, directly. To meet the positive definiteness constraint on Qu one may parameterize as Q = E E + cI and optimize E directly.
Theorem 4 (Triangular Weights), If g W - is triangular and Hurwitz, then (1) is contracting in diagonal metric.
Theorem follows from the fact that hierarchy of contracting systems also contracting.
Theorem (Singular Value Restricted Weights). If there exists a positive diagonal matrix P such that:
WW P
then (I) is contracting in metric P.
In the case of discrete-time RNNs, this contraction condition has been proved by many different authors in many different settings. When P ==I, is known as the echo-state condition for discrete- time RNNs [Jaeger, 2001]. This was then generalized to diagonal P by Buehner and Young [2006]. More recently, the original echo-state condition was rediscovered by Miller and Hardt [2018] in the machine learning literature. Following this rediscovery, the condition was generalized to P o I by Revay and Manchester [2020]. Here we show that applies to continuous-time RNNs as well.
3.1 What do the Jacobian Eigenvalues Tell Us?
Several recent papers in ML, c.g [Haber and Ruthotto, 2017, Chang et al., 2019], claim that a sufficient condition for stability of the nonlinear system:
x = f(x, 1)
is that the associated Jacobian matrix J(x, 1) = af has eigenvalues whose real parts are strictly negative, i.c:
8x
max Re(A, t)) < aa
with a > However, this claim is generally false see Section 4.4.2 [Slotine and Li, 1991].
In the specific case the RNN (I), appears that the eigenvalues of the symmetric part of W do provide information global stability in number of applications. For example, in [Matsuoka, 1992] was shown that if W ] (W + WT) has all its eigenvalues less than unity, and u is constant, then (1) has unique, globally asymptotically stable fixed point. This condition also implies that the real parts of the eigenvalues 2019] it shown that setting the symmetric part of W. & 1 (W + WT) almost equal to zero (yet slightly negative) led to rotational, yet stable dynamics in practice. This leads us to the following theorem, which shows that if the slopes of the activation functions change sufficiently slowly function of time, then the condition in [Matsuoka, 1992] in fact implies global contraction of (1).
the Jacobian are uniformly negative. Moreover, in [Chang et al.,
Theorem 6. Let be a positive, diagonal matrix with D dóg and let P be an arbitrary, positive dx,'
diagonal matrix. If:
(gW T)P P(g - I) - -PP and D eg
-D -BD
for c, B > 0, then (1) is contracting in metric with rate
We stress however, that it is an open question whether or not diagonal stability of W implies that (1) is contracting. It has been conjectured that diagonal stability of gW - iss sufficient condition for global contraction of (1) [Revay al., 2020], however this has been difficult to prove. To better characterize this conjecture, we present Theorem 7, which shows by way of counterexample that diagonal stability of g W does not imply global contraction in a constant metric for (1).
5

---

Theorem 7. Satisfaction of the condition 9W sgm - - is not sufficient to show global contraction of the general nonlinear RNN (1) in any constant metric. High levels of antisymmetry in W can make it impossible to find such meiric, which we demonstrate via a 2 x 2 counterexample of
0
the following form, with с 2 when g = 1: W
4 Stability-Constrained Network of Networks Perform Well on Benchmarks
A natural concern is that stability of an RNN may come at the cost of its expressivity, which is particularly relevant for integrating information over long timescales. To investigate whether this might be an issue for our model, we trained a stability-constraiie benchmark sequential image classification tasks: sequential MNIST, permuted seqMNIST, and sequential CIFARIO. These tasks are often used to measure information processing ability over long sequences [Le al.. 2015]. Images are presented pixel-by-pixxe and the network makes a prediction at the end of the sequence. In permuted seqMNIST, pixels are input in fixed but random order.
network-of-netwe
on three
All of experiments were done networks governed by (2). The nonlinear subnetwork RNNS were connected to each other via linear all-to-all negative feedback, given by (3). For all subnetworks we use the ReLU activation function. To enforce contraction of each individual subnetwork, focused on two stability constraints from our theoretical results: Theorems and 5. In the case of Theorem 1, did not train the individual subnetworks' weight matrices, but only trained the connections between subnetworks (Figure 2B). For Theorem 5, we trained all parameters of the model (Figure 2C).
We refer to networks with subnetworks constrained by Theorem as 'Sparse Combo Nets' and networks with subnetworks constrained by Theorem 'SVD Combo Nets'. Throughout the experimental results we use the notation 'p x n network'. such network consists of p distinct subnetwork RNNs, with each such subnetwork RNN containing units.
A
B)
Sparse Combo Net
SVD Combo Net
schematic)
schematic)
trzind
tralind
stabic
Uneer Nagatier Feadback
Linsur
Negstive Seedback
masteigs/W/y/ Pi P- posttive diagonal
manteigs
P Paoottiivecom daggonel
Figure 2: Summary of task structure and network architectures. Images from MNIST (or CIFARIO) were flattened into an array of pixels and fed sequentially into the modular 'network of networks' with classification based on the output the last time-step. For MNIST, each image also permuted in a fixed manner (A). The subnetwork 'modules' of our architecture were constrained to meet either Theorem via sparse initialization (B) Theorem 5 via direct parameterization (C). Linear negative feedback connections were trained between the subnetworks according to (3).
4.1 Network Initialization and Training
For the Sparse Combo Net we were not able find parameterization to continuously update the internal RNN weights during training in way that preserved contraction. However, it is easy to randomly generate matrices with a particular likelihood of meeting the Theorem 1 condition by selecting appropriate sparsity level and limit entry magnitude. Sparsity in particular is of interest

---

due to its relevance in neurobiology and machine learning. so it is convenient that the condition makes easy to verify stability of many different sparse RNNs. As g for ReLU activation, we check potential subnetwork matrices W by simply verifying linear stability of [W|- I.
Because every RNN meeting the condition has corresponding well-defined stable L.TI system contracting in the same metric, it is also easy to find metric to use in our training algorithm: solving for M in -1 MA + A" M will produce valid metric for any stable LTI system A [Slotine and Li, 1991]. We utilize the fact that Hurwitz Metzler matrices are diagonally stable to improve efficiency of computing M (as well as in our proof of Theorem 1).
We therefore randomly generated fixed subnetworks satisfying Theorem and trained only the linear connections between them (Figure 3), well as the linear input and output layers. More information on network initialization, hyperparameter tuning, and training algorithm is provided in Section D.
A)
B)
2028 Retuore in tett ioetto loeton
T0SE Antoorc she TAASTS
Sonrt e Linear Ves Matta
Exoraile Uncor Wayg HOT1
675
150 tpodo
150
122
420
-55
-50
-05
-35
Figure 3: Example 3x16 Sparse Combo Net. Nonlinear intra-subnetwork weights are initialized using sparsity, and do not change in training (A). Linear inter-subnetwork connections are constrained to be antisymmetric with respect to the overall network metric, and are updated in training (B).
For the SVD Combo Net on the other hand, we ensured contraction of each subnetwork RNN by direct parameterization (described in Section E), thus allowing all weights to be trained.
4.2 Results
The Sparse Combo Net architecture achieved the highest overall performance on both permuted seqMNIST and seqCIFAR10, with 96. 94% and 65.72% best test accuracies respectively thereby setting a new SOTA for stable RNNs (Table 1). Furthermore, we were able to reproduce SOTA scores over several repetitions, including 10 trials of seqCIFARIO, 10. Along with repeatability of results, we also show that the contraction constraint on the connections between subnetworks (L in (3)) is important for performance, particularly in the Sparse Combo Net (Section 4.2.3).
Additionally, we profile how various architecture settings impact performance of our networks. In both networks, we found that increasing the total number of neurons improved task performance, but with diminishing returns (Section 4.2.1). We also found that the sparsity of the hidden-to-hiddee weights in Sparse Combo Net had large impact on the final network performance (Section 4.2.2).
4.2.1 Experiments with Network Size
Understanding the effect of size network performance is important practical application of these architectures. For both Sparse Combo Net and SVD Combo Net, increasing the number of subnetworks while holding other settings constant (including fixing the size of each subnetwork 32 units) was able to increase network test accuracy on permuted seqMNIST to a point (Figure 4).
The greatest performance jump happened when increasing from module (37.1% Sparse Combo Net, 61.8% SVD Combo Net) to two modules (89.1% % Sparse Combo Net, 92.9% SVD Combo Net). After that the performance increased steadily with number of modules until saturating at ~ 97% for Sparse Combo Net and ~ 95% for SVD Combo Net.
As the internal subnetwork weights are not trained in Sparse Combo Net, it is unsurprising that its performance was substantially worse at the smallest sizes. However Sparse Combo Net surpassed SVD Combo Net by the 12 x 32 network size, which contains a modest 384 total units. Due to the better performance of the Sparse Combo Net, we focused additional analyses there. Note also that the SVD Combo Net never reached 55% test accuracy for CIFARIO in our early experiments.
7

---

sMNIST Repeats Mean (n)
PSMNIST
SCIFARIO
Seq MNIST
PerSeq MNIST Best
Seq CIFAR
Stable RNN?
Repeats
Repeats Mean (n)
Name
Params
Mean (n)
Best
Best
[inn]
[Min]
Min]
LSTM
[Chang et al., 2019]
68K
97.3%
92.7%
59.7%
Transformer
[Trinh et al..
0.5M
98.9%
97.9%
62.2%
2018]
Antisymmetric
[Chang et al.,
?
36K
98%
95.8%
58.7%
2019]
96.85
64.72%
Sparse Combo Net
130K
(4)
(10)
99.04%
96.94% %
65.72% %
196.65 %
[63.73%]
Lipschitz [Erichson et al., 2021]
99.2%
95.9%
134K
(10)
(10)
99.4%
96.3%
64.2%
[99.0%]
[95.6%]
CKConv
[Romero ct al., 2021]
IM
99.32%
98.54%
63.74%
$4 [Gu et al., 2022]
7.9M
99.63
98.7 %
91.13%
Trellis [Bai
8M
99.25
98.13%
73.42%
et al., 2019]
Table Published benchmarks for sequential MNIST, permuted MNIST, and sequential CIFARIO best test accuracy. Architectures grouped into three categories: baselines, best performing RNNs with claimed stability guarantee*, and networks achieving overall SOTA. Within each grouping. networks ordered by number of trainable parameters (for CIFAR1O if it differed across tasks). Our network is highlighted. Where possible, we include information on repeatability. *For more stability guarantees in machine learning, Section 3.1
We then evaluated task performance as the modularity of a Sparse Combo Net (fixed to have 352 total units) was varied. We observed an inverse U shape (Figure SIB), with poor performance of 1 x 352 net and an 88 x net, and best performance from a 44 x net. However, this experiment compared similar sparsity levels, while in practice subnetworks by leveraging sparsity in a way not possible for smaller ones.
achieve better performance with larger
callcctton Accurayy
Performance vs Subnetworks, Permuted segMNIST
96.3
Sparse Combo
96.0
90
SVD Camba Net
94.4
94
92
10
30
Number Subnewworke
Figure 4: Permuted SeqMNIST performance plotted against the number of subnetworks. Each subnetwork has 32 neurons. Results are shown for both Sparse Combo Net and SVD Combo Net.
4.2.2 Experiments with Sparsity
Because of the link between sparsity and stability as well as the biological relevance of sparsity, we explored detail how subnetwork sparsity affects the performance of Sparse Combo Net. We ran number of experiments on the permuted seqMNIST task, varying sparsity level while holding network size and other hyperparameters constant. Here we use "n% sparsity level" to refer to a network with subnetworks that have just n% of their weights non-zero.

---

We observed a large (> 5 percentage point) performance boost when switching from a 26.5% sparsity level 10% sparsity level in the 11 32 Sparse Combo Net (Figure 5), and subsequently decided to test significantly sparser subnetworks in 16 x 32 Sparse Combo Net. We trained networks with sparsity levels of 5%, 3.3,, and 1%, as well as 10% for baseline comparison (Figure S2A). 3.3% sparsity level produced the best results, leading to our SOTA performance for stable networks on both permuted SeqMTIST and seqCIFARIO. With a component RNN size of just 32 units, this sparsity level is small, containing only one or two directional connections per neuron average (Figure S7).
As sparsity had such a positive effect on task performance, we did additional analyses better understand why. We found that decreasing the magnitude of non-zero elements while holding sparsity level constant decreased task performance (Figure S2B), suggesting that the effect is driven part by the fact that sparsity enables higher magnitude non-zero elements while still maintaining stability.
Performance During Training, Permuted seqMNIST
Voltcation Accuracy
90-
80-
70-
26.5% Sparsity
10.0% Sparsity
20
40
100
140
Epochs
Figure 5: Permuted SeqMNIST performance over the course of training for two x 32 Sparse Combo Nets with different sparsity levels.
The use of sparsity in subnetworks to improve performance suggests another interesting direction that could enable better scalability of total network size enforcing sparsity in the linear feedback weight matrix (L). We performed pilot testing of this idea, which showed promise in mitigating the saturation effect seen in Figure 4. Those results are detailed in Section D.2.4 (Table S1).
4.2.3 Repeatability and Controls
Sparse Combo Net does not have the connections within its subnetworks trained, so network perfor- mance could be particularly susceptible random initialization. Thus we ran repeatability studies on permuted sequential MNIST and sequential CIFARIO using our best network settings (16 x 32 with subnetwork sparsity level of 3.3%) and an extended training period. Mean performance over trials of permuted seqMNIST was 96.85% with 0.019 variance, while mean performance over 10 trials of seqCIFAR10 10 was 64.72% with 0.406 variance. Note we also ran a number of additional experiments on size and sparsity settings, described in Section D.2.
Across the permuted SeqMNIST trials, best test accuracy always fell between 96.65% and 96.94%, a range much smaller than the differences seen with changing sparsity settings and network size. Three of the four trials showed best test accuracy 2 96.88%, despite some variability early training performance (Figure S3). Similarly, eight of the ten seqCIFARII trials had test accuracy 64.3%, with all results falling between 63.73% and 65.72% (Figure S4). This robustly establishes new SOTA for stable RNNs, comfortably beating the previously reported (single run) 64.2% test accuracy achieved by Lipschitz RNN [Erichson al., 2021].
As a control study, we also tested how sensitive the Sparse Combo Net was to the stabilization condition the interconnection matrix (L in (3)). To do so, we initialized the individual RNN modules in a 24 x 32 network as before, but set L = B and did not constrain B at all during training, thus longer ensuring contraction of the overall system. This resulted in 47.0% test accuracy on the permuted seqMNIST task, a stark decrease from the original 96.7% test accuracy thereby demonstrating the utility of the contraction condition.
Discussion
Biologists have long noted that modularity provides organisms with stability and robustness [Kitano, 2004]. The other direction that stability and robustness provide modularity well known to engineers [Khalil, 2002, Slotine and Li, 1991, Slotine, 2003], but has been less appreciated in biology.

---

We use this principle to build and train provably stable assemblies of recurrent neural networks. Like real brains, the components of our "RNN of RNNs" can communicate with one another through a mix of hierarchical and feedback connections. In particular, we theoretically characterized conditions under which an RNN of RNNs will be stable, given that each individual RNN is stable. We also provided several novel stability conditions for single RNNs that are compatible with these stability- preserving interareal connections. Our results contribute towards understanding how the brain maintains stable and accurate function in the presence of massive interareal feedback, as well as external inputs.
The question of neural stability is one of the oldest questions in computational neuroscience. Indeed, cybemeticists were concerned with this question before the term 'computational neuroscience" existed [Wiener, 1948, Ashby, 1952]. Stability is a central component in several influential neuroscience theories [Hopfelld, 1982, Seung, 1996, Murphy and Miller, 2009], perhaps the most well-known being that memories are stored as stable point attractors [Hopfield, 1982]. Our work shows that stability continues to be a useful concept for computational neuroscience as the field transitions from focusing on single brain areas to many interacting brain areas.
While primarily motivated by neuroscience, our approach is also relevant for machine learning. Deep learning models can be as inscrutable as they are powerful. This opacity limits conceptual progress applications like autonomous driving or buman-centered
and may be dangerous in safety-criticn robotics. Given that stability is a fundamental property of dynamical systems and is intimately linked to concepts of control, generalization, efficiency, and robustness the ability to guarantee stability of recurrent model will be important for ensuring deep networks behave as we expect them to [Richards et al., 2018, Choromanski et al., 2020, Revay et al., 2021, Rodriguez et al., 2022].
In the case of RNNs, one difficulty is that providing a certificate of stability is often impossible or computationally impractical. However, the stability conditions we derive here allow for recursive construction of complicated RNNs while automatically preserving stability. By parameterizing our conditions for easy optimization using gradient-based techniques, we successfully trained our architecture on challenging sequential processing benchmarks. The high test accuracy our networks achieved with small number of trainable parameters demonstrates that stability does not necessarily come at the cost of expressivity. Thus, our results likewise contribute towards understanding stability certification of RNNs.
In future work, we will explore how our contraction-constaal variety of neuroscience tasks, in particular tasks with multimodal structure Yang al., 2019]. Our approach is particulary compatible with "global workspace" models, in which different networks communicate via a shared latent space [Newell and Simon, 1972 Baars, 1993, Dehaene et al., 1998, VanRullen and Kanai, 2021, Goyal et al., 2021]. One desiderata for these future models is that they learn representations which are formally similar those observed in the brain [Yamins et al., 2014, Schrimpf et al., 2020, Williams et al., 2021]. in complement with the structural similarities already shared. Moreover, a "network of networks" approach will be especially relevant to challenging multimodal machine learning problems, such as the simultaneous processing of audio and video. Therefore the advancement of neuroscience theory and machine learning remain hand-in-hame for our next lines of questioning. Indeed, combinations of trained networks have already seen groundbreaking success in DeepMind's s's AlphaGo [Silver et al., 2016].
RNNs of RNNs perform on a
As well as the many potential experimental applications, there are numerous theoretical future directions suggested by our work. Networks with more biologically-plausibe weight update rules, such as models discussed in [Kozachkov et al., 2020], would be fruitful neuroscience context in which to explore our conditions. One promising avenue of study there is to examine nependdennnentt stability of the learning process. In the context of machine learning, our stability conditions could be applied to the end-to-end training of multidimensional recurrent neural networks [Graves et al., 2007], which have clear structural parallels to our RNNs of RNNs but lack known stability guarantees.
In sum, recursively building network combinations in effective and stable fashion while also allowing for continual refinement of the individual networks, as nature does for biological networks, will require new analysis tools. Here we have taken concrete step towards the development of such tools, not only through our theoretical results, but also through their application to create stable combination network architectures that perform well in practice on benchmark tasks.
10

---

Acknowledgments and Disclosure of Funding
This work benefited from stimulating discussions with Michael Happ, Quang-Cuong Pham, and members of the Ficte lab at MIT.
References
Larry Abbott and Karel Svoboda. Brain-wide interactions between neural circuits. Current Opinion in Neurobiology, 65:iii-, Dec 2020. ISSN 09594388. doi: c0.10016..cmmb22200122.2...
William Ashby. Design for bruin: The origin of adaptive behaviour. Springer Science & Business Media, 1952.
Bernard Baars. cognitive theory of consciousness. Cambridge University Press, 1993.
Shaojie Bai, Zico Kolter, and Vladlen Koltun. Trellis networks for sequence modeling. ICLR. 2019.
Nicholas M Boffi, Stephen Tu, Nikolai Matni, Jean-Jacques E Slotine, and Vikas Sindhwani. Learning stability certificates from data. CoRL, 2020.
Michael Buehner and Peter Young. A tighter bound for the echo state property. IEEE Transactions on Neural Networks, 17(3):820-83 824, 2006.
Thiago B Burghi, Timothy O' O'Leary, and Rodolphe Sepulchre. Distributed online estimation of biophysical neural networks. arXiv preprint arXiv:2204.01472, 2022.
Veronica Centorrino, Francesco Bullo, and Giovanni Russo. Contraction analysis of hopfield neural networks with hebbian learning. arXiv preprint arXiv:2204.05382. 2022.
Bo Chang, Minmin Chen, Eldad Haber, and Ed H Chi. Antisymmetricrnn A dynamical system view recurrent neural networks. arXiv preprint arXiv:1902.0963: 2019.
Krzysztof M Choromanski, Jared Quincy Davis, Valerii Likhosherstov, Xingyou Song, Jean-Jacques Slotine, Jacob Varley, Honglak Lee, Adrian Weller, and Vikas Sindhwani. Ode an ode. Advances in Neural Information Processing Systems, 33:3338-3333 2020.
Soon-Jo Chung and Jean-Jacques E Slotine. Cooperative robot control and concurrent synchronization of lagrangian systems. IEEE transactions on Robotics, 25(3):686-700, 2009.
Yimian Dai, Stefan Ochmeke, Fabian Gieseke, Yiquan Wu, and Kobus Barnard. Attention as activation. arXiv preprint arXi::2007.0772912 2020.
Peter Dayan and Laurence F Abbott. Theoretical neuroscience: computational and mathematical modeling of neural systems. 2005.
Stanislas Dehaene, Michel Kerszberg, and Jean-Pierre Changeux. A neuronal model of global workspace in effortful cognitive tasks. Proceedings of the national Academy of Sciences, 95(24): 14529-14534, 1998.
Rainer Engelken, Fred Wolf, and Larry F Abbott. Lyapunov spectra of chaotic recurrent neural networks. arXiv preprint 20Xiv:2006.02427, 2020.
N. Benjamin Erichson, Omri Azencot, Alejandro Queiruga, Liam Hodgkinson, and Michael W. Mahoney. Lipschitz recurrent neural networks. ICLR, 2021.
Jean Gallier al. The schur complement and symmetric positive semidefinite (and definite) matrices (2019). URL https:///ww. cis. upenn. edu/jean/schur-come pdf, 2020.
John Gerhart and Mare Kirschner. The thcory of facilitated variation. Proceedings of the National Academy of Sciences. 104(1):8582-858) 2007. doi: 10. 10.1073//mas.000113551000
Anirudh Goyal, Aniket Didolkar, Alex Lamb, Kartikeya Badola, Nan Rosemary Ke, Nasim Rahaman, Jonathan Binas, Charles Blundell, Michael Mozer, and Yoshua Bengio. Coordination among neural modules through shared global workspace. arXiv preprint 200.07710 2021.
11

---

Alex Graves, Santiago Fernández, and Jürgen Schmidhuber. Multi-dimensional recurrent neural networks. International conference on artificial neural networks, pages 549-558. Springer, 2007.
Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. ICLR, 2022.
Eldad Haber and Lars Ruthotto. Stable architectures for deep neural networks. Inverse Problems, 34 (1):014004, 2017.
John J Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 7955):2554-2558 1982.
Herbert Jaeger. The "echo state" approach to analysing and training recurrent neural networks-with an erratum note. 2001.
Saber Jafarpour, Alexander Davydov, Anton Proskumikov, and Francesco Bullo. Robust implicit networks via non-euclidean contractions. Advances in Neural Information Processing Systems, 34: 9857-9868, 2021.
Eric R Kandel, James Schwartz, Thomas M Jessell, Steven Siegelbaum, A James Hudspeth, and Sarah Mack. Principles of neural science, volume 4. McGraw-hill New York, 2000.
Hassan K Khalil. Nonlinear systems, volume 2002.
Hiroaki Kitano. Biological robustness. Nature Reviews Genetics, 5(1111):826-337, Nov 2004. ISSN 1471-0064. doi: 10. 1038//8n 1471.
Leo Kozachkov, Mikael Lundqvist, Jean-Jacques Slotine, and Earl Miller. Achieving stable dynamics in neural circuits. PLoS computational biology, 1608)_e 2020.
Leo Kozachkov, John Tauber, Mikael Lundqvist, Scott L. Brincat, Jean-Jacques Slotine, and Earl K Miller. Robust working memory through short-term synaptic plasticity. bioRxiv, 2022a.
Leo Kozachkov, Patrick M Wensing, and Jean lean-Jacques Slotine. Generalization supervised learning through riemannian contraction. arXiv preprint arXiv:2201.06655 2022b.
Dmitry Krotov and John Hopfield. Large associative memory problem in neurobiology and machine learning. arXiv preprint arXiv:2008.06900 2020.
Christopher Langdon and Tatiana Engel. Latent circuit inference from heterogeneous neural responses during cognitive tasks. bioRxiv, 2022.
Quoc V Le, Navdeep Jaitly, and Geoffrey E Hinton. simple way to initialize recurrent networks of rectified linear units. arXiv preprint 3xXiv15044.099 2015.
Mario Lezcano-Casado and David Martinez-Rubio. Cheap orthogonal constraints in neural networks: A simple parametrization of the orthogonal and unitary group. In International Conference on Machine Learning, pages 3794-3803. PMLR, 2019.
Winfried Lohmiller and Jean-Jacques E Slotine. On contraction analysis for non-linear systems. Automatica, 34(6):683-666, 1998.
Timothy A. Machado, Isaac V. Kauvar, and Karl Deisseroth. Multiregion neuronal activity: the forest and the trees. Nature Reviews Neuroscience, page 1-22, Oct 2022. ISSN 1471-0048, doi: 10. 10.10388/411533-02--0066
Ian R Manchester and Jean-Jacques E Slotine. Control contraction metrics: Convex and intrinsic criteria for nonlinear feedback design. IEEE Transactions on Automatic Control, 62(6): 30563,,3 2017.
George A Mashour, Pieter Roelfsema, Jean-Pierre Changeux, and Stanislas Dehaene. Conscious processing and the global neuronal workspace hypothesis. Neuron, 105(5):776-798, 2020.
12

---

Kiyotoshi Matsuoka. Stability conditions for nonlinear continuous neural networks with asymmetric connection weights. Neural networks, 5(3):495-500, 1992.
Jonathan A Michaels, Stefan Schaffelhofer, Andres Agudelo-Toro, and Hansjörg Scherberger. A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping. Proceedings of the national academy of sciences, 1 117/50):32124-32133 2020.
Earl K Miller and Jonathan D Cohen. An integrative theory of prefrontal cortex function. Annual review of neuroscience, 24(1):167-202, 2001.
John Miller and Moritz Hardt. Stable recurrent models. arXiv preprint arXiv: 10369, 2018.
Kenneth Miller and Francesco Fumarola. Mathematical equivalence of two common forms of firing rate models of neural networks. Neural computation, 24(1):25-311 20,
Brendan K Murphy and Kenneth D Miller. Balanced amplification: a new mechanism of selective amplification of neural activity patterns. Neuron, 61(4):635-648, 2009.
Kumpati S. Narendra and Robert Shorten. Hurwitz stability of metzler matrices. IEEE Transactions On Automatic Control, (5(6))1444418 1487, 2010.
Allen Newell and Herbert Alexander Simon. Human problem solving, volume 104. Prentice-hal hall Englewood Cliffs, NJ, 1972.
Merav Parter, Nadav Kashtan, and Uri Alon. Facilitated variation: How evolution learns from past environments to generalize new environments. PLOS Computational Biology, 4(11), 2008
Matthew G. Perich, Charlotte Arlt, Sofia Soares, Megan E. Young, Clayton P. Mosher, Juri Minxha, Eugene Carter, Ueli Rutishauser, Peter H. Rudebeck, Christopher D. Harvey, and Kanaka Rajan. Inferring brain-wide interactions using data-constrained recurrent neural network models. 2021. doi: 10. 101/2020. 12.1..2.....2....... 423348. URL https: /wwww//wwwww////////// biorxiv. org/ content/eer11//221/ 03/11 03/11/2020.1 12. 18. 423348.
Quang-Cuome Pham and Jean-Jacques Slotine. Stable concurrent synchronization in dynamic system networks. Neural networks, 20(1):62-77 2007.
Adityanarayanan Radhakrishnana, Mikhail Belkin, and Caroline Uhler. Overparameterized neural networks implement associative memory. PNAS, 117:27122-27170, 2020.
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008.02217 2020.
Max Revay and Ian Manchester: Contracting implicit recurrent neural networks: Stable models with improved trainability. In Learning for Dynamics and Control, pages 393-403, PMLR, 2020.
Max Revay, Ruigang Wang, and Ian Manchester. Lipschitz bounded equilibrium networks. arXiv e-prints, pages ar 2r10 2020.
Max Revay, Ruigang Wang, and Ian Manchester. Recurrent equilibrium networks: Unconstrained learning of stable and robust dynamical models. arXiv preprint arXiv:2104.0594. 2021.
Spencer M Richards, Felix Berkenkamp, and Andreas Krause. The Iyapumov neural network: Adaptive stability certification for safe learning of dynamical systems. In Conference on Robot Learning, pages 466-476. PMLR, 2018.
Ivan Dario Jimenez Rodriguez, Aaron Ames, and Yisong Yue. Lyanet: A lyapunov framework for training neural odes. In International Conference Machine Learning, pages 18687-18703. PMLR. 2022.
David W. Romero, Anna Kuzina, Erik J. Bekkers, Jakub M. Tomezak, and Mark Hoogendoomn Ckconv: Continuous kernel convolution for sequential data, arXiv preprint arXiv:2 102.2200200000 102.0202 /vl, 2021.
13

---

Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib J Majaj, Rishi Rajalingham, Elias B Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Franziska Geiger, et al. Brain-score: Which artificial neural network for object recognition is most brain-like? BioRxiv, page 407007, 2020.
João D Semedo, Amin Zandvakili, Christian K Machens, M Yu Byron, and Adam Kohn. Cortical areas interact through a communication subspace. Neuron, 102()):249-259 2019.
H Sebastian Seung. How the brain keeps the eyes still. Proceedings of the National Academy of Sciences, 93(23):13339-1334, 1996.
David Silver, Aja Huang, Chris Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittiesee,, Ioannis Antonogiou, Veda Panneershelvam,, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.
Herbert A. Simon. The architecture of complexity. Proceedings of the American Philosophical Society, 106(6):467-482, 1962. ISSN 0003-049X.
Jean-Jacques Slotine. Modular stability tools for distributed computation and control. Int. J. Adaptive Control and Signal Processing, 17(6), 2003.
Jean-Jacques Slotine and Yang- Yu Liu. The missing link. Nature Physics, 8(7):512-513 2012.
Jean-Jacques Slotine and Winfried Lohmiller. Modularity, evolution, and the binding problem: view from stability theory. Neural Networks, 14(2):137-445, 2001.
Jean-Jacques E Slotine and Weiping Li. Applied Nonlinear Control. Prentice-Has Hall, 1991.
Haim Sompolinsky. Andrea Crisanti, and Hans-Jurgen Sommers. Chaos in random neural networks. Physical review letters, (3):2299 1988.
Eduardo D Sontag. Contractive systems with inputs. In Perspectives in mathematical system theory, control, and signal processing, pages 217-228. Springer, 2010.
David Sussillo and Omri Barak. Opening the black box: low-dimensional dynamics in high- dimensional recurrent neural networks. Neural computation, 25(3):626-64)) 6499 2013.
Yaroslav Sych, Aleksejs Fomins, Leonardo Novelli, and Fritjof Helmchen. Dynamic reorganization of the cortico-basal ganglia-thalamoocortiiaa network during task learning. Cell Reports, 40(12): 11139,, 2022.
Nicolas Tabareau and -aan-eacuess Slotine. Notes on contraction theory. arXiv preprint mlin/0601011, 2006.
Trieu H. Trinh, Andrew M. Dai, Minh-Thang Luong, and Quoc V. Le. Learning longer-term dependencies in rmss with auxiliary losses. arXiv preprint arXiv:1803.00 40144v3, 2018.
Rufin VanRullen and Ryota Kanai. Deep learning and the global workspace theory. Trends in Neurosciences, 44(9):692-704, 2021.
Ryan Vogt, Maximilian Puelma Touzel, Eli Shlizerman, and Guillaume Lajoie. On Yyapunov exponents for mns: Understanding information propagation using dynamical systems tools. arXiv preprint arXiv:2006. 2020.
Eric W. Weisstein. Positive definite matrix. PosittiveDefinnttlaati...k.
URL https://mathworld. holfram.c com/
Patrick M Wensing and Jean- Slotine. Beyond convexity-contractioo and global convergence of gradient descent. Plos one, 15())e0236661, 2020.
Norbert Wiener. Cybernetics or Control and Communication in the Animal and the Machine. MIT press, 1948.
Alex Williams, Erin Kunz, Simon Kornblith, and Scott Linderman. Generalized shape metrics on neural representations. Advances in Neural Information Processing Systems, 34:4738-477 2021.
14

---

Hugh R. Wilson and Jack D. Cowan. Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical Journal, 12(1):1-24, 1972. ISSN 0006-3495, doi: https://doi.org
00.0110500062244(7786666554
Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. serformance-ottmmiize hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 1 111(23):8619 8624, 2014.
Guangyu Robert Yang and Manuel Molano-Mazón. Towards the next generation of recurrent network models for cognitive neuroscience. Current opinion in neurobiology, 70:182-192, 2021.
Guangyu Robert Yang, Madhura R Joglekar, H Francis Song, William T Newsome, and Xiao-Jing Wang. Task representations in neural networks trained perform many cognitive tasks. Nature neuroscience, (((2):977-366 2019.
15

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you descrihe the limitations of your work? [Yes] See Supplementary Section B.1
(c) Did you discuss any potential negative societal impacts of your work? [No] There are no unique potential negative societal impacts of this work
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See mathemat- ical details within the paper, Section 2 and Section 3
(b) Did you include complete proofs of all theoretical results? [Yes] See Supplementary Section C
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material as URL)? [Yes] Our code is linked in Supplementary Section D.1.2, and can run directly on Google Colab. We used only extremely common benchmark image datasets in experiments, which can be utilized directly via the provided Colab link.
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] All initialization and training details are reported in Supplemen- tary Section D.1.1. The results for all hyperparameter tuning trials are included in Supplementary Section D.4
(c) Did you report error bars (c.g., with respect to the random seed after running exper- iments multiple times)? [Ys]] Where applicable in the main text, reported mean, min, max, and variance test accuracy across our repeated trials (see Section 4.2.3 in particular). We also provide extended information repeatability tests in Supplemen- tary Section D.2.3, and tables that report the results each individual trial we ran in Supplementary Section D.4
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See information on the code in Supplementary Section D.1.2
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you eite the creators? [Yes] We cite the article that introduced the sequential image classification tasks when introducing them Section
(b) Did you mention the license the assets? [Yes] We include the licenses for MNIST and CIFARIO datasets part of Supplementary Section D.1.2
(c) Did you include any new assets either the supplemental material or as URL? [Yes]
Our code is linked in Supplementary Section D.1.2
(d) Did you discuss whether and how consent obtained from people whose data you 're
using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
16


Reviews:
[{"review_id": "NRjYdpPdJWR", "reviewer": "Reviewer_tmsC", "summary": "The authors analyze a model of interconnected recurrent neural networks, trained to perform various tasks. They emphasize the importance of network stability to obtain good performance, and analyze it using  tools from contraction analysis. Specifically, they derive conditions under which a network of RNNs can remain stable. These, in turn, are used to parametrize the network for a training protocol that preserves stability during training.\nThis setup is used to test two different variants of a network of networks on the sequential MNIST and CIFAR tasks. Performance is higher than several similar models. Finally, the effect of hyper-parameters such as sparseness is numerically explored. \n", "questions": "1.\tBenchmarks. I am not an expert in benchmarks of these form, but a quick search resulted in what seems like an RNN with a similar number of parameters and better performance (https://paperswithcode.com/paper/parallelizing-legendre-memory-unit-training/review/?hl=27784). Again - this is simply the first result I found.\n2.\tL289-290 “stability and modularity unexplored”. For instance: Aljadeff, Stern, Sharpee PRL 2015.\n3.\tL74-75. The linear coupling between nonlinear networks is not an intuitive choice from a neuroscience perspective. It is claimed that the nonlinear case will also be considered, but I didn’t find it.\n4.\tEquation 2 : is it x_i or x_j ?\n5.\tYou show that modular networks learn better when initialized sparsely - this is an interesting result (perhaps related to the advantage of sparse connectivity in reservoir networks). Unfortunately, no intuition or analysis of why this is the case is provided.\n\n", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "As recording from multiple brain areas simultaneously becomes more feasible in experiments, it is timely to study how networks of networks can collaborate to perform more complex and higher-level computations. This paper brings to the front the importance of making sure that certain properties of RNNs are retained when they are combined and assembled into a larger network. \nThe topic is thus timely and important.\nStability of trained networks has been recognized as important for performance (e.g., Sussillo & Abbott 2009), but mainly studied in the reservoir computing framework - where there is feedback to a single network. The current work extends this framework to the scenario of multiple networks.\n\nThe main weaknesses are the difficulty to understand the main objective of the paper, and a lack of reference to existing literature.\nThe motivation in the abstract is to better understand multi-area RNNs as models in neuroscience. It isn’t clear why benchmarks serve this purpose. On the other hand, if this is a proposal for a new ML architecture, it seems that performance isn’t better than state of the art.\nFurthermore, the improved performance in the modular case isn’t accompanied by any analysis of the resulting networks. Do different modules handle different parts of the computation? Is the optimal number of modules related to the task? If so, how? These questions are much more relevant to the neuroscience motivation, but are not explored at all.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "KHJKswwDcwu", "reviewer": "Reviewer_vKbB", "summary": "The authors consider the behaviour of networks of artificial recurrent neural networks (RNNs), specifically the stability of RNNs of RNNs. This is achieved using nonlinear control theory, specifically to consider the contraction properties of the networks (where a contracting system is one in which it's long-term behaviour is exponentially independent of initial conditions). \n\n", "questions": "Is it possible to discern the role of different sub-networks in forming learned representations? What is the effect of ablating sub-networks during training?  \n\nHow well does the results on stability and contractivity apply to other network architecture? Could a similar theoretical analysis be applied to other recurrent models e.g. neural cellula automata? ", "limitations": "It wasn't immediately clear to me how well their results generalise to other network architectures. I also note this as a question to the authors above. ", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors identify that understanding networks of neural networks is significant both for using all available neural data in neuroscience as well as for new deep learning architecture frameworks. They use nonlinear control theory to analyze the contraction of networks of RNNs. I believe that this, in itself is novel and is further improved on by them deriving several original results. They also review several recent results in machine learning theory, such as the sufficient conditions for stability of nonlinear systems and give clarification to their applicability. The authors implement a form of RNNs of RNNs which are expected to satisfy appropriate contractivity, given their theoretical exposition. They find strong performance when compared to other baselines. \n\nOverall, I did not find signficant weaknesses. The manuscript was well structured, on an original topic, and the authors were thorough in their investigation. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "IDAxhiXNYQE", "reviewer": "Reviewer_3ViV", "summary": "With the increasing ability of experimental neuroscientists to obtain simultaneous recordings of brain activity from multiple areas of the brain, it is important to develop computational theory for RNNs that can model such situations. In this context, the submission is focused on the question of the stability of networks of multiple RNNs that are connected and in particular the question of contractive stability. The paper presents a network of networks model that involves multiple continuous time rate-based vanilla RNNs whose state updates depend on the states of the other RNNs.  A parameterization of negative feedback between the multiple RNNs is presented which allows for optimization using gradient-based techniques and it is proved that this ensures contractive stability of the combined system (assuming each individual RNN is contracting). In addition, multiple sufficient conditions are presented to ensure contractive stability of the individual RNNs. \n\nTwo particular instantiations of the networks of networks model based on two of the stability results are empirically evaluated on the seqMNIST, permuted seqMNIST and seqCIFAR sequence classification tasks to determine if the stability constraints impede expressivity. One of the variants outperforms other recent stable RNNs on both permuted SeqMNIST and seqCIFAR and outperforms an LSTM baseline on all three. In addition, experiments are performed to study how the performance increases as more networks are added and the sparsity is adjusted.\n\n", "questions": "1. There seems to be a close connection with multidimensional RNNS (Graves 2007) and multidimensional state space models (https://en.wikipedia.org/wiki/Multidimensional_system), where the update equations for each of the multiple states of the system depend on the other states. Perhaps a more detailed discussion of this connection would be useful for the readers and potentially broaden the reach of this work? There could be relevant ideas in the multidimensional systems theory literature as well. \n\n2. Lines 74-75 say that later on extensions to nonlinear couplings will be discussed. This sounded interesting, but unless I am missing something, the only mention of this I saw was the last sentence of line 107, which I am not sure I would consider a discussion.  I would recommend either rewording lines 74-75 where you are not overpromising or expanding the discussion (or referring to its location more explicitly if I am missing it somewhere else).\n\n3. The freezing of the weights of the Sparse Combo Net has a connection to Echo State Networks which might also be included in a related works section.\n\n4. Why are the results for the SVD Combo net not presented in the results Table 1? It seems strange to specifically discuss this network and then not present the results. Some of the results (though I do not think all) appear to be in the text and appendix, but it seems including this in the table would reduce confusion.\n\n5. Why were only the two variants determined by Theorem 1 and 5  (Sparse combo net and SVD combo net) considered instead of other variants based on the other theorems as well? Unless I missed this discussion in the appendix, a more detailed discussion of this choice might be helpful.\n\n6. Section 4.2.1 discusses adjusting the network size. Increasing the number of modules increases the number of parameters, so one natural question would be how does the performance of 2 size 32 modules compare to the performance of 1 size 64 module, and so on...? It is somewhat unclear to me what this would look like for the Sparse combo net since many of the parameters are not trained, but it seems natural for the SVD combo net. \n\nMinor points:\n\n-Line 194-195 says \"only trained the connections between subnetworks (Figure 2A)\" but I think this should say Figure 2B (the sparse combo net)\n\n-Line 195 says \"we trained all parameters of the model (Figure 2B)\" but I think this should be Figure 2C (the SVD Combo net)\n\n-There are few places where being a bit more explicit about the shapes of some of the variables might help. For example, in Corollary 1 in the appendix, I think you would benefit from more explicitly stating that L is a matrix made up of matrices (perhaps explicitly providing the general dimensions) where each $L_{ij}$ is a matrix. Similarly, explicitly saying u(t) is made up of $u_i(t)$ could be helpful since in the main text u(t) is only defined for eq. 1 (a single RNN). Alternatively, you could maybe define $\\tilde{u}(t)$ analogously to $\\tilde{x}$. ", "limitations": "The authors discuss limitations in an extended discussion in the appendix.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "-Originality: While the proposed network of RNNs appears to fall within the class of multidimensional RNNs and multidimensional state space models (though see Question #1 below for clarification on this point) and the idea of using negative feedback to ensure stability of combined systems is well-known in the nonlinear systems literature, this appears to be the first work to study in detail how to parameterize these stability conditions in a way that is straightforward to optimize using deep learning techniques. In addition, the conditions derived to ensure contractive stability of a single RNN appear to be novel. It seems however that the paper would benefit from a more detailed related works section to help the reader have a better survey of where this work fits into the literature.\n\n-Quality: The theoretical results appear to be sound. The empirical results seem to support the basic claim that some of the theoretical stability conditions can be used in a way that does not severely inhibit the expressivity of the network. However, I have some lingering questions related to these results that could be clarified. See Questions section below. In addition, the major motivation for the approach in the paper is for modeling neuroscience data recorded from multiple brain areas, so some kind of modeling experiment related to this (even if synthetic) would strengthen the submission.\n\n-Clarity: The paper is in general well-written and clear. See Questions sections below for a few detailed suggestions for improvement.\n\n-Significance: This paper will be of interest to the computational neuroscience community as well as the general machine learning community interested in the stability of RNNs.  This paper seems to have connections to multidimensional RNNs and a more thorough discussion of this connection could broaden the reach to people interested in modeling multidimensional data.  The theoretical stability results could help guide future development of these methods. In addition, in my experience seqCIFAR is still a challenging task for most methods and good performance on this task tends to translate to good performance on other tasks requiring the modeling of long-range dependencies. So the fact that these combined networks perform better than many of the baseline methods is quite interesting.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "G34xXgOT_S", "reviewer": "Reviewer_HWhw", "summary": "This paper tries to address the stability issue of combining multiple stable RNNs. The motivation is that the stability of a single RNN is well studied, but the stability of this kind “RNNs of RNNs” is little studied, particularly, as compared to the recent rising research direction in neuroscience. The “RNNs of RNNs” is defined in eq.(2), which considers the simplest case: linear combination.  The authors provided some theoretical justification to prove that the global stability of “RNNs of RNNs” can be achieved. The authors tested one instance network: stability constrained network on sequential MNIST and Cifar10, showed that good performance can be achieved as compared to baselines.", "questions": "1. What is the \"stability\" of RNN meaning? it should be given and explained in a better way at the beginning of the paper. This causes confusion in the later stage.\n2. The term \"RNNs of RNNs\" is confusing, in eq (2), the connection between RNNs is not recurrent/recursive, but linear connection. \n3. Given the linear combination of RNNs, what is the difference to ensemble RNNs?\n4. How the empirical results support the main claims? Good accuracy presents stability? \n\n5.There are many methods give better performance on mentioned datasets, just give a few examples:   \n[1]Chang et al, Dilated Recurrent Neural Networks, NeuriPS 2017. \n\n[2] Tim Cooijmans et al, Recurrent Batch Normalization, ICLR 2017. \n\n[3] Wang et al, State-Regularized Recurrent Neural Networks, ICML 2019. \n\n[4] Trinh et al, Learning longer-term dependencies in rnns with auxiliary losses, ICML 2018. \n", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The core idea of studying RNN contracting stability is interesting as well as the connection to neuroscience.\n2. Some theoretical findings would be helpful in understanding the “network of network” stability.\n\nWeaknesses:\n1. There are some unclear points, which makes this paper become less and less understandable in the later stage.\n2. The experimental section is weak. Experiments on harder tasks such as language modelling, text classification, would make this section much stronger. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept