Learning rule influences recurrent network representations but not attractor structure in decision-making tasks
Authors: Brandon J McMahan
Michael Kleinman
Jonathan Kao
Venue: NeurIPS 2021 Poster (2021)
Keywords: RNN
learning rules
hyperparameters
RNN dynamics
decision-making
computational neuroscience
interpretability
biological models

Abstract: Recurrent neural networks (RNNs) are popular tools for studying computational dynamics in neurobiological circuits. However, due to the dizzying array of design choices, it is unclear if computational dynamics unearthed from RNNs provide reliable neurobiological inferences. Understanding the effects of design choices on RNN computation is valuable in two ways. First, invariant properties that persist in RNNs across a wide range of design choices are more likely to be candidate neurobiological mechanisms. Second, understanding what design choices lead to similar dynamical solutions reduces the burden of imposing that all design choices be totally faithful replications of biology. We focus our investigation on how RNN learning rule and task design affect RNN computation. We trained large populations of RNNs with different, but commonly used, learning rules on decision-making tasks inspired by neuroscience literature. For relatively complex tasks, we find that attractor topology is invariant to the choice of learning rule, but representational geometry is not. For simple tasks, we find that attractor topology depends on task input noise. However, when a task becomes increasingly complex, RNN attractor topology becomes invariant to input noise. Together, our results suggest that RNN dynamics are robust across learning rules but can be sensitive to the training task design, especially for simpler tasks.

Learning rule influences recurrent network representations but not attractor structure in decision-making tasks
Brandon McMahan Michael Kleinman Jonathan C. Kao Department of Electrical and Computer Engineering
University of California, Los Angeles
(bmcmahan2025, michael weleeimmmn@ucll....m
kaoêseas www..clllee...
Abstract
Recurrent neural networks (RNNs) are popular tools for studying computational dynamics in neurobiological circuits. However, due the dizzying array design choices, it is unclear if computational dynamics unearthed from RNNs provide reliable neurobiological inferences. Understanding the effects of design choices on RNN computation is valuable in two ways. First, invariant properties that persist in RNNs across wide range of design choices are more likely to be candidate neurobiological mechanisms. Second, understanding what design choices lead to similar dynamical solutions reduces the burden of imposing that all design choices be totally faithful replications of biology. We focus our investigation learning rule and task design affect RNN computation. We trained large populations of RNNs with different, but commonly used, learning rules
how RNN
tasks inspired by neuroscience literature. For relatively complex tasks, we find that attractor topology is invariant the choice of learning rule, but representational geometry is not. For simple tasks, we find that attractor topology depends on task input noise. However, when task becomes increasingly complex, RNN attractor topology becomes invariant to input noise. Together, our results suggest that RNN dynamics robust across learning rules but be sensitive the training task design, especially for simpler tasks.
Introduction
Computational neuroscientists have increasingly used recurrent neural networks (RNNs) to model cortical computation and gain insight into how the brain performs cognitive and motor tasks. Typically, neuroscientists design RNNs to model a particular brain region during a behavioral task by generating an artificial dataset that abstracts the task. RNNs are trained to perform this task and often exhibit similar representations and low-dimensional dynamics to neural population recordings. This similarity is used to argue that the RNN is good model of the brain circuit's 's computations and neural dynamics. In this way, RNNs have been successfully used to study cognitive tasks, including decision-making and working memory [17]. well motor tasks (8H12). Because RNN parameters are fully observed, trained RNNs can be analyzed to propose new hypotheses for how these computations are carried out in the brain [13]
One potential concern of this approach is that there are significant differences between RNNs and biological circuits. Further, RNNs may exhibit different representations and dynamics result of design choices, such hyperparameters and architectures. Maheswaranathan. Williams, al. 14 investigated whether there were any universal properties across recurrent architectures and activation functions. They found through numerical study that RNNs tend to exhibit different representations across architectures (RNN, GRU, LSTM, UGRNN) and activation function (relu, tanh)
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

but exhibited universality in attractor topologies. These results encouragingly suggest that, although RNN architecture and activation function may result in different representations, the dynamical mechanisms of these networks appear to be conserved irrespective of these choices.
A similarly important aspect of RNN modeling is how networks are trained, which differ in nature from biological neural circuits. Neural circuits are believed to employ Hebbian learning princi- ples [15] and while it is possible that they may employ a backpropagation-lic algorithm, typical training approaches in RNNs are not necessarily biologically plausible [6 17]. For example, backpropagation-tie -trrughh marough-time -time assumes global knowledge of all the parameter weights in the network. Computational neuroscience studies utilize a diverse range of learning algorithms. from stochastic gradient descent with packpropagation-trros
time T 5 6 [9 lebbian-inspired learning [8], FORCE learning [19 4, [20]. and evolutionary learning through genetic algorithms (2T]. Dif- ferent learning rules result in different learning trajectories through parameter space, leading to the possibility that RNN computation is learning rule dependent. If learning rule affects the learned solution, then conclusions of computational neuroscience studies may depend on the experimenter- chosen learning rule. We therefore sought to empirically study the similarities and differences in RNN representations and dynamics across learning rules.
In particular, we study the topological structure of RNN attractor (or fixed point) states and the representational geometry following training with different learning rules. We find that all learning rules resulted in similar attractor topologies across different decision -akiin tasks. However, while we find evidence that the attractor structure is invariant to the choice of learning rule, it is not invariant to task hyperparameters. We found that task hyperparameters for the contextual integration task from different studies resulted in RNNs with different attractor structure and dynamical mechanisms. Interestingly, we also find that as we train RNNs on progressively more complex tasks, the attractor topology remains invariant to the choice of learning rule even though representational geometries become more individualistic. The results of our study therefore provide insights into what properties of RRNN are likely conserved across design choices, which in turn supports the study of RNNs to probe dynamics in biological neural circuits.
2 Methods
2.1 Model Architecture
We trained continuous time RNNs, which when discretized via Buler's method, evolve as
X1+1 (1 a)xx + a (Xx, + U:)
where x, E R250 is the pre-activation state, u, IR. M is the input to the RNN, f(-) is the activation function, and a At/T is the Euler time step, At, divided by the network time constant, т. When 1, the continuous time RNN simplifies to a vanilla RNN. Unless noted otherwise, the activation function is f()) = 1 + tanh(-), and we define r, (xx). We shifted the tanh(-) by 1 so that its output is always positive, as a non-negative activation is typically used for computational neuroscience studies 2 The RNN output y. is linear readout of the hidden states,
Y: = W fx.:.
(2)
W rec was initialized with weights drawn from N00, {) where g = 1 is the network gain and N = 50 the number of artificial neurons in the RNN. Win weights were initialized from (O, 0.5). W out were initialized from (0, 0.1). At the beginning of all trials, the initial hidden state Xg E R was drawn from A (0, 0.3). All networks were trained using the mean- mean-squared error (MSE) loss. The tasks and learning rules are described in the next sections. In total, we trained and analyzed 1,920 RNNs this numerical study]] []]
2.2 Tasks
We used three tasks based on commonly studied perceptual decision making tasks from the neuro- science literature [22 18).
Training was carried out in approximately 7 days on an RTX 2080. H and FF RNNs were trained using CPUs both locally and on AWS

---

Random dots motion (RDM): RNNs received a I-dimensional Gaussian noise imput [22 For each trial, we drew a random number μ U[-0.1857, +0.18577, which modeled the "coherence" of a traditional RDM stimulus [23 I. At each timestep. / = 0, 1...., 750 ms, the input to the network was drawn from u, ~ Ni, 1). The RNN was constrained to output y; = 0 at = 0 and y; = 1 (or Y: 1) at = 750 ms if was positive (negative). These input and output values were chosen to be consistent with a previously implemented context-dependent integration task, described next [.]
Context-dependent integration (CDI): We implemented the CDI task in the same way as M. RNNs received two static binary context inputs and two time-varying Gaussian noise inputs. For each trial, one context input was 0 and the other was I, providing stream should be integrated to compute the output. For each trial, two Gaussian noise inputs where both independently sampled from N (1g, 1) and (Hz2, 1) where μ1 and #2 where both independently sampled from 0 0.1857]. Samples were drawn at = 0.,,.... 750 ms. RNNs were trained to output zero at t 0 and 1 at i = 750 ms depending on the sign of ja for the relevant input.
me-hot encoding of which Gaussian input
We generalized the CDI task by considering N noisy input streams and N static context inputs, which we call the N-CDI task. Only one of the N static context inputs was I, indicating which input should be integrated compute the output. Each noisy input stream was generated the same way as described for the CDI task. As in the CDI task, the RNN was trained to output zero at O, and #1 at 750 ms depending on the sign of μ for the relevant input. We trained networks from N = 2 to N = 6 contexts.
N-input context-dependers
integration (N-CDI):
2.3 Learning Rules
We trained RNNs with four different learning rules: stochastic gradient descent with gradients computed using backpropagation-tte wowppaagatiotttruug--iiit through-time (BPTT), a genetic learning rule (GA) [21] a Hebbian (H) learning algorithm (18), and the Full Force (FF) algorithm [20]. For all learning rules, training was terminated when accuracy on a validation dataset of 2,000 trials exceeded 90%.
BPTT: Gradients were computed using backpropagation through time over the entire trial length. After computing gradients, we updated parameters using the Adam optimizer with a mini-batch size of 500 trials and a learning rate of 0.0005.
GA: Each iteration of the GA involves producing more accurate generation of RNNs from the prior generation. We started with an initial population of 50 RNNs with random initializations as described Section 2.1, termed generation zero. We then computed the loss of generation zero RNNs over 500 trials. The five RNNs with the lowest loss were then used to produce the next generation (generation one). The next generation of RNNs was constructed by randomly choosing one of the five best RNNs from the prior generation and applying small amount of noise to all the weight matrices. The noise was drawn from N (0, 0.005). This resulted in new RNNs. The original RNNs (without perturbation) were also included in the next generation. This process was repeated until our validation criterion was met. The final RNN was the RNN with the lowest MSE in the final generation.
H: We used the learning rule described in Miconi [8]. In brief, during training trials, all weights, accumulated an eligibility potential. The weight Wiyj which the (i. element of W Wree had an eligibility potential Cij, given by,
((,))= (r)(t11) 1) (1) 2.)) 3
(3)
where ry is the activation of artificial unit j and x; is the pre-activation value of artificial unit T. designated the average pre-activation value of artificial unit i across the trial. At the end of each trial, the weights were updated according to the cligibilities and the discrepancy between the current trial loss. R, and running average of past losses for this trial type, R.
Vw,, = me., (R- RR)
for a learning rate, 17 = 0.01. This learning rule therefore alters synapses (weights) when the activity of post-synaptic and pre-synaptic neurons correlated during training, mimicking Hebbian plasticity. In this learning rule, the RNN output is a readout of a single unit [18). These Hebbian networks used /(-) tanh(-), since the shifted + tanh(-) resulted in training difficulties.
FF: We used the learning rule proposed by DePasquale et al. 20 In short, this learning rule uses recursive least squares algorithm to minimize the difference between the RNN's s output and the
3

---

target output. Additionally, recurrent activations are constrained to match the activations of a second randomly initialized RNN that receives both task imputs and outputs. These FF networks used Ι(+) tanh(.), since the shifted 1 + tanh(.) resulted in training difficulties.
2.4 Finding RNN attractor states
To study the dynamical mechanisms used by RNNSs we computed their attractor states. An attractor state is an activity state x' E R where X X,, We therefore solved for attractor states by optimizing over x" satisfying Equation []]
x' = (1-aax a (W (weee((")) Win
(5)
Because x" depends on the input u, to the RNN time t, set the input to be static value when solving for attractor states. We solved for these fixed points by using a numerical solver, based on the algorithm [24]. Due to numerical precision,
termed any states satisfying [|x++1 x+Il < e to be an attractor, where ε = 10- In some analyses, we also computed slow points along line attractor, which corresponded to states with = 1. After finding fixed points, clustered them based on their topology, described further in Section 2.6.
2.5 SVCCA to assess similarity of RNN activity
We quantified how similar the representational geometry, that is, the activity of two different RNNs, was to the same inputs using singular value canonical correlation analysis (SVCCA) as described in Raghu al. (25). SVCCA was also used by Maheswaranathan, Williams, et al. [44 quantify differences and similarities in representational geometry between RNNs. To compute the SVCCA, we provided multiple test inputs the RNN. RNN activity was then projected onto its top K principal components (PCs), with K chosen so that the PCs explained over 95% of the variance. Canonical correlation analysis (CCA) was then performed on the projected activity to determine a correlation between the PC-embedded activations of two RNNs. The SVCCA was the maximum correlation between the two projected representations subject to linear transformation.
2.6 Multi-dimennional scaling visualize attractor topology and representational geometry
We compared an embedding of attractor topologies across RNNs in a similar manner as 14 For additional details, refer to the Appendix. To compare representational geometry across RNNs, we computed pairwise distance matrix between all RNNs using SVCCA. Each element of the distance matrix was minus the SVCCA correlation between pairwise RNNs. We used MDS to qualitatively visualize how clustered this pairwise distance matrix was for different tasks. To quantify clustering. we computed the Silhouette cluster score for the pairwise distance matrix using the learning rule as the cluster label. All Silhouette cluster scores were computed on the high-dimensional data, not the MDS projections. MDS projections were used solely for visualization purposes.
3 Results
We numerically investigated the impact that learning rule, task complexity, and input noise have on RNN representations and dynamics during perceptual decision-making tasks. We make the following contributions. First, we show that across four learning rules, RNNs adopt a similar attractor topology regardless of the learning rule used. Second, we show that as task complexity increases, the representational geometry of RNNs becomes more individualistic (unique) across different learning algorithms while attractor topologies remain universal. Third, we show that task input noise, which affects task difficulty, can result in different RNN integration dynamics. Fourth, we show evidence that as tasks become more complex, task input noise has a smaller effect on RNN attractor topologies. These results suggest that RNN attractor topologies, and therefore dynamical mechanisms, are similar across learning rules and tasks despite differences in representations.
3.1 RNN attractor topologies are universal across learning rules
We trained RNNs to perform the RDM task and the CDI tasks using the previously described learning rules. All RNNs successfully performed the tasks with greater than 90% accuracy following training

---

a
SVCCA distances
b
SVCCA MDS clustering
RDM
CDI
RDM silhouette: -0.05
CDI silhouette: 0.004
BPTT
03
Full FORCE
BPTT
02
Genatic
Hebbion
H
BPTT
GA
H
FF
C
RDM example attractors
e
Attractor structure MDS clustering
RNN Inputs
BPTT
GA
H
FF
RDM sihoouett: -0.023
0.2
-0.5
PC2
PCJ
CDI example attractors
CDI sihouette: 2.9e-3
ANN activity (PCA)
context #1
PCy (au)
Figure 1: Representation and attractor structure across learning rules, a) SVCCA distance matrices for RDM (left) and CDI (right). There are 50 trained networks for each learning rule. BPTT, GA, and FF exhibit similar SVCCA distances, but H is different. (b) SVCCA MDS clustering shows that BPTT, GA, and FF networks are highly overlapping, while H representations are more dispersed. Each dot corresponds to one RNN. c) Attractor calculation intuition. Different inputs to the network (top) lead to different RNN equilibrium representations (bottom, which shows stable activity along the first principal component). We show 5 different attractor states corresponding to inputs: (-0.5, 0..2, O, 0.2,0.5). 0.5). d) Example attractors for the 5 different input levels for a single RNN trained with all four learning rules for RDM (top) and CDI (bottom). Attractors are visualized in the top two principal components. e). Clustering across RNNs trained for BPTT, GA, H, and FF (50 networks each). Networks are highly clustered in attractor structure.
with any of the four learning rules. We assessed the similarity of RNN representations by computing a pairwise distance matrix between all RNNs based on their SVCCA score, shown in Figure TT. We used MDS clustering to visualize RNNs based on similarity of neural representations, revealing that network representations were largely overlapping following training with BPTT, GA, and FF learning rules (Figure p)). Interestingly, Hebbian RNNs exhibited less overlap with RNNs trained with other learning rules, which may because Hebbian RNNs use an arbitrary recurrent neuron as the output [18). We therefore performed these analyses after training non-Hebbian RNNs to use single neuron readout, and observed consistent results (Appendix Figure II.. Overall, this suggests that representations in RNNs are similar across BPTT, GA, and FF for the RDM and CDI tasks.
We analyzed the attractor states of all trained RNNs to investigate their dynamical mechanisms and whether they differed across learning rules. We solved for attractor states of each network under five different probe input conditions: u {0.5, 0.2, 0, 0.2, -.53 (Figure D We visualized these attractor states in the principal components (PCs) of the activity, with example networks for each learning rule shown Figure Dt. In the RDM task, non-zero inputs resulted a single attractor state. We observed that at zero input, the network had multiple attractor states, which we elaborate on in Section 3.3. In the CDI task, RNNs instantiated relatively similar attractor topologies, with a set of based attractors for each context, previously shown in Mante et al. [[] and Maheswaranathan et al. 14. All RNNs also exhibited a line attractor of slow points, discussed further in Section 3.3.
Visualization in the PCs is limited, since these attractors are property of the high- high-dimensional RNN activity space. We therefore computed attractor structure similarity in the space of the RNN activity. We clustered RNN attractors associated with static inputs near zero based on the topology of their nearest neighbors and used MDS to visualize the resultant clusters (see Appendix and [44.. These results are shown in Figure De. We found attractor topologies were indistinguishable across RNNs trained with different learning rules. Silhouette cluster scores were close to zero across all learning rules. Our results contribute to literature finding evidence that there is universality and invariance of
5

---

SVCCA distances for N-CDI task
3-CDI
4-CDI
5-CDI
6-CDI
BPTT
02
01
BPTT
GA
SVCCA MDS clustering for N-CDI task
SVCCA Silhouete
Silhouette: 0.10
0.087
0.37
0.44
0.4-
BPTT
0.2
GA
for N-CDI task
Attractor structure MDS clustering for N-CDI task
Attractor Sihauette
Silhouetle: 0.030
0.002
0.03
-0.001
7erd
N for N-CDI task
Figure 2: Effect of task complexity on representational geometry and attractor topology. (a) SVCCA distance matrices for N-CDI task varying N = (left) to = 6 (right) comparing BPTT and GA networks. As the task became more complex, block structure was observed, indicating larger distances between BPTT and GA representations. (b) Visualization of SVCCA MDS clusters for N (left) to N 6 (right). As N increases, BPTT and GA RNNs representations cluster. (c) SVCCA cluster silhouette score increased as the number of contexts in N-CDI increased. (d) Visualization of attractor MDS clusters. As N increases, BPTT and GA attractor structures remain overlapping. (e) Attractor topology silhouette score did not show a trend with the number of contexts.
attractor structure across different RNN architectures and activation functions show that different learning rules exhibit universal attractor structure.
14] In particular, we
3.2 Increasing task complexity results in more individualistic representations, but not dynamics
To assess the degree to which representations and attractor topologies varied with task complexity, we performed the N-CDI task, varying the number of contexts and inputs in the task. We speculated that increasing task complexity may affect neural representations following training with different learning rules since more task-related information would need be stored in the network. Concurrently, increasing task complexity likely reduces the space of task solutions, constraining possible solutions. We found that, tasks became more complex, H and FF training did not meet the accuracy termination criterion when N 2, that is, these learning rules did not lead to high high-performing networks. We therefore report results comparing BPTT and GA.
BPTT and GA trained RNNs had larger MDS distances (Figure as the number of CDI contexts increased from N 2 to N
indicating different representations. As result, observed stronger SVCCA MDS clustering, shown in 2b. We quantified this by measuring the Silhouette scores of the BPTT and GA SVCCA clusters, finding they generally increased with increasing N, as shown in Figure p. These results suggest that increasing task complexity leads to representations that form distinctive clusters depending on the choice of learning rule. That is, RNN representa- tions became more individualistic as task complexity increased since trained RNNs had distinct (individual) representations instead of the universally shared representations. This is important to consider, especially as computational neuroscience studies typically compare RNN representations to neurophysiological representations. contrast, we found that across values N, BPTT and GA RNNs shared universal attractor structure (Figure 1.e). These results suggest that even as representations become more individualistic for complex tasks, attractor topologies may remain universal across learning rules.

---

Example RDM attractor topologies at varying input noise
variance = 0.1
0.25
0.5
0.75
1.0
0000 000000
0000
1000000
PCa
Multiple attractors for each input
RDM attractor topology MDS across noise levels
Attractor sincuutte
Silhouette: 0.45
0.37
0.22
-0.03
0.5
Full FORCE
Genatic
Hebbian
0.1
1.0
input noise variance
d
Example CDI attractor topologles varying Input noise
variance 0.1
0.25
0.5
0.75
1.0
00
0 D
vi
mm
000
00 3
30 o: D
Multiple attractors each input
CDI attractor topology MDS across noise levels
Attractor Silhouette
Silhouette: 0.13
0.03
-0.01
-0.03
4.3e-3
0.5
0.1
1.0
input noise variance
Figure 3: Effect of input noise on attractor structure. (a) Example RNN attractor structure at different input noise variance. When the noise variance is sufficiently large, the RNN instantiates multiple attractors for low inputs. (b) As the noise increases, there is more universality in attractor structure across learning rules. (c) Attractor structure clustering decreases with increasing input noise. (d-f) Same as (a-c) but for the CDI task.
3.3 Task input noise can alter RNN integration strategies
Because RNNs are used to propose candidate mechanisms for neural computation, it is appealing that RNNs exhibit similar dynamics across different training approaches and task complexities, suggesting robustness to particular hyperparameter settings, network architectures (as detailed in [14]) and learning rule. However, this led us to ask: under what settings, if any, might a network exhibit distinct dynamics? We reasoned that an important consideration task input design, which is determined by the experimenter. For example, in the CDI task, Mante et al. [ chose the standard deviation of the input noise to be 31.623 At, where At is the time step of the Euler update for the RNN, resulting in noise variance of But Miconi [8], also modeling the CDI task, set the noise to 0.5 and the mean to (instead of at most 0.1875), leading to difference in task difficulty. However, the input noise may dramatically affect the integration strategies: tasks with high input noise may employ more robust integration strategies. We ultimately find that, depending on the input noise, RNNs may employ different integration mechanisms, including for the CDI task [T]
We trained RNNs to perform the RDM and CDI tasks under varying input noise levels with the four learning rules. As input noise increased, there was greater multiplicity of attractors for small input values (shaded attractor topologies in Figure d)). Whereas low input noise levels instantiated one attractor for each input, increasing levels of noise resulted in the emergence of multiple attractors for small inputs. Attractors associated with small input values are of particular interest since they correspond to the networks state the absence of any meaningful input. Examples these attractor topologies are visualized in Figure where representative attractor topologies are shown for BPTT. Interestingly, the topological structure of attractors was invariant across learning rules for the high noise tasks. However, following training the lower noise task variants, attractor topologies were
7

---

Transient integration (noise var = 0.1)
one stable attractor for zero input
C
Dynamiis drive trajectory to attractor
Trajectory transiently relaxes to singlc zero attractor
Imputs on
inputs off
Output activatioon
Inputs
Inputs
0-
0.2
X
X X
X A
PCB
Ince attracter slow dynamics
500
Time (ma)
PC2
Stable integration (noise var = 1.0)
Trajectory stably relexes zero attractors
d
f
Inputs on
inguts off
Inputs on
inputs off
xxxxxx
XxXXX
-1
500
stable attrastors for zero input
Figure 4: Two different input strategies based on input noise. Panels (a-c) illustrate transient integration, while (d-f) is for stable integration. (a) RNN output activation for positive (red) and negative (blue) inputs. After inputs turn off, the RNN output slowly decays. (b) Attractor topology for a transient RNN. Dots are attractors. Numbers next to them correspond the setting of the input, u. There was a single attractor for u, = O. There was also a line attractor of slow dynamics under input, denoted by gray x's. (c) When the inputs turn on, the RNN state is driven towards attractors (blue, yellow dots). When the input turns off, the trajectories relax to a single attractor corresponding zero input. This decay relatively slow along line attractor of slow dynamics. (d) RNN output activation for a RNN with stable integration. After the inputs turn off, the RNN output maintains a stable value. (e) Attractor topology for stable RNN. Now, at an input of zero (purple), there are two attractors. (f) After the dots turn off, the trajectory stably relaxes to one of the two attractors corresponding zero input, holding memory of the input stably.
not universal across learning rules (Figure The effects of input noise and task complexity on attractor topologies is further discussed in Section 3.4. Together, these results suggest that task noise affects attractor topology, and when the task is too easy (low noise), learning rules may converge to different dynamical mechanisms.
We found that this bifurcation from single to multiple attractors results in two fundamentally different modes of integration, which we term "transient" and "stable" integration for the single and multiple attractor structures, respectively. We interrogated these dynamical mechanisms by providing a pulse of input evidence to the RNN, In these experiments, the inputs to the network are turned off (set to zero) midway through the trial (Figure 4p.d "inputs off"). Single attractor networks employed a transient integration mechanism, which is a form of leaky integration, causing the output to slowly forget its integrated state (FigureFaa, When the input was pulsed on, the RNN state evolved to the single attractor corresponding to the input level (Figure p. left). However, when the input was turned off, the network instantiated a single attractor at the zero input level, which caused the network state to slowly relax to an attractor corresponding to zero output (Figure Ap. right). The network, under zero input, also, instantiated a line attractor with slow dynamics (Figure 4p.c), consistent with the dynamics reported in Mante, Sussillo, et al. [1]. The network therefore decayed to the single attractor at zero, albeit slowly along the line attractor. This corresponded to a slowly decaying output value, reflecting leaky integration of the input.
In contrast, the multiple attractor networks employed a stable integration mechanism. particular, small inputs resulted in multiple fixed points, shown in Figure 4b. When turning off the input, the RNN state converged to a nearby attractor that maintains a stable memory of the sign of the pulsed input (Figure 4). This enabled the RNN to stably and indefinitely output non-zero value (Figure 4 corresponding to the sign of the prior integrated evidence, in contrast to the leaky integration mechanism. In both mechanisms, there is line attractor of slow dynamics that the RNN
8

---

a
RDM
CDI 6ax4
3-CDI
4-CDI
5-CDI
6-CDI
Number of contoots
Sinowett Booen $20
0.10
4044
-000
0.507
Figure 5: Input noise and task complexity. We show SVCCA and attractor topology clustering for BPTT networks trained at input noise variances of 0.1, 0.5, and 1.0. (a, b) As task complexity increases (more contexts), representations are highly overlapping. (c, d) Increasing task complexity results in similar attractor topologies across different input noise variances.
relaxes along. However the key difference between these mechanisms is the presence of multiple attractors low inputs. The same analysis for the RDM task Figure is also shown for the CDI task in the Appendix. Together, these results show that task design, in particular, input variance, can have a stronger effect on the dynamical mechanism than learning rule.
3.4 Attractor topologies are robust input noise as tasks become more complex
We previously found that, as tasks become more complex, networks trained with different learning rules maintain similar attractor topologies. This may reflect that as tasks become more complex, this constrains the space of potential solutions. Additionally, we found that RNNs trained ar different levels of input noise instantiated different attractor topologies and therefore dynamical mechanisms. We therefore investigated the effect of input noise on attractor topology in the setting of increasingly complex tasks.
We trained BPTT networks at three different noise levels: variances of 0.5, task, RNNs trained at different levels of input noise instantiated distinct attractor topologies. For the 2-CDI task, however, attractor topologies were less strongly clustered by input noise and appeared more universal. We subsequently trained the N-CDI task for these three input noise levels up to N 6. We found that as N increased beyond 3, attractor topologies were indistinguishable across all three noise levels. Figure 5 depicts attractor topologies following training at different levels of input noise for all tasks. These results suggest that although input noise may affect RNN dynamics, these effects on attractor topology diminish for sufficiently complex tasks.
0.5, and 1.0. For the RDM
Discussion
We studied the effects of learning rule, input noise, and task complexity RNN dynamics. We found that an RNN's attractor topologies are invariant to the choice learning rule across different decision- making tasks. We also observed this conclusion in simple working memory task (Appendix). We found the amount of noise present the task during training can change an RNN's dynamics when the task is sufficiently simple. These results suggest that when task is simple, hyperparameters may affect RNN dynamics. Consistent with these results, we also found that activation function affects attractor topology for RDM, but not the complex CDI task (see Appendix).
We characterized two distinct dynamical mechanisms used for integrating the inputs to an RNN: a transient and stable integration mechanism. In the case of high input noise, find a dynamical mechanism consistent with that reported by [T] following training in the CDI task. Alternatively, training with low input noise led to a mechanism with transient integration. While the topological structure of attractors did not vary, the representational geometry of RNNs was different following training with different learning rules with those by Maheswaranathan, Williams, et al. [14] that dynamics and computational scaffold are invariant over various RNN design choices while representational geometries not.
complex N-CDI tasks. Our findings here are consistent
These results contribute to laying a foundation for interpretability in using RNNs to study computa- tional neuroscience. We focused here traditional RNNs on relatively simple neuroscience tasks to

---

better understand the effect of learning rules attractor topologies and representational geometries. Our main contribution showing that dynamical mechanism, as assessed through attractor structure, remains conserved across different learning rules and task designs for sufficiently complex tasks, even if representational geometry not. This is important finding since many existing computa- tional neuroscience studies directly compare representations from artificial neural networks to neural recordings. While our results suggest that for tasks as complex as N-CDI, we observe universality in attractor topology, future work should extend these results to more complex tasks. More broadly, having a framework that allows better understanding of the validity of neurobiological inferences from modern neural networks during realistic tasks is important for building foundation for RNN use in neuroscience studies.
10

---

Acknowledgments
We thank Niru Maheswaranathan for helpful discussions on the attractor topology comparison. BM was supported by the NIH NIGMS training grant GM008042. MK was supported by the National Sciences and Engineering Research Council (NSERC). JCK was supported by NSF CAREER 1943467, NIH DP2NS 122077, the Hellman Foundation, and UCLA Computational Medicine AWS grant. We gratefully acknowledge the support NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.
References
[1] V. Mante, D. Sussillo, K. V. Shenoy, and W. T. Newsome, "Context-dependee computation by recurrent dynamics in prefrontal cortex," Nature, vol. 503, no. 7474, pp. 78 84, Nov. 2013.
[2] H. F. Song, G. R. Yang, and X. J. Wang, "Training excitatory-nniibitoyy recurrent neural networks for cognitive tasks: a simple and flexible framework," PLOS Comput. Biol., vol. 12,
no. 2, pp. 1-30, 2016.
[3] W. Chaisangmongkon, S. K. Swaminathan, D. J. Freedman, and X. Wang. "Computing by robust transience: how the fronto-parietal network performs sequential, category- decisions," Newron, vol. 93, no. 6, pp. 1504-1517.e4, 2017.
[4] K. Rajan, C. D. Harvey, and D. W. Tank, "Recurrent network models of sequence generation and memory," Neuron, vol. 90, no. 1, pp. 128-142, Apr. 2016.
[5] G. R. Yang, M. R. Joglekar, H. Francis Song, W. T. Newsome, and X.-J. Wang. "Task represen- tations in neural networks trained to perform many cognitive tasks," pp. 297-306, 2019.
[6] A. E. Orhan and W. J. Ma, "A diverse range of factors affect the nature of neural representations underlying short-term memory," Nat. Neurosci., vol. 22, no. pp. 275-283, Feb. 2019.
[7] M. Kleinman, C. Chandrasekaran, and C. Kao, "A mechanistic multi-area recurrent network model of decision-making." Advances in Neural Information Processing Systems, 2021.
[8] G. Hennequin, T. Vogels, and W. Gerstner, "Optimal control of transient dynamics in balanced networks supports generation complex movements," Neuron, vol. 82, no. 6, pp. 1394-1406, Jun. 2014.
[9] D. Sussillo, M. M. Churchland, M. T. Kaufman, and K. V. Shenoy, "A neural network that finds a naturalistic solution for the production of muscle activity," Nat. Neurosci,, vol. 18, no. 7, pp. 1025-1033, 2015.
[10] A. Michaels, B. Dann, and H. Scherberger, "Neural population dynamics during reaching are better explained by dynamical system than representational tuning." PLoS Comput. Biol., vol. 12, no. 11, p. e1005175, 2016.
[11] C. Kao, "Considerations in using recurrent neural networks to probe neural dynamics," " bioRxiv, 2018. [Online]. Available:
httt/////eaaa222000000000033333 8/0708/33-4489
[12] J. A. Michaels, S. Schaffelhofer, A. Agudelo-Tore and H. Scherberger, "A goal-driven modular neural network predicts parietofrronal neural dynamics during grasping," Proc. Natl. Acad. Sci. U. S. A., vol. 117, no. 50, pp. 32 124-33 32 135, Dec. 2020.
[13] D. Sussillo and O. Barak, "Opening the black box: low-dimensional dynamics in high- dimensional recurrent neural networks," Neural Comput., vol. 25, no. 3, pp. 626-649, 2013.
[14] N. Maheswaranathan, A. H. Williams, M. D. Golub, S. Ganguli. and D. Sussillo, "Universality and individuality in neural dynamics across large populations recurrent networks." Advances in Neural Information Processing Systems, 2019.
[15] N. Caporale and Y. Dan, "Spike meinng-dependent plasticity: A hebbian learning rule," Annual Review of Neuroscience, vol. 31, no. 1, pp. 25-46, 2008 PMID: 18275283. [Online]. Available: 3...0....0000.550000

---

[16] T. P. Lillicrap and A. Santoro, "Backpropagation through time and the brain," Current Opinion in Neurobiology, vol. 55, pp. 82-89, 2019.
[17] P. Lillicrap, A. Santoro, L. Marris, C. J. Akerman, and G. Hinton, "Backpropagation and brain," Nat. Rev. Neurosci. Apr. 2020.
[18] T. Miconi, "Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks," Elife, vol. 6, Feb. 2017.
[19] D. Sussillo and L.. F. Abbott, "Generating coherent patterns of activity from chaotic neural networks," Neuron, vol. 63(4), pp- 544-557, 2009.
[20] B. DePasquale, C. J. Cueva, K. Rajan, G. S. Escola, and L. F. Abbott, "full-force: A target-based method for training recurrent networks," PLOS ONE, vol. 13, no. 2, p. e0191527, Feb 2018. [Online]. Available:
0191557
[21] P. Such, Madhavan, Conti, Lehman, K. O. Stanley, and J. Clune, "Deep neuroevo- lution: Genetic algorithms
competitive alternative for training deep neural networks for reinforcement learning," " arXiv preprint arXiv: 171..06667, 2017.
[22] A. H. Williams, T. H. Kim, F. Wang, S. Vyas, S. I. Ryu, K. V. Shenoy, M. Schnitzer, T. G. Kolda, and S. Ganguli, "Unsupervised discovery of demixed, Low-Dimensional neural dynamics across multiple timescales through tensor component analysis," Neuron, vol. 98, no. 6, pp. 1099-1115.e8, Jun. 2018.
[23] W. Newsome, K. H. Britten, and J. A. Movshon, "Neuronal correlates of perceptual decision," Nature, vol. 341, no. 6237, pp- 52-54, Sep. 1989.
[24] D. Sussillo, M. M. Churchland, M. T. Kaufman, and K. V. Shenoy, "A recurrent neural network that produces EMG from rhythmic dynamics," in Frontiers in Neuroscience. Computational and Systems Neuroscience (COSYNE), Salt Lake City, UT, 2013.
[25] M. Raghu, J. Gilmer, J. Yosinski, and J. Sohl-Dickstein, "Sveca: Singular vector canonical correlation analysis for deep learning dynamics and interpretabilityy" Advances in Neural Information Processing Systems, 2017.
12


Reviews:
[{"review_id": "zDkoySoXV9E", "reviewer": "Reviewer_T4eL", "summary": "This paper aims to characterize the impact of design choices in RNN-based experiments on learning dynamics and representation geometries, both of which are important data sources when using RNNs to model biological intelligence. It begins by setting up the problem of investigating RNNs’ fitness to model cognitive and neural systems, and drawing a distinction from the most similar work (Maheshwaranathan et al). It then describes a standard continuous-time RNN architecture, three tasks, one of which is a complexity-modular version of another, and four learning rules of which I believe one is local. Along with the model setup, it details evaluation and visualization methods. The bulk of the paper appropriately focuses on results. This goes through the following conclusions: dynamics do not vary with learning rule (implicitly, for more complex tasks), representations vary with task complexity, dynamics do not vary with task complexity, dynamics do vary with learning rule under low input noise with simpler tasks, and finally dynamics are confirmed to not vary with learning rule under low input noise with more complex tasks (suggested explanation being that more complex tasks have a more constrained solution space). The paper ends with a summary of these conclusions and their implications for the use of RNNs in computational neuroscience.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Great paper! Impressively clear, well-scoped, doesn’t overpromise, delivers something interesting. \n\n*Originality:* The work seems sufficiently original. It is a contribution to the community and is very clear and honest in comparing itself even to the most similar work, distinguishing itself specifically by focusing on learning rules. It includes a good range of related work and positions itself as a helpful addition.\n\n*Quality:* This work seems technically sound. It scopes itself well and lists out certain claims, then goes on to provide experimental backing for each claim. The experiments are well-designed and clear, and answer the questions asked. Particular compliments to Fig 1, which illustrates the primary claim of the paper almost independently. Though some of this and other figures ask for visual inspection, they are backed up by descriptive statistics in the text. I have no qualms about experimentation quality. My only concern would be that I would have liked to see more tasks even of these complexities, so that there were more examples for various experiments (especially the ones using only N-CDI). Without these, the paper does seem to be putting all its eggs in one basket for certain claims, but the thoroughness of the experiments that are present mitigates this a lot. \n\nThe methods are intuitive and thorough. One concern is that some of the selected learning rules become incompatible with certain experiments, so it could be that a larger set of learning rules would have been valuable so that certain experiments weren’t limited to one or two rules, but I could see this being left for future work (and I realize that options are limited). \n\nI don’t know that the paper discusses limitations sufficiently to satisfy the current NeurIPS guidelines (see below), but the paper also doesn’t overstate its contributions. I think it’s relatively intellectually honest. I do see it as a work in progress in that it leaves a lot of space for expansion, but as a paper in computational neuroscience I think it’s sufficient. \n\n*Clarity:* This paper is really well-written. Many papers start off strong and proceed to fall apart writing-wise in the results, but this one has great organization, signposting, and writing throughout. It also follows the rule of “say what you’re going to say, say it, say what you said”, which was very useful for my understanding. My only concern is that I interpret the findings to be the following in order: “dynamics do not vary with learning rule (implicitly, for more complex tasks), representations vary with task complexity, dynamics do not vary with task complexity, dynamics do vary with learning rule under low input noise with simpler tasks, dynamics are confirmed to not vary with learning rule under low input noise with more complex tasks (suggested explanation being that more complex tasks have a more constrained solution space)” (quote from my summary). It’s a little confusing to understand how the beginning and end of this flow fit together. These aren’t exactly organized by either design choice (LR, input noise, etc.) or output (dynamics, representations) and might benefit from such an organization. I may also not fully understand the flow, so I am open to the existing ordering. \n\nMinor: the paper starts off referring to task complexity as “task hyperparameters” exclusively; use of “task complexity” starts around halfway through the paper. Terms should be consistent regardless, but this is especially confusing because the paper is about learning rules. So, I recommend only using “task complexity” so no one has the opportunity to get confused even if they are skimming, even if the N in N-CDI could be considered a hyperparameter. \n\n*Significance:* I think this paper is a useful contribution to understanding of how RNNs should be used when modeling biological intelligence. To my knowledge, the results are unique and set up both lots of ground for expansion and framework for experimentation meant to model the brain. To me, this seems like a worthwhile addition to the field. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The checklist claims that this work discusses its limitations, but the referenced limitation is “mechanisms in the RNNs should be interpreted as a hypothesis for biological mechanisms”. I wouldn’t consider this a discussion of limitations of this work. I’m not even sure how it’s a limitation at all, but if the idea is that RNNs aren’t a perfect model for biological intelligence and are primarily theorized as one, then that’s a critique of the whole field (in fact, one that this paper is trying to rectify). The one limitation that this work does discuss in the main text is that experiments should extend to more complex tasks, but this is given half a sentence and is posed as a future work. I totally agree and think the paper is complete enough to leave that for future work, but that’s not a sufficient discussion of limitations as per the NeurIPS checklist guidelines. I think the paper should discuss the limited space of tasks (even in the range of complexity of RDM and (N-)CDI) and learning rules, or biological implausibility of some of the learning rules. It’s certainly possible that I misinterpreted the note in the checklist, but I don’t think this paper really discusses the limitations. I also don’t think it would be hurt by a discussion - it’s a good paper that does enough to survive an honest discussion of limitations. \n\nI agree that this paper is theoretical enough to not need a discussion of societal impacts. Since the work “involves understanding deep neural networks”, I’m not sure that it “inherits the broader positive and negative societal impacts of deep learning”. It does very indirectly (which is what I think the authors are saying), but (also indirectly, but less so) it inherits more of the impacts of neural network interpretability than neural network application. One thing that isn’t touched is the societal impact of having good models of biological intelligence, which are huge and rarely discussed. I don’t think this warrants a flag for ethical review, since 1) there is so much ground between the state of the field and a societally impactful model and 2) I rarely see such discussion in comp neuro papers, but I do believe it warrants consideration if it is the ultimate goal.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "ewbxwzsqh8p", "reviewer": "Reviewer_GYy3", "summary": "The authors extend the recent work of Maheswaranathan et al to explore how learning rule and task complexity affect the solutions obtained by recurrent neural networks trained on neuroscience-inspired tasks.\nSpecifically, four different learning rules are used to train networks on a suite of similar tasks with systematically increasing complexity. The authors show that learning rule has some effect on representation geometry, and this effect grows with task complexity. Attractor topology, however, remains universal for almost all tasks and learning rules.\nThe authors also show some instances in which network dynamics are influenced by input noise. This sensitivity, however, decreased as task complexity increased.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The question of individuality and universality of solutions obtained by trained recurrent neural networks is an important and timely one. These networks are used to compare with experimental data, and it is important to understand how the nature of solution is affected by various design choices.\nThe specific extensions studied here – learning rule and task complexity – are widely used to optimize networks, so it is critical to understand how each criterion biases the obtained solutions, especially when claiming similarity between RNNs and brains. Of particular note is the use of genetic algorithms, that are not often used in this context, and could hence provide novel viewpoints on the space of solutions. Furthermore, the systematic approach to task complexity is a promising approach to understand its effect.\nDespite these strengths, there are several major issues with the current submission: The novel insights beyond the work of Maheswaranathan et al are not large; The effects uncovered by the authors are not explored; The dynamical system analysis is non-standard and not justified appropriately.\n1.\tAs the authors state, this work is very much in line with that of Maheswaranathan et al.  As such, the added value should be much more clearly stated. In particular, the reasons for discrepancy should be studied. The authors speculate that “increasing task complexity… constraining possible solutions”. And yet the empirical finding is that solutions of complex tasks are more individualistic. This seems like a promising avenue for exploration – but it is not pursued.\n2.\tHebbian learning is distinct from all other rules. This is interesting. Why is it so? The topic is not pursued.\n3.\tThe analysis used to arrive at the conclusion is composed of many processing steps. At the end of all these steps, we are left with a single number (Silhouette score) that describes variability. Compressing a complex phenomenon like the solution found by a network to a single number is desirable in order to produce figures like 2C and 2E. But it also precludes an understanding of what the underlying reasons for these observations are. It would be very helpful to accompany such analyses with several intermediates like comparing PCA (or other dimensionality reduction methods) of various networks. \n4.\tThe authors define fixed points as having a speed of 1e-8 and below, and slow points of a speed of 1 and below. This is a very high threshold for the slow points (compare to Sussillo & Barak 2013). Furthermore, the analysis used inputs that are close to zero to define some ordering between fixed points. If I understand correctly, the authors actually find slow points that have a speed considerably lower than 1. By slightly perturbing the input from zero, these slow points cross the threshold of being considered fixed points. First – this is my guess. Second – this could be an interesting heuristic for reverse engineering trained networks. Third – none of this is explained or justified in the appendix. Standard analysis requires using zero input. Other choices might be useful, but should be justified and clearly stated.\n\nSpecific points:\n5.\tThe authors analyze one task with varying complexity and find that the dynamics are almost invariant. It could be that other tasks have more variability, and hence could be better suited to answer the main question posed here – does learning rule affect solution?\n6.\tSilhouette score is used extensively. It could be useful to provide some intuition on the meaning of the various values.\n7.\tFigure 1 caption “networks are highly structured in attractor structure”. This sentence is somewhat confusing when combined with a low silhouette score\n8.\tFigure 2B 5-CDI panel shows two clusters for the GA. Why is that? Such observations are not mentioned or analyzed in the paper.\n9.\tFigure 3A the attractor changes shape from a line to a sigmoid. Why is that?\n10.\tFigure 3a: It is unclear where the slow areas appear in relationship to the attractors, which differences arise when the Silhouette score is high (e.g. figure 3B for low variance) and what is the contribution of the learning rule to this variability. This is especially needed because the Hebbian learning rule appears to dominate the variability. \n11.\tL271: “memory of the pulsed input”. Should this be memory of the sign of the input?\n12.\tFigure 4. Perhaps looking at the actual speed values of points along the attractor can be informative here.\n13.\tFigure 5 caption is hard to understand\n14.\tL302: “two distinct dynamical mechanisms used for integrating the inputs”. I’m not sure whether it was demonstrated how these mechanisms are used for the task.\n15.\tFigure S6 in appendix. The panel D, which represents stable integration looks quite similar to the  transient panel A. This is not commented upon.\n16.\tIn general, interpretation and explanation of the numerical results would greatly strengthen the paper.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Societal impact – not relevant.\nLimitations: mentioned above. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "bNBS_6vYoKq", "reviewer": "Reviewer_obz4", "summary": "A popular paradigm for modeling the brain using task-optimized deep learning models identifies three key features governing the solutions learned by neural networks: architecture, task, and learning rule. Building on work by Maheswaranathan et al. [1], which showed that architecture influences the representations learned by trained recurrent neural networks (RNNs), but not their attractor structure, the authors of this work extend the analysis to study the effect of learning rule and task on the representations and attractor structure of trained RNNs — with broadly consistent results. The authors show that the attractor structure is largely invariant to the choice of learning rule (at least on two simple tasks: random dot motion and context-dependent integration, with large input noise), but the learned representations are not invariant, especially as the complexity of the task (the number of individual contexts to integrate) increases. However, the authors also exhibit a setting where the attractor structure is *not* invariant to the choice of learning rule: when the input noise is low in either task. They further show that the learned attractor structure depends sensitively on the amount of noise injected into the network, with a transition between one solution in the low-noise regime and another in the high-noise regime. While these results may seem contradictory, they may be consistent under the hypothesis that the attractor structure of trained RNNs exhibits universality across architectures and learning rules *when the task is sufficiently difficult*.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "If this is indeed a point the authors wish to make, it would have been nice to see the models trained on a greater diversity of tasks, with more interesting attractor topologies. [1] studied three tasks with fairly unique attractor topologies, which were further investigated and linearized. Of these the present work studies only the context dependent integration task, whose attractor topology has been studied exhaustively in previous works ([2], although the authors here identify a new solution as a function of input noise), and additionally studies a random dot motion task. But the only analysis of attractor structure reveals either one or two fixed points (for zero input) in each case. It is perhaps not so surprising that different learning rules converge on these same simple solutions (although it would be interesting to investigate how the attractor topologies differ in the case of low input noise! The authors could consider including this analysis). When the task is made more difficult by including N contexts, only two learning rules lead to successful training (backprop and a genetic algorithm). The two learning rules lead to consistent attractor structure as evaluated by the MDS analysis, but the attractor structure itself is not investigated.\n\nThis work includes several interesting conceptual insights beyond previous works, including the sensitive dependence of learned solutions on input noise. And the demonstration of settings where trained networks can exhibit universality to learning rules is an important step towards better understanding and interpreting neural networks, and using them as models of the brain. Overall the analyses are well performed and the presentation is clear. That said, the extensions this work presents beyond [1] are relatively modest, and no new methods or analysis techniques are introduced which could be of independent interest (aside from the silhouette score, the methods in the present work appear to be identical to those introduced in [1]).\n\nFinally, a “cautionary tale” discussed in [1] is that CCA may actually be a poor metric of representational similarity, given that in some settings the canonical correlation between trained and untrained RNNs can be greater than the canonical correlation between two trained RNNs with different design choices (e.g. nonlinearities). It would be interesting for the authors here to discuss this concern, and/or to try out a different metric for representational similarity.\n\n\n\n[1] N. Maheswaranathan, A. H. Williams, M. D. Golub, S. Ganguli, and D. Sussillo, “Universality and individuality in neural dynamics across large populations of recurrent networks,” 2019.\n\n[2] H. F. Song, G. R. Yang, and X. J. Wang, “Training excitatory-inhibitory recurrent neural networks for cognitive tasks: a simple and flexible framework,” PLoS Comput. Biol., vol. 12, no. 2, pp. 1–30, 2016.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The results are fairly presented and the authors are clear throughout the paper about the limitations of their work as well as potential future directions and impact on neuroscience.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "SCwruRPgCdU", "reviewer": "Reviewer_MbCb", "summary": "This paper studies the effect of learning rules on the algorithms learned by recurrent networks on context-dependent integration tasks. A number of continuous time vanilla RNNs are trained on integration tasks with different learning rules: backprop, genetic algorithms, hebbian learning, and full-force. The paper finds that across these learning rules, all networks learn consistent attractor topologies in order to solve the task. As the complexity of the task is increased (increasing number of contexts), the representations start to differ, but the attractor structure is preserved. The paper finds that using line attractors to solve integration seems to be a universal property of these networks, independent of learning rule. Finally, the paper shows how properties of the learned attractors (both the topology and whether the attractors are continuous manifolds or discrete points) vary with task input noise.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I enjoyed reading this paper. It is well written and addresses an important problem: what aspects of the algorithms learned by networks trained to solve particular tasks are sensitive to the choice of learning rule used to train the network?\n\n### Questions / concerns\n- The tasks are presented as being a part of three separate types (RDM, CDI, and N-CDI), but I feel like it makes more sense to present the tasks as all being part of the same family of tasks. They are all context-dependent integration tasks, but what varies is the number of contexts: ranging from one (RDM) to two (CDI) to multiple (N-CDI). Is my understanding correct?\n- L184/185: \"This is possibly due to the fact that Hebbian RNNs do not compute a linear readout ... but instead use an arbitrary recurrent neuron as the output\". This seems like a significant difference, and should be controlled for. Why not train the other networks (BPTT / GA) in this fashion, by picking an arbitrary neuron to be the readout (as opposed to having a linear readout). That would help control for this difference.\n- It would be nice to see results for other tasks (besides integration). Are there other simple tasks that might be good candidates?\n- I thought the results showing a transition from transient to stable integration with increasing task noise particularly interesting. Are there other aspects of the task that might induce similar transitions? This point is particularly relevant for neuroscientists that might observe continuous or discrete integration in the brain.\n- In Figure 2a, it looks like the BPTT representations are all similar, all the GA networks are different (not only from BPTT, but different amongst themselves as well). Any intuition for this? It looks like some GA networks are similar to the BPTT networks, especially for smaller numbers of contexts. Is that variability due just to the random seed? Can you identify what it is about certain random seeds that makes some GA networks look like BPTT networks?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper is an empirical study of the impact of learning rule and input noise on the RNN solutions in a small family of tasks. The reviewers agree that writing is clear, and showed excitement in seeing the results, but concerned with the narrow range of tasks. Reviewers made excellent constructive suggestions  that I strongly encourage the authors to incorporate in the final, stronger manuscript.
Decision: Accept (Poster)