Extracting computational mechanisms from neural data using low-rank RNNs
Authors: Adrian Valente
Jonathan W. Pillow
Srdjan Ostojic
Venue: NeurIPS 2022 Accept (2022)
Keywords: RNN
computational neuroscience

Abstract: An influential framework within systems neuroscience posits that neural computations can be understood in terms of low-dimensional dynamics in recurrent circuits. A number of methods have thus been developed to extract latent dynamical systems from neural recordings, but inferring models that are both predictive and interpretable remains a difficult challenge. Here we propose a new method called Low-rank Inference from Neural Trajectories (LINT), based on a class of low-rank recurrent neural networks (lrRNNs) for which a link between connectivity and dynamics has been previously demonstrated. By fitting such networks to trajectories of neural activity, LINT yields a mechanistic model of latent dynamics, as well as a set of axes for dimensionality reduction and verifiable predictions for inactivations of specific populations of neurons. Here, we first demonstrate the consistency of our method and apply it to two use cases: (i) we reverse-engineer "black-box" vanilla RNNs trained to perform cognitive tasks, and (ii) we infer latent dynamics and neural contributions from electrophysiological recordings of nonhuman primates performing a similar task. 

Extracting computational mechanisms from neural data using low-rank RNNs
Adrian Valente"
Jonathan W. Pillow
Srdjan Ostojic École Normale Supérieure PSL Research University
École Normale Supérieure PSL Research University
Princeton Neuroscience Institute Princeton University
Abstract
An influential framework within systems neuroscience posits that neural com- putations can be understood in terms of oww-dimessional dynamics in recurrent circuits. A number of methods have thus been developed to extract latent dynamical systems from neural recordings, but inferring models that are both predictive and interpretable remains difficult challenge. Here propose a method called Low-rank Inference from Neural Trajectories (LINT), based on a class low-rank recurrent neural networks (IRRNNs) for which link between connectivity and dy- namics has been previously demonstrated. By fitting such networks to trajectories of neural activity, LINT yields mechanistic model of latent dynamics, as well a set of axes for dimensionality reduction and verifiable predictions for inactivations of specific populations of neurons. Here, we first demonstrate the consistency of our method and then apply it to two use cases: (i) we reverse-engings engginerr -black-box" vanilla RNNs trained to perform cognitive tasks, and (ii) we infer latent dynam- ics and neural contributions from ddectrophysiologiie recordings of nonhuman primates performing a similar task.
Introduction
As large-scale neural recordings behaving animals are becoming commonplace, a pressing question is how computational principles be extracted from the electrical activity of thousands of cells. An influential framework posits that neural computations rely on latent low-dimensional dynamics [54 600 distributed across populations of neurons (66 43 In line with this proposal, number of data-
analysis methods have been developed to infer latent dynamics from neural recordings [65] 255 BR 32]
12
45 [2] (reviewed in (II.. While these statistical approaches often provide compelling descriptions of the recorded data, they generally lack a mechanistic interpretation that would, for instance, allow them to make predictions for responses to novel interventions the underlying neural circuits. How to identify mechanistic, predictive models from neural data thus currently remains an important challenge.
31
24
33)
10
19
16. 40
41
Recurrent neural networks (RNNs) have emerged as key models for studying neural computations [64]. Indeed, RNNs can be trained to solve various cognitive tasks, and lead dynamics surprisingly 39 and have also successfully been
similar to those observed neural recordings 29 trained to reproduce the activity of neurons recorded in vivo B8 14
56
5
51
While potentially 3S
predictive, the obtained networks are however typically challenging to understand mechanistically [55]
28 27
26
63 [7), and ongoing research directions aim reducing them to simplified, interpretable
models, for example through the use of linearized dynamical systems 12
50 or through "network
distillation" methods 44 22
*Correspondence to adrian. www.maaweweeeeeeee.e.eem
psl. psl.
Code available at http:///hhhh...ooo.ooaaaaaaa/aaa iineeeeiffffeeeeeeeem
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Here, we exploit particular class of interpretable RNNs, namely low-rank RNNs (IrRNNs) BC B
to develop a new method that extracts mechanistic and predictive low-dimensional models from observed neural activity, and can be applied to both artificial and biological neural networks. Previous work has performed theoretical analyses of low-rank RNNs, either designed or trained on specific tasks 30 91, but has not introduced methods for fitting them to neural activity. Our method, Low-rank Inference from Neural Trajectories (LINT) infers a minimal rank IrRNN from dataset and then exploits the theory of low-rank networks to relate the obtained connectivity with now-dimensional dynamics and computations. dimensionality reduction of neural population activity, effective connectivity that implements latent dynamical system, and predictions for inactivations on specific subsets recorded neurons (fig. Ma).
96 ! 58
produces three outputs: a set of axes for
Our contributions can be summarized in three steps: first, we verify the consistency of LINT by applying it to data simulated from IrRNNs, and show that it recovers the effective part of the connectivity that reproduces the dynamics and computations. Second, we apply LINT to data generated from full-rank RNNs trained on cognitive tasks, and demonstrate that the resulting IrRNNs can capture their dynamics and offer mechanistic insights into how they function. Notably, we identify novel population-based mechanism enabling context-dependent switching in vanilla RNN, and verify by targeted inactivation experiments. Finally, we apply LINT to electrophysiological recordings in nonhuman primates performing context- neeeext-deeendent decision making task [29]: this reveals that a rank-1 network can capture most aspects of the dynamics present in the data, and that computations in this neural circuit appear to be supported by small proportion of all recorded neurons.
2 Approach
Here we describe our method for extracting an interpretable low-dimensional projection and dynami- cal model from neural trajectories by fitting a low-rank recurrent network to data.
Low-rank RNNs (IRRNNs). We start from rate-based recurrent neural networks of N units, each characterized by an activation variable x; which follows the dynamics:
dzg dt
-2; + Σ (Zz Σ (t) +x()).
Here J represents the network connectivity matrix and a nonlinear transfer function defining the neural firing rate d(x;)), which we take here to be lanh. Each network also receives N₁, input signals (t) via set of weights (*) that we refer to input vectors.
Low-rank RNNs represent subclass of models in which the connectivity matrix is constrained to be of finite rank R < N 30 3 9.. In this case, J can be written as sum of outer products of connectivity vectors n (r) and ())
R N
Joj
( (r)
To define unique representation, we take m. (r) the left singular vectors of the connectivity matrix, and as (r) the right singular vectors with an appropriate normalization.
Network inference. We infer IrRNNs from data by training the connectivity parameters to reproduce recorded neural trajectories (either single-trial condition-averaged trajectories). Specifically, we use back proppaappoganonnn on connectivity vector parameters n²t") and m." and input parameters (()) to minimize the squared difference between target and reproduced trajectories:
C=
Σ (0(2)) (t) -030))0-(((000 (())2
o(
(3)
where (1) represents the target trajectory in condition e, for neuron i and timestep and a! (.( the corresponding trajectory produced by the model.

---

b.
C.
Neural data
Low-rank RNN
Low-d
dynamics
Infer
20
J =
connectivity & mechanism
Effective
-20
-20
20
Juj
J
Figure 1: LINT pipeline: Low-rank RNNs fitted to either simulated or recorded neural trajectories. The obtained IrRNNs provide interpretable low-dimensional dynamics as well as computational mechanism based trajectories simulated from a rank-1 IrRNN trained to perform the CDM task (see Table 1 for results across tasks). b. For each pair of neurons, relationship between the original (Jiz) and inferred (Jiz) synaptic connectivity (512² pairs, r = 0.57). c. For the same networks, relationship between the original effective connectivity (J see text) and the inferred one (J+ff 0.99)
effective connectivity structure. b-c. LINT tested on
Dimensionality reduction. One property of IrRNNs is that, in the absence noise, they constrain the activity vector x(t) to evolve in a low-dimensiona subspace spanned by the R connectivity vectors ml(t) and the Ni. input vectors ((s) The trial- trajectories therefore explore most R + Nin dimensions and can be parametrized as:
B
N
x x01)= = Σ Kr (1)m + +Σ (.(
(4)
where U, (t) are the low-pass filtered input signals ut(() (see Appendix and Ky are a set of latent variables generated by recurrent activity. Thus, IrRNNs provide by design reduction of N-dimensional neural activity to at most R + Nin dimensions that can be directly interpreted in terms of components on recurrent subspace spanned by the connectivity vectors m (r) and on an input subspace spanned by inputs vectors (s) 61..
Latent dynamics. In IrRNNs, the latent variables k, (t) form non-linear low-dimensional dynamical system, described by:
d
di x(1) F(x()
(5)
where <(t) = {Ky (t))==i..) is an R-dimensional vector representing the activity on the recurrent subspace, ut( represents the input signals, and is a non-linear function that can be directly determined from network connectivity parameters 3 9 (Appendix A Moreover, it can be shown that rank- R networks are universal approximators of R-dimensional dynamical systems K and thus that the function F can approximate any non-linear mapping in R dimensions.
Task-optimized RNNs. We applied our method first trajectories produced by task-optimized RNNs, which produce an output signal z(t) from the recurrent dynamics (I via a linear readout:
N =(1) Σ wx)(x(())
(6)
In task-optimizae networks, the parameters are trained with backpropagation through time to minimize the squared eтrоr between the output 2(t) and target 2n =t)) In this work, we consider four cognitive Decision Making (CDM,
tasks; Decision Making (DM), Working Memory (WM), fig. 2 and Delayed Saamhe (DMS) (see Appendix B for task definitions). analyze computations underlying these tasks, we generated neural trajectories from both low-rank (in section 3.1) and full-rank (see section 3.2) task-optimized RNNs. All networks were implemented in pytorch
+ training details can be found in Appendix C

---

Table 1: Synthetic data validation results for LINT
(CC: connectivity correlation, ECC: effective connectivity correlation)
Task
Rank
Trajectory R²
CC
ECC
Decision Making (DM)
1
0.97
0.50
0.99
Working Memory
2
0.96
0.43
0.93
Context-dependent DM
1
0.91
0.57
0.99
Delayed Match-to-Sample
2
0.98
0.39
0.63
3 Results
3.1 Validation with synthetic data and effective aspects of connectivity
We first validated the capacity of the LINT method to recover low-rank connectivity features and reproduce neural trajectories on data simulated from IrRNNs trained on cognitive tasks. For this, we first trained IrRNNs with 512 neurons produce correct behavioral output on four systems neuroscience tasks, each time retaining the minimal rank solution ? (see Table 1). For each task- optimized IrRNN, we then generated simulated trajectories corresponding to trials in the trained task, and applied our method to fit these trajectories using equivalent models with the same rank and number of neurons. We found that the inferred networks were able reliably reproduce the original trajectories when fed with the same inputs, as quantified by the R² scores between original and fitted trajectories. Although the inferred networks were not explicitly trained on behavioral outputs, they were able to perform their task accurately when plugged to the original readout vector, implying that they also captured behavioral aspects of the original networks.
The output of LINT is an inferred low-rank connectivity that we compared with the original one. We noted that the inferred connectivity weights not identical to the original ones, although certain degree of correlation is present (fig. Dp and Table 1, column CC for connectivity correlation). However, from a theoretical point of view, range of different connectivity matrices can lead to identical dynamics 37
The low-rank framework allows for a precise characterization of the relevant part of the connectivity, and in particular allows us define an effective connectivity matrix Jell that captures the minimal features required to obtain certain dynamics. Indeed, for this class of networks, the low-dimensional dynamics described by equations 4 depend on the exact entries on the (r) and [(s) vectors, but not the individual entries on the (r))) vectors. Instead, (r) vectors influence the dynamics only through their global projections on the m (t) and (()) vectors (see details in Appendix A. Thus, the components of (r) orthogonal to the space spanned by (r) and (() are irrelevant for the dynamics, and removing them leads to an effective connectivity matrix Jeff which captures the minimal features required for obtaining particular dynamics. Our results on the synthetic data showed indeed a very high degree of similarity between original and inferred effective connectivities, demonstrating that LINT recovered the relevant aspects of the neural connectivity (Table 1, ECC for Effective Connectivity Correlation, and fig. IP).
[K] [s]
In conclusion, LINT is able to accurately fit neural trajectories generated by low-rank RNNs, to infer networks that give rise to similar trajectories and behavioral performance, and to recover the essential features of the connectivity.
3.2 Application to reverse-engineering full-rank RNNs
We next ask to which extent our method can infer computational mechanisms from activity generated by unconstrained, full-rank networks. Indeed, RNNs trained on simple tasks without a rank constraint often exhibit www-dimeniionl dynamics that can be related to the way computations unfold in the
network [55] 28 27
It has been found that such -iw-iimesiionl dynamics can be reproduced by low-rank networks 3 yet the best approach for inferring the corresponding low-rank connectivity remains to be determined. Here we show that our method vastly outperforms direct low-rank approximation of the connectivity matrix based on truncating the SVD 477) We then demonstrate how the inferred low-rank models can be used interpret the computational mechanisms in the original full-rank networks.
26
577.

---

a.
b.
(erea(t)
1.0
Task accuracy
1.0
task
(t((t) wym
output
&
0.5
truncated Riteed
truncated
ua((t) Muy
fited
neural
0.0
4
5
0.0
1
2
5
(ttze(t)
trojectories
Rank
Rank
d.
1.0
1.0
Task accuracy
1.0
&
0.6
truncated
truncoted
0.4
rank
full-rank
0.0 1
2
5
0.0
2
5
10-2
10
10
Rank
Rank
subsampling ratio
Figure 2: LINT applied to full-rank task-optimized RNNs. a. Description of the context-dependent decision making (CDM) task inputs and outputs. Neural trajectories correspond to the neural firing rates, task output refers to a linear readout as eq. b. For the CDM task, similarity between trajectories produced by original full-rank networks and series of fitted networks of increasing rank (red), or low-rank networks obtained by truncating the original connectivity matrix (grey). Each line corresponds to randomly initialized full-rank network. c. Task accuracy of the same networks when associated with the original task readout. d-e. Same as b-c. for the delayed match-to-sampl (DMS) task. f. For the CDM task, similarities between trajectories of the original and fitted networks of rank one (blue) or full-rank (orange), when networks are fitted only to random subset of neurons of the original network. Error bars: mean + 1std over 10 random subsamples for each ratio value (see also sup. fig. D
3.2.1 Extracting low-dimensional dynamics through low-rank connectivity
We considered full-rank, vanilla RNNs trained on two systems neuroscience tasks, respectively context-dependent matiiiionmmaiigl222 (CDM) and delayed match-to-sample(3 (DMS). The full- rank RNNs reached 100% accuracy each task, and-as dynamies. From these RNNs, we generated trajectories corresponding to trials in the trained tasks, and then used LINT to infer IrRNNs of increasing rank.
low-dimensional
In the CDM task, network received two signal inputs which varied randomly around positive or negative mean, as well as two binary context cues, one of which was set to in each trial. The network was trained to output the average sign of the signal indicated by the active context, while ignoring the other signal (fig. 2 see Appendix B for task details). Applying LINT, we found that rank- networks are sufficient to accurately reproduce the trajectories of the trained full-rank networks. Increasing the rank improved the goodness-of-ff only marginally (fig. Pp). In contrast, low-rank networks obtained by truncating the SVD of the trained full-rank connectivity matrix required a rank of or to reach comparable accuracy (fig- ) and much higher for different hyperparameters (sup. fig. D.. This shows that LINT captures simplified connectivity structure that could not be trivially extracted from the original connectivity. Moreover, even though not explicitly trained on the behavioral outputs, the inferred rank-1 I networks performed the task correctly when plugged onto the original readout, implying that they faithfully captured the task-related dynamics (fig. 2
For the DMS task, we similarly found that rank-2 networks were able both to capture the dynamics of the trained full-rank network and to accurately perform the task, with no notable improvement when the rank is increased (fig. Ile) As for the CDM task, LINT vastly outperformed direct truncation of the full-rank connectivity matrix. For both these tasks, reproducing the experiment over larger number of full-rank RNNs, trained with diverse hyperparameters and random seeds led to similar results (sup. fig. g.
5

---

Finally, it is important to note that the CDM and DMS tasks are particularly prone to low-dimensions implementations, which is not necessarily the case for all computations of interest in systems neuroscience [8] To probe the capacities of LINT with higher-dimensionll tasks, we applied to full-rank networks trained on the K-back task, where a network receives random number P of stimulations in 1, and has to output the K-th last stimulation received, task directly inspired from sequence working memory paradigms [22] This task requires the implementation of an internal working memory with capacity of at least K bits, and should thus require rank at least K. We indeed observed that the rank identified by LINT increased with K, but only linearly (see details in appendix F and sup. fig. 2 These results show that LINT is also helpful in probing the dimensionality required for more complex tasks.
Subsampling. In experimental settings, one cannot expect to have access to all units of a network. It thus important to assess whether we can still recover the relevant low-dimensional dynamies and information when only a handful of neurons from network are recorded. We considered a full-rank network trained on the CDM task, and fitted networks to trajectories, either without constraining their rank or by fixing it to one. Without subsampling, the inferred full-rank networks slightly outper- formed rank-1 I networks in reproducing the original trajectories. Yet when considering subsamples of neurons, rank-1 networks appear to be more robust than unconstrained ones, maintaining good performance until subsampling ratios as low as 1% of original neurons (fig. 2 This experiment was reproduced on the DMS task setting, showing a similar advantage for low-rank fitted networks (sup. fig. S).
3.2.2 Extracting computational mechanisms from inferred low-rank connectivity
Low-rank models of behavioral tasks open the door to mechanistic interpretations of the underlying dynamics 30 g
Here we show how LINT allows extract computational mechanisms from full-rank networks performing the CDM task.
A first output of LINT is a set of interpretable axes defining a task-related subspace for dimen- sionality reduction. The inferred rank-1 model of the CDM task specifically yields five axes that correspond to the two stimulus inputs, the two context inputs and an internal latent variable, generated by recurrent connectivity, which rep- resents integrated evidence and therefore choice. Projecting the activity of the full-rank network along these axes shows how low-dimensional dynamics transform inputs into the choice out- put (fig. 3) In this case, the axes determined by LINT are closely related to those obtained by standard targeted dimensionality reduction (TDR, sup. fig. 2 [29] However, in contrast with TDR, the axes inferred by LINT corre- spond to connectivity features causing the low- dimensional dynamics, and provide method for an unsupervised discovery of recurrently- generated latent variables. In particular, we did not a priori specify that the activity along m should encode choice, but directly observed that choice was generated along that axis through recurrent dynamics.
Context
Context B
input la
Inout An
signal A coh.
rec. output m
rec. output m
input 1s
input j
signal B coh.
rec. output m
rec. output m
Figure 3: Low-dimensional projections produced by LINT. Trial-averaged trajectories for differ- ent task conditions, in the original full-rank net- work (full lines) and the inferred low-rank network (dashed lines), projected on axes obtained from the fitted IRRNN connectivity: input A and B axes, and the output axis of the rank-1 recurrent connectivity m, which encodes choice. All trajectories start at center.
Going beyond dimensionality reduction, LINT extracts an effective low-dimeniional model of the task. For the CDM task, the inferred model -ne-dimensional, non-linear latent dynamical system, where the latent variable integrates the relevant evidence. To understand how context- coneext-dependent selection of the evidence was performed, we analyzed the connectivity parameters of the inferred rank-1 1 RNN. Indeed, recent work has introduced a clustering method for analyzing low-rank connectivity, and has showed that in trained IrRNNs, context- dependent integration relies on gain-modulation mechanism mediated by two distinct populations of neurons [9] Applying similar clustering approach to rank-1 networks inferred by LINT led to

---

a.
b.
Ctx
C.
unperturbed No pop, A No pop.
Rank-one (infer (inferred)
task perf.
((x)
Full-rank (result)
task perf.
((x)
-2
0
2
-2
0
2
ctx A
ctx B
ctx B
ctx B
pop. A pop. B
X;
Xi
ctx
ctx
Figure 4: Extracting mechanisms and testing predictions from a low-rank network (top) inferred from a full-rank network trained on the CDM task (bottom). a. Three populations of neurons (grey, green and purple) identified by clustering connectivity parameters in the fitted rank-1 network (top). Here, we plot the same populations in joint distributions of contextual input vectors in the rank-1 (top) vs full-rank (bottom) networks. The populations are not directly identifiable from input parameters in the original network (bottom). b. Neural transfer function (tanh) and smoothed distributions of mean neural activity for populations and B in the inferred rank-1 network (top) and the full-rank network (bottom). Inactivation experiments: task performance on each context for unperturbed networks and after inactivating populations and in the fitted rank-1 network (top) and inactivating the same neurons in the original unconstrained network (bottom). Red dots: chance level.
comparable mechanism. Specifically, this method identified two populations distinguished by strong values of the fitted weights for the context-cue inputs (fig. Fa top and sup. fig. 2 In each context, different positions on the non-linear transform function ((x) (fig. 4p top). thereby modulating in opposite ways their gains (defined for each neuron the local slope ø' (z)). Combined with different statistics of the connectivity parameters on each of the populations, this modulation is sufficient to implement the desired context-switching behavior of the network (see details in Appendix D) The same pattern of gain modulation was found in the original full-rank network, indicating that is using a similar mechanism (fig- p bottom).
these context-eue inputs place the neurons belonging to the two populations
The computational mechanism extracted using LINT produces predictions for inactivations that can be directly tested in the original full-rank model. The analysis described above assigns neurons to different populations that have specific computational roles. Specifically, in the inferred rank-1 network, inactivating separately each population leads to a specific loss of performance in single context, but not the other one (fig. 4 top and sup. fig. 2 for detailed error patterns). Since individual neurons in the original networks directly correspond to-one basis, the identified populations can be directly mapped back onto the original network to reproduce the inactivation experiment. The predicted effects of inactivations on context-dependent performance were directly replicated the full-rank network, demonstrating that it relies on the identified computational mechanism (fig. 4 bottom and sup. fig. 2 One can note that although the two identified populations clearly stand out in the inferred rank- connectivity, they are not directly apparent in the original full-rank connectivity (fig. 4h bottom), showing that although the mechanism extracted by LINT applies to the full-rank network, using an IrRNN was necessary step to uncover it.
neurons in the rank-1 1 network
one-
3.3 Application to neural recordings
We next applied our method vivo recordings. We considered here electrophysiological recordings from non-human primates performing a context-dependent decision-making task similar to that studied in previous sections 29 More specifically, two macaques (designed by A and F) were presented with random dots stimuli that varied along two dimensions: the overall motion direction of the dots, and their overall color, ranging from two extremes with set of intermediary coherences in between. At the beginning of each trial, cue indicated context for the trial, ordering the
7

---

Context A
Context B
R² accuracy
motion coh.
Imotion
Imation
global
accuracy
3
full
m
b.
rank
1.0
color coh.
0.5
lcoar
lcator
0.0
-0.5
-1.0
10
m
m
Mean FR
Figure LINT applied to neural recordings from non-human primate performing context- dependent task (monkey A) 29 a. Global R2 of model trajectories with respect to original ones, and task accuracy of inferred networks of increasing ranks. b. For the rank-1 inferred network, R² of each recorded neuron plotted against its average data firing rate, median Rt in red, with marginal box-plot of R values. 14 neurons with an R² <- -1 are not shown, all with mean firing rate < 2.2 (see sup. fig. 4.. c. Trial-averaged trajectories in the recorded data (full lines) and for the rank-1 model (dashed lines), for different task conditions (only one coherence value for each coherence sign is plotted here), projected axes identified by LINT (motion and color input axes and the output axis m of the rank- recurrent matrix). Trajectories start at the center.
subject to report either the average motion or color with eye saccade, while ignoring the irrelevant evidence. Non-simultancous recordings were performed in the frontal eye field (FEF) an area of the macaque prefrontal cortex, with respectively 727 neurons for monkey A and 574 neurons for monkey recorded in all 72 conditions the task presented (ignoring error trials).
For the analysis, we started by binning (5 ms) and smoothing (50 ms std Gaussian window) spike train data, and then computed the treal-averaged response of every neuron in each of the task conditions, forming a pseudo- pow.ulatioiotaiinnnnoooo tensor of size N x T 72 with N the number of neurons in each monkey and = 150 the number of discrete time steps. We denoised this tensor by projecting its first axis the subspace spanned by the top 12 principal components of the pseudo-populatio activity, and then projecting back to the high-dimensional space spanned by all neurons [29]. Finally, in order consider only task-related neural activity, subtracted from each neuron's trajectory its condition -vveraee mean. We denote the entries of the final obtained tensor by it,t.e.
We applied LINT to the obtained trajectories, training the activation of each unit in the low-rank RNN (noted 2,,,,e) to match the corresponding pre-processed activity. Importantly, the inferred networks received inputs following the same structure than in the CDM task of previous sections, that is two noisy signal inputs and two contextual cue inputs (fig. 2 and were left unconstrained for the first 350 ms of each trial while receiving only contextual cues, account for the original task procedure. In particular, we made the implicit hypothesis that choice signals are generated by the recurrent activity of the network from received inputs. The quality of fits was quantified by leaving out random subset of 8 conditions during network inference, and evaluating the R² of fitted networks on these left-out conditions.
We found that for both monkeys the neural activity was well reproduced by rank-1 network, with minimal improvements when the rank was increased (fig. 5 and sup. fig. 5 R² = 0.66 for monkey A, and R 0.57 for monkey F). Moreover, when simulated independently, the inferred rank- networks were able to perform the task by adding linear readout (accuracy curves in fig. Dh. sup. fig.55. (D. This was the case even though the recurrent and input connections were not trained on task performance, demonstrating that the reproduced trajectories contained information about the choice made by the monkey. The activity of both original and reproduced trajectories can moreover be projected on the axes identified by LINT, providing geometrical picture of task exccution (fig. 5b). These axes were

---

closely related to those found by targeted dimensionality reduction: notably the context, motion, and color axes identified by TDR closely match the corresponding input axes in the inferred network, while the choice TDR axis could be identified to the m axis of the rank-1 recurrent connectivity (sup. figs. D The projections of the original activity on the connectivity axes (full lines on fig. 5))
therefore showed how inferred connectivity explains the geometry of recorded data.
closer look at the distribution of inferred connectivity weights provided more information on the neural mechanisms at play (sup. figs. 6 D. In contrast to the fits of task-optimized networks, we observed for both monkeys a clearly non- nor-normal, meavy-taalldd distribution of inferred connectivity parameters (Fisher's ' kurtosis between 4.9 and 62.7 for different connectivity parameters), echoing some past observations on biological synaptic weights [51 D. Separating with clustering algorithm (GMM) large and small-weight neurons showed that the latter could be inactivated without affecting the task performance of the network in significant way (sup. figs. 6 7. even though they repre- sented majority of neurons in the circuits (570/727 neurons in monkey A, 389/574 in monkey F). This suggests that minority of neurons with large connectivity weights supports the computation performed by these networks.
4
Discussion
In this paper, we introduced LINT, a new method for inferring a latent dynamical system from neural activity based on the theory of low-rank RNNs. This method yields mechanistic, interpretable and predictive model of neural trajectories, and bridges different levels of analysis, from state-space geometry to neural connectivity. After verifying the consistency of our method, we demonstrated its potential to extract the mechanisms used by "black-box box" vanilla RNNs trained to perform cognitive tasks. In particular, we found that low-rank RNNs reproduced well the dynamics of full-rank RNNs trained on a variety of tasks and across range of hyperparameters. Moreover, the obtained IrRNNs provided interpretable low-dimensional projections of the fitted activity, as well as predictions for the effects of inactivations of specific populations of neurons. These predictions were verified in the original network, validating computational mechanisms derived from inferred low-rank connectivity weights. Finally, we applied LINT to neural activity recorded in the prefrontal cortex of nonhuman primates during a context-dependent decision decision-making task. LINT was able to infer rank-1 I networks that reproduced both neural activity and task performance, from which low-dimensional projections and interpretable connectivity could be extracted.
Our results pave the way for a better understanding of how RNNs perform computations. In particular, LINT complementary to other approaches seeking to reverse-engineer RNNs 555] 29 50 in that it infers an effective connectivity that links low-dimensional dynamics with interactions at the single neuron level, opening possibilities for goal-oriented interventions and a better control on the behavior of trained networks. Moreover, LINT leads to compressed, distilled versions of large networks that can be of practical interest to engineers, as suggested by recent research in low-rank training of deep neural networks (00). Extensions to engineeinginn more complex architectures like GRUs or
LSTMs, which have already been shown to rely on low-dimensional trajectories [28. 26 21
would
be an interesting direction for future research.
It worth noting that other statistical methods such as mTDR can provide a better to the same data, but do not identify dynamical system that generates the trajectories. LINT is related to switching latent dynamical systems ICC 50 24
IT 12, 58 in that it bridges descriptive latent- space analyses like mTDR with methods that identify a predictive model of the dynamics, such as LFADS B3 A distinctive fcature of LINT is however that (unlike LFADS) it retains a mechanistic description at the level of neural connectivity of how the dynamics emerge from interactions between neurons.
As number of other data analysis methods for neuroscience, our approach relies on the ubiquitous observation that neural dynamics appear to be confined to low-dimensional manifolds within the activity state-space 7 17 43 Recent works have proposed that this low-dimensionality might be an artifact stemming from the simplicity of the examined tasks [8], or have reported higher-dimensional
59 This raises the question of how far our method could account for activity in more complex tasks. Our results on the K-back task suggest that LINT scales well with task dimensionality, and hint to a possible compositional implementation of internal computations
activity patterns ( 2

---

Further confronting our and related methods to nigher-dimensional task designs however remains an important endeavour.
that could be unpacked by breaking them into simpler primitives 63 8
Distinguishing signals generated by dynamics within a given area from external inputs receives is major challenge for any interpretation of neural recordings, and focus of recent work T6 25 42
Most methods for inferring latent dynamics from activity data rely on ad hoc assumptions about the inputs received by circuit. Following previous work 2 when analyzing recordings from the prefrontal cortex we hypothesized that choice was recurrently generated within the recorded area.
LINT however potentially provides a new approach for testing hypotheses on input structure, by comparing models fits obtained from single-tri trial recordings, or by building multi-area models [2]
from large-scale recordings that are increasingly available nowadays 40 48 52 35
Acknowledgments and Disclosure of Funding
The authors would like to thank Valerio Mante for kindly providing access to the monkey data, and João Barbosa for relevant discussions. AV and SO were supported by the NIH Brain Initiative project 121-NS122123 and the program "Ecoles Universitaires de Recherche" launched by the French Gov- ermment and implemented by the ANR, with the reference ANR-I7-EURE-0017, JWP was supported by grants from the Simons Collaboration the Global Brain (SCGB AWD543027), the NIH BRAIN initiative ER01690))44)), and U19 NIH-NINDS BRAIN Initiative Award (5U19NS10466)
10

---

References
[1] Mikio C Aoi, Valerio Mante, and Jonathan W Pillow. Prefrontal cortex exhibits multidimensional dynamic encoding during decision- decision-making. Nature neuroscience, 23(11); 11:1414-1442) 2020.
[2] Joao Barbosa, Remi Proville, Chris C Rodgers, Srdjan Ostojie, and Yves Boubenec. Flexible selection of task-relevant features through across-area population gating. bioRxiv, 2022.
[3] Manuel Beiran, Alexis Dubreuil, Adrian Valente, Francesca Mastrogiuseppe, and Srdjan Os- tojic. Shaping dynamics with multiple populations in low-rank recurrent networks. Neural computattin, 336)):112611 2021.
[4] György Buzsáki and Kenji Mizuseki. The log-dynamic brain: how skewed distributions affect network operations. Nature Reviews Neuroscience, 15(4):264 2014.
[5] Warasinee Chaisangmongkon, Sruthi K Swaminathan, David J Freedman, and Xiao-Jing Wang. Computing by robust transience: how the fronto-parietal network performs sequential, category- based decisions. Neuron, 93(6):1504-1517 2017.
[6] Zach Cohen, Brian DePasquale, Mikio C Aoi, and Jonathan W Pillow. Recurrent dynamics of prefrontal cortex during context-dependent decision-making. bioRxiv, 2020.
[7] John P Cunningham and M Yu Byron. Dimensionality reduction for large-scale neural record- ings. Nature neuroscience, 17(1 1): 1500-1555, 2014.
[8] Laura Driscoll, Krishma Shenoy, and David Sussillo. Flexible multitask computation in recurrent networks utilizes shared dynamical motifs. bioRxiv, 2022.
[9] Alexis Dubreuil, Adrian Valente, Manuel Beiran, Francesca Mastrogiuseppe, and Srdjan Ostojic. The role of population structure in computations through neural dynamics. Nature Neuroscience, 25:783-794, 2022.
[10] Lea Duncker, Gergo Bohner, Julien Boussard, and Maneesh Sahani. Learning interpretable continuous-time models of latent stochastic dynamical systems. In International Conference Machine Learning, pages 7726-1734, PMLR, 2019.
[11] Lea Duncker and Maneesh Sahani. Dynamics on the manifold: Identifying computational dynamical activity from neural population recordings. Current opinion in neurobiology. 70:163- 170, 2021.
[12] Daniel Durstewitz. A state space approach for piecewise-linear recurrent neural networks for identifying computational dynamics from neural measurements. PLoS computational biology, 1366):e100554, 2017.
[13] Chris Eliasmith and Charles H Anderson. Neural engineering: Computation, representation, and dynamics in neurobiological systems. MIT press, 2003.
[14] Arseny Finkelstein, Lorenzo Fontolan, Michael N Economo, Nuo Li, Sandro Romani, and Karel Svoboda. Attractor dynamics gate cortical information flow during recision-making Nature Neuroscience, 24(6):843 -40-8500, 2021.
[15] Timo Flesch, Keno Juechems, Tsvetomira Dumbalska, Andrew Saxe, and Christopher Summer- field. Orthogonal representations for robust context-dependent task performance in brains and neural networks. Neuron, 2022.
[16] Aniruddh R Galgali, Maneesh Sahani, and Valerio Mante. Residual dynamics resolves recurrent contributions to neural computation. bioRxiv, 2021.
[17] Juan A Gallego, Matthew G Perich, Lee E Miller, and Sara A Solla. Neural manifolds for the control of movement. Newron, 94(5):978-98) 2017.
[18] Peiran Gao, Eric Trautmann, Byron Yu, Gopal Santhanam, Stephen Ryu, Krishna Shenoy, and Surya Ganguli. theory of multineuronal dimensionality, dynamics and measurement. BioRxiv, page 214262, 2017. 2017.
11

---

[19] Joshua Glaser, Matthew Whiteway, John P Cunningham, Liam Paninski, and Scott Linderman. Recurrent switching dynamical systems models for multiple interacting neural populations. Advances in neural information processing systems, 33:14867-14878, 2020.
[20] Siddhartha Rao Kamalakara, Acyr Locatelli, Bharat Venkitesh, Jimmy Ba, Yarin Gal, and Aidan N Gomez. Exploring low rank training of deep neural networks. arXiv preprint urXiv:2209.13566 2022.
[21] Kamesh Krishnamurthy, Tankut Can, and David J Schwab. Theory of gating in recurrent neural networks. Physical Review X, 12(1):01 1011, 2022.
[22] Christopher Langdon and Tatiana A Engel. Latent circuit inference from heterogeneous neural responses during cognitive tasks. bioRxiv, 2022.
[23] Frederic Lanore, Alex Gayeo, Harsha Gurnani, Diccon Coyle, and R Angus Silver. Cerebellar granule cell axons support high- -igge-imensional representations. Nature Neuroscience, 24(8):1142-11555 2021.
[24] Scott Linderman, Matthew Johnson, Andrew Miller, Ryan Adams, David Blei, and Liam Paninski. Bayesian learning and inference in recurrent switching linear dynamical systems, In Artificial Intelligence and Statistics, pages 914-922. PMLR, 2017.
[25] Jakob H Macke, Lars Buesing, John P Cunningham, Byron M Yu, Krishna V Shenoy, and Mancesh Sahani. Empirical models of spiking in neural populations. Advances in neural information processing systems, 24, 2011.
[26] Niru Maheswaranathan and David Sussillo. How recurrent networks implement contextual processing in sentiment analysis. In Proceedings of the 37th International Conference on Machine Learning, pages
2020.
[27] Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, and David Sussillo. Re- verse engineering recurrent networks for sentiment classification reveals line attractor dynamics. Advances in neural information processing systems, 32, 2019.
[28] Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, and David Sussillo. Universality and individuality in neural dynamics across large populations of recurrent networks. Advances in neural information processing systems, 32, 2019.
[29] Valerio Mante, David Sussillo, Krishna V Shenoy, and William Newsome, Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature, 503(7474):78-84, 2013.
[30] Francesca Mastrogiuseppe and Srdjan Ostojic. Linking connectivity, dynamics, and computa- tions in low-rank recurrent neural networks. Neuron, 99(3):609-623, 2018
[31] Marcel Nonnenmacher, Srinivas C Turaga, and Jakob H Macke. Extracting low-dimensional dy- namics from multiple large-scale neural population recordings by learning to predict correlations. Advances in Neural Information Processing Systems, 30, 2017.
[32] Marius Pachitariu, Biljana Petreska, and Maneesh Sahani. Recurrent linear models of simultaneously-recoreedd neural populations. Advances in neural information processing systems, 26, 2013.
[33] Chethan Pandarinath, Daniel J O' Shea, Jasmine Collins, Rafal Jozefowicz, Sergey D Stavisky, Jonathan C Kao, Eric M Trautmann, Matthew Kaufman, Stephen Ryu, Leigh Hochberg, al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nature methods, 15(10):805-815. 2018
[34] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
[35] Matthew G Perich, Charlotte Arlt, Sofia Soares, Megan Young, Clayton Mosher, Juri Minxha, Eugene Carter, Ueli Rutishauser, Peter H Rudebeck, Christopher D Harvey, et al. Inferring brdin-wide interactions using data-constrained recurrent neural network models. bioRxiv, pages 2020-12,4 2021.
12

---

[36] Biljana Petreska, Byron M Yu, John P Cunningham, Gopal Santhanam, Stephen Ryu, Krishna V Shenoy, and Maneesh Sahani. Dynamical segmentation of single trials from population neural data. Advances in neural information processing systems, 24, 2011.
[37] Eli Pollock and Mehrdad Jazayeri. Engineering recurrent neural networks from task-relevant manifolds and dynamics. PLoS computational biology, 16(8):e1005) 1288 2020.
[38] Kanaka Rajan, Christopher D Harvey, and David W Tank. Recurrent network models of sequence generation and memory. Neuron, 90(1))128-144) 2016.
[39] Evan D Remington, Devika Narain, Eghbal A Hosseini, and Mehrdad Jazayeri. Flexible senso- rimotor computations through rapid reconfiguratio of cortical dynamics. Neuron, 9805):1005- 1019, 2018.
[40] Omid G Sani, Hamidreza Abbaspourazad, Yan Wong, Bijan Pesaran, and Maryam M Shanechi. Modeling behaviorally relevant neural dynamies enabled by preferential subspace identification. Nature Neuroscience, 14(1):140-1499 2021.
[41] Omid G Sani, Bijan Pesaran, and Maryam M Shanechi. Where all the nonlinearity: flexible nonlinear modeling of behaviorally relevant neural dynamics using recurrent neural networks. bioRxiv, 2021.
[42] Britton A Sauerbrei, ShonZZhong Guo, Jeremy D Cohen, Mattco Mischiati, Wendy Guo, Mayank Kabra, Nakul Verma, Brett Mensh, Kristin Branson, and Adam W Hantman. Cortical pattern generation during dexterous movement is input-driven. Nature, 577(7790):3863311 2020.
[43] Shreya Saxena and John P Cunningham. Towards the neural population doctrine. Current opinion in neurobiology, 55:103-111, 2019.
[44] Rylan Schaeffer, Mikail Khona, Leenoy Meshulam, Ila Fiete, et al. Reverse-engineerin recurrent neural network solutions to hierarchical inference task for mice. Advances in Neural Information Processing Systems, 33:4584-4596 2020.
[45] Marine Schimel, Ta-Chu Kao, Kristopher T Jensen, and Guillaume Hennequin. ilqr-vae: control-based learning of input-driven dynamics with applications to neural data. bioRxiv, 2021.
[46] Friedrich Schuessler, Alexis Dubreuil, Francesca Mastrogiuseppe, Srdjan Ostojic, and Omri Barak. Dynamics of random recurrent networks with correlated low-rank structure. Physical Review Research, 2(1):01311,, 2020.
[47] Friedrich Schuessler, Francesca Mastrogiuseppe, Alexis Dubreuil, Srdjan Ostojic, and Omri Barak. The interplay between randomness and structure during learning in rnns. Advances in neural information processing systems. 33:13352-13362. 2020.
[48] João D Semedo, Amin Zandvakili, Christian K Machens, M Yu Byron, and Adam Kohn. Cortical areas interact through a communication subspace. Neuron, 102(1))249-259, 2019.
[49] Markus Siegel, Timothy J Buschman, and Earl K Miller. Cortical information flow during flexible sensorimotor decisions. Science,
2015.
[50] Jimmy Smith, Scott Linderman, and David Sussillo. Reverse engineering recurrent neural networks with jacobian switching linear dynamical systems. Advances in Neural Information Processing Systems, 34, 2021.
[51] Sen Song, Per Jesper Sjöström, Markus Reigl. Sacha Nelson, and Dmitri B Chklovskii. Highly nonrandom features of synaptic connectivity in local cortical circuits. PLoS biology, 3(3)=66, 2005.
[52] Nicholas A Steinmetz, Peter Zatka-Haas, Matteo Carandini, and Kenneth D Harris. Distributed coding of choice, action and engagement across the mouse brain. Nature, 576(7786):266-775 2019.
[53] Carsen Stringer, Marius Pachitariu, Nicholas Steinmetz, Matteo Carandini, and Kenneth D Har- ris. High-dimensional geometry of population responses visual cortex. Nature, (71(6765):361-
365, 2019.
13

---

[54] David Sussillo. Neural circuits as computational dynamical systems. Current opinion in neurobiology. 25:156-16:3 2014.
[55] David Sussillo and Omri Barak.
Opening the black box: low-dimensional dynamics in high- dimensional recurrent neural networks. Neural computation, 25(3):626-649, 2013.
[56] David Sussillo, Mark M Churchland, Matthew T Kaufman, and Krishna V Shenoy. neural net- work that finds naturalistic solution for the production musele activity. Nature neuroscience, 18(7):0225-1033, 2015.
[57] Elia Turner, Kabir V Dabholkar, and Omri Barak. Charting and navigating the space of solutions for recurrent neural networks. Advances in Neural Information Processing Systems, 34:25320-25333, 2021.
[58] Adrian Valente, Srdjan Ostojie, and Jonathan Pillow. Probing the relationship between latent linear dynamical systems and low-rank recurrent neural network models. Neural computation, 34(():1871-18)) 2022.
[59] Ivan Voitov and Thomas D Mrsic-Flogel. Cortical feedback loops bind distributed representa- tions of working memory. Nature, 608(7922)3381 2022.
[60] Saurabh Vyas, Matthew D Golub, David Sussillo, and Krishna V Shenoy. Computation through neural population dynamics. Anmual Review of Neuroscience, 43:249-275, 2020.
[61] Jing Wang, Devika Narain, Eghbal A Hosseini, and Mchrdad Jazayeri. Flexible timing by temporal scaling of cortical responses. Nature neuroscience, 21 110, 2018.
[62] Yang Xie, Peiyao Hu, Junru Li, Jingwen Chen, Weibin Song, Xiao-Jing Wang, Tianming Yang. Stanislas Dehaene, Shiming Tang, Bin Min, et al. Geometry of sequence working memory in macaque prefrontal cortex. Science, 375(6581) (116668)))662-63) 2022.
[63] Guangyu Robert Yang, Madhura Joglekar, H Francis Song. William T Newsome, and Xiao- Jing Wang. Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience, 22(2):297-306, 2019.
[64] Guangyu Robert Yang and Xiao-Jing Wang. Artificial neural networks for neuroscientists: A primer. Neuron, 107(6))0048-)) 2020.
[65] Byron M Yu, Afsheen Afshar, Gopal Santhanam, Stephen Ryu, Krishna V Shenoy, and Maneesh Sahani. Extracting dynamical structure embedded in neural activity. Advances in neural information processing systems, 18, 2005.
[66] Rafael Yuste. From the neuron doctrine to neural networks. Nature reviews neuroscience, 16(8):487-497, 2015.
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] The three contributions exposed the introduction are detailed in the results section. The specific scope is detailed in the introduction and the discussion.
(b) Did you describe the limitations of your work? [Yes] Limitations are exposed in the discussion, notably with respect hypotheses made about the inputs.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See discussion.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [N/A] No novel theoretical results are exposed here, the theoretical results on low-rank RNNs already exist the literature, notably in [3]
(b) Did you include complete proofs of all theoretical results? [N/A] Same answer.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [No] Code will be made available upon publication.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] In the supplemental material, Appendix C
(c) Did you report eπrоr bars (c.g., with respect to the random seed after rumning exper- iments multiple times)? [Yes] Figure 2 contains whole distributions for the fitting experiments, standard deviation for the subsampling experiment, and the material exposed in section 3.3 and figure 5 has been fully reproduced on data from a second macaque subject. Moreover, additional hyperparameters
explored in sup. fig. (d) Did you include the total amount compute and the type of resources used (c.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] In the supplemental material, Appendix C
4. If you using existing assets (e.g., code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] Providers of the dataset 29 cited section 3.3
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [No] Trained network weights will be provided upon publication.
(d) Did you discuss whether using/curating? [N/A] No human participants.
how consent was obtained from people whose data you're
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A] No human participants not offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A] No human participants.
(b) Did you deseribe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A] No human participants.
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A] No human participants.
15


Reviews:
[{"review_id": "wiJgKauMfkj", "reviewer": "Reviewer_ittd", "summary": "This paper introduces low-rank RNNs, parameterized by the left and right eigenvectors of the connectivity matrix. The authors first show on simulated full-rank vanilla RNNs trained to perform some cognitive tasks that low-rank RNNs can accurately extract the essential dynamical and task structure learned by the full-rank systems. They thereby offer a computationally interpretable low dimensional representation of task activity. The authors then test their approach on neural recordings from two monkeys performing a context-dependent decision making task. Again the trained low-rank RNNs seem to extract the behaviorally relevant low-dimensional dynamics, at the same time producing synaptic weight distributions reminiscent of those observed in the brain.", "questions": "1) One may perhaps still ask what is really novel about this approach, given that low-rank RNN-type models have been employed in computational neuroscience for a number of years now to obtain insight into neural dynamics and task performance. I assume the authors are the first to actually train such a model directly on simulated and real neural data and to systematically explore its abilities in identifying the dynamical structure?\n\n2) Directly inferring a lrRNN seems to be more powerful than training a full-rank RNN and using SVD posthoc. But why actually? Can the authors offer some theoretical insight why this is more efficient? Does this simply have to do with the much lower number of parameters that need to be inferred on lrRNNs, easing the training process?\n\n3) The authors call the CDM and DMS tasks ‘complex’, but compared to many real world tasks and those in use elsewhere in the ML community I find them almost simplistic. Could the low-rank structure be an ‘artifact’ or result of the very simple nature of these tasks? For instance, say you were to construct a very high-dimensional space where dimensions are somewhat modulated by task features like inputs, would a lrRNN always infer this same structure cos it’s really a property of the task, not of the recorded neural activity?\n\nI think this is a relevant question since I was surprised that, despite the very simple nature of the tasks, hundreds of neurons were used for the full-rank RNNs trained on the tasks. More importantly, the neural recording data sets appear to consist likewise of hundreds of neurons which were however drawn *independently* from different sessions and trials? This appears to imply that the neurons are *not* truly dynamically connected, and do not have any correlations beyond that induced by the task structure. Thus, the inferred lrRNNs may simply reflect this task structure but not really the processing and (missing) dynamics that goes on in the neural population during a task.\n\n4) A related question: As I assume hundreds of RNN units are not really necessary for learning these tasks, couldn’t we just train a much lower-dimensional system to begin with and thereby obtain a much more parsimonious representation of dynamics? I would guess for these tasks 5 units may actually already do the job?\n\n5) Is the phenomenon of mixed selectivity in cortical networks at odds with the low-rank structures identified?\n\n6) I would like to see more examples of original and fitted neural firing rates and neural trajectories. For instance, in sect. 3.1 the claim is made that essential connectivity features and neural trajectories were well reproduced by the lrRNN, but Table 1 shows only a single summary statistic w/o confidence limits. Same goes for the real neural recordings, more unit examples that illustrate neural behavior on different trials of the task would be great.\n\n7) Since Fig. 3-5 appear to show just examples of a single training run, I would like to see more stats on how robust the training process is, i.e. how likely the same structures are to be identified again when starting from many different initial conditions for parameters?\n\nI think point #3 is really my main issue. In any case it would have been nice to see the application of this method to at least one other data set with a very different task. But given the third point, in my mind it’s essential to see an application on a higher-dimensional set of truly simultaneously recorded neurons. \n\nMinor points:\n\n- Line 129: Universal approximation properties in my mind don’t apply here: To assure this, you would need to allow for a an arbitrarily long basis expansion, commonly implying that J would need to be of full rank and, in addition, you may need many more latent dim. than observed variables.\n\n- My understanding is that for any real physiological data set one would need to re-train the lrRNN for a number of different R’s?\n\n- Line 146-147: says that at least rank of 9 is necessary, but Fig. 2 only shows up to rank 5.\n\n- Would be good to have connectivity matrices from the Suppl. in main text. I think they add a lot to the understanding of the inner workings of the trained lrRNNs.\n\n- I’m somewhat puzzled why so many units are used for lrRNNs to begin with, and then kept after training? The results on the CDM, for instance, seem to imply one just needs 5 units?\n\n- Does the lrRNN trained on the monkey data have as many dimensions as units? Isn’t this a constraint that could also prevent detection of latent structure in the data, since there are no latent variables in the lrRNN that could account for unobserved processes?\n", "limitations": "Partly, see questions above.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nGenerally speaking, I think this is a powerful approach that may have important implications in neuroscience. Building a low-rank structure right away into the models used for inference may profoundly ease detection of computational mechanisms from data.\n\nWeaknesses:\nSee detailed questions below. My major point probably is that I don't think it's clear whether the model really extracts the neural dynamics and how an animal is truly solving the task, or whether it merely finds a parsimonious solution to the task.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "rrlH_jJfgV_", "reviewer": "Reviewer_98wL", "summary": "This paper studies the problem of modeling dynamical systems using low-rank RNNs. The paper motivates a number of domains in which low-rank RNNs have given insight into the computations underlying a particular task. The paper proposes directly fitting low-rank RNNs (meaning the connectivity matrix in the RNN is restricted to be low-rank). The paper demonstrates how this restriction allows one to learn the correct effective connectivity for a number of synthetic tasks, and also applies the method to recordings from macaque prefontal cortex while the animals performed a context-dependent decision making task. This method reveals that a rank one network is sufficient to capture the neural activity.", "questions": "- Why use an explicit (hard) rank constraint, rather than an implicit (soft) constraint, for example by penalizing the nuclear norm of the connectivity matrix? My understanding is that the latter is admits an easier optimization problem, as the nuclear norm is the convex relaxation of a hard rank constraint.\n- There are multiple gray and red curves in Fig 2. What do they correspond to? Different random seeds?\n- Why is there no equivalent of Fig 2f. for the DMS task?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is well written, and does a good job of motivating the problem and summarizing relevant work.\n2. The paper's focus on low-rank structure as an important inductive bias for neuroscience is not particularly new, but is nonetheless important. For example, Fig 2f. is a powerful reminder of the utility of the appropriate inductive biases for systems neuroscience, where we are (generally) only able to sample a tiny fraction of the relevant neurons participating in a given task.\n3. The experiments on synthetic tasks as well as the application to neural data are clear and high quality.\n\nWeaknesses:\n1. The main contribution of the paper is essentially advocating for a type of regularization (low-rank structure), one that has been discussed plenty in the literature (for example by Mastroguiseppe and Ostojic, which is already cited by the paper). While important, the overall significance and novelty does not seem that high.\n2. It is not clear how one generalizes the method to RNNs beyond the ones described by equation (1). For example, what about RNNs with multiple weight matrices (such as gated architectures: GRUs or LSTMs)? These are commonly used in machine learning and neuroscience, I am curious if or how the authors would apply their method to those networks.\n3. The paper studies tasks which can readily be solved by a low-rank vanilla RNN. But the space of tasks that we are interested in understanding is much greater! What happens if you apply this method to a dynamical system that is solving a more complicated task? For example, one that does not admit a low-rank solution? It would be good to get a sense of what to expect from the method in these scenarios.\n\nMinor suggestions for improvement:\n- Should explain the \"effective connectivity\" in the main text. It is used throughout, and thus seems important enough to put in the main text (currently the explanation is in appendix A.4).\n- The entire PDF is low resolution (for exmample text) which makes it hard to read.\n- This is a nitpick, but replace $J^{eff}$ with $J^{\\text{eff}}$. That will prevent the \"eff\" from being italicized (which should only be done for mathematical variables).\n\nOverall, I am not so worried about point (1) under weaknesses, but look forward to see if the authors can address (2) and (3) in the rebuttal.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "nSXAXj4HwXT", "reviewer": "Reviewer_db5J", "summary": "In this paper, the authors use low-rank RNNs for modelling neural data. The connectivity matrix is approximated using a low-rank representation, which is then inferred by minimising the loss between the predicted neural outputs and the target ones. In task-optimised RNN, relevant parameters to are trained based on task targets. The method is then applied to various cognitive tasks to validate and extract insights.  \n", "questions": "- The assumption is 'm' vectors are orthogonal to each other (and similarly for 'n'). How is this constraint enforced during training? \n\n- 3.2 assumes that m,n, I, and w are all zero-centred. Please expand on why this is a valid assumption.\n\n- One can first obtain a full-ranked connectivity matrix and then approximate it using the low-ranked method suggested here (similar to what is done for SVD in the paper) rather than learning a low-ranked approximation directly. How is this alternative compared to what is suggested in the paper? On a similar note, one can train a truncated SVD directly (rather than making the approximation post-training); how is this method's performance compared with the proposed method?", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \nThe models used are rather simple, which benefits from good interpretability. \nThe experiments considered are interesting and elucidate different aspects of the model.\n\nWeaknesses:\nNovelty. The novelty of the paper in terms of the model is very limited. The paper shows that low-rank RNNs can help analyse neural data, but the contributions of the work beyond these experimental evaluations and applications of the low-rank RNNs are unclear.\n\nMinor weaknesses:\nPresentation: From the main text, it is unclear how \\kappa and \\v enter the analysis.\nSimilarly, the role of section A.2 is rather unclear. Is it to justify the effective connectivity metric?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DgKRFPYA06u", "reviewer": "Reviewer_8DBW", "summary": "This paper describes results from a method for estimating low rank RNNs from observed trajectories.  The method itself is just gradient descent on the set of weights that specify a low-rank RNN to match observed trajectories.  The paper demonstrates that the method can recover the effective connectivity of known low-rank RNNs, does a good job describing the behavior of general RNNs trained on a set of four tasks, and does a reasonable job recovering neural trajectories from NHPs performing one of those tasks.", "questions": "It would be very informative to include a situation where the dynamics are in fact relatively high-dimensional.  Note that a full-rank RNN trained on the tasks used here does not necessarily exhibit the growth in dimensionality.  One possibility is to compare to an RNN trained to distinguish a large number of temporal intervals.  ", "limitations": "\nThe paper should do a better job at pointing to the computational limitations of low-rank RNNs and their possible limitations in describing brain data and cognition.\n\n", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nGiven the widespread interest in RNNs in machine learning and neuroscience, this method may find broad applications in both of those fields.  The low-rank description is more readily interpretable.\n\nWeaknesses\n\nI think the paper seems to take as a given that low rank RNNs can provide a veridical description of brain dynamics and learn many tasks.  This is not at all well established.  The tasks chosen here are well-described by low-dimensional RNNs, but many tasks and natural networks appear not to have that property.  For instance, Cueva et al., (2020, PNAS) estimate the dimensionality of neural populations during the delay period of a variety of experiments (see especially their figure 7).  They find a number on the order of 5-10 for the dimensionality over a few seconds.  Leaving aside technical questions about whether a particular number is meaningful, one can read their result as decelerating but growing without bound (e.g., dimensionality $\\propto \\log t$), or as saturating at some time $T$.  The first interpretation argues that a low-rank description is not appropriate---or at least that rank grows to be much larger than 10.  The other possibility---that dimensionality saturates at some time T---suggests it would be impossible for the network to distinguish times $> T$.  But that seems inconsistent with the data (e.g., Lewis & Miall, 2009, Trans Royal Soc B). \n\nKind of a minor issue, but submitting a set of images instead of a searchable PDF made the job of reviewing this paper more difficult than it needed to be.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept