Title: Partial observation can induce mechanistic mismatches in data-constrained models of neural dynamics
Venue: NeurIPS 2024
Accepted: yes
Keywords: neuroscience

recurrent neural networks

dynamical systems
Abstract: One of the central goals of neuroscience is to gain a mechanistic understanding of how the dynamics of neural circuits give rise to their observed function. A popular approach towards this end is to train recurrent neural networks (RNNs) to reproduce experimental recordings of neural activity. These trained RNNs are then treated as surrogate models of biological neural circuits, whose properties can be dissected via dynamical systems analysis. How reliable are the mechanistic insights derived from this procedure? While recent advances in population-level recording technologies have allowed simultaneous recording of up to tens of thousands of neurons, this represents only a tiny fraction of most cortical circuits. Here we show that observing only a subset of neurons in a circuit can create mechanistic mismatches between a simulated teacher network and a data-constrained student, even when the two networks have matching single-unit dynamics. In particular, we show that partial observation of models of low-dimensional cortical dynamics based on functionally feedforward or low-rank connectivity can lead to surrogate models with spurious attractor structure. In total, our results illustrate the challenges inherent in accurately uncovering neural mechanisms from single-trial data, and suggest the need for new methods of validating data-constrained models for neural dynamics.

Partial observation can induce mechanistic mismatches in data-constrained models of neural dynamics

Abstract

One of the central goals of neuroscience is to gain a mechanistic understanding of how the dynamics of neural circuits give rise to their observed function. A popular approach towards this end is to train recurrent neural networks (RNNs) to reproduce experimental recordings of neural activity. These trained RNNs are then treated as surrogate models of biological neural circuits, whose properties can be dissected via dynamical systems analysis. How reliable are the mechanistic insights derived from this procedure? While recent advances in population-level recording technologies have allowed simultaneous recording of up to tens of thousands of neurons, this represents only a tiny fraction of most cortical circuits. Here we show that observing only a subset of neurons in a circuit can create mechanistic mismatches between a simulated teacher network and a data-constrained student, even when the two networks have matching single-unit dynamics. In particular, partial observation of models of low-dimensional cortical dynamics based on functionally feedforward or low-rank connectivity can lead to surrogate models with spurious attractor structure. Our results illustrate the challenges inherent in accurately uncovering neural mechanisms from single-trial data, and suggest the need for new methods of validating data-constrained models for neural dynamics.

1 Introduction

In recent years, advances in recording techniques have brought forth a deluge of neural data. Simultaneous measurements of the activity of hundreds to thousands of neurons can now be obtained at high spatiotemporal resolution [1-3]. These methods are increasingly deployed to perform longitudinal recordings in animals executing quasi-naturalistic behaviors or complex tasks [2-7], meaning that one may not have recourse to repeatable trial structure when analyzing these data [8]. A critical question for contemporary systems neuroscience then arises: How can mechanistic insights about the neural dynamics underlying animal behavior be extracted from large-scale recordings [3, 5, 7, 9, 10]?

Given access to only a single, non-repeatable measurement of neural activity, a natural question is whether one could construct a reliable in silico surrogate model for the dynamics of the measured neural circuit. As this surrogate model would not be subject to measurement limitations, it could be used to generate hypotheses about the corresponding biological neural populations, and to simulate how such populations might behave under various external inputs or perturbations. A natural approach to constructing a surrogate model is to optimize a recurrent neural network (RNN) to mimic the

recorded neural activity. In recent years, this approach has gained broad popularity, and has been applied to data from many species and recording modalities [9, 11-20].

Data-driven models of neural dynamics are constructed under a number of less-than-ideal conditions. First, in single-trial settings, one has access to measurements of dynamics only in a restricted condition or set of conditions, so it may be impossible to observe how changes in input or in internal state affect the circuit of interest [8]. Second, the system of interest is only partially observed: one can only usually record from a subset of the neurons in a given circuit, and certainly cannot simultaneously record all of their inputs [7]. Third, measurements of neural activity are noisy, and may be biased to capture only certain components of neural activity. For example, intrinsic indicator dynamics mean that calcium imaging may effectively low-pass filter neural activity, even if one is only interested in firing rates and not precise spike times [21, 22]. Fourth, it can be challenging to account for the presence and structure of intrinsic neuronal noise. Finally, there will always be a significant mismatch between the single-unit dynamics of a model and biology, as models abstract away or ignore biophysical details to enable efficient optimization and simulation [17, 18].

Even in the unrealistic scenario where the activity of every relevant neuron is recorded, exactly inferring synaptic weights from dynamical measurements alone is extremely challenging [23]. A more modest hope is that data-constrained models should be able to capture the mechanistic dynamical properties of ground-truth circuits at a qualitative level-that is, to recapitulate slow time scales, unstable directions, oscillatory dynamics, and attractors [9, 12-14, 17, 19, 24, 25]. Given a dataconstrained model, one can identify attractor properties using dynamical systems analysis [13, 26, 27]. These macroscopic dynamical properties are of substantial neuroscientific interest, as low-dimensional attractors are believed to underlie observed neural activity across a variety of neural circuits and tasks [24]. In particular, line attractors-sets of stable fixed points organized along lines in neural activity space-have been proposed to underlie cognitive functions requiring short-term or working memory, including sensory integration, decision making, and even aggressive behavior [12, 13, 24, 26, 28, 29]. Importantly, low-dimensional structures in data should be relatively robustly detectable even under partial observation [7], so there is reason to be optimistic that data-constrained models could correctly recover line attractor dynamics. Indeed, several recent papers have used data-constrained models with low-dimensional latent RNN dynamics to propose that line attractors underlie the accumulation of internal drives and of external reward [12, 13, 30, 31].

However, despite some positive examples [14, 19], previous works have not mapped out how partial observation affects whether data-driven modeling can accurately recover low-dimensional attractor structure. To address this question, in this paper, we consider a teacher-student setup in which activity from one RNN is imitated by another, and show that partial observation can induce mechanistic mismatches even under relatively ideal conditions where the input to a circuit is either perfectly known or white noise, and where the single-unit dynamics of the student match the teacher. Our primary contributions are as follows:

In §2, we begin with a motivating example: we show that data-constrained modeling fails to distinguish between two mechanistically-distinct models in a stimulus-integration task. Both a line attractor network [28] and a functionally feedforward chain [32] are identified as line attractors.

We then turn to the analytically-tractable setting of noise-driven linear RNNs (§3). In §3.1, we show that when the teacher is an approximate line attractor, the student will recover this structure. In contrast, when the teacher connectivity is non-normal, the student may learn spurious approximate attractor structure. We illustrate this with two biologically-motivated examples: functionally-feedforward integrators (§3.2), and networks with low-rank connectivity (§3.3).

Then, in §4, we explore how these insights generalize beyond the linear setting. Focusing on the example of nonlinear low-rank networks, we show that partial observation once again can induce overestimation of eigenvalue magnitude. Here, though, this can result in spurious attractor structure including additional stable fixed points and limit cycles.

Our results begin to illuminate the inductive biases of data-constrained RNNs trained under partial observation towards particular mechanisms of generating long timescales. They suggest that caution is warranted in inferring mechanism from data-constrained models, and underscore the primacy of direct activity perturbations for validating putative attractor dynamics [25].

a

i)

ii)

iii)

iv)

v)

vi)

Time

Activity

. . .

. . .

mode 1

mode 2

mode D

b

i)

ii)

iii)

iv)

v)

vi)

Time

Activity

. . .

. . .

mode 1

mode 2

mode D

GroundTruth

1.0

200

100

0.5

0

100

-100

0

100

200

0.0

Learned

40

PC2 (A.U.)

20

Q

20

-20

0

20

40

Learned

20

10

口

10

-10

0

10

20

GroundTruth

1.0

100

50

0.5

Q

50

0.0

-50

0

50

100

2 A motivating example: data-constrained modeling of integrator circuits

The circuit basis for temporal integration of scalar sensory inputs is a longstanding question in systems neuroscience [12, 24, 28, 29, 32-41]. Though many models for integrator circuits have been proposed [28, 32, 37, 40, 42], two linear RNN models are perhaps the most prominent: the line attractor [28, 40], and the feedforward chain [32, 37]. Both of these models have extremely simple dynamics

for state z ∈ R D , recurrent weights J ∈ R D × D , and input u ( t ) ∈ R encoded through b ∈ R D . However, they posit structurally distinct mechanisms for how memories can be maintained beyond the single-unit time constant τ . In classic line attractor networks, the recurrent weights are chosen to be symmetric, and one eigenvalue of J is tuned to be precisely equal to one, with the rest being less than one. Then, by choosing the input weights b to align with the corresponding eigenvector, one obtains a perfect integrator of the signal u ( t ) [28] (App. A). However, this model suffers from a substantial fine-tuning problem: slight mis-tuning of the weights causes exponentially large deviations from perfect integration [28, 32, 37]. In contrast, a functionally feedforward chain maintains a memory by iteratively passing signals from one mode of activity to the next (Fig. 1; App. A) [32, 37]. Most

simply, a literal feedforward chain has connectivity J ij = δ i +1 ,j . However, one can encode modes in distributed patterns of neural activity rather than single neurons, so that this structure is not obviously apparent in recordings (see App. A for details). Such networks are more robust to mistuning of synaptic strengths than line attractor networks, but they can only sustain a memory over O ( τD ) time. Importantly, the dynamics of such a network are highly non-normal; the recurrent connectivity matrix J has all eigenvalues equal to zero. Here, inspired by [36], we add skip connections from each mode to the last mode in the chain (Fig. 1; App. A). This guarantees that, like the line attractor network, the activity produced by the functionally feedforward network is approximately low-dimensional.

Given the simplicity and ubiquity of these models, we first asked whether data-constrained modeling could robustly distinguish between them. We constructed a model sensory integration task, which networks of both architectures could effectively solve (Fig. 1). Using standard variational inference methods [13, 18], we fit recordings of 5% of the neurons from each network with a latent linear dynamical system (LDS), which models the neural activity as a linear projection of a low-dimensional latent RNN [18] (App. G). 1 These models explicitly encode the prior belief that population activity is low-dimensional. In this case, we used 5 latent dimensions.

Though the data-constrained models do an excellent job of capturing the activity recorded from both the line attractor and the feedforward chain, analyzing the latent dynamics matrices reveals that both networks are interpreted as approximate line attractors (Fig. 1). In particular, the spectrum of eigenvalues ˆ λ i of each LDS dynamics matrix induces a spectrum of decay time constants ˆ τ i = τ/ | 1 -ℜ ˆ λ i | (in continuous time; see App. A and B.3) [13, 27]. Previous works have identified networks with large gaps between the top two timescales as approximate line attractors [13, 27]. As a simple metric, Nair et al. [13] defined a 'line attractor score' log 2 (ˆ τ 1 / ˆ τ 2 ) , and interpreted scores greater than 1 as indicative of approximate line attractors. The LDS models fitted to these mechanistically different integrator circuits each have a single slow direction, with a line attractor score in excess of 6 (Fig. 1). However, visualising the flow fields of the ground truth and data-constrained models shows that the dynamics of the line attractor are qualitatively recovered well, while the model fit to recordings of the feedforward chain shows a strong mismatch as it discovers a spurious line attractor (Fig. 1). Therefore, data-driven modeling fails to distinguish circuit hypotheses for this simple task.

An intuitive explanation for why the data-driven model fails to mechanistically reproduce the functionally-feedforward chain is immediate: if the single-unit time constants are fixed and the dataconstrained model has fewer neurons, it cannot realize a feedforward chain with sufficiently long memory [32, 37]. The only way to manufacture long memory timescales with a small number of latent neurons is through large eigenvalues. This is potentially a fundamental obstacle to the ability of latent space models to recover neural mechanisms; we will return to this point in the Discussion.

3 A tractable model setting: noise-driven linear networks

Motivated by the observations of the previous section, we now seek a setting in which we can analytically study the structure of the student RNN's weight matrix. Whereas in §2 we assumed the teacher networks were driven by a known low-dimensional signal, here we consider the case in which the teacher and student are driven by isotropic Gaussian noise. This is an optimistic assumption, as it means that the teacher network will explore all directions in its phase space evenly over the course of a single long trial [37].

Concretely, we consider a teacher-student setup in which both networks are rate-based linear RNNs driven by isotropic Gaussian noise, or, equivalently, their activity evolves according to multivariate Ornstein-Uhlenbeck (OU) processes [43]. The teacher has D neurons and a recurrent weight matrix B , such that the dynamics of its firing rate vector z ( t ) ∈ R D is

where ξ ( t ) is white Gaussian noise. The student's dynamics are identical, except that it has d neurons, recurrent weights A , and driving noise η ( t ) , such that its rate x ( t ) ∈ R d evolves as

Then, the task is to estimate the student's dynamics matrix A given access only to partial observations of the teacher's activity. For simplicity, we assume that we observe the first d neurons of the teacher

a

i)

ii)

iii)

iv)

b

i)

ii)

iii)

iv)

GroundTruth

1.C

20

10

0.5

0

-10

20

-20

0

20

0.0

Learned

6

4

2

0

-2

P

-5

0

5

GroundTruth

1.0

20

10

0.5

0

-10

-20

-20

0

20

0.0

Learned

6

4

2

0

-2

C

P

-4

-5

0

5

network for time T , i.e., we observe

Assuming an isotropic Gaussian prior A ij ∼ i . i . d . N (0 , 1 / ( ρT )) scaled such that the long-time limit is well-defined, we show in Appendix B that the maximum a posteriori (MAP) estimate of A can be computed explicitly in terms of empirical covariances of x obs ( t ) [44-47]. To make the problem analytically tractable, we focus on the limit T →∞ , where these covariances can be computed using classical results on stationary states of OU processes [43, 48]. We assume that the eigenvalues of the teacher's weight matrix B have real part strictly less than one, such that it admits a stable stationary state with covariance S = lim t →∞ E [ z ( t ) z ( t ) ⊤ ] . Then, in the limit of a long observation window T →∞ , the MAP estimate of the student's dynamics matrix can be written in terms of the stationary covariance S as (App. B)

This result is stated in continuous time; we also give the corresponding result for discretized dynamics in Appendix B. In the fully-observed case, the zero-ridge limit of the MAP recovers the teacher dynamics matrix, i.e., lim ρ ↓ 0 ˆ A ∞ | d = D = B . The stability condition means that we can consider at best approximate line attractors with arbitrarily large but not infinite time constants, but this is not a substantial limitation [28, 32, 37].

To determine when this data-driven modeling approach recovers the mechanistic structure of the teacher, our task is then to analyze the spectrum of ˆ A ∞ for various choices of B , as for linear networks the eigenspectrum fully determines the (approximate) attractor structure [28]. Concretely, letting λ i and ˆ λ i be the eigenvalues of the teacher and student dynamics matrices, respectively, we want to compare the resulting spectra of timescales τ i = τ/ | 1 -ℜ λ i | and ˆ τ i = τ/ | 1 -ℜ ˆ λ i | . We are primarily interested in whether the existence or non-existence of slow timescales can be accurately recovered.

3.1 Normal dynamics

We begin by considering teacher networks with normal connectivity matrices ( BB ⊤ = B ⊤ B ). This includes attractor networks like the idealized line attractor, which have symmetric connectivity ( B = B ⊤ ), and when driven by noise have an equilibrium stationary state [43, 48]. As such matrices

a

10

2

10

1

10 0

Subsampling fraction

0

1

2

3

4

5

Line attractor score

D

=100

D

=200

D

=300

D

=400

D

=500

D

=600

D

=700

D

=800

D

=900

D

=1000

b

10 2

10 3

D (Teacher size)

0.0

0.2

0.4

0.6

0.8

1.0

(

)

1

2

c

10 2

10 3

D (Teacher size)

0

20

40

60

80

Time constant

1

2

have orthogonal eigenspaces, the dynamics of a normal teacher network can be viewed as a set of noninteracting modes with decay timescales determined by the real parts of the eigenvalues (App. A).

For such teachers, we show in Appendix C that partial observation does not lead to overestimation of timescales under MAP inference. Ordering the eigenvalues of B in descending order of their real parts as 1 > ℜ ( λ 1 ) ≥ ℜ ( λ 2 ) ≥ · · · ≥ ℜ ( λ D ) , the eigenvalues ˆ λ i of the student's dynamics matrix ˆ A ∞ satisfy ℜ ( λ 1 ) ≥ ℜ ( ˆ λ i ) ≥ ℜ ( λ D ) for all 1 ≤ i ≤ d . However, this positive recovery result does not exclude the possibility that the spectrum of the student's dynamics matrix will have qualitatively distinct gap structure, which would lead to incorrect inference of approximate attractor mechanisms.

In the special case of an ideal line attractor, this does not happen: if the teacher is a symmetric approximate line attractor, then the student will be as well. Concretely, suppose that B is symmetric, with eigenvalues satisfying λ 1 = 1 -ε , ε ≪ 1 , and λ i ≪ 1 for i ≥ 2 , and that the eigenvector u 1 corresponding to the leading eigenvalue (the direction of the approximate line attractor) is randomly oriented or delocalized. Then, the eigenvalues of the student dynamics matrix satisfy ˆ λ 1 ≥ λ 1 -O ( εD/d ) and ˆ λ 2 ≤ λ 2 (App. C.3). This implies that approximate line attractors can be recovered even under heavy partial observation so long as the deviation ε of the teacher dynamics from a perfect line attractor is small. In Figure 2, we illustrate this successful recovery, and show that it is not qualitatively affected even if the observation time is finite. This successful recovery is consistent with what we found in the driven setting in Figure 1.

3.2 Non-normal dynamics: Feedforward amplification

Our results for normal teacher dynamics in §3.1 show that the student can correctly recover line attractor dynamics, matching our motivating observation in Figure 1. However, we recall that we found that a non-normal network performing integration through feedforward amplification was incorrectly recognized as also being a line attractor. While it is challenging to analyze general nonnormal teacher matrices in the noise-driven setting [43, 48], we can show that this mismatch again emerges for feedforward chains. In particular, we show in Appendix D that the dynamics of a student of fixed size approach that of a line attractor as teacher size increases. Assume that the teacher is a perfect feedforward chain with connectivity B ij = δ i +1 ,j . Then, as D →∞ for fixed d , the student dynamics matrix ˆ A ∞ in the limit of long observation time and vanishing regularization approaches δ i +1 ,j + δ id δ ij , hence its leading eigenvalue approaches 1 , while the others tend to zero (App. D).

As illustrated in Figure 2, the qualitative conclusion that partial observation leads to timescale overestimation does not change even when the observation time is finite. We remark that the fact that the student becomes closer and closer to a line attractor as D increases is consistent with the intuitive argument given at the end of Section 2: if the number of observed neurons is fixed and small, the only way for the student network to capture the long integration window of the feedforward chain is through tuning its eigenvalues to create long timescales. In Figures 3 and G.3, we substantiate this intuition by showing how the estimated timescales depend on the size of the teacher network relative to the student.

a

i)

ii)

iii)

iv)

5%

50%

GT

b

i)

ii)

iii)

iv)

5%

50%

GT

3.3 Low-rank non-normal dynamics

As a second neuroscience-inspired example of non-normal teacher dynamics, we consider low-rank connectivity. In recent years, low-rank RNNs have emerged as popular models for cortical dynamics [15, 16, 25, 49, 50]. Importantly for our purposes, they yield low-dimensional population activity, and hence are again a relatively ideal scenario for data-constrained modeling under partial observation [7, 50]. However, we find that connectivity that is both non-normal and low-rank can also give rise to severe timescale overestimation in the student network.

As a particularly simple example of low-rank teacher dynamics, we consider the case in which B = MN ⊤ is rank r ≪ D , with M,N ∈ R D × r having null overlap M ⊤ N = 0 r × r and orthogonal columns M ⊤ M = N ⊤ N = γ 2 I r . Then, B is a non-normal matrix with all-zero eigenvalues. When γ ≫ 1 , the stationary covariance S of the teacher network's activity will have precisely r large eigenvalues of order γ 4 , separated from a bulk of eigenvalues that are of O (1) with respect to γ (App. E). In this largeγ regime where the teacher's activity is approximately low-dimensional, the student's learned dynamics matrix has r eigenvalues approaching 1, with the rest approaching zero (App. E). Therefore, the student learns an r -dimensional hyperplane attractor. Importantly, this can occur when γ 2 is chosen such that B has order-1 elements. We show how this effect depends on subsampling fraction in G.4.

In simulations, we observe a finite observation time effect whereby only r -1 of the learned eigenvalues are near 1 when process noise is small (Fig. G.1). Consequently, fitting a student network to a non-normal teacher with null overlap connectivity of rank r as described above can result in the spurious discovery of approximate ( r -1) -dimensional hyperplane attractors. We illustrate this explicitly for the cases r = 2 and r = 3 , where observing only 5% of the neurons in the teacher network leads to the spurious discovery of approximate line attractor and plane attractor dynamics, respectively, despite nearly perfectly recapitulating the observed activity (Fig. 4). Consistent with these observations, we show that latent LDS models fit to the same teacher activity via more sophisticated variational inference methods also learn a few vastly enlarged timescales (Fig. G.5).

4 Mismatched attractor structure in data-constrained nonlinear networks

Though the linear networks studied in §3 are analytically tractable, they are of course inherently limited in the types of attractor dynamics they can display. We therefore asked which qualitative insights from the linear setting carry over to nonlinear networks where one allows for a nonlinear

firing rate transfer function ϕ . That is, we again consider a student-teacher setup, but now the teacher and student dynamics are τ ˙ z = -z + Bϕ ( z )+ ξ ( t ) and τ ˙ x = -x + Aϕ ( x )+ η ( t ) , respectively (App. B). Our focus is again on low-rank networks, both for their usage as models for cortical processing and for the fact that their approximately low-dimensional activity makes them a natural candidate for data-driven modeling under partial observation [15, 16, 49].

While it is less straightforward to relate attractor dynamics to the eigenspectrum of B in the nonlinear setting, one can still use spectral information to gain insight into the dynamics near the trivial fixed point at the origin. Specializing to ϕ ( z ) = tanh( z ) , the Jacobian of the teacher dynamics at the origin is simply -I D + B , and that of the student is analogously -I d + A . Therefore, connectivity matrix eigenvalues of real part greater than 1 would imply that the fixed point at the origin is unstable, and thus the network must support other dynamical behavior (e.g, other stable fixed points, limit cycles, and/or chaos). In the linear case, eigenvalues of real part greater than 1 were never spuriously discovered, as that would yield exponentially divergent activity. However, with a saturating nonlinearity, this extreme eigenvalue overestimation is no longer pathological. Indeed, when we infer the weights of a student network using MAP estimation, we find that at small subsampling fractions the student can learn eigenvalues of real part greater than one from a teacher with no such eigenvalues if the teacher connectivity is non-normal. Strikingly, this can lead to the discovery of spurious limit cycles (Fig. 5a) and fixed points (Fig. G.6).

At this point, one might ask whether the eigenvalue overestimation phenomenon we have observed is an artefact of the estimation methods (thus far, MAP and LDS variational inference) on which we have focused. The conceptual argument given at the end of §2 suggests that this should hold more broadly for low-dimensional student networks learning from partial observations of high-dimensional teachers, but this is a heuristic argument, not a rigorous test. We therefore applied several other commonly-used inference methods [14, 51, 52] to fit student dynamics in the low-rank nonlinear teacher setting. All of the methods produced mismatched attractor structure, with many showing a propensity to overestimate eigenvalues, yielding spurious limit cycles (Fig. 5). We show that these effects persist for more general teacher weight matrices and for student networks with additional hidden units to account for unobserved neurons in Fig. G.2.

5 Discussion

In this paper, we have shown partial observation can lead data-constrained models to incorrectly identify the mechanistic basis for slow recorded neural dynamics. We found that, while attractorlike networks can be faithfully recovered even when only a small fraction of neurons are recorded, data-constrained models can learn spurious attractor structure from non-normal transient dynamics.

As noted in §2, an intuitive explanation of our results for linear networks is that low-dimensional dynamical systems are limited in the longest timescales they could generate through functionally feedforward integration, and thus are inherently biased towards line-attractor-like mechanisms when fit to observations of slow dynamics. Though our focus has been on partial observation as a driver for this dimensional restriction, most approaches to data-constrained modeling with latent dynamics explicitly bias model selection towards smaller latent spaces. In particular, it is standard to select the smallest latent space dimension that captures more than a certain threshold fraction of the variance in the data [13, 18]. This will necessarily favor approximate-attractor-like solutions. Indeed, if one applied such a model selection procedure to the integrator models studied in Figure 1, one would select at most a two-dimensional latent space (see Figure G.8), and thus fall victim to the failure mode noted there. This bias in model selection procedures illustrates a wider issue: benchmarking and model selection based on explained variance for a restricted set of measured dynamics alone are not necessarily sufficient to diagnose mechanistic mismatches [23, 53]. It highlights a tension between the desire to recapitulate mechanism and our intuitive conception of low dimensionality as a signature of model parsimony.

Here, we have focused on the setting in which one measures dynamics over a single trial for inputs that are either fully known or white noise. Previous works have shown how failure to account for unobserved inputs can lead to incorrect eigenvalue estimation [19]. Moreover, most previouslyproposed methods to disentangle input-driven versus autonomous dynamics require repeatable trial structure [10, 19, 54]. Our work focuses only on the effects of unobserved neurons, and does not attempt to address this additional source of mechanistic mismatch. Observation of a network under

FORCE

BPTT

CORNN

MAP

b

a

c

d

i)

ii)

iii)

i)

ii)

iii)

i)

ii)

iii)

i)

ii)

iii)

10%

50%

GT

5%

50%

GT

10%

50%

GT

10%

50%

GT

a restricted set of input conditions poses a particularly striking challenge if it has a spectrum of heterogeneous integration timescales. If inputs drive activity along only a subset of dimensions, it is easy to imagine how a heterogeneous spectrum of time constants could be reductively interpreted as low-dimensional attractor structure [38, 39, 55]. When studying circuits whose upstream inputs are not well-understood dynamically, inferences about circuit-intrinsic attractor structure become even more tenuous.

As our work shows that data-constrained models can fail to correctly distinguish between mechanistically different hypotheses for the circuit basis of slow dynamics, an important question is how one should validate putative attractor structure. Though we do not address this issue in the present work, the obvious candidate for conclusive validation of attractor dynamics is of course direct experimental perturbation of neural activity. Daie, Svoboda, and colleagues have identified feedforward amplification (in lieu of line-attractor-like dynamics) in anterior lateral motor cortex using the correlation structure of responses to optogenetic perturbations [36]. O'Shea, Duncker, and colleagues have used targeted optogenetic perturbations to interrogate putative low-dimensional dynamics in primate motor cortex [25]. There, they show that data-constrained models with unconstrained weight matrices do not readily predict perturbation responses, while those with weight matrices constrained to be lowrank capture the fast recovery of the dynamics to a low-dimensional subspace. After the completion of our work, Vinograd et al. [56] have begun to interrogate putative line attractor dynamics in hypothalamus using similar perturbations. An important question for future work will be to determine how specifically targeted a patterned optogenetic perturbation must be in order to distinguish between the line attractor and functionally-feedforward integrator networks studied here. Another important

question is how to relate patterned stimuli in silico and in vivo , particularly for latent variable models where a possibly ill-posed inversion of the latent state to observed neuron mapping would be required.

Finally, we remark that our work relates to a broader question of inference for partially-observed dynamical systems. There is a substantial literature on performance guarantees for parameter estimation for linear systems, which largely focuses on predictive accuracy rather than qualitative features [44-47, 57]. There are also a host of methods which leverage delay embeddings; in addition to predictive accuracy, these methods prioritize accurately inferring the dimensionality and topological structure of an underlying system [20, 58-60]. However, such methods are 'equation-free", and thus are not ideal for identifying how dynamical variables are coupled [61]. Consequently, such methods might not be suitable for distinguishing between mechanistically distinct circuit hypotheses that generate similarly low dimensional neural activity. Indeed, delay embeddings do not appear to distinguish between the two integrator circuits studied here (Fig. G.7). We leave a more detailed investigation of delay embedding-based approaches for future work.

Acknowledgments and Disclosure of Funding

We thank Farhad Pashakhanloo and Mitchell Ostrow for helpful comments on a previous version of our manuscript. JAZV and CP were supported by NSF Award DMS-2134157 and NSF CAREER Award IIS-2239780. JAZV was further supported by a Junior Fellowship from the Harvard Society of Fellows. CP was further supported by a Sloan Research Fellowship. This work has been made possible in part by a gift from the Chan Zuckerberg Initiative Foundation to establish the Kempner Institute for the Study of Natural and Artificial Intelligence.

References

Shreya Saxena and John P Cunningham. Towards the neural population doctrine. Current Opinion in Neurobiology , 55:103-111, 2019. ISSN 0959-4388. doi:https://doi.org/10.1016/j.conb.2019.02.002. URL https://www.sciencedirect.com/ science/article/pii/S0959438818300990 . Machine Learning, Big Data, and Neuroscience.

Paul Masset, Shanshan Qin, and Jacob A Zavatone-Veth. Drifting neuronal representations: Bug or feature? Biological Cybernetics , pages 1-14, 2022. doi:doi.org/10.1007/s00422-02100916-3.

Anne E. Urai, Brent Doiron, Andrew M. Leifer, and Anne K. Churchland. Large-scale neural recordings call for new insights to link brain and behavior. Nature Neuroscience , 25(1):11-19, 01 2022. ISSN 1546-1726. doi:10.1038/s41593-021-00980-9. URL https://doi.org/10. 1038/s41593-021-00980-9 .

Ashesh K Dhawale, Rajesh Poddar, Steffen BE Wolff, Valentin A Normand, Evi Kopelowitz, and Bence P Ölveczky. Automated long-term recording and analysis of neural activity in behaving animals. eLife , 6:e27702, 2017. doi:10.7554/eLife.27702.

Saurabh Vyas, Matthew D. Golub, David Sussillo, and Krishna V. Shenoy. Computation through neural population dynamics. Annual Review of Neuroscience , 43(1):249-275, 2020. doi:10.1146/annurev-neuro-092619-094115.

Markus Meister. Learning, fast and slow. Current Opinion in Neurobiology , 75:102555, 2022. ISSN 0959-4388. doi:https://doi.org/10.1016/j.conb.2022.102555. URL https://www. sciencedirect.com/science/article/pii/S0959438822000496 .

Peiran Gao, Eric Trautmann, Byron Yu, Gopal Santhanam, Stephen Ryu, Krishna Shenoy, and Surya Ganguli. A theory of multineuronal dimensionality, dynamics and measurement. bioRxiv , 2017. doi:10.1101/214262. URL https://www.biorxiv.org/content/early/2017/11/ 12/214262 .

Alex H. Williams and Scott W. Linderman. Statistical neuroscience in the single trial limit. Current Opinion in Neurobiology , 70:193-205, 2021. ISSN 0959-4388. doi:https://doi.org/10.1016/j.conb.2021.10.008. URL https://www.sciencedirect.com/ science/article/pii/S0959438821001203 . Computational Neuroscience.

Lea Duncker and Maneesh Sahani. Dynamics on the manifold: Identifying computational dynamical activity from neural population recordings. Current Opinion in Neurobiology , 70:163170, 2021. ISSN 0959-4388. doi:https://doi.org/10.1016/j.conb.2021.10.014. URL https:// www.sciencedirect.com/science/article/pii/S0959438821001264 . Computational Neuroscience.

Aniruddh R. Galgali, Maneesh Sahani, and Valerio Mante. Residual dynamics resolves recurrent contributions to neural computation. Nature Neuroscience , 26(2):326-338, Feb 2023. ISSN 1546-1726. doi:10.1038/s41593-022-01230-2. URL https://doi.org/10.1038/ s41593-022-01230-2 .

Chethan Pandarinath, Daniel J. O'Shea, Jasmine Collins, Rafal Jozefowicz, Sergey D. Stavisky, Jonathan C. Kao, Eric M. Trautmann, Matthew T. Kaufman, Stephen I. Ryu, Leigh R. Hochberg, Jaimie M. Henderson, Krishna V. Shenoy, L. F. Abbott, and David Sussillo. Inferring singletrial neural population dynamics using sequential auto-encoders. Nature Methods , 15(10): 805-815, October 2018. ISSN 1548-7105. doi:10.1038/s41592-018-0109-9. URL https: //doi.org/10.1038/s41592-018-0109-9 .

Emily L. Sylwestrak, YoungJu Jo, Sam Vesuna, Xiao Wang, Blake Holcomb, Rebecca H. Tien, Doo Kyung Kim, Lief Fenno, Charu Ramakrishnan, William E. Allen, Ritchie Chen, Krishna V. Shenoy, David Sussillo, and Karl Deisseroth. Cell-type-specific population dynamics of diverse reward computations. Cell , 185(19):3568-3587.e27, 2022. ISSN 00928674. doi:https://doi.org/10.1016/j.cell.2022.08.019. URL https://www.sciencedirect. com/science/article/pii/S0092867422011138 .

Aditya Nair, Tomomi Karigo, Bin Yang, Surya Ganguli, Mark J. Schnitzer, Scott W. Linderman, David J. Anderson, and Ann Kennedy. An approximate line attractor in the hypothalamus encodes an aggressive state. Cell , 186(1):178-193.e15, 2023. ISSN 0092-8674. doi:https://doi.org/10.1016/j.cell.2022.11.027. URL https://www.sciencedirect.com/ science/article/pii/S0092867422014714 .

Fatih Dinc, Adam Shai, Mark Schnitzer, and Hidenori Tanaka. CORNN: convex optimization of recurrent neural networks for rapid inference of neural dynamics. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, pages 51273-51301. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/ a103529738706979331778377f2d5864-Paper-Conference.pdf .

Adrian Valente, Jonathan W Pillow, and Srdjan Ostojic. Extracting computational mechanisms from neural data using low-rank RNNs. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 24072-24086. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ 9877d915a4b4f00e85e7b4cfdf41e450-Paper-Conference.pdf .

Adrian Valente, Srdjan Ostojic, and Jonathan W. Pillow. Probing the Relationship Between Latent Linear Dynamical Systems and Low-Rank Recurrent Neural Network Models. Neural Computation , 34(9):1871-1892, 08 2022. ISSN 0899-7667. doi:10.1162/neco_a_01522. URL https://doi.org/10.1162/neco_a_01522 .

Matthew G. Perich, Charlotte Arlt, Sofia Soares, Megan E. Young, Clayton P. Mosher, Juri Minxha, Eugene Carter, Ueli Rutishauser, Peter H. Rudebeck, Christopher D. Harvey, and Kanaka Rajan. Inferring brain-wide interactions using data-constrained recurrent neural network models. bioRxiv , 2021. doi:10.1101/2020.12.18.423348. URL https://www.biorxiv.org/ content/early/2021/03/11/2020.12.18.423348 .

Scott Linderman, Annika Nichols, David Blei, Manuel Zimmer, and Liam Paninski. Hierarchical recurrent state space models reveal discrete and continuous dynamics of neural activity in c. elegans. bioRxiv , 2019. doi:10.1101/621540. URL https://www.biorxiv.org/content/ early/2019/04/29/621540 .

Parsa Vahidi, Omid G. Sani, and Maryam M. Shanechi. Modeling and dissociation of intrinsic and input-driven neural population dynamics underlying behavior. Proceedings of the National Academy of Sciences , 121(7):e2212887121, 2024. doi:10.1073/pnas.2212887121. URL https: //www.pnas.org/doi/abs/10.1073/pnas.2212887121 .

Daniel Durstewitz, Georgia Koppe, and Max Ingo Thurm. Reconstructing computational system dynamics from neural data with recurrent neural networks. Nature Reviews Neuroscience , 24 (11):693-710, Nov 2023. ISSN 1471-0048. doi:10.1038/s41583-023-00740-7. URL https: //doi.org/10.1038/s41583-023-00740-7 .

Yan Zhang, Márton Rózsa, Yajie Liang, Daniel Bushey, Ziqiang Wei, Jihong Zheng, Daniel Reep, Gerard Joey Broussard, Arthur Tsang, Getahun Tsegaye, Sujatha Narayan, Christopher J. Obara, Jing-Xuan Lim, Ronak Patel, Rongwei Zhang, Misha B. Ahrens, Glenn C. Turner, Samuel S.-H. Wang, Wyatt L. Korff, Eric R. Schreiter, Karel Svoboda, Jeremy P. Hasseman, Ilya Kolb, and Loren L. Looger. Fast and sensitive GCaMP calcium indicators for imaging neural populations. Nature , 615(7954):884-891, March 2023. ISSN 1476-4687. doi:10.1038/s41586023-05828-9. URL https://doi.org/10.1038/s41586-023-05828-9 .

Carsen Stringer and Marius Pachitariu. Computational processing of neural recordings from calcium imaging data. Current Opinion in Neurobiology , 55:22-31, 2019. ISSN 0959-4388. doi:https://doi.org/10.1016/j.conb.2018.11.005. URL https://www.sciencedirect.com/ science/article/pii/S0959438818300977 . Machine Learning, Big Data, and Neuroscience.

Abhranil Das and Ila R. Fiete. Systematic errors in connectivity inferred from activity in strongly recurrent networks. Nature Neuroscience , 23(10):1286-1296, October 2020. ISSN 1546-1726. doi:10.1038/s41593-020-0699-2. URL https://doi.org/10.1038/s41593-020-0699-2 .

Mikail Khona and Ila R. Fiete. Attractor and integrator networks in the brain. Nature Reviews Neuroscience , 23(12):744-766, December 2022. ISSN 1471-0048. doi:10.1038/s41583-02200642-0. URL https://doi.org/10.1038/s41583-022-00642-0 .

Daniel J. O'Shea, Lea Duncker, Werapong Goo, Xulu Sun, Saurabh Vyas, Eric M. Trautmann, Ilka Diester, Charu Ramakrishnan, Karl Deisseroth, Maneesh Sahani, and Krishna V. Shenoy. Direct neural perturbations reveal a dynamical mechanism for robust computation. bioRxiv , 2022. doi:10.1101/2022.12.16.520768. URL https://www.biorxiv.org/content/early/ 2022/12/16/2022.12.16.520768 .

David Sussillo and Omri Barak. Opening the Black Box: Low-Dimensional Dynamics in High-Dimensional Recurrent Neural Networks. Neural Computation , 25(3):626-649, 03 2013. ISSN 0899-7667. doi:10.1162/NECO_a_00409. URL https://doi.org/10.1162/NECO_ a_00409 .

Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, and David Sussillo. Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/ d921c3c762b1522c475ac8fc0811bb0f-Paper.pdf .

H. S. Seung. How the brain keeps the eyes still. Proceedings of the National Academy of Sciences , 93(23):13339-13344, 1996. doi:10.1073/pnas.93.23.13339. URL https://www. pnas.org/doi/abs/10.1073/pnas.93.23.13339 .

Valerio Mante, David Sussillo, Krishna V. Shenoy, and William T. Newsome. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature , 503(7474):78-84, November 2013. ISSN 1476-4687. doi:10.1038/nature12742. URL https://doi.org/10.1038/ nature12742 .

Mengyu Liu, Aditya Nair, Nestor Coria, Scott W. Linderman, and David J. Anderson. Encoding of female mating dynamics by a hypothalamic line attractor. Nature , Aug 2024. ISSN 1476-4687. doi:10.1038/s41586-024-07916-w. URL https://doi.org/10.1038/ s41586-024-07916-w .

George Mountoufaris, Aditya Nair, Bin Yang, Dong-Wook Kim, Amit Vinograd, Samuel Kim, Scott W. Linderman, and David J. Anderson. A line attractor encoding a persistent internal state requires neuropeptide signaling. Cell , 2024/09/24 XXXX. ISSN 0092-8674. doi:10.1016/j.cell.2024.08.015. URL https://doi.org/10.1016/j.cell.2024.08.015 .

Mark S. Goldman. Memory without feedback in a neural network. Neuron , 61(4):621-634, 2009. ISSN 0896-6273. doi:https://doi.org/10.1016/j.neuron.2008.12.012. URL https: //www.sciencedirect.com/science/article/pii/S0896627308010830 .

Timothy Doyeon Kim, Thomas Zhihao Luo, Tankut Can, Kamesh Krishnamurthy, Jonathan W. Pillow, and Carlos D. Brody. Flow-field inference from neural data using deep recurrent networks. bioRxiv , 2023. doi:10.1101/2023.11.14.567136. URL https://www.biorxiv. org/content/early/2023/11/16/2023.11.14.567136 .

Thomas Zhihao Luo, Timothy Doyeon Kim, Diksha Gupta, Adrian G. Bondy, Charles D. Kopec, Verity A. Elliot, Brian DePasquale, and Carlos D. Brody. Transitions in dynamical regime and neural mode underlie perceptual decision-making. bioRxiv , 2023. doi:10.1101/2023.10.15.562427. URL https://www.biorxiv.org/content/early/ 2023/11/20/2023.10.15.562427 .

Kayvon Daie, Karel Svoboda, and Shaul Druckmann. Targeted photostimulation uncovers circuit motifs supporting short-term memory. Nature Neuroscience , 24(2):259-265, Feb 2021. ISSN 1546-1726. doi:10.1038/s41593-020-00776-3. URL https://doi.org/10.1038/ s41593-020-00776-3 .

Kayvon Daie, Lorenzo Fontolan, Shaul Druckmann, and Karel Svoboda. Feedforward amplification in recurrent networks underlies paradoxical neural coding. bioRxiv , 2023. doi:10.1101/2023.08.04.552026. URL https://www.biorxiv.org/content/early/ 2023/08/07/2023.08.04.552026 .

Surya Ganguli, Dongsung Huh, and Haim Sompolinsky. Memory traces in dynamical systems. Proceedings of the National Academy of Sciences , 105(48):18970-18975, 2008. doi:10.1073/pnas.0804451105. URL https://www.pnas.org/doi/abs/10.1073/pnas. 0804451105 .

Emre Aksay, Itsaso Olasagasti, Brett D. Mensh, Robert Baker, Mark S. Goldman, and David W. Tank. Functional dissection of circuitry in a neural integrator. Nature Neuroscience , 10(4):494504, 04 2007. ISSN 1546-1726. doi:10.1038/nn1877. URL https://doi.org/10.1038/ nn1877 .

Andrew Miri, Kayvon Daie, Aristides B. Arrenberg, Herwig Baier, Emre Aksay, and David W. Tank. Spatial gradients and multidimensional dynamics in a neural integrator circuit. Nature Neuroscience , 14(9):1150-1159, 09 2011. ISSN 1546-1726. doi:10.1038/nn.2888. URL https://doi.org/10.1038/nn.2888 .

Stephen C. Cannon, David A. Robinson, and Shihab Shamma. A proposed neural network for the integrator of the oculomotor system. Biological Cybernetics , 49(2):127-136, Dec 1983. ISSN 1432-0770. doi:10.1007/BF00320393. URL https://doi.org/10.1007/BF00320393 .

Natalie A Steinemann, Gabriel M Stine, Eric M Trautmann, Ariel Zylberberg, Daniel M Wolpert, and Michael N Shadlen. Direct observation of the neural computations underlying a single decision. bioRxiv , 2024. doi:10.1101/2022.05.02.490321. URL https://www.biorxiv.org/ content/early/2024/05/07/2022.05.02.490321 .

Alexei A. Koulakov, Sridhar Raghavachari, Adam Kepecs, and John E. Lisman. Model for a robust neural integrator. Nature Neuroscience , 5(8):775-782, Aug 2002. ISSN 1546-1726. doi:10.1038/nn893. URL https://doi.org/10.1038/nn893 .

Crispin W Gardiner. Handbook of stochastic methods , volume 3. Springer Berlin, 1985.

Max Simchowitz, Horia Mania, Stephen Tu, Michael I. Jordan, and Benjamin Recht. Learning without mixing: Towards a sharp analysis of linear system identification. In Sébastien Bubeck, Vianney Perchet, and Philippe Rigollet, editors, Proceedings of the 31st Conference On Learning

Theory , volume 75 of Proceedings of Machine Learning Research , pages 439-473. PMLR, 0609 Jul 2018. URL https://proceedings.mlr.press/v75/simchowitz18a.html .

Tuhin Sarkar, Alexander Rakhlin, and Munther A. Dahleh. Finite time lti system identification. Journal of Machine Learning Research , 22(26):1-61, 2021. URL http://jmlr.org/papers/ v22/19-725.html .

Anastasios Tsiamis and George J. Pappas. Linear systems can be hard to learn. In 2021 60th IEEE Conference on Decision and Control (CDC) , pages 2903-2910, 2021. doi:10.1109/CDC45484.2021.9682778.

Tuhin Sarkar and Alexander Rakhlin. Near optimal finite time identification of arbitrary linear dynamical systems. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pages 5610-5618. PMLR, 09-15 Jun 2019. URL https://proceedings. mlr.press/v97/sarkar19a.html .

Claude Godrèche and Jean-Marc Luck. Characterising the nonequilibrium stationary states of Ornstein-Uhlenbeck processes. Journal of Physics A: Mathematical and Theoretical , 52(3): 035002, 2018. doi:10.1088/1751-8121/aaf190.

Alexis Dubreuil, Adrian Valente, Manuel Beiran, Francesca Mastrogiuseppe, and Srdjan Ostojic. The role of population structure in computations through neural dynamics. Nature Neuroscience , 25(6):783-794, June 2022. ISSN 1546-1726. doi:10.1038/s41593-022-01088-4. URL https: //doi.org/10.1038/s41593-022-01088-4 .

Francesca Mastrogiuseppe and Srdjan Ostojic. Linking connectivity, dynamics, and computations in low-rank recurrent neural networks. Neuron , 99(3):609-623.e29, 2018. ISSN 08966273. doi:https://doi.org/10.1016/j.neuron.2018.07.003. URL https://www.sciencedirect. com/science/article/pii/S0896627318305439 .

David Sussillo and L.F. Abbott. Generating coherent patterns of activity from chaotic neural networks. Neuron , 63(4):544-557, 2009. ISSN 0896-6273. doi:https://doi.org/10.1016/j.neuron.2009.07.018. URL https://www.sciencedirect.com/ science/article/pii/S0896627309005479 .

Timothy P Lillicrap and Adam Santoro. Backpropagation through time and the brain. Current Opinion in Neurobiology , 55:82-89, 2019. ISSN 0959-4388. doi:https://doi.org/10.1016/j.conb.2019.01.011. URL https://www.sciencedirect.com/ science/article/pii/S0959438818302009 . Machine Learning, Big Data, and Neuroscience.

Poornima Ramesh, Basile Confavreux, Pedro J. Gonçalves, Tim P. Vogels, and Jakob H. Macke. Indistinguishable network dynamics can emerge from unalike plasticity rules. bioRxiv , 2023. doi:10.1101/2023.11.01.565168. URL https://www.biorxiv.org/content/early/ 2023/11/04/2023.11.01.565168 .

Mitchell Ostrow, Adam Eisen, Leo Kozachkov, and Ila Fiete. Beyond geometry: Comparing the temporal structure of computation in neural circuits with dynamical similarity analysis. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, pages 33824-33837. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/ 6ac807c9b296964409b277369e55621a-Paper-Conference.pdf .

Mati Joshua, Javier F. Medina, and Stephen G. Lisberger. Diversity of neural responses in the brainstem during smooth pursuit eye movements constrains the circuit mechanisms of neural integration. Journal of Neuroscience , 33(15):6633-6647, 2013. ISSN 0270-6474. doi:10.1523/JNEUROSCI.3732-12.2013. URL https://www.jneurosci.org/content/ 33/15/6633 .

Amit Vinograd, Aditya Nair, Joseph H. Kim, Scott W. Linderman, and David J. Anderson. Causal evidence of a line attractor encoding an affective state. Nature , Aug 2024. ISSN 1476-4687. doi:10.1038/s41586-024-07915-x. URL https://doi.org/10.1038/ s41586-024-07915-x .

Yuyang Zhang, Shahriar Talebi, and Na Li. Learning low-dimensional latent dynamics from high-dimensional observations: Non-asymptotics and lower bounds. arXiv , 2024.

William Gilpin. Deep reconstruction of strange attractors from time series. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 204-216. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/ 021bbc7ee20b71134d53e20206bd6feb-Paper.pdf .

William Gilpin. Chaos as an interpretable benchmark for forecasting and data-driven modelling. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2021. URL https://openreview.net/forum?id=enYjtbjYJrf .

William Gilpin. Generative learning for nonlinear dynamics. Nature Reviews Physics , 6(3): 194-206, March 2024. ISSN 2522-5820. doi:10.1038/s42254-024-00688-2. URL https: //doi.org/10.1038/s42254-024-00688-2 .

George Stepaniants, Alasdair D. Hastewell, Dominic J. Skinner, Jan F. Totz, and Jörn Dunkel. Discovering dynamics and parameters of nonlinear oscillatory and chaotic systems from partial observations. arXiv , 2023.

Rajendra Bhatia. Perturbation bounds for matrix eigenvalues . SIAM, 2007.

Suk-Geun Hwang. Cauchy's interlace theorem for eigenvalues of hermitian matrices. The American mathematical monthly , 111(2):157-159, 2004.

DLMF. NIST Digital Library of Mathematical Functions . http://dlmf.nist.gov/, Release 1.1.1 of 2021-03-15, 2021. URL http://dlmf.nist.gov/ . F. W. J. Olver, A. B. Olde Daalhuis, D. W. Lozier, B. I. Schneider, R. F. Boisvert, C. W. Clark, B. R. Miller, B. V. Saunders, H. S. Cohl, and M. A. McClain, eds.

Murray Dow. Explicit inverses of Toeplitz and associated matrices. ANZIAM J. , 44(E):E185E215, January 2003. URL http://anziamj.austms.org.au/V44/E019 .

Lloyd N Trefethen. Pseudospectra of matrices. Numerical Analysis , 91:234-266, 1991.

Alexander Atanasov, Jacob A Zavatone-Veth, and Cengiz Pehlevan. Risk and cross validation in ridge regression with correlated samples. arXiv preprint arXiv:2408.04607 , 2024.

Charles R. Harris, K. Jarrod Millman, Stéfan J van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature , 585:357-362, 2020. doi:10.1038/s41586-020-2649-2.

Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, ˙ Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods , 17:261-272, 2020. doi:10.1038/s41592-019-0686-2.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, highperformance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'AlchéBuc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_ files/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf .

A Introduction to integrator models

In this appendix, we provide a brief, pedagogical introduction to the integrator models used as motivating examples in §2. We recall from the main text that both models have dynamics

for state z ∈ R D , recurrent weights J ∈ R D × D , and input u ( t ) ∈ R encoded through a vector b ∈ R D . They differ only in the choice of weight matrix J . These linear dynamics are of course exactly solvable, yielding

A.1 Line attractor

The construction of the classic line attractor network as popularized by Seung [28] starts by assuming that J is symmetric, such that it admits an orthogonal eigendecomposition with real eigenvalues

for OO ⊤ = O ⊤ O = I D and Λ = diag( λ 1 , . . . , λ D ) for λ j ∈ R . We assume that the eigenvalues are ordered as λ 1 ≥ λ 2 ≥ · · · ≥ λ D . For the system to be stable, we must of course have λ j ≤ 1 for all j . Then, letting

be the projections of the state and encoding vector into the eigenvector basis, we have

where we have introduced the timescales

Then, it is easy to see that if for some j we have λ j = 1 , the corresponding timescale τ j will be infinite and the activity ˜ z j ( t ) along that dimension will perfectly integrate u ( t ) . If integrating u ( t ) in a way that is stable to perturbations of the network is our only goal, then activity along other dimensions should decay in time, meaning that we should have all other eigenvalues be strictly less than one, i.e., 1 = λ 1 > λ 2 ≥ · · · ≥ λ D . Moreover, we should have ˜ b k = 0 for all k > 1 , i.e., the input should be aligned to the top eigenvector of J . For the decay to be fast, we want the gap between λ 1 and λ 2 to be large. The classic line attractor network achieves this very simply, choosing

̸

such that it has eigenvalue 1 with multiplicity 1, corresponding to an eigenvector proportional to 1 D , and eigenvalue -1 / ( D -1) with multiplicity D -1 [28, 40].

However, in a realistic setting, it will not be possible to fine-tune the top eigenvalue exactly to 1, and there will be some decay along the integration dimension. Therefore, one must consider approximate line attractor dynamics, for which λ 1 = 1 -ε for some error ε > 0 , while the other eigenvalues are far smaller, i.e., λ 1 ≫ λ 2 ≥ · · · ≥ λ D [13, 27, 28, 40]. This network is exceptionally sensitive to the error ε , as with λ 1 = 1 -ε one has τ 1 = τ/ε , and the error between the true integral of u ( t ) and the readout from the approximate attractor network is exponentially large in time. Yet, so long as λ 1 ≫ λ 2 , perturbations along the approximate integration dimension will still decay exponentially more slowly than those along other dimensions.

and

In Figure 1, we generated connectivity J such that the largest eigenvalue is close to 1 , and all other eigenvalues are < 1 . Specifically, we used J = Q Λ Q -1 for

̸

and Q a matrix generated with entries Q ij ∼ N (0 , 1 √ D ) . Note that for realism, we have relaxed the symmetry constraint, and instead use connectivity that can be related to a corresponding symmetric approximate line attractor via a similarity transform. We use D = 500 as the size of the network.

A.2 Functionally-feedforward integrator

The exquisite sensitivity of the line attractor network to small perturbations of the synaptic weights has motivated theoretical investigation of a panoply of alternative integrator circuits. Restricting our attention to simple linear networks, the most prominent proposal is approximate integration through functionally-feedforward non-normal integration [32, 37]. This model starts with the following linear-algebraic observation: if J is non-normal (i.e., JJ ⊤ = J ⊤ J ), though one loses orthogonal diagonalizability, one can still consider the Schur decomposition

where O is orthogonal and T is upper triangular. As proposed by Goldman [32], the Schur decomposition is a more conceptually useful tool for interpreting non-normal dynamics than the eigendecomposition, as it preserves the orthogonality of the modes. In particular, while if the dynamics are normal T is diagonal and each mode only excites itself, if J is non-normal a given mode may interact 'later' modes in a hidden feedforward structure, revealing a circuit basis for non-normal amplification.

As the simplest example of this structure, Goldman [32] considered a hidden chain structure

As T is strictly upper triangular, all eigenvalues of J vanish. Considering the mode decomposition

as we did in the symmetric case, we have the mode-wide dynamics

This gives sequential low-pass filtering of the input, which allows approximate maintenance of a memory over O ( τD ) time [32, 37]. Importantly, this mechanism is inherently far less sensitive to small variations in the weights than the line attractor.

For the functionally feedforward network in Figure 1, we use

Here, β controls the strength of skip connections that further amplify the output mode of activity. We select β = 0 . 5 so that, like the line attractor network, the activity produced by the functionally feedforward network is approximately low-dimensional. We generate O as an orthonormal matrix uniformly at random with respect to the Haar measure, and use D = 500 as the size of the network. For input weights, we use the sum of the Schur modes b = ∑ D i =1 O : ,i , where O : ,i denotes the i th Schur mode. Then, any readout proportional to the mean Schur mode will then solve the integration task up to a constant rescaling. To achieve the correct readout scale for β = 0 . 5 , we used readout weights 0 . 7 · O , where O = 1 D ∑ D i =1 O : ,i denotes the mean Schur mode.

B MAPinference of connectivity in noise-driven RNNs

In this Appendix, we lay out the procedure sketched in §3 for maximum a posteriori (MAP) inference of connectivity in noise-driven RNNs that underlies our analytical results. We first consider the continuous-time setting directly, and then the discretized case.

and

̸

B.1 Continuous time

We first consider the continuous-time setting. We recall from the main text that we consider a teacherstudent setup, where the teacher has D neurons and a recurrent weight matrix B , such that the dynamics of its firing rate vector z ( t ) ∈ R D is

where ξ ( t ) is uncorrelated Gaussian noise with E [ ξ ( t )] = 0 and E [ ξ ( t ) ξ ( s ) ⊤ ] = 2 σ 2 ξ δ ( t -s ) I D , and ϕ is a possibly nonlinear transfer function, which we take to act elementwise. Again, we assume a d -dimensional student with recurrent weights A , such that its rate x ( t ) ∈ R d evolves as

where η ( t ) is d -dimensional white noise with E [ η ( t )] = 0 and E [ η ( t ) η ( s ) ⊤ ] = 2 σ 2 η δ ( t -s ) I d . Assuming d < D , we observe the first d neurons of the teacher:

Our goal is to infer the student's weight matrix A given these observations.

To do so, we use MAP inference. Our starting point is the likelihood of observing a trajectory { x obs ( t ) : t ∈ [0 , T ] } given a particular weight matrix A , which using the path integral representation of an Itô process can be written non-rigorously as

Here, we have used that

to simplify the notation. To make the problem analytically tractable, we choose an isotropic Gaussian prior over the elements of A :

where ρ > 0 . We have chosen this parameterization of the prior variance because it makes the logposterior density particularly simple:

We remark that we have proceeded rather cavalierly in our treatment of the functional density, but this procedure can equally well be viewed as ridge-regularized least-squares estimation. We will also arrive at the same characterization of the log-posterior density as the continuous-time limit of the discrete setting in the subsequent subsection.

As the log-posterior density is quadratic, it is easy to read off that the MAP estimate of A is

where we add a subscript T to emphasize the observation window. Using the dynamics of x obs ( t ) = P z ( t ) , we can re-write this in terms of the teacher's dynamics as where

is the empirical covariance of the teacher network activity.

So far, we have let ϕ be general. However, we now specialize to the linear setting ϕ ( z ) = z , in which the student and the teacher are Ornstein-Uhlenbeck (OU) processes. Then, we have the formal solution

and, at least in the long-time limit, we can leverage the classical theory of such processes [43, 48].

Provided that all eigenvalues of the dynamics matrix -I D + B have negative real part, this process will converge to a Gaussian stationary state with equal-time covariance

which solves the Lyapunov equation

or equivalently is given by the matrix integral

In the stationary state, the time-lagged correlation

Moreover, if one adds infinitesimal linear perturbations to the dynamics as

one has that the linear response to perturbations of the system in the stationary state is given by

The vanishing of this term follows from the observation that

is given by so that

Thus, we will have and we claim that

while by the Itô isometry

Thus, from (B.1), we conclude that the MAP estimated student dynamics matrix in the long time limit takes the form

As an aside, if B is a symmetric matrix, the process will be reversible, and the stationary state an equilbrium. In this case, setting σ 2 ξ = 1 for brevity, the stationary covariance takes the relatively simple form

In this case, we can gain some intution for the effect of partial observation directly from considering the stationary covariance. Consider a generic symmetric weight matrix, partitioned according to the

observed and non-observed neurons:

We can then interpret

We can then write the marginal covariance matrix of the observed neurons as

as a sort of effective weight matrix that accounts for the effect of feedback through the unobserved neurons on the stationary state.

B.2 Discrete time

We now consider the discrete-time setting, in which the teacher and student are both AR(1) processes. Our goal here is to show that taking the continuum limit of the resulting estimate of the dynamics matrix recovers the result obtained directly in continuous time. Letting

be the discretization scale, the teacher's dynamics are now

where ξ t ∼ N ( 0 , σ 2 ξ I D ) is isotropic Gaussian noise, while those of the student are

where η t ∼ N ( 0 , σ 2 η I d ) is isotropic Gaussian noise. The likelihood of some observed data { x obs t } t ∈ [ T ] is then given by

which is precisely the time-sliced analogue of the functional density considered above. Again assuming an isotropic Gaussian prior on the entries of A , we obtain the corresponding loss function

where ρ corresponds to the strength of the prior/regularization. We then can arrive at the MAP estimate of the dynamics matrix

Again assuming that the observed data { x obs t } t ∈ [ T ] are produced via partial observations of the teacher activity

we can then describe the learned dynamics matrix solely in terms of properties of the teacher RNN:

where

It is now easy to see that the continuum limit of this discrete-time estimate converges in distribution to the continuous-time result.

In discrete time, it is easy to see that and

̸

as z t -1 is independent of ξ t . Then, so long as C T remains bounded, this correlator tends in probability to zero as T →∞ .

We thus arrive at the MAP estimate of the student dynamics matrix in the long time limit:

the discrete time analog of (B.2). If we specialize to the linear case, letting

B.3 A note on time constants

We note an equivalence between the time constants

used in this work and the discrete time analog used in previous work [13, 27, 30],

where λ ′ i are the eigenvalues of the discrete-time dynamics matrix J = (1 -α ) I D + αB , which in terms of the eigenvalues λ i of B has eigenvalues λ ′ i = 1 -α + αλ i . Thus, | λ ′ i | = √ (1 -α + α ℜ λ i ) 2 +( α ℑ λ i ) 2 . Taylor-expanding the logarithm yields

such that we have the formal solution

or, in the true continuous-time limit,

which matches the continuous-time time constants. For α ≪ 1 -ℜ λ i , we therefore may use the continuous-time result with negligible error.

C Normal teacher

In this Appendix, we derive the two results on normal teachers claimed in §3.1 of the main text: that the student eigenvalues are contained within the support of the teacher spectrum, and that an approximate line attractor is recovered by MAP inference even under severe partial observation.

Assuming that BB ⊤ = B ⊤ B , for large T , we can simplify the teacher covariance as follows:

as only the diagonal noise terms should contribute. Further, the first sum above should also tend to 0 assuming that J only has eigenvalues with modulus < 1 . Note that if α is small, this condition is equivalent to the real parts of eigenvalues of B being < 1 . Simplifying the remaining term, we have

where the first equality above follows from normality of B . Summing this Neumann series, this simplifies to

The only term that remains in the limit T →∞ is

yielding

Expanding,

where we have defined B s = B + B ⊤ 2 as the symmetric part of B . This yields the simplification

For α ≪ 1 , we recover the continuous time result for symmetric B stated in (B.3).

Thus, we arrive at the expression

where the regularization ˜ ρ has been re-scaled appropriately to absorb constants.

Since B is normal, it can be diagonalized over C as B = U Λ U ∗ . Observe that B + B ⊤ 2 = B + B ∗ 2 = U ℜ (Λ) U ∗ where ℜ denotes the real part. We then have that

where u i 1: d represents the truncated/projected i th eigenvector of B . For brevity, define K = ( ˜ ρI d + ∑ D i =1 1 1 -ℜ ( λ i ) u i 1: d ( u i 1: d ) ∗ ) .

We first analyze the symmetric case B = B ⊤ , where the argument is slightly simpler. The more general normal case is addressed in C.1. In this case, we have:

Using the fact that for general matrices P and Q , PQ and QP have the same eigenvalues, we can instead analyze the spectrum of

Using that λ 1 ≥ λ i for all i , we have

where P ⪯ Q denotes that P -Q is negative semidefinite.

If we further suppose that ˜ ρλ 1 ≥ 0 , we have the relation

Thus, if ˜ ρλ 1 ≥ 0 , all eigenvalues of ˆ A ∞ satisfy ˆ λ i ≤ λ 1 . Similarly, if ˜ ρλ D ≤ 0 , all eigenvalues of ˆ A ∞ satisfy ˆ λ i ≥ λ D . Both upper and lower bounds are necessarily satisfied simultaneously in the ridgeless limit ρ → 0 .

C.1 More general normal matrix case

Suppose B has p pairs of complex eigenvalues λ c j , λ c j with corresponding eigenvectors u c j , u c j , as well as D -2 p real eigenvalues λ r j with corresponding eigenvectors u r j . Wecan then rewrite (C.1) as

. Similarly, we can also express K in terms of real components:

We then study the spectrum of

ˆ A ′ ∞ is no longer symmetric because of the skew-symmetric components G c j . However, we can still analyze the symmetric component

As before, under the condition ˜ ρ ℜ ( λ 1 ) ≥ 0 , we have the ordering

We can then observe that v ⊤ ( ˆ A ′ ∞ - ℜ ( λ 1 ) I d ) v ≤ 0 for arbitrary v ∈ R d , since the symmetric component of ˆ A ′ ∞ -ℜ ( λ 1 ) I d is NSD. This implies that ℜ ( ˆ λ j ) ≤ ℜ ( λ 1 ) for all j .

Similarly, under the condition ˜ ρλ D ≤ 0 , ℜ ( ˆ λ j ) ≥ ℜ ( λ D ) for all j . Both upper and lower bounds again hold simultaneously for ρ → 0 , regardless of the teacher spectra.

C.2 Stronger result for symmetric teachers

In the symmetric case B = B ⊤ with ρ → 0 , we can also show a stronger result that ˆ λ j ≤ λ j for all j ∈ { 1 , . . . d } :

Observe that for any j > 1 , that

Let λ j ( · ) denote the j th largest eigenvalue of · . Applying Weyl's inequality, we have

which follows since the first term of the RHS is of rank ≤ j -1 .

We also have that

Thus, we can bound (C.3) above by λ j , yielding the result

C.3 Line attractor recovery

Suppose the teacher is a near perfect symmetric line attractor. In particular, let B = B ⊤ have eigenvalues λ 1 = 1 -ε , ε ≪ 1 , and λ i ≪ 1 for i ≥ 2 . For simplicity, assume ρ → 0 . In this case, we can express (C.2) as

Denote P 1 = ∑ D i =2 λ i 1 -λ i u i 1: d ( u i 1: d ) ⊤ and P 2 = ∑ D i =2 1 1 -λ i u i 1: d ( u i 1: d ) ⊤ . From Weyl's perturbation bounds on symmetric matrices [62], we can bound the eigenvalues of the "numerator" as follows:

where (C.6) follows from the Cauchy interlacing theorem [63]. This yields the bound on the top eigenvalue of the numerator,

We can obtain a similar bound on the largest eigenvalue of the "denominator":

In the case where λ i ≥ 0 (e.g., no timescale is faster than the intrinsic timescale of a single neuron), we can use bounds on the eigenvalues of products of PSD matrices to obtain the following:

where we have used 'Num' and 'Den' as shorthand for the factors in (C.5). Assuming eigendirections are randomly oriented, || u 1 1: d || 2 2 = O ( d D ) .

From result (C.4), we have an upper bound on the second largest eigenvalue

Thus, under the stated assumptions, we can conclude ˆ λ 1 ≥ λ 1 -O ( εD d ) , and ˆ λ 2 ≤ λ 2 .

D Feedforward chain

In this Appendix, we derive the approximation for the learned dynamics matrix resulting from partial observations of a feedforward chain that we state in §3.2.

Suppose the teacher matrix has structure B = QMQ ⊤ for M ij = δ i +1 ,j , and QQ ⊤ = Q ⊤ Q = I D . For convenience, we focus on the continuous-time limit. In this limit, the stationary covariance

satisfies the relation

By the nilpotency of B , we have that

We then have that

Defining Σ M as

we can express the stationary covariance as

The learned dynamics matrix is then given by

For simplicity, we consider the Q = I D case, with ρ → 0 . ˆ A will satisfy:

Observe that [ P Σ M P ⊤ ] ij = [Σ M ] ij for 1 ≤ i, j ≤ d , and that

And thus, for d < D , [ PM Σ M P ⊤ ] ij = [Σ M ] i +1 ,j for 1 ≤ i, j ≤ d . We can then make the ansatz that ˆ A ij = δ i +1 ,j + δ id ˆ a j for some constants ˆ a j . This yields the following:

The first d -1 rows of D.1 are equal under this ansatz. The elements ˆ a ∈ R 1 × d are then chosen such that the d th row of D.1 matches, yielding that they must satisfy the following linear relation:

Then, using the bounds [64]

we have that

Using asymptotics for generalized harmonic numbers [64], we have

For any fixed i , this immediately yields

̸

Also note that ˆ A has the form of a companion matrix, and thus has eigenvalues given by the roots of the polynomial f ( λ ) = λ d -∑ d k =1 λ k -1 ˆ a k . Since ˆ a = 0 , we can say that ˆ A will have nonzero eigenvalues.

D.1 Structure of the subsampled stationary covariance

When d ≪ D and D is very large, [ P Σ M P ⊤ ] is well approximated as having a Toeplitz structure with constant differences between diagonals. Specifically, we claim that for 1 ≤ i, j ≤ d ≪ D ,

To show this, we must obtain asymptotics for

when 1 ≤ i, j ≤ d as D →∞ for fixed d . It is easy to confirm that this sum is symmetric in i and j , as

Consider the lower triangular elements, letting j = i -q for q ∈ { 0 , 1 , 2 , . . . , i -1 } . After shifting k ← k -i , we have

It is then easy to see that the diagonal elements ( q = 0 ) are weighted sums of central binomial coefficients:

Now, consider the off-diagonal elements, for q ∈ { 1 , 2 , . . . , i -1 } . We remind ourselves that the sum of interest is

Using the recurrence we have

so

which shows that the matrix elements are non-increasing as one moves away from the diagonal:

Moreover, we have from the same recurrence the weak lower bound

These bounds show that all elements of the truncated covariance matrix must be of the same order. To show that the subleading term is of the desired form, we consider the difference between successive diagonals, which using the above identities may be expressed as

Using the abovementioned bounds on central binomial coefficients, we have the bound

√

which shows that the series is convergent as D →∞ , with an O (1 / D ) remainder. In particular, letting n = D -i +1 , as this bound is monotone decreasing in k , we have

whence

What remains is to compute the infinite sum, which evaluates to

for q ≥ 1 . Therefore, we have

hence in combination with our previous result for the diagonal terms we conclude that

or, restoring the indices, we obtain the claimed result that

This shows that the subsampled stationary covariance matrix is approximately Toeplitz.

D.2 Structure of the student dynamics matrix under heavy subsampling

Now, we consider the structure of the student's dynamics matrix in the d ≪ D regime. The inverse of the form of Toeplitz matrix by which the stationary covariance is approximated is known to take the form [65]:

where c = √ D/π . Wealso have that PM Σ M P ⊤ ≈ P Σ M P ⊤ + 1 2 R where R ij = 1 ( i < j ) -1 ( i ≥ j ) . Thus,

Taking the large c approximation, we find that the learned student dynamics approaches the form

In other words, ˆ A approaches a feedforward chain of size d , except with the activity of the start of the chain never decaying. The largest learned eigenvalue in this limit is 1 , while the others vanish identically.

We note that in practice, the sensitivity of the eigenvalues of feedforward chain connectivity matrices to small perturbations would cause multiple of the learned eigenvalues to be significantly larger than 0 . In particular, the ε -pseudospectrum of a feedforward chain of length d has a radius on the order ε 1 /d [66].

E Low rank

In this Appendix, we derive the results on MAP inference for low-rank null teachers stated in §3.3.

Consider a low-rank teacher of the form B = MN ⊤ , M ∈ R D × r , N ∈ R D × r . If N ⊤ M = 0 r × r and N ⊤ N = M ⊤ M = γ 2 I r , then B has all 0 eigenvalues, but is nonnormal. Here γ 2 is a scale parameter, which in some sense controls the degree of non-normality (scales the norm of the

commutator [ B,B ⊤ ] = BB ⊤ -B ⊤ B ). We compute the stationary covariance of the teacher process, suppressing factors of σ 2 ξ by setting σ 2 ξ = 1 :

Observe that m i n ⊤ i commutes with m j n ⊤ j due to the N ⊤ M = 0 constraint. Thus, we can write

where B s = B + B ⊤ 2 . Performing this integral yields the solution

E.1 Spectrum of the stationary covariance

Our first goal is to determine the eigenvalues and eigenvectors of Σ ∞ . To do so, suppose that u ∈ R D is a unit-norm eigenvector of Σ ∞ with eigenvalue λ . Then, it must satisfy

As M and N span orthogonal r -dimensional subspaces of R D , one possibility is that u lies in the ( D -2 r ) -dimensional complement of those subspaces, in which case it must have eigenvalue 1 . Thus, Σ ∞ has eigenvalue 1 with multiplicity D -2 r . Now consider the case in which u lies in the union of the subspaces spanned by M and N . Make a decomposition

where a , b ∈ R r . The unit-norm condition is

while the eigenvector condition becomes

Acting with M ⊤ , we have while acting with N ⊤ , we have

Together these conditions imply that b = t a , which gives a coupled set of equations for t and λ :

which each must correspond to orthogonal r -dimensional eigenspaces. Therefore, we at last conclude that the eigenvalues of Σ ∞ are 1 with multiplicity D -2 r and λ ± , each with multiplicity r . When γ ≫ 1 , this gives an r -dimensional 'signal' eigenspace with eigenvalue

a ( D -2 r ) -dimensional 'null' eigenspace with eigenvalue 1, and an r -dimensional 'suppressed' eigenspace with eigenvalue

As a result, increasing γ will push the effective dimensionality of activity in the stationary state closer to r .

E.2 Spectrum of the learned dynamics matrix for large γ

We now turn to our main goal, which is to approximately determine the eigenvalues of the learned dynamics matrix after subsampling. Using our result for the stationary covariance, we find that the learned dynamics matrix in the infinite time limit is given by

where ˜ M = PM denotes M truncated to the first d rows. Since MN ⊤ is of rank r , ˆ A will have at most r non-zero eigenvalues.

The relevant regime is when γ ≫ 1 , such that the activity is approximately low-dimensional. Because of the normalization condition N ⊤ N = M ⊤ M = γ 2 I r , in any fixed dimension we must have N ij = O ( γ ) , M ij = O ( γ ) . We can then consider making γ parametrically large, in which case we have

where Π ˜ M ˜ M ⊤ is the orthogonal projector onto the r -dimensional span of ˜ M ˜ M ⊤ . Here, we have used the fact that γ 2 ˜ M ˜ M ⊤ ∼ O ( γ 4 ) and ˜ M ˜ N ⊤ ∼ O ( γ 2 ) , so the former terms will dominate at large γ . Therefore, it follows that as γ becomes large the r non-zero eigenvalues of ˆ A tend to one. This argument relies on fixing all dimensions.

A case of interest is when γ 2 ∼ O ( D/ √ r ) for D ≫ r ; with this scaling, the elements of B are O (1) with respect to D and r .

This linear system has solutions

E.3 Low rank teachers with nontrivial eigenspectra

While the null overlap case is analytically tractable, we would like to also consider non-normal low rank teachers with nontrivial eigenspectra. To do this, we can specify an overlap matrix Q with the desired spectra, such that N ⊤ M = Q . For a fixed M (e.g. selected with random entries), we can satisfy the desired overlap via selecting N ⊤ as N ⊤ = QM † where M † = M + + V ( I D -MM + ) is a generalized pseudoinverse. Here, V ∈ R r × D is arbitrary, and M + is the Moore-Penrose pseudoinverse. Observe that in the case where M has orthogonal columns and V = 0 r × D , M † = M + = M ⊤ , yielding that B = MN ⊤ is normal so long as Q is normal.

Thus, there are then two ways to make B = MN ⊤ = MQM † non-normal; one is to choose choose V = 0 r × D and make Q non-normal. Another way is to select nonzero V , e.g. with entries drawn from N (0 , σ 2 V ) . Then, one can construct a highly non-normal B even with a normal overlap matrix. We use the latter method to generate non-normal low rank teacher connectivity that supports oscillatory dynamics. Specifically, for Fig. 5, we construct teacher connectivity by generating M

F Regularization

In the analysis thus far, we have focused on learning the dynamics matrix in the ridgeless limit. However, using a high enough regularization may help prevent the discovery of spurious slow modes of dynamics from non-normal transients. Indeed, this is necessarily true by observing that in the opposing limit ρ ↑ ∞ , the learned dynamics matrix collapses to the zeros matrix. One can attempt to estimate an optimal regularization parameter that yields a predictive fit via cross-validation, but it is unclear whether this would necessarily resolve the eigenvalue overstimation issues described here. Obtaining an accurate estimate of the optimal ridge from cross-validation on windows of correlated timeseries data is statistically challenging [67], and ill-defined if the timeseries is non-stationary. Furthermore, such problems arising from an inherent expressivity issue of the student network-such as when fitting a low-dimensional LDS model to a functionally feedforward chain-clearly cannot simply be resolved through regularization. We leave a more detailed investigation of regularizationbased strategies for ablating spuriously discovered attractor-like dynamics to future work.

G Numerical methods and supplemental figures

All of our numerical simulations are implemented in Python 3.9.18 using NumPy 1.26.2 [68], SciPy [69], and PyTorch [70]. They were not computationally-intensive, and required less than 12 hours in total to run on a consumer Dell XPS laptop equipped with an Intel Core™ i7-13700H processor. Code to reproduce all experiments is available at https://github.com/wqian0/ DataConstrainedRNNs/ .

For simplicity, we use τ = 1 in all numerical simulations. Unless stated otherwise, we use a teacher network size of D = 500 in all numerical experiments.

We integrate the student and teacher RNN dynamics via Euler integration with a timestep ∆ t = 0 . 01 . Under the discretization scheme of B.4, in all experiments, we select the noise parameters of the student and teacher dynamics as σ η = σ ξ = 0 . 02 √ 2 .

In the examples of Fig. 1, we generate ground truth network activity by iterating the dynamics for a duration T = 5000 × ∆ t .

In the purely noise-driven experiments with finite observation time windows, we fit student networks to ground truth teacher activity generated over a duration T = 30000 × ∆ t .

For MAP inference, we use a regularization parameter ρ = 0 . 001 in all experiments. In experiments involving the long time limit T →∞ , we use SciPy [69]'s built-in Lyapunov solver to compute the stationary covariance of the teacher activity.

For all LDS models, we run the fitting procedure for 200 iterations using the implementation provided by the authors of [18] under an MIT License on GitHub. 2 For the experiments in Fig. 1, the input

signal was explicitly passed to the fitting procedure. For the purely noise-driven teacher setting of Fig. G.5, no input signal was provided.

Fig. 5: We note that CORNN is applicable only for fitting RNNs that obey slightly different dynamics (leaky-rate instead of leaky-current), of the form

We thus modify student and teacher accordingly in experiments involving CORNN [14]. We run the CORNN fitting procedure for 1000 iterations using the implementation provided by the authors as free software on GitHub. 3

For backpropagation-through-time, we train student network dynamics matrices on observed teacher activity via truncated BPTT, splitting the observed 30000 timepoints into segments of 500 contiguous timepoints. Student networks were then trained for 100 epochs at a learning rate 0 . 01 , batch size of 16 , and a teacher forcing ratio of 0 . 5 .

For experiments with FORCE, we train student networks for 1000 iterations with parameters g = 1 . 5 , P 0 = 0 . 01 . We adapt the implementation provided by the authors of [17] under a GNU GPL license on Github. 4 To increase agreement of the fitted activity, we restricted the observation window of the fitted network to the first 5000 timepoints; fitting to the full observation window yielded fits that failed to capture prominent initial transients of the observed activity.

For experiments involving low-rank teacher networks with null overlap, we use γ 2 = 0 . 2 D √ r for the linear networks (Fig. 4) and γ 2 = 0 . 4 D √ r for the nonlinear networks (Fig. G.6). This scaling ensures elements of B are order-1 (see E.2). For low-rank teacher networks with nontrivial eigenspectra (Fig. 5), see E.3.

0.0

0.5

1.0

Rank 2

Rank 3

5

10

0.0

0.5

1.0

Rank 4

5

10

Rank 5

Dimension

(

)

Finite time

T

BPTT

MAP

BPTT + hidden

10%

GT

10%

10%

10

2

10

1

10 0

Subsampling fraction

0

1

2

3

4

5

6

Line attractor score

10

2

10

1

10 0

Subsampling fraction

0

1

2

3

4

5

6

Singular value

1

2

1

2

10

2

10

1

10 0

Subsampling fraction

0.0

0.2

0.4

0.6

0.8

1.0

Overlap

O

([

u

1]

1:

d

,

u

1)

O

([

v

1]

1:

d

,

v

1)

spurious

1

d

10

2

10

1

10 0

Subsampling fraction

0

1

2

3

4

5

6

Line attractor score

10

2

10

1

10 0

Subsampling fraction

0

2

4

6

8

10

Singular value

1

2

1

2

10

2

10

1

10 0

Subsampling fraction

0.0

0.2

0.4

0.6

0.8

1.0

Overlap

O

([

u

1]

1:

d

,

u

1)

O

([

v

1]

1:

d

,

v

1)

spurious

1

d

10

2

10

1

10 0

Subsampling fraction

0

1

2

3

4

5

6

Line attractor score

10

2

10

1

10 0

Subsampling fraction

0

5

10

15

20

Singular value

1

2

1

2

10

2

10

1

10 0

Subsampling fraction

0.0

0.2

0.4

0.6

0.8

1.0

Overlap

O

([

u

1]

1:

d

,

u

1)

O

([

v

1]

1:

d

,

v

1)

spurious

1

d

10

2

10

1

10 0

Subsampling fraction

0

2

4

6

Line attractor score

10

2

10

1

10 0

Subsampling fraction

0

10

20

30

40

50

Singular value

1

2

1

2

10

2

10

1

10 0

Subsampling fraction

0.0

0.2

0.4

0.6

0.8

1.0

Overlap

O

([

u

1]

1:

d

,

u

1)

O

([

v

1]

1:

d

,

v

1)

spurious

1

d

10

2

10

1

10 0

Subsampling fraction

0

2

4

6

Line attractor score

10

2

10

1

10 0

Subsampling fraction

0

20

40

60

80

100

Singular value

1

2

1

2

10

2

10

1

10 0

Subsampling fraction

0.0

0.2

0.4

0.6

0.8

1.0

Overlap

O

([

u

1]

1:

d

,

u

1)

O

([

v

1]

1:

d

,

v

1)

spurious

1

d

10

2

10

1

10 0

Subsampling fraction

0

2

4

6

Line attractor score

10

2

10

1

10 0

Subsampling fraction

0

50

100

150

200

Singular value

1

2

1

2

10

2

10

1

10 0

Subsampling fraction

0.0

0.2

0.4

0.6

0.8

1.0

Overlap

O

([

u

1]

1:

d

,

u

1)

O

([

v

1]

1:

d

,

v

1)

spurious

1

d

a

b

a

5%

50%

GT

i)

ii)

iii)

b

5%

50%

GT

i)

ii)

iii)

a

0

100

0

100

100

0

100

0

100

100

0

100

100

0

100

0.0

0.5

1.0

t/T

PC1 (A.U.)

PC2 (A.U.)

b

0

50

0

50

0

100

0

50

0

100

0

100

0.0

0.5

1.0

t/T

PC1 (A.U.)

PC2 (A.U.)

a

10

2

10

1

Subsampling fraction

0

1

2

3

4

5

6

Line attractor score

b

2

3

4

5

6

Latent dim.

0

1

2

3

4

5

6

Line attractor score

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state the claims we make in our paper, and its limitations.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We emphasize the limitations of the settings of our experiments and theoretical results throughout, as well as the fact that many of our theoretical results are asymptotic or approximate. We test finite-size deviations from those approximations in our experiments. Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide complete derivations of our results in Appendices B, C, D, and E. We note that many of our analytical results are approximate, as we emphasize throughout.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We describe our numerical results in detail in Appendix G. Moreover, we include code to reproduce all results with the initial submission.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.

In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: For initial submission, we include code to reproduce all experiments as a ZIP file. For publication, we will make code available on GitHub.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide details in Appendix G and in the code included with our submission.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For plots showing average metrics over multiple instantiations, we include either shaded regions or error bars denoting one standard error of the mean. In some cases, the error bars are too small to see. For other results, we do not make claims of statistical significance, so we do not include error bars. Our empirical results are proofs of concept rather than claims regarding frequency.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide details in Appendix G.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Yes, we affirm that our research conforms with the NeurIPS Code of Ethics. Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: We do not anticipate our work will have societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper involves neither data nor models that have a risk of misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We provide citations and license information for all code used in Appendix G.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors investigate whether dynamical systems models that are statistically fit to neural data recapitulate the same dynamical mechanisms of the circuit. Commonly, neuroscientists only measure activity from a small fraction of neurons within a circuit. Under this constraint, the authors show examples where the model learns fundamentally different dynamical mechanisms. If true, this would be of great significance to the computational neuroscience community and an important cautionary tale.\n\nSTRENGTHS:\nThere are a lot of good things to say about this paper. The motivating question is very interesting and important. The mathematical framing and analysis is elegant. The introduction and discussion sections are skillfully written to target the computational neuroscience community, and I think the paper has the potential to be very impactful within this community after some revisions.\n\nWEAKNESSES:\nI very much liked the big picture idea of this paper. However, in various parts of the paper, the writing is unclear and left me confused on details. The interpretation of certain findings should be clarified. I am optimistic about being to raise my score during the discussion period if these weaknesses can be addressed.\n\n* In section 2, the authors fit a linear dynamical system model with 5 latent dimensions (line 113) to 5% of neurons sampled from a feedforward chain circuit. The model fails to recapitulate the feedforward chain and instead learns a line attractor. Critically, there are two distinct but non-mutually exclusive explanations for this result: (a) 5 latent dimensions is not enough to capture the line attractor, or (b) the partial observation of 5% of the circuit is insufficient. The title and abstract of the paper exclusively favor explanation (b). However, on lines 127-132 the authors do not mention (b) and instead put forth (a), \"[if] the data constrained model has fewer neurons, it cannot realize a feedforward chain with sufficiently long memory\". Here, the recurrent dynamics in the LDS happen within a 5-dimensional latent space, so I assume that the authors mean \"fewer neurons\" to mean the relatively low dimensionality of the latent space.\n\n* I would like to see the result in section 2 more systematically investigated as follows. Let `N` denote the number of neurons in the teacher circuit (feedforward chain). Let `d` denote the latent dimensionality of the LDS model, and let `n` denote the number of neurons that are measured / observed for fitting the LDS model. I am interested to see whether the LDS model still fails to learn a feedforward chain when `N` is large (e.g. maybe 1000) and `n = d` is big but still a small fraction of the overall system (e.g. `n = d = 100`). Additionally, it would be elucidating to see an example where `N = n` (i.e. fully observed circuit), but a low-dimensional dynamical model is assumed `d < n`. The purpose of these additional experiments is to clarify whether the problem is that `n` $\\ll$ `N` (as suggested by the title / abstract), or whether the real problem is that `d` $\\ll$ `N`, or whether both are a problem.\n\n* Figure 3 seems to get at some of these concerns, but I think the paper will be much clearer if the above comments are addressed in Figure 1 as well.\n\n* In Figure 3, it is still a little unclear to me whether subsampling, per se, is the problem. If I understand the construction of the teacher network properly, as `D` increases, the timescale of the feedforward chain also increases. It is harder for the student network to match this timescale with a chain, leading it to instead learn a line attractor. I would like to see an experiment where the timescale of the feedforward chain is kept constant while increasing `D` (e.g. by inserting multiple copies of each neuron in the feedforward chain). In this case, when you randomly sample `d` neurons for the student network, do the learned dynamics still fail to capture a feedforward chain for large `D`? An affirmative answer would seem to be more in agreement with the main claims of the title and abstract. A negative answer would still be interesting, but would change the interpretation of the paper to be more in line with the idea that a mismatch in dimensionality is the core of the problem instead of sub-sampling.\n\nQUESTIONS:\nSee \"Weaknesses\".\n\nLIMITATIONS:\nLimitations are adequately discussed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors address provide a cautionary tale for how data-constrained RNN models can mis-identify the dynamical structure underlying neural population computations when only a subset of the neural population is observed (\"partial observation\").  Empirical case studies are provided, whereby data-constrained student networks are trained from partial observation of synthetic teacher networks. The analyzed dynamics of the trained student networks differed from those of the teacher network in several teacher network setups. When teacher networks performed integration of a stimulus input via a feedforward chain dynamic, student networks were found to be biased to recover a line attractor dynamic under certain conditions. Mismatched student-teacher dynamics were also shown when teachers were linear noise-driven networks with non-normal dynamics or low-rank connectivity. The final case study provided details similar propensities for mismatched student-teacher dynamics when all networks are non-linear.\n\nSTRENGTHS:\n- Originality: The overall message and cautionary tale is indeed original (although see related work in Weaknesses).\n\n- Quality: The case study examples are well chosen and illustrative of the potential for mismatched student-teacher dynamics. The experiments appear well controlled and visualizations are intuitive and elegant.\n\n- Clarity: Overall, the writing is clear and appropriately didactic. \n\n- Significance: Recent years have seen a proliferation of data-constrained model development and interest in interpreting the corresponding trained models. Thus, this cautionary tale message is quite significant and timely. While perhaps beyond the scope of this work, solutions to the problems exposed, even if only hypothesized, could improve the significance of this work.\n\nWEAKNESSES:\n- Minor: There is a line of related work from Chethan Pandarinath's group on interpretable data-constrained models that should be referenced if not explicitly discussed. For example: Sedler, Andrew R., Christopher Versteeg, and Chethan Pandarinath. 2023. “Expressive Architectures Enhance Interpretability of Dynamics-Based Neural Population Models.” Neurons, Behavior, Data Analysis, and Theory, March, 1–22. https://doi.org/10.51628/001c.73987.\n\nQUESTIONS:\n- If I understood correctly, the student models applied throughout were designed to provide a one-to-one mapping from hidden units in the student RNN to observed teacher unit activity. Other data-constrained architectures do not rely on such design constraints--in particular, LFADS [11] and Neural ODEs [33] attempt to reproduce single-trial neural recordings (or teacher activity) via a readout from the RNN hidden activity space, thus allowing the RNN expressivity to be decoupled from the size of the observed neural population.  To what extent might some of the described mismatches be artifacts of the limited expressivity of RNN models whose individual units are constrained to directly match observed teacher unit activity? To be concrete, in the integration example, too few units in the student network was explained as a cause for the student network mis-identifying a line attractor motif when the teacher actually employed a feedforward chain (lines 127-132; and Figure 1b). Would a mismatch still appear by when fitting partial observations using a less-constrained architecture like LFADS?\n- Minor: The intro paragraph beginning on line 34 lists a number of challenges for data-constrained models. Why is it important for us to consider all of these limitations when only the partial observability is addressed in this work? There may be an opportunity to clarify the exposition here.\n- Minor: Line 66: \"even under relatively ideal conditions where the input to a circuit is either perfectly known or white noise\". In the white noise case, were known inputs not provided to the student networks? This sentence could benefit from clarification.\n\nLIMITATIONS:\nThe authors did not include an explicit \"Limitations\" section, and beyond a brief sentence in the author checklist, limitations of the techniques are not discussed explicitly in the manuscript. I would encourage the authors to consider stating some of the limitations of their studies and the particular model class chosen for the student RNNs.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nDeriving a mechanistic understanding of neural circuits from observations (neural recordings) is a fundamentally ill-posed problem. This paper explores and exposes this issue in controlled theoretical settings. Specifically, the authors focus on two aspects: the intrinsic biases of data-constrained surrogate models and the partial observability of the data. They demonstrate these pitfalls in both linear and non-linear systems.\n\nSTRENGTHS:\nThis is a beautifully written paper, clear and concise. The examples are well motivated. The theoretical and experimental analyses are mathematically tight. Moreover, the issue of fitting models to partially-observed data and deriving mechanistic insights from such surrogate models is a pertinent one for neuroscience today, given the developments in recording techniques and data-driven modeling.\n\nWEAKNESSES:\nWhile the problem formulation is intuitive and well-motivated, the settings considered here are exceptionally constrained. This paper would greatly benefit from and be practically more useful if the authors provided scenarios where observed data has more than one plausible mechanistic explanation and how one would go about falsifying each hypothesis. The authors allude to perturbation analyses as a solution, but once again, the interpretations are unclear when we lack a \"ground truth.\" \n\nAre there existing biological neural datasets that the authors can use to expose possible misinterpretations of circuit mechanisms?\n\nThis question of system identification has been of long-standing interest in neuroscience. Coupled with surrogate modeling, there is a big literature on simulation-based inference and adjacent topics that would be worth adding as discussion in this current manuscript. Some representative references:\n\nFlexible and efficient simulation-based inference for models of decision-making. Boelts et al. (2022)\n\nFast Inference of Spinal Neuromodulation for Motor Control using Amortized Neural Networks. Govindarajan et al. (2022)\n\nThe frontier of simulation-based inference. Cranmer et al. (2020)\n\nQUESTIONS:\nPlease refer to the weaknesses section above.\n\nLIMITATIONS:\nPlease refer to the weaknesses section above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-21"}