Back to the Continuous Attractor

Authors: ábel  ságodi
guillermo  martín-sánchez
piotr  sokół
memming  park
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 21:29:03

Abstract: Continuous attractors offer a unique class of solutions for storing continuous-valued variables in recurrent system states for indefinitely long time intervals.
Unfortunately, continuous attractors suffer from severe structural instability in general---they are destroyed by most infinitesimal changes of the dynamical law that defines them.
This fragility limits their utility especially in biological systems as their recurrent dynamics are subject to constant perturbations.
We observe that the bifurcations from continuous attractors in theoretical neuroscience models display various structurally stable forms.
Although their asymptotic behaviors to maintain memory are categorically distinct, their finite-time behaviors are similar.
We build on the persistent manifold theory to explain the commonalities between bifurcations from and approximations of continuous attractors.
Fast-slow decomposition analysis uncovers the existence of a persistent slow manifold that survives the seemingly destructive bifurcation, relating the flow within the manifold to the size of the perturbation. Moreover, this allows the bounding of the memory error of these approximations of continuous attractors.
Finally, we train recurrent neural networks on analog memory tasks to support the appearance of these systems as solutions and their generalization capabilities.
Therefore, we conclude that continuous attractors are functionally robust and remain useful as a universal analogy for understanding analog memory.

Main paper: 
 introduction: 
 Biological systems exhibit robust behaviors that require neural information processing of analog variables such as intensity, direction, and distance. Virtually all neural models of working memory for continuous-valued information rely on persistent internal representations through recurrent dynamics. The continuous attractor structure in their recurrent dynamics has been a pivotal theoretical tool due to their ability to maintain activity patterns indefinitely through neural population states 131, 28, 66, 32 . They are hypothesized to be the neural mechanism for the maintenance of eye positions, heading direction, self-location, target location, sensory evidence, working memory, and decision variables, to name a few 111, 113, 107 . Observations of persistent neural activity across many brain areas, organisms, and tasks have corroborated the existence of continuous attractors 107, 88, 95, 139, 23, 87, 86, 129, 76 .Despite their widespread adoption as models of analog memory, continuous attractors are brittle mathematical objects, casting significant doubts on their ontological value and hence suitability in accurately representing biological functions. Even the smallest arbitrary change in recurrent dynamics can be problematic, destroying the continuum of fixed points essential for continuous-valued working memory. In neuroscience, this vulnerability is well-known and often referred to as the "fine-tuning problem" 111, 103, 16, 78, 92, 61 . There are two primary sources of perturbations in the recurrent network dynamics: (1) the stochastic nature of online learning signals that act via synaptic plasticity, and (2) spontaneous fluctuations in synaptic weights 44, 114 . Thus, additional mechanisms are necessary to compensate for the degradation in particular implementations, by bringing the short-term behavior closer to that of a continuous attractor 74, 75, 11, 72, 103, 51, 53 . However, we lack the theoretical basis to understand how much this matters in practice, i.e. what are the effects of different levels of degradation on memory. This is fundamental to justify relying on the brittle concept of continuous attractors for understanding biological analog working memory.In this study, we explore perturbations and approximations of continuous attractors in the space of dynamical models. We first report on the differences and similarities between the various structurally stable dynamics in the vicinity of continuous attractors in the space of dynamical systems models. Our analysis reveals the presence of a "ghost" continuous attractor (a.k.a. slow manifold) in all of them (Sec. 2). By assuming normal hyperbolicity we separate the time scales to obtain a decomposition of the dynamics by separating out the fast flow normal to and the slow flow within the slow manifold. We derive theoretical results that ensure the existence of a slow manifold and determine its closeness to a continuous attractor (Sec. 3). We explore task-trained recurrent neural networks (RNNs) to show that these systems appear naturally as solutions to the task (Sec. 4) and that their generalization capabilities can easily be studied as the distance to the continuous attractor (Sec. 5). The proposed decomposition applied to theoretical models and task-trained RNNs reveals a "universal motif" of analog memory mechanism with various potential topologies. This leads to the connection of different systems with different topologies as approximate continuous attractors (Sec. 6). Our theory guarantees that systems close to a continuous attractor (in the space of vector fields) will have similar behavior to it, implying that the concept of continuous attractors remains a crucial framework for understanding the neural computation underlying analog memory (Sec. 3.4). 
a critique of pure continuous attractors: 
 We will first lay out a number of observations about the dynamics of bifurcations and approximations of continuous attractors used in theoretical neuroscience. Ordinary differential equations (ODEs) are commonly used to describe the dynamical laws governing the temporal evolution of firing rates or latent population states 131 . In this framework, neural systems are viewed as implementing the continuous time evolution of neural states to perform computations. We will consider a continuous attractor as a mechanism that implements analog memory computation: carrying a particular memory representation over time. To define it formally, let x(t) ∈ R d denote the neural state, and ẋ = f (x) represent its dynamics. Let M ⊂ R d be a manifold. We say M is a continuous attractor, if (1) every state on the manifold is a fixed point, ∀x ∈ M, f (x) = 0, and (2) the fixed points are marginally stable tangent to the manifold and stable normal to the manifold. In other words, the continuous attractor is a continuum of equilibrium points such that the neural state near the manifold is attracted to it, and on the manifold, the state does not move. Marginal stability implies that continuous systems are structurally unstable, meaning that small perturbations or variations in the system's parameters lead to significant changes in the system's behavior or stability 94, 91, 105, 106 . We will now study some examples of continuous attractors and how perturbations change their dynamics.As an example, we can construct a line attractor (a continuous attractor with a line manifold) as follows:EQUATIONwhere W = [0, -1; -1, 0] and b = [1; 1], and [•] + = max(0, •) is the threshold nonlinearity per unit. We get ẋ = 0 on the x 1 = -x 2 + 1 line segment in the first quadrant as the manifold (Fig. 1A , left; black line). Linearization of the fixed points on the manifold exhibits two eigenvalues, 0 and -2; the 0 eigenvalue allows the continuum of fixed points, while -2 makes the flow normal to the manifold attractive (Fig. 1A , left; flow field).In general, continuous attractors are not only structurally unstable 80 , they bifurcate almost certainly for an arbitrary perturbation. Small changes to the parameters (b, W) perturb the eigenvalues and any perturbation to the 0 eigenvalue destroys the continuous attractor: it bifurcates to either a single stable fixed point (Fig. 1A , top) or two stable fixed points separated with a saddle-node in between (Fig. 1A , bottom). However, interestingly, after bifurcation, continuous attractors seemingly tend to leave a "ghost" manifold topologically equivalent to the original continuous attractor (note the slow speed). Furthermore, the flow after the bifurcation is contained in the ghost manifold, i.e., it is an invariant manifold. This phenomenon, wherein a continuous attractor is approximated by a manifold within the neural space along which the drift occurs at a very slow pace, has previously been commented on 112, 81, 109 . Figure 1 : The critical weakness of continuous attractors is their inherent brittleness as they are rare in the parameter space, i.e., infinitesimal changes in parameters destroy the continuous attractor implemented in RNNs 111, 103 . Some of the structure seems to remain; there is an invariant manifold that is topologically equivalent to the original continuous attractor. (A) Phase portraits for the bounded line attractor (Eq. ( 1)). Under perturbation of parameters, it bifurcates to systems without the continuous attractor. (B) The low-rank ring attractor approximation (Sec. (S3.4)). Different topologies exist for different realizations of a low-rank attractor: different numbers of fixed points (4, 8, 12), or a limit cycle (right bottom). Yet, they all share the existence of a ring invariant set.For circular variables such as the goal-direction (e.g. for navigation 119, 132 and working memory for communication in bees 130 ) or head-direction, the temporal integration, and working memory functions are naturally solved by a ring attractor (continuous attractor with a ring topology) 67, 68, 125, 126, 59, 123, 122, 3, 41 . Other examples include integration of evidence for continuous perceptual judgments, e.g. a hunting predator that needs to compute the net direction of motion of a large group of prey 37 . In this section, we investigate the bifurcations of various implementations of continuous attractors. Continuous attractor network models of the head direction are based on the interactions of neurons that (anatomically) form a ring-like overlapping neighbor connectivity 142, 88, 1, 128, 12, 70, 117, 137 . Similarly to the line attractor, the ring attractor bifurcates with almost any perturbation. However, the resulting dynamics continue to follow a familiar pattern: they remain confined to a ghost manifold that closely approximates the original continuous attractor.Piecewise-linear ring attractor model of the central complex: Firstly, we discuss perturbations of a continuous ring attractor proposed as a model for the head direction representation in fruit flies 88 . This model is composed of N heading-tuned neurons with preferred headings θ j ∈ { 2πi /N} i=1...N radians (Sec. S3.2). For sufficiently strong local excitation (given by the parameter J E ) and broad inhibition (J I ), this network will generate a stable bump of activity corresponding to the head direction. This continuum of fixed points forms a N -sided polygon.We evaluate the effect of parametric perturbations of the form W ← W+V with V i,j iid ∼ N (0, 1 /100) on a network of size N = 6 (forming an hexagon, see also Sec. S3). We found that the ring (consisting of infinite fixed points) can be perturbed into systems with between 2 and 12 fixed points (Fig. 2A ). As far as we know, this bifurcation from a ring of equilibria to a saddle and node has not been described previously in the literature. The probability of each type of bifurcation was numerically estimated (Sec. S3.2). Surprisingly, the number of fixed points is maintained throughout a range of perturbation sizes and hence depends only on the direction of the perturbation (Fig. S8 ).A well-established approach, consistent with the principles observed in the fly head direction network 64 , involves utilizing a circulant matrix as the connectivity matrix 9, 108 . This type of ring attractor network can support a stable "activity bump" that can move around the ring in correspondence with changes in head direction. This can be accomplished with a connection matrix W with entries that follow a circular Gaussian function of i -j 110, 101, 50, 22 (see for more details Sec. S3.3). For finite-sized networks, the dynamics are constrained to an attractive invariant ring, covered with N stable fixed points for a network of size N (Fig. 2B ). For such networks the number of fixed points does change with the size of the perturbation (Fig. S8 ). Low-rank ring attractor model: Low-rank networks can be used to approximate ring attractors 82, 7 .In the limit of infinite-size networks, one can construct a ring attractor through a rank 2 network by constraining the overlap of the right-and left-connectivity vectors (see for more details Sec. S3.4). However, in simulations of finite-size networks with this constraint, the dynamics instead always converge to a small number of stable fixed points arranged on a ring (Fig. 1B ).Embedding Manifolds with Population-level Jacobians: Approximate ring attractors can be constructed by constraining the connectivity so that the networks Jacobian satisfies certain requirements for a ring attractor to exist 96 (see for more details Sec. S3.5). The models fitted with this method also can contain an invariant ring manifold on which the dynamics contain stable and saddle fixed points (Fig. 2C ). It has been observed that approximate continuous attractor emerge in networks trained on sampled points with other methods as well 27 .Similarity between all bifurcations and approximations of continuous attractors: In all discussed models of ring attractors, we verify that they suffer from the fine-tuning problem. However, importantly, we also observe in all the systems the existence of ghosts of the continuous attractor (either through bifurcation or from finite-size effects) in the form of an attractive invariant manifold. Therefore, while they are not strictly a continuous attractor in the mathematical sense, they are approximate ring attractors in the sense that the fixed points and connecting orbits still form a circle.Is this lawful degradation a universal phenomenon? And if so, how does it relate to the size of the perturbation? And what are the implications for the memory performance of these approximations? (Sec. 3). Do these approximations appear as natural solutions to the memory storage problem? (Sec. 4). And if so, how well do they generalize to longer time requirements? (Sec. 5). Finally, are continuous attractors in practice still useful as an idealized model of how animals represent continuous variables? (Sec. 6). 
theory of approximate continuous attractors: 
 In this section, we theoretically answer in an implementation-agnostic manner the degradation questions posed from the exploration. To do so, we apply invariant manifold theory to continuous attractor models and translate the results for the neuroscience audience (see also Sec. S1).First, we argue that the lawful degradation into a system with a slow manifold is universally guaranteed (as long as the perturbation is small, and the continuous attractor was normally hyperbolic). Let l be the intrinsic dimension of the manifold of equilibria that defines the continuous attractor. Given a perturbation p(x) to the ODE that induces a bifurcation,EQUATIONwhere ∥p(•)∥ ∞ = 1 and ϵ > 0 is the bifurcation parameter, we can reparameterize the dynamics around the manifold with coordinates y ∈ R l and the remaining ambient space with z ∈ R d-l . To describe an arbitrary bifurcation of interest, we introduce a sufficiently smooth function g and h, such that the following system is equivalent to the original ODE:EQUATION) where ϵ = 0 gives the condition for the continuous attractor ẏ = 0. We denote the corresponding manifold of l dimensions M 0 = {(y, z) | h(y, z, 0) = 0}.We say the flow around the manifold is normally hyperbolic, if the flow normal to the manifold is hyperbolic: the Jacobians ∇ z h evaluated at any point on the M 0 has d -l eigenvalues with their real part uniformly bounded away from zero, and ∇ y g has l eigenvalues with zero real part. More specifically, for continuous attractors, the real part of the eigenvalues of ∇ z h will be negative, representing sufficiently strong attractive flow toward the manifold. Equivalently, for the ODE, ẋ = f (x), the variational system is of constant rank and has exactly (d -l) eigenvalues with negative real parts uniformly away from zero and l eigenvalues with zero real parts everywhere along the continuous attractor.Figure 3 : Persistent manifold theorem applied to compact continuous attractor guarantees the flow on the slow manifold M ϵ is invariant and continues to be attractive. The dashed line is a trajectory "trapped" in the slow manifold (which has the same topology as the continuous attractor).For any parameterization g, ϵ > 0 induces a bifurcation of the continuous attractor. What can we say about the fate of the perturbed system? The continuous dependence theorem 17 says that the trajectories will change continuously as a function of ϵ without a guarantee on how quickly they change. However, the topological structure and the asymptotic behavior of trajectories change discontinuously due to the bifurcation. Yet, surprisingly, there is a strong connection in the geometry due to Fenichel's theorem 40 . 1 We informally present a special case due to Jones 63 :Theorem 1 (Persistent Manifold). Let M 0 be a connected, compact2 , normally hyperbolic manifold of equilibria originating from a sufficiently smooth ODE. For a sufficiently small perturbation ϵ > 0, there exists a manifold M ϵ diffeomorphic to M 0 and invariant under the flow of Eq. (3)-( 4).The manifold M ϵ is called the slow manifold which is no longer necessarily a continuum of equilibria. However, the invariance implies that trajectories remain within the manifold except potentially at the boundary. Furthermore, the non-zero flow on the slow manifold is slow and given in the ϵ → 0 limit as dy dτ = g(c ϵ (y), y, 0) where τ = ϵt is a rescaled time and c ϵ (•) parameterizes the l dimensional slow manifold. In addition, the stable manifold of M 0 is similarly persistent 63 , implying that the manifold M ϵ remains attractive. Finally, the persisting invariant manifold is very close in space to the original continuous attractor (see also Theorem 3). These conditions are met for the examples in Fig. 1 (see Sec. S2.1 for the fast-slow rerparametrization of the BLA 3 ). As the theory predicts, BLA bifurcates into a 1-dimensional slow manifold (Fig. 1 , dark-colored regions) that contains fixed points and connecting orbits and is overall still attractive. Furthermore, Fenichel's Persistent Manifold theorem explains the bifurcation structure of the theoretical models discussed in Sec. 2.2. Because continuous ring attractors are bounded, they persist as invariant manifold and remain attractive under small perturbations 133 .Second, we relate the flow tangent to the slow manifold to the size of the perturbation needed to bring it back to a continuous attractor. Consider a behaviorally relevant timescale for working memory, for example, roughly up to a few tens of seconds. If the dynamical system is orders of magnitude slower, for example, 1000 sec or longer, its effect is too slow to have a practical impact on the behavior that relies on the working memory mechanism. This clear gap in the fast and slow time scales can be recast as normal hyperbolicity of the slow manifold by relaxing the zero real part to a separation of time scales (reciprocal of eigenvalues or Lyapunov exponents). In other words, the attractive flow normal to the manifold needs to be uniformly faster than the flow on the slow manifold. By taking the limit of the slow flow on the manifold to arbitrarily long time constant (i.e., to zero flow), we achieve the reversal of the persistent manifold theorem. Proposition 1 (Revival of continuous attractor). Let M ϵ be a connected, compact, normally hyperbolic slow manifold (as parametrized by Eq. (3)-( 4)). Further, assume that the real part of the eigenvalues of ∇ z h are negative. Let the uniform norm of the flow tangent to the manifold be ∥ ẏ∥ ∞ = η. There exists a perturbation with uniform norm at most η that induces a bifurcation to a continuous attractor manifold.An explicit perturbation is derived in Sec. S5.1. This makes the uniform norm of the vector field on a (slow) manifold a useful measure to express the distance of an approximation to a continuous attractor. Prop. 1 can be extended to the case where the invariant manifold has additional dynamics to which the output mapping is invariant (see Theorem 7). These systems can be perturbed onto a decomposable system where one of the subsystems has a slow flow.Third, we relate the flow of the manifold (and, through Prop. 1, the size of the perturbation) to the memory error of the approximation in short-time scale. We also discuss the implications of the theoretical insights on the memory error in the asymptotic time scale.In the short-time scale the memory performance is bounded by the uniform norm of the flow tangent to the manifold. Let x 0 ∈ M, and φ = p(•)| M be the vector field restricted to the manifold (following the notation in Eq.2). The average deviation from initial memory x 0 over time is bounded linearly (for a derivation, see Sec. S6):EQUATIONNote that this bound is the worst case and tighter for sufficiently small t ≥ 0. Furthermore, for compact invariant manifolds the error is bounded by the diameter of the manifold and hence this bound becomes irrelevant for t large.While the uniform norm gives insight on the short-time scale behavior of the perturbed ODE, we expect that working memory tasks generalize to longer durations 92 . The long-time scale behavior on the slow manifold is dominated by the stability structure, i.e., the topology of the dynamics. Although we have seen numerous topologies in Sec. 2, the Persistent Manifold Theorem says that this variability is fundamentally limited, especially in low dimensions (see for more details Sec. S4). This is especially relevant as previous works have identified a low-dimensional organization of neural activity to explain the brain's ability to adapt behavioral responses to changing stimuli and environments 8, 2, 38 . For a ring attractor, this implies that the stability structure of the invariant manifold is either (1) composed of an equal number of stable fixed points and saddle nodes, placed alternatingly and with connecting orbits, or (2) a limit cycle. These different strability structures have different generalization properties (see Sec. 5). In more complex scenarios, such as two-dimensional attractors, fixed points can coexist with limit cycles, creating a rich tapestry of possible neural states.Animal behavior exhibits strong resilience to changes in their neural dynamics, such as the continuous fluctuations in the synapses or slight variations in neuromodulator levels or temperature. Hence, any theoretical model of neural or cognitive function that requires fine-tuning, such as the continuous attractor model for analog working memory, raises concerns, as they are seemingly biologically irrelevant. Moreover, unbiased data-driven models of time series data and task-trained recurrent network models cannot recover such continuous attractor theories precisely. Our theory shows that this apparent fragility is not as devastating as previously thought: despite the "qualitative differences" in the phase portrait, the "effective behavior" of the system can be arbitrarily close, especially in the behaviorally relevant time scales. We show that as long as the attractive flow to the memory representation manifold is fast and the flow on the manifold is sufficiently slow, it represents an approximate continuous attractor 4 . Furthermore, our theory bounds the error in working memory incurred over time for such approximate continuous attractors. Therefore, the concept of continuous attractors remains a crucial framework for understanding the neural computation underlying analog memory, even if the ideal continuous attractor is never observed in practice.Experimental observations that indicate the slowly changing population representations during the "delay periods" where working memory is presumably required, do not necessarily contradict the continuous attractor hypothesis. Perturbative experiments can further measure the attractive nature of the manifold and their causal role through manipulating the memory content.4 Numerical Experiments on Task-optimized Recurrent NetworksWhile our theory describes the abundance of approximate continuous attractors in the vicinity of a continuous attractor, it does not imply that there are no approximate solutions away from continuous attractors. In this section, we use task-optimized RNNs as a means to search for plausible solutions for analog memory for a circular variable. We train a diverse set of RNNs, and then identify the solution type of trained RNNs to gain insights into its performance, error patterns, generalization capabilities, and, ultimately, proximity to a continuous attractor.Understanding the implemented computation in neural systems in terms of dynamical systems is a well-established practice 111, 116, 131 . Researchers have analyzed task-optimized RNNs through nonlinear dynamical systems analysis 121, 120, 4, 29, 79, 25, 26 and to compare those artificial networks to biological circuits 81, 102, 45 . Previously, systematic analysis of the variability in network dynamics has been surveyed in vanilla RNNs, and variations in dynamical solutions over architecture and nonlinearity have been quantified 121, 81, 140, 79, 29 . Furthermore, working memory mechanisms in RNNs had tendencies to find sequential or persistent representations through training depending on the task specification 89 . We therefore investigated to what extent training RNNs on a task uniquely determines the low-dimensional dynamics, independent of neural architectures. We see that all the solutions have a slow invariant manifold, making all of them an instatiation of approximate continuous attractors. 
model architectures and training procedure: 
 Building upon prior work, which has shown their capabilities on such tasks, we trained RNNs to either (1) estimate head direction through integration of angular velocity 25, 26 or (2) perform a memory-guided saccade task for a ring variable 134, 138 (details in Sec. S7.1 and see Sec. S7.3 for how RNNs relate to Eq. 2.). We numerically minimized the mean squared error loss L MSE between the network output y(t) and the target output ŷ(t): L MSE := y i,t -ŷi,t ) 2 . For each activation function and each network architecture (vanilla RNN with ReLU, tanh, and rectified tanh activation functions, LSTM, and GRU), we trained 10 networks per hidden size: 64, 128, and 256 with state noise.For each trained network, we find the slow manifold by integrating the autonomous dynamics, then selecting the parts of the trajectories that have speed slower than a threshold (Sec. S7.9.1). We identify the points on the invariant manifold from the simulated trajectories that are projected closest to a set of points in the output space relevant to the task after convergence, i.e. on the target ring. We parametrize the one-dimensional invariant manifold by fitting a cubic spline with periodic boundary constraints to these points (black line in Fig. 4A and B ). Normal hyperbolicity is measured by a gap in the timescales of the system (measured from the eigenvalue spectrum of the linearization along points on the invariant manifold, Fig. 4E and F ).We find the fixed points on the invariant ring by identifying regions where the direction of the flow flips (Sec. S7.6.3). Stable fixed points are identified where the flow directions are both pointing towards this flip point, while saddle nodes are identified where they are pointing away (Fig. 4A, B .)To understand what solutions the RNNs found to solve the task, we investigate their memory mechanism. For this, we dissect the dynamics of RNNs by segregating time scales to delineate the rapid flow normal to the slow manifold, and the flow on the manifold (Sec. S7.6.3). All solutions involve a slow manifold with the same topology as the relevant variable in the task. The different solutions are different in their asymptotic dynamics (Fig. 4 ). The most often found solution is of the type fixed point ring manifold (Fig. 4A and B ). These solutions are consistent with observations that persistent activity relies on discrete attractors 13, 60 . Less commonly found topologies includes the slow torus around a repulsive ring invariant manifold (Fig. 4D ). This solution in turn is consistent with both observations of the possibility of using non-constant dynamics for memory storage 56, 92 and neuronal circuits underlying persistent representations despite time-varying activity 30 . All stability structures (fixed points and limit cycles) are mapped close to the target output circle (Figs. S15, S19, S20).We verify that this gap exists in the trained networks shown in Fig. 4B and C . The largest eigenvalue of the Jacobian is non-zero (the invariant manifold is not a continuous attractor), but it is removed from the second largest, see Fig. 4E and F . This implies that normal hyperbolicity for these trained networks holds. The fixed point topologies show a lot of variation across networks (Fig. 4B ,C, Fig. 5 and Fig. S20 ), much like the systems next to continuous attractors (Fig. 1 and Fig. 2 ). Previously, it has been observed that fixed point analysis has a major limitation, namely, that the number of fixed points must be equal across compared networks 79 . Our methodology effectively addresses and overcomes this limitation. The universal structure of continuous attractor approximations as slow invariant manifolds allows us to connect different topologies as approximate continuous attractors (Sec. 3.3). For results on LSTMs and GRUs and a higher dimensional task, see Sec. S7.7 and Sec. S7.9, respectively.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: We report on all the analysis step decisions that might affect the results. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
generalization analysis: 
 In this section, we use task-trained RNNs to study the relationship between dynamics and generalization capabilities. When neuroscientists study neural computations in animals, tasks have finite durations, leaving it unclear whether animals learn the intended computation or a finite-time approximation. The same issue applies to trained neural networks. We will explore whether the networks possess the necessary memory for perfect recall or only perform the task within the timescale of their training.The two possible approximations of a ring attractor discussed in Sec. 2 exhibit markedly distinct generalization characteristics. Approximating the system as a limit cycle results in a memory trace that gradually diminishes over time. Conversely, the alternative approximation's memory states are contingent upon the quantity and positioning of stable fixed points within the system. We describe in detail the generalization properties of the trained networks5 on the angular velocity integration task at two different time scales: asymptotic and finite time. Finite time: Aside from the angular velocity integration component of the task, the trained networks learn to store a memory of an angular variable. We assess the performance of the network to store the memory of the angle over time. The networks typically perform well on the timescale on which they have been trained T 1 = 256 time steps (Fig. 5C ). This loss is, as theoretically predicted (Prop. 2), bounded by the uniform norm of the vector field on the invariant manifold, and therefore by the distance to a continuous attractor (Prop. 1, Fig. 5A , see Sec. S7.6.3 and S6).EQUATIONAsymptotic time: Looking beyond the finite timescale provides valuable insights into the network's ability to store information. For the asymptotic time scale, we capture the asymptotic behavior of the system by identifying to what part of the system evolves to in the limit t → ∞ (see also Sec.S7.6.2). For a one-dimensional system, this will either be fixed points or a limit cycle. For the fixed-point type solution, the maximal error is given by the maximal distance to the next fixed point, while for a limit cycle, this will always be π. We calculate the average fixed point distance by takeing the average of the inter-fixed-point interval for each neighboring pair of fixed points. We identify the location of the fixed points as described in Sec. 4.2.Besides the maximal error that the network will make in the asymptotic limit, we can also characterize how many different angles the network would confuse in the asymptotic limit. We construct a probability distribution of what part of state space we end up in an infinite time through the calculation of the size of the basins of attraction of stable fixed points as a proportion of the ring. Finally, we characterize the memory capacity of the network by calculating the entropy of this probability distribution (see Sec. S7.6.2).Error Accumulation in Neural Networks: The mean accumulated error at the time at which the task was trained has an exponential relationship with the number of fixed points (Fig. 5A ). Furthermore, this error is bounded by the mean distance between stable and unstable fixed points (red dots in Fig. 5D ). This is another indication that the networks rely on a ring invariant manifold to implement the task. Networks with different numbers of fixed points might have the same performance on the finite time scale (bounded by T 1 ∥φ∥ ∞ ) but have vastly different generalization properties because they differ in the number of fixed points (Fig. 5C ).Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: Although we do not provide open access to the data and code, we strive to make it available as soon as possible.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/public/ guides/CodeSubmissionPolicy) for more details. • While we encourage the release of code and data, we understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark). • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/ guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc. • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
approximate slow manifolds are near continuous attractors: 
 In Sec. 2.2, we presented a theory of approximate solutions in the neighborhood of continuous attractors. When are approximate solutions to the analog working memory problem near a continuous attractor? We posit that there are four conditions (see for more detail Sec. S5.3): (C1) sufficiently smooth approximate bijection between neural activity and memory content, (C2) the speed of drift of memory content is bounded, (C3) robustness against state (S-type) noise, and (C4) robustness against dynamical (D-type) noise. The correspondence implied by (C1) translates to the existence of a manifold in the neural activity space with the same topology as the memory content. 6 Persistence (C2) requires that the flow on the manifold is slow and bounded. S-type robustness (C3) implies nonexpansive flow, i.e., non-positive Lyapunov exponents. Along with D-type robustness (C4), it implies the manifold is "attractive", and normally hyperbolic (see also Sec. S5.3.1).If these four conditions hold, for example for task-trained RNNs, there exists a smooth function with a uniform norm matching the slowness on the manifold such that when added, the slow manifold becomes a continuous attractor (Prop. 1 and Theorem 7, see also Sec. S5.4). For the RNN experiments, we added state-noise while training using stochastic gradient descent, satisfying (C3) and (C4). We have also verified that (C2) holds (Fig. 5A ). Although the stochastic optimization cannot lead to the continuous attractor solution, it gets to the neighborhood where all approximate solutions share the same main feature: having a subsystem that has a slow flow.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: These are reported in detail in the Supplemental.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
discussion: 
 Continuous attractors are highly prone to bifurcation under arbitrary perturbations unless they exist in special parametric forms. This sensitivity to perturbations has traditionally made them seem unsuitable for modeling neural computation in noisy biological systems, according to conventional views on robustness. Nevertheless, we demonstrate that continuous attractors can exhibit functional robustness, making them a crucial concept in explaining the neural computation underlying analog memory. We show that approximations of analog memory (i.e., theoretical models that satisfy conditions (C1)-(C4)) must possess slow manifold dynamics, placing them near continuous attractors within the space of dynamical systems. This implies that both biological systems and artificial neural networks only need to be near a continuous attractor to effectively solve problems in a manner similar to the ideal theoretical model, on behaviorally relevant timescales.Limitations Although, we only explicitly describe the topology and dimensionality of the identified invariant manifolds for a representative set, the results indicate that most solutions have a ring invariant manifold with a slow flow on it. Our numerical analysis relies on identifying a time scale separation from simulated trajectories. If the separation of time scales is too small, it may inadvertently identify parts of the state space that are only forward invariant (i.e., transient). However, this did not pose a problem in our analysis of the trained RNNs, which is unsurprising, as the separation is guaranteed by state noise robustness (due to injected state noise during training).To identify solutions with a fast-slow decomposition only rely on the generalization property of the network (in terms of the normalized mean square error for ten times longer trials). The possible solutions that the networks can find are restricted by having a linear output mapping. For a nonlinear output mapping, a possible solution for analog memory is the quasi-periodic toroidal attractor, but this is not possible for a linear output mapping. Our analysis methods can identify these limit sets, but we do not have a simple way to parametrize the two dimensional torus invariant manifold.In order to demonstrate the implications of the theory of the persistence of continuous attractors, we rigorously test the predictions of the theory on the stability of the Bounded Line Attractor (BLA). Our objective is to assess the practical implications of the theoretical findings of bounded continuous attractors in a small and tractable system, and second, to contribute empirical evidence that can help refine and extend existing theoretical frameworks.The BLA has a parameter that determines step size along line attractor α. Analogously as for UBLA, these parameters determine the capacity of the network. The inputs push the input along the line attractor in two opposite directions, see below. The BLA needs to be initialized at β(1, 1) and β 2 (1, 1), respectively, for correct decoding, i.e., output projection.EQUATIONParameter that determines step size along line attractor α. The size determines the maximum number of clicks as the difference between the two channels. This pushes the input along the line "attractor" in two opposite directions, see below.The results from such low-dimensional system can be extended to higher-dimensional systems through reduction methods from center manifold theory. On the center manifold the singular perturbation problem (as is the case for continuous attractors) restricts to a regular perturbation problem 39 . Furthermore, relying on the Reduction Principle 69 , one can always reduce all systems (independent of dimension) to the same canonical form, given that they have the same continuous attractor.First of all, we will show how to transform the BLA network to the slow-fast form in Eq. 3-4 to explicitly demonstrate that the theory applies to it. To achieve this, we transform the state space so that the line attractor aligns with the y-axis. So, we apply the affine transformation R θ (x -1 2 ) with the rotation matrixR θ = cos θ -sin θ sin θ cos θ = 1 √ 2 1 1 -1 1where we have set θ = -π 4 . So we perform the transformationx → x ′ = R θ (x -12 ) and so we haveEQUATION)For a perturbed connection matrixEQUATIONWe will now identify all possible bifurcations from the BLA, to show that indeed all perturbations preserve the continuous attractor as an invariant manifold.We consider all parametrized perturbations of the form W ← W + V for a random matrix V ∈ R 2×2 to the BLA. The BLA can bifurcate in the following systems, characterized by their invariant sets: a system with single stable fixed point, a system with three fixed points (one unstable and two stable) and a system with two fixed points (one stable and the other a half-stable node) and a system with a (rotated) line attractor. Only the first two bifurcations (Fig. 1A ) can happen with nonzero chance for the type of random perturbations we consider. The perturbations that leave the line attractor intact or to lead to a system with two fixed points have measure zero in the parameter space. The perturbation that results in one fixed point happen with probability 3 4 , while perturbations lead to a system with three fixed points with probability 1 4 , see Sec. S2.2.2. The (local) invariant manifold manifold is indeed persistent for the BLA and homeomorphic to the original (the bounded line).We investigate how perturbations to the bounded line affect the Lyapunov spectrum. We calculate the eigenspectrum of the Jacobian:det[W ′ -(1 + λ)I] = (ϵ11 -1 -λ)(ϵ22 -1 -λ) -(ϵ12 + 1)(ϵ21 + 1) = λ 2 -(2 + ϵ11 + ϵ22)λ -ϵ11 -ϵ22 + ϵ11ϵ22 -ϵ12 -ϵ21 -ϵ12ϵ21 Let u = -(2 + ϵ11 + ϵ22) and v = -ϵ11 -ϵ22 + ϵ11ϵ22 -ϵ12 -ϵ21 -ϵ12ϵ21There are only two types of invariant set for the perturbations of the line attractor. Both have as invariant set a fixed point at the origin. What distinguishes them is that one type of perturbations leads to this fixed point being stable while the other one makes it unstable.Stability of the fixed points on the axes We perform the stability analysis for the part of the state space where W x > 0. There, the Jacobian isEQUATIONWe apply the perturbationEQUATIONwithEQUATIONThe eigenvalues are computed asEQUATIONCase 1:EQUATIONWe give some examples of the different types of perturbations to the bounded line attractor. The first type is when the invariant set is composed of a single fixed point, for example for the perturbation:EQUATIONThe second type is when the invariant set is composed of three fixed points:EQUATION)The third type is when the invariant set is composed of two fixed points, both with partial support.EQUATIONThe fourth and final type is when the line attractor is maintained but rotated:EQUATIONEQUATIONWe will now state our previous observations as a Theorem that characterizes the possible bifurcations of the BLA. Theorem 2. All perturbations of the bounded line attractor are of the types as listed above.Proof. We enumerate all possibilities for the dynamics of a ReLU activation network with two units. First of all, note that there can be no limit cycle or chaotic orbits. Now, we look at the different possible systems with fixed points. There can be at most three fixed points 85 Corollary 5.3. There has to be at least one fixed point, because the bias is non-zero.General form (example):EQUATIONOne fixed point with full support:In this case we can assume W to be full rank.ẋ = ReLU ϵ11 ϵ12 ϵ21 ϵ22 x1 x2 + 1 1 - x1 x2 = 0 Note that x > 0 iff z1 := ϵ11x1 + (ϵ12 -1)x2 -1 > 0. Similarly for x2 > 0.So for a fixed point with full support, we haveEQUATIONwithA := ϵ11 -1 ϵ12 -1 ϵ21 -1 ϵ22 -1 .Note that it is not possible that x1 = 0 = x2.B := A -1 = 1 det A ϵ22 -1 1 -ϵ12 1 -ϵ21 ϵ11 -1 with det A = ϵ11ϵ22 -ϵ11 -ϵ22 -ϵ12ϵ21 + ϵ12 + ϵ21.Hence, we have that x1, x2 > 0 if B11 + B12 > 0, B21 + B22 > 0 and det A > 0 or if B11 + B12 < 0, B21 + B22 < 0 and det A < 0.This can be satisfied in two ways, If det A > 0, this is satisfied if ϵ22 > ϵ12 and ϵ11 > ϵ21, while if det A < 0, this is satisfied if ϵ22 < ϵ12 and ϵ11 < ϵ21. This gives condition 1.Finally, we investigate the condition that specify that there are fixed points with partial support. If x1 = 0 then (ϵ22 -1)x2 + 1 = 0 and z1 < 0. From the equality, we get that x2 = 1 1-ϵ 22 . From the inequality, we get (ϵ12 -1)x2 + 1 ≥ 0, i.e. Then, we get the following conditions for the different types of bifurcations:1. If condition 1 is violated, but condition 2 is satisfied with exactly one strict inequality, there are two fixed points on the boundary of the admissible quadrant.2. If condition 1 is violated, and only one of the subconditions of condition 2 is satisfied, there is a single fixed point on one of the axes.3. If condition 2 is violated, there is a single fixed point with full support.4. If both conditions are satisfied, there are three fixed points.We now look at the possibility of the line attractor being preserved. This is the case if v = 0. It is not possible to have a line attractor with a fixed point off of it for as there cannot be disjoint fixed points that are linearly dependent 85 Lemma 5.2We will now calculate which proportion proportion of the bifurcation parameter space is results in the different bifurcation types. The conditions that result in three fixed points are Therefore, because ϵ22 ≤ ϵ12, ϵ11 ≤ ϵ21.we always have that0 < ϵ11ϵ22 -ϵ11 -ϵ22 -ϵ12ϵ21 -ϵ12 -ϵ21.This implies that this bifurcation happens with probability 1 4 in a ϵ-ball around the BLA neural integrator with ϵ < 1. We conclude that the single stable fixed point type perturbation happens with probability 3 4 .We will present the structure of the bifurcation space through a slice in which we fix ϵ11 and ϵ12. First, we summarize which conditions result in which bifurcation in Table 1 . We derive that the local bifurcation in this slice has the structure as shown in Fig. S6 . ε 21 det A > 0 det A < 0 3FPs ε 11 -ε 12 ε 11 -1 b = ε 12 -1 ε 11 -1 a = ε 2 2 = a ε 2It is well-known that activation functions (σ in Eqs. 61 and 64), which can take many forms, play a critical role in propagating gradients effectively through the network and backwards in time 62, 100, 54 . Activation functions that are C r for r ≥ 1 are the ones to which the Persistence Theorem applies. The Persistence Theorem further specifies how the smoothness of the activation can have implications on the smoothness of the persistent invariant manifold. For situations where smoothness of the persistent invariant manifold is of importance, smoother activation functions might be preferable, such as the Exponential Linear Unit (ELU) 21 or the Continuously Differentiable Exponential Linear Units (CELU) 5 .To computationally investigate the neighborhood of recurrent dynamical systems that implement continuous attractors, we investigate 5 RNNs that are known a priori to form 1 or 2 dimensional continuous attractors.We define a local perturbation (i.e., a change to the ODE with compact support) through the bump function Ψ(x) = exp 1 ∥x∥ 2 -1 for ∥x∥ < 1 and zero outside, by multiplying it with a uniform, unidirectional vector field. All such perturbations leave at least a part of the continuous attractor intact and preserve the invariant manifold, i.e. the parts where the fixed points disappear a slow flow appears.The parametrized perturbations are characterized as the addition of a random matrix to the ODE.We further analyzed a simple (non-biological) ring attractor, defined by the following ODE: ṙ = r(1r), θ = 0. This system has as fixed points the origin and the ring with radius one centered around zero, i.e., (0, 0) ∪ {(1, θ) | θ ∈ [0, 2π)}. We investigate bifurcations caused by parametric and bump perturbations of the ring invariant manifold (see Sec. S3), which is bounded and boundaryless. All perturbations maintain the topological structure of the invariant manifold.The networks proposed in 88 are composed of N heading-tuned neurons whose preferred headings θj uniformly tile heading space, with an angular separation of ∆θ = 2π /N radians. These neurons can be arranged topologically in a ring according to their preferred headings, with neurons locally exciting and broadly inhibiting their neighbors. The total input activity hj of each neuron is governed by:EQUATIONwithEQUATIONwhere JE and JI respectively control the strength of the tuned and untuned components of recurrent connectivity between neurons with preferred headings θj and θj and vin is an angular velocity input which the network receives through asymmetric, velocity-modulated weightsEQUATION)Fixed points In the absence of an input (vin = 0) fixed points of the system can be found analytically by considering all submatrices W sym σ for all subsets {σ ⊂ [n]} with[n] = {1, . . . , N }. A fixed point x * needs to satisfyEQUATIONWe bruteforce check all possible supports to find all fixed points. We use the eigenvalues of the Jacobian to identify the stability of the found fixed points. We evaluate the effect of parametric perturbations of a network of size N = 6 with JE = 4 and JI = -2.4 by identifying all bifurcations (Fig. 2A ).We approximate connecting orbits through numerical integration of the ODE intialized in close to the identified saddle points along the unstable manifold.Measure zero co-dimension 1 bifurcations of the ring attractor network fall into two types, see Fig. S7 .Figure S7 : Measure zero co-dimension 1 bifurcations of the ring attractor network 88 .Measure zero co-dimension N bifurcation The limit cycle is the only bifurcation that we found that can be achieved on only a measure zero set of parameter values around the parameter for the continuous attractor.As we can see in Fig. S8 , the topology of the system is maintained through a range of bifurcation sizes when the bifurcation direction is fixed. We investigated the bifurcations around the approximate ring attractor constructed with a symmetric weight matrix for a tanh network 22, 110 . The functional form of W is the sum of a constant term plus a Gaussian centered at θi -θj = 0:EQUATIONwith the dimensionless parameter J -representing the strength of the weak crossdirectional connections, J + the strength of the stronger isodirectional connections, and σ the width of the connectivity footprint.Such ring attractor approximations are similar to the ones in 50, 108, 101, 124 . However, some have other nonlinearities, e.g., the sigmoid is used in 50 . Another line of related models can be found in 14 , 24 and 118 .Loss of function: Sensitivity of continuous attractors to perturbations We will show that there are differences at how well approximations perform at different timescales. We measure how performance of different models for the representation of an angular variable drop as a function of perturbation size Fig. S9 through the memory capacity metric (Sec.S7.6.2). For each perturbation size, we sample a low rank (rank 1,2 or 3) random matrix with norm equal to that perturbation size. We determine the location of the fixed points through the local flow direction criterion as described in Sec. 4.2 This invariant manifold was found to be consistently close the the original invariant ring attractor. The initial ring had 2N fixed points (N stable, N saddle) on this invariant ring manifold. The memory capacity of this initial configuration is N log(N ) for the 2N uniformly spaced fixed points.Figure S9 : Degradation of performance across perturbation sizes. System behavior at the asymptotic time scales measured through memory capacity.Another line of models 83, 136, 43 also relies on connection weights with translational invarianceEQUATIONwhere J(x, x ′ ) is the neural interaction from x ′ to x and the ensemble of infinite neurons are lined up so that x ∈ (-∞, ∞).Weak random spatial fluctuations in the connection strength are to be expected when learning the coupling function with Hebbian plasticity. In the 1D case, it is well known that the presence of such synaptic heterogeneity causes a drift of an input-induced activity pattern to one of a finite number of attractor positions which are randomly spread over representational space 103, 61 . Similarly, in 2D, spatial fluctuations in the connection strengths cause a slight perturbation of the bump shape 135 . Frozen stabilisation has been proposed as alternative method to construct a neural networks to self-organise to a state exhibiting (high-dimensional) memory manifolds with arbitrarily large time constants 15 . 73 analyzes an Ising network perturbed with a specially structured noise at the thermodynamic limit. Although their analysis elegantly shows that the population activity of the perturbed system does not destroy the Fisher information about the input, they do not consider a scenario where the ring attractor is used as a working memory mechanism, it is rather used to encode instantaneous representation. In contrast, our analysis involves understanding how the working memory content degrades over time due to the dynamics. We are not aware of any mean field analysis that covers this aspect.The networks consisted of N firing rate units with a sigmoid input-output transfer function 82 :EQUATIONwhere xi(t) is the total input current to unit i, Jij = gχij + Pij is the connectivity matrix, ϕ(x) = tanh(x) is the current-to-rate transfer function, and Ii is the external, feedforward input to unit i. The random component gχ is considered unknown except for its statistics (mean 0, variance g 2 /N ). A general structured component of rank r ≪ N can be written as a superposition of r independent unit-rank termsEQUATIONi n(1) jEQUATIONand is in principle characterized by 2r vectors m (k) and n (k) .To approximate a ring attractor we can consider structured matrices where the two connectivity pairs m (1) and n (1) , m (2) and n (2) share two different overlap directions, defined by vectors y1 and y2. We set:EQUATIONEQUATIONEQUATIONEQUATIONwhere Σ 2 is the variance of the connectivity vectors and r 2 1 and r 2 2 quantify the overlaps along the directions y1 and y2.We keep the following parameters for the analysis: Σ = 2, ρ = 1.9 and g = 0.1. Our theory explains the phenomenon of the existence of a ring invariant manifold in these low-rank networks as follows. We can think of the finite size realization as a small perturbation to the infinite size network on the reduced dynamics in the m 1 , m 2 plane (independent of the parameter g for the random part of the matrix) (Fig. 2B ). For very small networks the ring structure is destroyed and only the plane persists as a slow manifold.We fit three networks with the Embedding Manifolds with Population-level Jacobians (EMPJ) method 96 .The networks are RNNs of N = 10 neurons with a tanh activation function and τ = 0.05 time constant. To use EMPJ we need to specify our desired k fixed points and m eigenvectors and eigenvalues per fixed point.To do this we choose k = 3 points {(xi, yi)} N i=1 in the 2D ring with radius r = 1 that are equally spaced from 0.1 to 2π radians. Then we specify the vectors orthogonal to the ring in those points (Vo = {(xi, yi)} N i=1 , same as the points) and tangent (Vt = {(-yi, xi)} N i=1 ). Finally we associate the negative eigenvalue -1/τ to the orthogonal eigenvectors and different eigenvalues to the tangent eigenvectors. We use -1, 0 or 1 depending on whether we want the points to be stable, center or unstable in the direction of the slow ring manifold.Since these determined dynamics are only in a 2D plane, we use a random linear mapping D : R 2 → R N to map the fixed points and the eigenvectors to a 10-dimensional space. First, we sample a random from orthogonal matrix A from the O(N ) Haar distribution and then we take the first two columns of this matrix to be D = [A]12.EMPJ then returns a network parameters that satisfy these constraints. Furthermore, due to the particular solver used, that regularizes the magnitude of the parameters, we get that all the other eigenvalues not specified are also set to -1 /τ (for details see 96 ).Finding fixed points As we remark in Sec. S3.5.1, EMPJ networks are not robust to S-type noise, therefore we cannot apply our analysis of identifying the invariant set through the convergence criterion of numerically integrated trajectories. We therefore find fixed points through the Newton-Raphson method. We iteratively solveEQUATIONwhere J(xi) is the Jacobian of the system at xi and xi+1 = xidxi. The iteration stops when |dx| < δ for a tolerance threshold δ. We initialize x0 on the invariant ring uniformly. The maximum number of iterations was set to 10.000 and the tolerance level to δ = 10 -8 .We remark that the resulting invariant manifold is not robust to S-type perturbations. Although the fixed points that are constrained in the fitting procedure are attractive in all directions, some of the points along the ring might not be. For on-manifold perturbations (in the plane in which the ring is embedded), S-type perturbations do lead to flow towards the invariant ring (Fig. S11A ). However, for (small) off-manifold perturbations, the trajectories typically diverge away from the invariant ring (Fig. S11B ). This indicates that the basin of attration is very small and hence this approximation is not robust to S-type noise.All of the perturbations were sampled as x(0) = x(0) + η with x(0) ∈ span(D). For the on-manifold perturbations η ∈ span(D) and ∥η∥2 = 10 -2 . For the off-manifold perturbations η ∈ R 10 and ∥η∥2 = 10 -5 . We furthermore fit a torus and a sphere continuous attractor with the EMPJ. The networks we used have N = 100 neurons. For finding fixed points with the Newton-Raphson method, we used 1.000 as the maximum number of iterations and a tolerance level of δ = 10 -5 .Torus Figure S12 illustrates the stability structures of the approximate torus attractor fitted with EMPJ. The ratio of the radii of the two rings is adjusted for visualization purposes. Sphere Fig. S13 illustrates the stability structures of the approximate sphere attractor fitted with EMPJ.Similar to the torus, points initialized off the sphere converge onto the sphere attractor (Fig. S13 ).X Y 1 -1 Z 1 -1 X 1 -1 Y 1 -1 Z 1 -1 Y Z X Figure S13:The approximate sphere attractor. (A) Points initialized on a grid off of the sphere (blue) converge onto the torus attractor (red). (B) The found fixed points on the approximate sphere (green: stable, red: saddle). The two subfigures show rotated versions of the location of the fixed points on the sphere.Understanding the long-term behavior of dynamical systems is a fundamental question in various fields, including mathematical biology, ecological modeling, and neuroscience. Fenichel's Persistence Theorem provides critical insights into the behavior of such systems, particularly in relation to the stability and persistence of invariant manifolds under perturbations.Fenichel's Persistence Theorem extends the classical theory of invariant manifolds, offering conditions under which normally hyperbolic invariant manifolds persist despite small perturbations. This theorem is particularly powerful in analyzing systems where perturbations are inevitable, providing a framework for understanding how qualitative features of the system's dynamics are maintained. In this section, we delve into the specifics of Fenichel's Persistence Theorem, outlining its key components, assumptions, and implications.Invariance One of the main concepts in the Persistence Theorem is the notion of an invariant manifold.Intuitively, this just means that trajectories stay inside the manifold for all time. Local invariance is a bit more involved and allows for the possibility of leaving the manifold, but only through it's boundary.A set M is locally invariant under the flow from Eq. 3-4 if it has neighborhood V so that no trajectory can leave M without also leaving V . In other words, it is locally invariant if for all x ∈ M, φ(x, [0, t] ⊂ V implies that φ(x, [0, t]) ⊂ M , similarly with [0, t] replaced by [t, 0] when t < 0. Definition 2. A set S is said to be forward invariant under a flow φt if for every point x in S and for all t ≥ 0, the image of x under the flow at time t, denoted φt(x), remains in S. This can be written as: φt(x) ∈ S for all x ∈ S and t ≥ 0.Small perturbation In the context of Fenichel's Persistence Theorem, a "sufficiently small C r perturbation of the vector field f " refers to perturbations that are small in the C r norm. The C r norm measures the size of a function and its derivatives up to order r.Formally, let f : R n → R n be the original smooth vector field and let f be a perturbed vector field. The perturbation f is a sufficiently small C r perturbation if the difference f -f has a small C r norm. Mathematically, this can be expressed as:f -f C r < ϵ,where ϵ is a small positive number, and the C r norm is defined as:f -f C r = max 0≤k≤r sup x∈R n D k ( f (x) -f (x)) .Which gives a constraint on how much each of the k-th derivatives D k of the perturbed vector field can differ from the original.In summary, a perturbation f is considered sufficiently small in the C r sense if the difference between f and f , along with their derivatives up to order r, is uniformly small across the domain. For C 1 perturbations, this defines a C 1 neighborhood of functions, within which the persistence of the manifold is ensured by Fenichel's theorem. For r = 0, we get the uniform norm (defined as ∥f ∥)∞ := sup(|f |), see also Sec.S6).Closeness Finally, to formalize what is meant by that the persistent invariant manifold is very close to the original continuous attractor, we need a notion of distance between the manifolds. quantifies the distance from a point a ∈ X to the subset B ⊆ X.In conclusion, the Hausdorff distance provides a rigorous mathematical framework to quantify the "closeness" or "similarity" between two sets.This section will introduce the original Fenichel's Persistent Manifold Theorem, laying the groundwork for understanding how normally hyperbolic invariant manifolds persist under perturbations in systems with distinct time scales. By examining this foundational theorem, we can build a deeper understanding of the stability and behavior of complex dynamical systems.In the study of dynamical systems, particularly those involving multiple time scales, understanding the behavior of solutions near invariant manifolds is crucial. These manifolds often determine the long-term dynamics of the system and their stability properties. The Fenichel's Persistent Manifold Theorem provides a powerful framework for analyzing such systems by demonstrating the persistence of normally hyperbolic invariant manifolds under small perturbations. This theorem is particularly relevant in systems where variables evolve on different time scales-typically referred to as "slow" and "fast" dynamics. By reformulating the system with a change of time-scale, we can explore how the dynamics on these manifolds behave, especially when perturbed. Consider the system given by Equations 3-4, which can be rewritten using a rescaled time variable, τ , to distinguish between the fast and slow dynamics as follows:EQUATIONwhere ′ = d dτ and τ = t /ϵ. The time scale given by τ is said to be fast whereas that for t is slow, as long as ϵ ̸ = 0 the two systems are equivalent.The functions g and hin Eq. 3-4 are both assumed to be C r (for r > 0 on a set U × I where U ⊂ R d is open, and I is an open interval, containing 0.Suppose that the set M0 is a subset of the set {h(x, y, 0) = 0} and is a compact manifold, possibly with boundary, and is normally hyperbolic relative to 38.Theorem 3 (Theorem 1 in 63 ). If ϵ > 0, but sufficiently small, there exists a manifold Mϵ that lies within O(ϵ) of M0 and is diffeomorphic to M0. Moreover it is locally invariant under the flow of Eq. 3-4, and C r , including in ϵ, for any 0 < r < ∞. Finally, Mϵ has O(ϵ) Hausdroff distance to M0 and has the same smoothness as g and h.If the invariant manifold M0 is attractive, then the only way trajectories can escape the invariant set Mϵ after perturbation, is in negative time through the boundaries. This guarantees that the persistent manifold Mϵ is still attractive.Theorem 4 (One Dimensional Equivalence). Two flows and in are topologically equivalent iff their equilibria, ordered on the line, can be put into one-to-one correspondence and have the same topological type (sink, source or semistable).The flow on a ring can be described by the differential equation 57 :EQUATIONwith θ ∈ [0, 2π].The simplest type of flow on a ring is the uniform circular flow, where each point moves with a constant angular velocity f (θ) = ω. If |f (θ)| > 0 for all θ, there is a circular flow with a variable speed.There is a fixed point for each unique θ for which f (θ) = 0. The nature of the flow around these fixed points can be classified into:• Stable fixed points: Points where the flow tends to as time progresses (f (θ) > 0 for all θ ∈ [0, θ] ∩ V and f (θ) < 0 for all θ ∈ [θ, 2π] ∩ V for some open ball V around θ).• Unstable fixed points: Points from which the flow diverges (f (θ) < 0 for all θ ∈ [0, θ] ∩ V and f (θ) > 0 for all θ ∈ [θ, 2π] ∩ V for some open ball V around θ).For a detailed discussion of how bifurcations of a continuous attractors depend on the symmetry of the continuous attractor, see for example The Equivariant Branching Lemma (Lemma 1.31 in 49 ). If the solutions are symmetric under rotations (a circular symmetry such as for a ring attractor). As you change the bifurcation parameter, you find that new solutions appear that also respect this rotational symmetry. The lemma tells you that these new solutions will align with specific symmetries (like different rotation angles), which are described by the irreducible representations of the symmetry group.Fenichel's Persistence Theorem has several significant implications for modelling and system identification in dynamical systems. Because the theorem provides a guarantee that small perturbations in the system do not lead to significant changes in the qualitative behavior of the system, we can be (slightly) wrong for example about the exact nonlinearity of a neuron's transform function. For example, if neurons are only approximately ReLU the theory developed in 10 still holds (at the behaviorally relevant timescales). More generally, when reconstructing computational system dynamics to understand how cognitive functions are implemented in the brain, our theory shows that small deviations in the identified system can still lead to behaviorally equivalent models for neural computation 32 .We will give some clarifications and proofs of the claims on systems near perfect analog memory systems.We provide a proof of Prop. 1 which is dependent on the ϵ-Neighborhood Theorem7 , which we state here.Theorem 5 (ϵ-Neighborhood Theorem 52 ). Let Y ⊆ R n be a smooth, compact manifold. Then for a small enough ϵ > 0, each w ∈ Yϵ := {w ∈ R n | ∃y ∈ Y : ∥y -w∥ < ϵ} has a unique closest point in Y , denoted π(w). Moreover, π : Yϵ → Y is a submersion with π|Y = id.We now prove Prop. 1.Proof. It is sufficient to show that there is a perturbation p that has zero flow off of Mϵ but for which f + p = 0 on Mϵ for the full system f as defined in Eq. 2. Definep(x) = Mϵ -δ(x -y)f (y)dy,with the Dirac delta function δ(x) = δ(x1)δ(x2) . . . δ(x d ). It is then easy to check that f (x) + p(x) = 0 for all x ∈ Mϵ and f (x) + p(x) = f (x) for all x ̸ ∈ Mϵ. Hence, we have a continuous attractor at Mϵ. If smoothness is important, we can construct the following perturbation. From the ϵ-Neighborhood Theorem 42 , we get that there exists a smooth positive function δ : Mϵ → R + , such that if we let N δ be the δ-neighborhood of Mϵ, Mϵ := {y ∈ R n : |y -x| < δ(x) for some x ∈ Mϵ}, then each y ∈ N δ possesses a unique closest point π δ (y) in Mϵ with the map π δ : M δ → Mϵ being a submersion.We can then define a bump functionEQUATIONThen the perturbation p(y) = -ψ(y)f (π δ (y))is smooth and creates a continuous attractor at Mϵ.This paper focuses on a linear output mapping for simplicity. Errors can be minimized with a nonlinear mapping, if there is an output that is mapped off of the output manifold, we can always adjust the output mapping to correct for this, if the space of output mappings is general enough. However, this can only be applied for errors off of the output manifold. If there is memory degradation along the output manifold, it is not possible to choose another mapping that corrects for this error. Therefore, we choose a linear output mapping for our analysis. In some cases, linear output mappings are found to support neural computation, for example for motion direction-discrimination 141 .We will now discuss the conditions for when approximate solutions to an analog working memory problem are near a continuous attractor. We consider approximate solutions to an analog working memory problem to be systems of the form 61 or 64 (in both cases following a linear decoder), which have a small memory error over time in output space.Noise, practically defined as unpredictable components of the system's behavior, comes from many sources. The concept of S-and D-type noise is based on 92 .S-type robustness S-type noise encapsulates reversible changes in the neural state such that the deterministic part of the dynamics itself remains unchanged. Neural dynamics must be robust to perturbations and stimuli that push the neuronal activity away from the continuous attractor 31 .D-type robustness D-type noise, in the space of recurrent network dynamics parameterized by the synaptic weights. This corresponds to slight changes to the ODE, i.e. perturbations. Previously, it robustness to D-type noise was considered to correspond to structurally stability 92 .Here we shortly discuss what we consider to be a necessary and sufficient condition for D-type robustness. We can use the concept of Lipschitz persistence to define D-type robustness. Mañe showed that if an invariant manifold is Lipschitz persistent then it must be normally hyperbolic 77 . To understand the concept of Lipschitz persistence, we need to define the Lipschitz section and Lipschitz constant.Definition 4. Let M be a C ∞ boundaryless manifold and V ⊂ M a C 1 compact boundaryless submanifold. Assume that M is a submanifold of R n . Let N V be a C 1 subbundle of T M |V satisfying T V ⊕ N V = T M |V . If η is a section of N V , define the Lipschitz constant of η by Lip(η) = sup ∥η(x) -η(y)∥ ∥x -y∥ | x, y ∈ V, x ̸ = y .We say that η is a Lipschitz section if Lip(η) < +∞. Let ΓL(N V ) be the space of Lipschitz sections of N V endowed with the norm ∥η∥L = sup {∥η(x)∥ | x ∈ V } + Lip(η). Let Diff 1 (M ) be the space of C 1 diffeomorphisms with the topology of the C 1 convergence on compact subsets. Definition 5. Let f ∈ Diff 1 (M ). We say that V is a Lipschitz persistent invariant manifold of f if there exists a neighborhood U of V such that for all δ > 0 there exists a neighborhood U δ of f such that if g ∈ U δ there exists η ∈ ΓL(N V ) with ∥η∥L < δ satisfying Vg = graph(η), where graph(η) = {exp x (η(x)) | x ∈ V }, Vg = g(U ).Observe that this definition implies V f = V , hence f (V ) = V . Moreover, the Lipschitz persistence is independent of the bundle N V .For a flow φt (coming from the solutions of an ODE) we can fix t = τ ∈ R>0 so that we get a homeomorphism φτ . This allows us to apply this result to apply to our case.In the case where V is a point, then it is a hyperbolic fixed point and the persistence follows trivially from the implicit function theorem. Remark 1. It is sufficient to take a persistent manifold that is uniformly locally maximal. If V is persistent and uniformly locally maximal, then V has to be normally hyperbolic 77 .Definition 6 (Uniformly locally maximal). There exist neighborhoods U of V in M and U of f in the space Diff 1 (M ) of C 1 -diffeomorphisms of M , such that for any g ∈ U , Ng = k∈Z g k (U ) is a C 1 -submanifold close to V , with N f = N . The latter property implies the uniqueness of the invariant submanifold.We now prove a more general statement about the kind of systems that are close to perfect analog memory systems. The theory guarantees that a system that satisfies conditions (C1)-(C4) will have a continuous attractor in the following sense. For such a system there exists a decomposition such that the system can be effectively decomposed into a continuous attractor (attractive invariant manifold with zero flow) and a component on which a (possibly "fast" flow) can exist buy which get quenched by the out put projection. These additional dynamics orthogonal to decoding have been observed for motor movement preparation 65, 20 .For this part of the theory, we need to consider the output manifold Moutput, the manifold on which we determine the error over time as in Sec. S6. For this section, we will consider the output mapping to be a smooth (possibly nonlinear) mapping g : X → Y between the neural state space X = R d X and the output space Y = R d Y . For a circular variable this will be the ring S 1 .The construction of a perturbation relies on finding the necessary minimal structure in the invariant manifold for which we can guarantee closeness to a continuous attractor. Therefore, first of all, we need this closeness in terms of the geometry of the manifold, which we guarantee through the notion of a fibration of the output mapping. Second, we need to guarantee that the flow is bounded in a sense so that our perturbation is also bounded by this amount. We will characterize this by the vector field normal to the fibers of the fiber bundle. A fiber bundle is a mathematical structure that allows us to study spaces that are locally like a product space but globally may have a different structure. So we first state the definition of a fiber bundle and a trivial fibration. Definition 7. A fiber bundle is a structure (E, B, π, F ) where:• E is the total space,• B is the base space,• π : E → B is a continuous surjection called the projection map, and• F is a topological space called the fiber.This structure must satisfy the local triviality condition: for each b ∈ B, there exists an open neighborhood U of b such that there is a homeomorphismφ : π -1 (U b ) → U b × F bthat commutes with the projection onto U , meaning that the following diagram commutes:π -1 (U b ) U b × F b U b U b φ π pr 1 idwhere pr 1 : U b × F b → U b is the projection onto the first factor.So, around every point in the base space, you can "zoom in" and see that the bundle looks like a straightforward product of the base space and the fiber and φ is providing this trivialization.Definition 8. We say that a projection map π : E → B is locally trivial if each point b ∈ B is contained in an open set U having the property that EU := π -1 (U ) is trivial over U .We will rely on the concept of a submersion to characterize how the output space needs to relate to our invariant manifold.Definition 9 (Submersion). Let M and N be differentiable manifolds and f : M → N be a differentiable map between them. The map f is a submersion at a point p ∈ M if its differential Dfp : TpM → T f (p) N is a surjective linear map.This allows us to characterize what kind of structure the invariant manifold needs to have, namely it can be see as the direct product of the output manifold and some other manifold which described over what part of the invariant manifold the output mapping is invariant.Theorem 6 (Ehresmann's lemma 33 ). If a smooth mapping f : M → N , where M and N are smooth manifolds, is 1. a surjective submersion, and 2. a proper map then it is a locally trivial fibration.Remark 2. If a manifold M is compact, then the above smooth map is a proper map. If we do not have a submersion or that the output mapping is transversal to the output manifold, we have a situation in which the flow on the invariant manifold can be arbitrarily fast (even though memory is degrading slowly). This happens when the flow is in a singularity of the output mapping.For our statement we want that the invariant manifold can be decomposed into a space that is diffeomorphic to the output manifold and another compact manifold: Mϵ = Mslow × Mnull with M slow ≃ MY . We can relax this to allow for the possibility of some torsion along the invariant manifold. For this to hold, it is sufficient that the output mapping must be a submersion because this makes it a locally trivial fibration. So we get that g : X → Y defines a locally trivial fiber mapping.The second assumption we need is to have a bound for the speed of trajectories along the fibers, which will correspond to a speed along the output manifold, resulting in memory degradation. We need to assume that there is a slow flow (in the direction of Mslow). We characterize the relevant maximal size of the vector field that needs to be perturbed as the supremum over the uniform norms of the vector field normal to the fibers of the fiber bundle.Theorem 7. Let Mϵ be a connected, compact, normally hyperbolic slow manifold (as parametrized by Eq. (3)-( 4)) with the real part of the eigenvalues of ∇zh all negative. Further, assume that this manifold can be decomposed Mϵ = Mslow × Mnull with M slow ≃ MY and that the uniform norm of the flow tangent to Mϵ restricted to Mslow be ∥ (y) slow ∥∞ = η. Then, there exists a perturbation with uniform norm at most η that induces a bifurcation to a system that is decomposable into a continuous attractor and a system with a non-zero flow.In other words, after applying this perturbation, the slow component of the perturbed system satisfies ẋ′ |slow = 0 for the system ẋ′ = f (x) + p(x). Furthermore, the trajectories of the resulting system form a fiber bundle where the output projection serves as the bundle projection. Each fiber consists of trajectories that are mapped to the same value in Moutput, meaning that the fibers describe an invariance under the output projection.Proof. Assume that the output mapping is a submersion from the invariant manifold Mϵ to the output manifold Moutput. Ehresmann's lemma implies that this implies that we have a locally trivial fibration (Mϵ, Moutput, g, Mnull). We construct the perturbation p as the part of the vector field that us normal to the fibers g -1 (y) for each y ∈ Moutput. By construction, this perturbation has uniform norm at most η. This perturbation makes the vector field normal to each fiber zero. That implies that each fiber is an invariant submanifold of Mϵ. From the structure of the fiber bundle it follows that there is a continuum of such invariant submanifolds. Hence, the perturbed system is decomposable into a system that is a continuous attractor and a system with a non-zero flow.Remark This perturbation can be made smoothness, along similar lines as above. We can take aϵ-Neighborhood around the invariant manifold on which we extend the above vector field with bump functions to get a smooth vector field that still results in an attractive invariant manifold.Example An example of an approximate continuous attractor solution of the form with a decomposable system of which one of the subsystems is close to a continuous attractor is the torus solution in Fig. 4D . In this case, the system can be perturbed slightly such that there exists a continuum of limit cycles laid out over a ring.S6 Upper bound for the memory performance on a short time scaleWe will now formalize the statement about an upper bound dependent on the uniform norm of the vector field on the slow manifold in Sec.3.2 and provide a proof. Proposition 2. Let M be a normally hyperbolic slow manifold as in Prop. 3.2. Let x0 ∈ M, and φ = f |M be the flow restricted to the manifold. The average deviation from initial memory x0 over time is bounded linearlyEQUATIONProof. Numerical integration of the ODE givesx(t, x0) = t 0 φ(x(τ ))dτ + x0 ≤ t 0 ∥φ∥∞dτ + x0 = t∥φ∥∞ + x0From this, we get1 vol M M |x(t, x0) -x0| dx0 ≤ 1 vol M M t∥φ∥∞ = t ∥φ∥ ∞ .We formulate here a theory of continuous attractor approximations in terms of memory loss over time. It can be used uniform norm of vector field on the manifold to bound the memory performance on the short-time scale. Let x0 ∈ M, and φ = p|M be the flow restricted to the manifold. We will show that the average deviation from initial memory x0 over time is bounded linearly as in Eq. 5.For a ring attractor we can give more tight bounds on the accumulated error for the angular memory. Suppose we have a dynamical system x ∈ R N with autonomous dynamics ẋ = F θ (x) and solutions x(t, x0) uniquely defined for each x0. Let us define the error (i.e.the average deviation from initial memory) for an attractor asEQUATIONIf we further assume that the memory is not simply the state of the network, we need to take into consideration a decoder of the memory. Suppose that there is an invertible decoder mapping f : U → R N . For a ring variable, we can take this to be the projection onto the plane Wout and then applying the arctan:EQUATIONIn the case of the ring attractor, the memory we would like to encode is α ∈ U = [0, 2π), the error is defined as |x -y|o = oπ(|x -y|) where:EQUATIONIf we call αθ (α0, t) = g(φ θ (f (α), t)) we get the expression of the memory loss for this kind of memory as:EQUATIONGeneral bounds Define the following functions:EQUATIONThen we get the bounds for the loss L(T ) as:EQUATIONWe can define the maximum, average and minimum memory error speed as:EQUATIONNotice then that since:EQUATIONthen,EQUATIONmanifold case Let's assume that we have managed the system F θ have a slow manifold M ∈ R N in bijection with U, i.e. f |M is not a mapping but a bijective function and:EQUATIONThen we have a slow manifold in the form of a ring attractor, we have f (0) = f (2π) and:EQUATIONThen:EQUATIONwhere we used that α0 ∈ [0, 2π) ⇒ α0 = α0 mod 2π and that |x mod 2π -y mod 2π| = |(x -y) mod 2π|.The final equation of the loss in this case has the form:EQUATIONIn this case, if we have N fixed points in the ring-like slow manifold, we know that:EQUATIONand therefore:EQUATIONWe will provide a detailed description of the tasks, architectures, training methods, and analysis techniques used in our numerical experiments with trained RNNs.Memory guided saccade task The total time length of a trial is 512 steps. The time delay to the output cue was sampled from T delay ∼ U (50, 400).(59) We applied a mask mi,t = 0 for 5 time steps (t = T delay + j for j = 0, . . . 4) after the go cue (Eq. 65).The time length of a trial is 256 steps. The input is an angular velocity and the target output is the sine and cosine of the integrated angular velocity. Velocity at every timestep is sampled from as a Gaussian Process (GP) for smooth movement trajectories, consistent with the observed animal behavior in flies and rodents.EQUATIONwith length scale ℓ. The length scale of the kernel was fixed at 1. The output of the angular velocity integration in the output space, color coded according to the integrated angle. An example of an input is shown with constant velocity and it is provided until one turn is completed.The Double Angular Velocity Integration Task is an extension of the Angular Velocity Integration Task, where two independent instances of the task are performed simultaneously. In this case, you have two separate angular velocities, each sampled from its own Gaussian Process, representing two distinct movement trajectories.For each of the two angular velocities, the integration over time is performed separately, resulting in two sets of outputs: one for each angular velocity, making the output space four dimensional.Grid cells are known to exhibit periodic firing patterns that form a hexagonal grid across an environment, and these patterns are often modeled as existing on a toroidal surface (a doughnut-shaped surface) 55 . The reason for this is that the activity patterns of grid cells are continuous and wrap around seamlessly, meaning that if you move far enough in one direction, the grid pattern will repeat itself. This toroidal structure allows for the continuous representation of space without boundaries, which is crucial for efficient path integration. In the context of the Double Angular Velocity Integration Task, where two independent angular velocities are integrated, the resulting four-dimensional output space can be considered as two 2D subspaces (one for each angular velocity).Unbounded tasks Regarding tasks with an unbounded range, such as navigation tasks, two points bear mentioning.For planar attractors are diffeomorphic to R 2 , note that they do not conform to the assumptions on normally hyperbolic invariant manifolds, since R 2 is not compact. There are suitable generalizations of this theory to noncompact manifolds 34 , but we do not pursue them since they require more refined tools, which would only obscure the point that we are trying to make. Tangentially, we would also like to point out that we assume that neural dynamics are naturally bounded (e.g. by energy constraints) and hence sufficiently well described by compact invariant manifolds. 4B ). (B) An example of a found solution to the angular velocity integration task (Fig. 4C ). (C) An example slow-torus type solution to the memory-guided saccade task. The colored curves indicate stable limit cycles of the system (Fig. 4D ).The output projection of the invariant manifolds and stability structures (fixed points and limit cycles) is very close to the target output circle, as shown in Fig. S15 .Computational neuroscientists often train RNNs as models of neural computation and interpret them as dynamical systems 81, 121, 138 . Our experiments connect to existing literature. We examined vanilla continuous-time RNNs for this work. The time-discretized version of RNN network activity is given by xtEQUATIONwhere xt ∈ R d is the hidden state, yt is the readout, It ∈ R K is the input, ϕ : R → R is an activation function that acts on each of the hidden dimensions, ζt iid ∼ N (0, σ 2 = 1 /100I) is a state noise variable, and W, b, Win, Wout, bout are parameters. We will shortly explain the discretization procedure, i.e., the steps for going from Eq. 64 to Eq. 61. Let tn = n∆t.The Euler-Maruyama method for a stochastic differential equation (Eq. 64) dx = (-x + ϕWinI(t) + Wx + b)) dt + σdWt is given by : So, assuming Euler-Maruyama integration with unit time step, the discrete-time RNN of (61) corresponds to the stochastic differential equation:EQUATIONEQUATIONEQUATION)where dW is a Wiener process that models the intrinsic state noise in the brain. See for more detail on correspondences between discrete-and continuous-time RNNs in 84 and 90 . Our experiments connect to existing literature. In future studies, it would be interesting to perform experiments with Neural SDEs Figure S17 : The projected vector field on the found invariant manifold for the system Fig 4C . (A) The found vector field aligns well with the ring in the projected output space. (B) The norm of the vector field is low around found fixed points as expected, but is higher for points that are just slow points.see also Sec. S6.For LSTMs and GRUs the vector field was approximated by taking the difference the initial and the next state after initializing the network from states on the identified invariant manifold.The trained LSTMs and GRUs share the same pattern observed in the trained vanilla RNNs: a ring slow invariant manifold (Fig. S18C and D ). The fixed point topologies in the LSTM and GRU networks show a lot of variation in the number of fixed points, paralleling the systems adjacent to continuous attractors from Fig. 1 , as seen in Figure 5D . These variations are similar to those discussed in Figures S18B. Additionally, the angular error and memory capacity measures across different time scales are comparable to those illustrated in Figure S18A , highlighting the generalization properties influenced by the topology of the solutions. These results underline the universality of our findings beyond vanilla RNNs. EQUATIONFixed point analysis 121 and 48 are primarily concerned with a pointwise definition of slowness, by comparison normal hyperbolicity requires a uniform separation of timescales over the entire invariant manifold. In 121 , it was observed that structural perturbations (random gaussian noise in the parameters with zero mean and standard deviation) still leads to the same approximate plane attractor dynamical structure is still in place, however no explanation is provided for these observations. Our theory can explain why perturbations to the trained RNN. The Persistence Manifold Theorem (Theorem 1) guarantees that for small perturbations (of size ϵ) the persistent invariant manifold will be at the approximate same place (it will be at a distance of order O(ϵ)).Piecewise linear recurrent neural network 109 identifies asymptotic behaviors in dynamical systems, fixed point dynamics and more general cases cycles and chaos. We look beyond asymptotic behavior and characterize attractive invariant manifolds, thereby also identifying connecting orbits (or heteroclinic orbits) between fixed points. Although we developed new analysis methods for dynamical systems to find slow manifolds in them, we do not propose a new general framework for analysis of all dynamical systems. Finally, 109 provides analysis tools for Piecewise-Linear Dynamical Systems, while our methods are generally applicable to RNNs with any activation function.• All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: Yes, we have reported the computer resources in the Supplemental.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.The Persistent Manifold Theorem has been successfully applied previously in neuroscience71,36 , for example to reduce the dimensionality of the Hodgkin-Huxley model104,35 .Seefor results of persistence of noncompact invariant manifolds.3 As a technical note, for the theory to apply to a continuous piecewise-linear system, it is required that the invariant manifold is globally attracting115 , which is also the case for the BLA (see also98,99 for a discussion of geometric singular perturbation theory for piecewise linear dynamical systems). Therefore, we consider systems that are at least continuous, but some extra conditions apply if a system is not differentiable.This correspond a type of "ideal pattern" in the vocabulary of Chirimuuta18 . Our framework proposes a general approach to abstract away irrelevant details in models for analog memory97 .We tested all networks with a validation set and took a cutoff for the normalized MSE for the networks we consider for the analysis at -20 dB (Fig.5B).Note that effectively feedforward solutions47 do not satisfy (C1).< ϵ11ϵ22 -ϵ11 -ϵ22 -ϵ12ϵ21 -ϵ12 -ϵ21, ϵ22 ≤ ϵ12, ϵ11 ≤ ϵ21.Storing multiple continuous attractors has been worked out in6 , with a similar approximation as our theory suggests. The different stored patterns form a discrete approximation of the continuous map (resulting in quasicontinuous maps, which corrspond to the approximate contintuous attractors in our theory).Not to be confused with the perturbation parameter ϵ. 
 
 Appendix:  


Decision: This work revives the neuroscience idea of using manifold attractors for storing continuously varying quantities in long memory of recurrent systems. The common critique with such systems is that they are structurally unstable, but anchored in solid and elegant theoretical work and topological considerations, the authors show this actually may not be so much of a problem on functionally relevant time scales. These are important observations that may give new impetus to a powerful concept in theoretical neuroscience and beyond. While some of the referees initially had some reservations about this work, all of them after a strong rebuttal unanimously support publication.

I would like to add, however, that the idea that the vicinity of a manifold attractor may still be good enough to support behaviorally relevant time scales is not entirely new [1,2]. This should be mentioned and discussed in the revision.

[1] https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2011.00040/full

[2] https://www.cell.com/cell-reports/fulltext/S2211-1247(22)00360-6

Reviews: review: paper_summary: paper_summary: The study explores some bifurcations from continuous attractors in neuroscience models, revealing various structurally stable forms. Through fast-slow decomposition analysis, they uncover the persistent manifold surviving destructive bifurcations. Additionally, they analyze RNNs trained on analog memory tasks, showing approximate continuous attractors with predicted slow manifold structures. An important takeaway of their work is that continuous attractors demonstrate functional robustness and serve as a valuable universal analogy for understanding analog memory.
strength_weakness: strength_weakness: 1. They discussed some interesting theoretical techniques (e.g., Theorem 1, Proposition 1) in their study. However, their theoretical investigation and results are limited to a few very simple systems, low-dimensional systems either in Section S4 or low-dimensional RNNs with specific activation functions and restrictive settings, i.e., specific parameter values (e.g., equations (1) and (10)). The bifurcation analysis of the line attractors and fast-slow decomposition in Section S2 are also studied for very simple systems. Therefore, it is difficult to determine how general their theoretical discussion is and whether it can be applied to investigate and obtain results for more general and high-dimensional cases.

2. In Sect. 3.1, the perturbation p(x) is not clear enough. Specifically, it is unclear:1) Under what conditions the perturbation function p(x) induces a bifurcation?  2) What types of (generic) bifurcations can arise from the perturbation p(x)? Likewise, the functions h and g are also not clear enough. It is unclear how one can obtain/choose the functions h and g such that the two systems defined by Eq. (2) and Eqs. (3) & (4) are equivalent.

3. What does **sufficiently smooth** mean in Theorem 1? As mentioned by the authors after this theorem, it applies to continuous piecewise linear systems. However, it cannot be applied to all piecewise smooth (PWS) systems , such as Filippov systems. In particular, for these systems, bifurcations involving non-hyperbolic fixed points can be analyzed using similar slow (center) manifold approaches, but only for part of the phase space. However, discontinuity-induced bifurcations cannot be examined in the same way, as there is no slow manifold in these cases.

4. It is unclear under what conditions RNN dynamics can be decomposed into slow-fast form to which we can apply Theorem 1.

5. In Sect. 4.1, line 213,  it is vague how assuming an Euler integration with unit time step, the discrete-time RNN of (6) transforms to eq. (7). Is this transformation independent of the function f and matrix W in eq. (6)?

6. In S4, the sentence "All such perturbations leave at least a part of the continuous attractor intact and preserve the invariant manifold, i.e. the parts where the fixed points disappear a slow flow appears." needs more clarification. Could you explain the mathematical reasoning behind this assertion?
questions: questions: How does the selection of a specific threshold value influence the identification and characterization of slow manifolds in neural networks with continuous attractors as discussed in the first lines of section 4.2? Could you elaborate on how different threshold settings impact the dynamics of network states and the emergence of persistent manifolds?
limitations: limitations: - The authors have discussed most limitations of the analysis in the discussion section, but I suggest making them more explicit. This could be done by either incorporating a dedicated (sub)section on limitations or adding a bold title "**Limitations**" at the beginning of the relevant paragraph within the discussion section. 

- As mentioned above, another important limitation is that it is difficult to determine how general their theoretical discussion is and whether it can be applied to investigate and obtain results for more general and high-dimensional cases.
paper_summary: paper_summary: This paper studies the fragility of continuous attractors, which have been used to explain various computations or functions in the brain related to memory and navigation, to perturbations. The authors mainly focus their analyses on ring attractors, which have been used to model continuous-valued memory. Under perturbation, the authors find that the bifurcations of continuous attractors exhibit structurally stable forms with different asymptotic behaviors but similar finite-time behaviors as the original attractor. For example, a stable limit cycle arising from a bifurcation of a ring attractor would be functionally similar to the ring attractor in finite time. Thus, the authors posit that signatures of the continuous attractor persist even under perturbation in the form of a persistent, attractive manifold which serves as an approximate continuous attractor. Experiments and analyses on recurrent neural networks performing analog memory tasks show that the networks learn approximate continuous attractors. Thus, the authors conclude from their theory and numerical experiments that approximate continuous attractors do remain a good model of analog memory.
strength_weakness: strength_weakness: 1. The experiments and associated analyses focus solely on networks that approximate 1D ring attractors. This is quite simplistic, and at least for the numerical expreiments, the authors could consider tasks like navigation where a planar 2D attractor is approximated by the networks.
2. The authors have only qualitatively characterized the variations in the topologies of the networks. It is perhaps possible to quantitatively characterize this by using Dynamical Similarity Analysis [1] on various trained networks.
3. For the generalization analysis, the authors could evaluate generalization performance by the nature/type of the approximate attractor as well. Furthermore, although I may have missed this, could the authors comment on what networks hyperparameters lead to which approximations?
4. The figures and presentation could be improved:
    1. On line 107 there is a comment that should be removed ("add link to details").
    2. Fig. 4C, caption should indicate the nature of the solution found.
    3. Fig. 5B, y-axis label is missing.
    4. Fig. 5D, could also show the mean $\pm$ std for classes of networks.
    5. Fig. 5E, y-axis label is missing. Also, the authors could just use the normalized MSE on the axis could just follow the convention used in Fig. 5A instead of using dB.
    6. Overall, the writing could be improved in several places to improve clarity. For example, the conclusions of the generalization analysis and their implications are not very clear, and how this connects to the various types of approximate attractors is not clear (related to W3).

**References:**
1. Ostrow et al. "Beyond geometry: Comparing the temporal structure of computation in neural circuits with dynamical similarity analysis." Advances in Neural Information Processing Systems 36 (2024).
questions: questions: See the Weaknesses section. I have some additional questions:
1. How do the authors identify the various kinds of approximations of the attractors? Can this be automated, perhaps by using to DSA to cluster the various types?
2. At what level of performance are all trained networks compared? Are they all trained until the same loss value and how close is this MSE to 0?

**Rebuttal update:** Score increased from 5 to 7.
limitations: limitations: There is no limitations section. Tthe authors have mentioned in the discussion that they do not explicitly characterize all possible invariant manifolds identified. However, there are other limitations as well such as the lack of diversity in the tasks/attractors explored. I would also encourage the others to unpack the limitations related to the numerical fast-slow analysis, such as how sensitive the results are to the threshold hyperparameter, cases or specific attractors where it does not work as well, etc.
paper_summary: paper_summary: The manuscript investigates the stability and robustness of continuous attractors to small deformations in vanilla recurrent neural networks. Continuous attractors were once heavily studied in the context of working memory, but they are inherently fragile and susceptible to even small perturbations and lead to very different topologically distinct attractor landscapes. This study, on the other hand, focuses on finite time behavior around these distinct attractor landscapes. Specifically, the authors theoretically discuss and empirically show that continuous attractors exhibit slow manifold structures under perturbations, which persist despite their theoretical instability.
strength_weakness: strength_weakness: - The presentation in section 5 becomes unclear, and perhaps too dense. The authors may want to expand on this section significantly.
- Some more control experiments need to be added for proving the generality (See below).
questions: questions: I believe the work warrants a borderline accept as is, yet I would feel very supportive of its publication (up to a strong accept) if the authors performed the following changes:

- I believe the slow manifold picture the authors are introducing here is not too different from [1], specifically Fig. 7. Can the authors please clarify the differences in the main text?

- The fast-slow decomposition does not seem to be specific to vanilla RNNs. Can the authors please include experiments with LSTMs and GRUs, which would support their claims on generality.

- Similarly, the experiments on Section 4 are centered around ring-like attractors. Can you please show an example with other structures? For example, you can consider the delayed addition/multiplication tasks discussed in [2]. Relatedly, similar slow dynamics arguments are made in [2], which I believe the authors should list as a close related work and explain the differences in their approach. 

- A more detailed discussion of Section 5 is desired. I was able to understand the main takeaways, but could not evaluate the validity of the claims. Perhaps the authors may want to explain in simpler terms the evidence presented in Fig. 5.

- The title is not descriptive of the manuscript's content and feels like as if it belongs to a blog post. Could you please update the title to be representative of the paper's conclusions?

Citations

[1] Opening the Black Box: Low-Dimensional Dynamics in High-Dimensional Recurrent Neural Networks, David Sussillo and Omri Barak, Neural Computation, 2013

[2] Schmidt, D., Koppe, G., Monfared, Z., Beutelspacher, M., & Durstewitz, D. (2019). Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies. arXiv preprint arXiv:1910.03471.


**Edit:** My main remaining concern is the presentation, which I find to be unnecessary complicated. That being said, this work is an important piece of contribution to neuroscience and I support its acceptance.
limitations: limitations: Yes.
paper_summary: paper_summary: The authors study continuous attractor networks, and their famous instability under noise. They show that continuous attractors, despite being unstable, are functionally robust, and analyse some behaviours when noise is introduced.
strength_weakness: strength_weakness: I got very confused by what most of the paper was trying to show. The main result, section 3, is, almost draw-droppingly, nice. (Though I'm partly trusting that the authors are applying the results correctly; they seem to be, because the step from the stated theorem to their claims is not large) The rest is, ..., a bit more meh, especially in how it was presented. I would have liked a lot more signposting: why are these sections the right ones to care about? What are you showing, and why?

For example, section 2 shows that noise degrades CANs - a nice literature review - and then shows a categorisation of perturbed fixed point dynamics. I guess the idea was to show that noise-instability was a problem for all CAN types in the literature? If so you could do with signposting that. If not, why did you include all of that? Why was it important that some patterns appear and not others in the perturbed dynamics?

Then there were a lot of quite confusing things, especially in the figures: 

Fig 1) A was a great figure, B was never mentioned in the text, despite being very pretty.

Fig 2) Why were there hexagons everywhere? I never could find any reason for there to be hexagonal figures, did you just make the ring a hexagon for fun? If so, tell the reader! Further, in B and C, are the blue lines numerical? How did you choose what to colour blue? Should I be using it as evidence for your big theoretical claim? Or is it just illustrative? 

Fig 3) What am I looking at in figure 4A2: it says example trajectory? But there are many trajectories, no? The dots in B and C are presumably fixed points, why then is it called a limit cycle (line 246)? It doesn't look like that? Why is 4D described as around a repulsive ring invariant manifold? Are you describing the ring inside the torus? (rather than the presumably attractive limit cycles that are marked on the figure) What does the colouring of the torus (the greys) denote? I didn't get told what the task was for figure A1 so had to go to the appendix to see that this is apparently the output of the integration? Why are you plotting the output space, and not the recurrent space as in all the other figures? 

On that last point, why include details of the (standard) discretisation scheme, MSE loss, and network details, when key steps to understand what I am looking at (e.g. figure A1 = output space) are missing?

Figure 5A) Why did one of the finite time results go above the line? Shouldn't this be an exact theoretical result, yet it appears not to be true? 

Seemed obtuse to claim a linear relationship then show a log-linear plot, fig 5C? How should I see this?

Did you define the memory capacity? 

Did you need to introduce omega-limit set, especially given the likely audience for this paper?

Finall, some other points:

You should definitely cite and discuss the relationship to this paper: Information content in continuous attractor neural networks is preserved in the presence of moderate disordered background connectivity, Kuhn & Monasson, 2023.

What was section 5.2 trying to show? First it claims that 2.2 presents a theory of approximate solutions in the neighbourhood of continuous attractors (news to me, as far as I could tell, that section showed me that all CAN models were unstable to noise and turn into a variety of different fixed points under noise, that doesn't sound like a theory at all? Section 3 seems to be the theory?) Then you list four conditions on what sounds like exactly the same problem? What is the difference between dynamical systems having robust manifolds, and the working memory task being solved, isn't the whole point of the model that these two are the same? (i.e. you can solve a working memory task with a CAN). Is this supposed to be a concluding section that says when working memory can be solved? Then why have you suddenly defined state and dynamical noise that haven't been used before, I thought we had a perfectly nice definition of perturbations on the network (equation 2)? This section seemed... strange, in my eyes the paper would be improved by removing it

Smaller things:
- line 107 - comment left in doc
- Figure 2 caption line 1, missing 'of' and pluralisation of 'implementation'

So all in all, I think the exposition, despite, as I said, often being very clear when describing a single idea, is, on a macro scale, a mess. I had a very hard time following, and the figures were quite tough-going. I think this paper should probably get in, but I think it should be given a good clean up first, at least for a humble neuro-theorist, rather than a dynamical systems guru, to understand.
questions: questions: My, many, questions and confusions ended up being in the weaknesses section.
limitations: limitations: The limitations were somewhat discussed.

Comments: title: References
Response
comment: I appreciate the authors' comprehensive responses. Assuming that the authors will make all mentioned changes to the revised version of the manuscript, I have increased my score to 6.
Thanks again for the updates – looks good to me!
I believe this new addition will be an important step towards increasing the work's impact. I support the acceptance, though I will keep my score as is. I feel slightly concerned about author's commitment to making more changes, as it took a long while to answer a simple question. So, I make this judgement based on the submitted version and clarifications during the rebuttal.
We thank the reviewer for their additional comments.
We would like to point the Reviewer to our response to reviewer xmvZ for our notes on how we will explicitly describe the steps in the Euler-Maruyama integration to demonstrate the relationship between (6) and (7): [see](https://openreview.net/forum?id=fvG6ZHrH0B&noteId=oMNXw7U8k7).
[1a] Fenichel, N. (1979). Geometric singular perturbation theory for ordinary differential equations. Journal of differential equations, 31(1), 53-98.

[2a] Kirchgraber, U., & Palmer, K. J. (1990). Geometry in the neighborhood of invariant manifolds of maps and flows and linearization. (No Title).

[3a] https://docs.sciml.ai/DiffEqDocs/stable/solvers/sde_solve/

[4a] Mante, V., Sussillo, D., Shenoy, K. V., & Newsome, W. T. (2013). Context-dependent computation by recurrent dynamics in prefrontal cortex. nature, 503(7474), 78-84.

[5a] Sussillo, D., & Barak, O. (2013). Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural computation, 25(3), 626-649.

[6a] Tzen, B., & Raginsky, M. (2019). Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit. arXiv preprint arXiv:1905.09883.
We thank the reviewer for the additional comments.


## Bifurcation and stability analysis
> Bifurcation and stability analysis in Appendix (S2) are not only restricted to a low-dimensional system but also to a system with specific parameters

### Analytically tractable examples
We would first like to clarify that in Supplementary Sections 1 and 2 the provided examples are illustrative rather than exhaustive.
Recognizing that the supplementary material previously lacked clear structure and signposting, we will revise the text to better convey the motivation behind the examples and to provide a clearer explanation of the analysis.
The analysis can furthermore be easily extended to a more general form $W = [w\_{11} w\_{12}; w\_{21} w\_{22}]$ that has a bounded line attractor, through a coordinate transformation.


### Extension of bounded line attractor analysis to higher dimensional systems
We agree that this analysis in S2 can be extended to higher dimensions, and we will incorporate this remark into the supplementary text.
An extension of these results of a low-dimensional system can be easily achieved by the 'addition' of dimensions that have an attractive flow normal to the low-dimensional continuous attractor or invariant manifold.


More generally, the results from a low-dimensional system can indeed be extended to higher-dimensional systems through reduction methods from center manifold theory.
On the center manifold the singular perturbation problem (as is the case for continuous attractors) restricts to a regular perturbation problem [1a].
Furthermore, relying on the Reduction Principle [2a], one can always reduce all systems (independent of dimension) to the same canonical form, given that they have the same continuous attractor. We thank the reviewer for pointing this out and will add a remark on this possibility to extend results.






## Perturbation and dimensionality
> Regarding the perturbation ...
We understand how that may have lead to confusion. We changed the statement in Sec.3.1 to be $l\\neq 0$ for clarity.


## Discretization of SDE
We appreciate the suggestion to better explain the discretization procedure, i.e., the steps for going from Eq.(7) to Eq.(6).


### Steps
In the supplementary material, we will include a note on the discretization, which goes as follows.

**Discretize the time variable:** Let $t\_n = n \\Delta t$.

The Euler-Maruyama method for a stochastic differential equation

$$\\mathrm{d}{\\mathbf{x}} = (-\\mathbf{x} + f( \\mathbf{W}\_{\\text{in}} \\mathbf{I}(t) + \\mathbf{W} \\mathbf{x} + \\mathbf{b} )) \\mathrm{d}{t} + \\sigma\\mathrm{d}{W}\_{t}$$

is given by :

$$\\mathbf{x}\_{n+1} = \\mathbf{x}\_n + ( - \\mathbf{x}\_n + f ( \\mathbf{W}\_{\\text{in}} \\mathbf{I}\_n + \\mathbf{W} \\mathbf{x}\_{n} + \\mathbf{b} ) ) \\Delta t + \\sigma \\Delta W\_{n}$$

with $\\Delta W\_{n}=W\_{(n+1)\\Delta t}-W\_{n\\Delta t}\\sim \\mathcal{N}(0,\\Delta t).$

**Subsitute $\\Delta t = 1$:**

$$\\begin{aligned}
 \\mathbf{x}\_{t+1} &= \\mathbf{x}\_t + ( -\\mathbf{x}\_t + f(\\mathbf{W}\_{\\text{in}} \\mathbf{I}\_t + \\mathbf{W} \\mathbf{x}\_t + \\mathbf{b}) ) + \\sigma \\Delta W\_t, \\\\
 &= f(\\mathbf{W}\_{\\text{in}} \\mathbf{I}\_t + \\mathbf{W} \\mathbf{x}\_t + \\mathbf{b}) + \\sigma \\Delta W\_t.
 \\end{aligned}
$$

**Introduce the noise term** $\\zeta\_t = \\sigma \\Delta W\_t$, which represents the discrete-time noise term.
Thus, we have derived the discrete-time equation:

$$\\mathbf{x}\_t = f(\\mathbf{W}\_{\\text{in}} \\mathbf{I}\_t + \\mathbf{W} \\mathbf{x}\_{t-1} + \\mathbf{b}) + \\zeta\_t$$


We thought that $\\Delta t=1$ would simplify the presentation, however, it seems to be misleading the readers.
In our numerical experiments, we used a $\\Delta t<1$.
We will update our manuscript to include $\\Delta t$ for clarity.



### Integration scheme
Numerical integration of a stochastic differential equation is an extensive field by itself [3a].
We chose to use the simplest Euler-Maruyama discretization form because this leads to the standard RNN form.
Although it is generally inferior to other methods in terms of efficiency, systems with a fast-slow decomposition are stiff which presents additional challenges in their solution.

Computational neuroscientists often train RNNs as models of neural computation [4a,5a]
and interpret them as dynamical systems.
Our experiments connect to existing literature.
In future studies, it would be interesting to perform experiments with Neural SDEs [6a].
We would like to thank the reviewer again.

For our proposal to increase the readability of the paper and to ameliorate the audience problem, we would like to point the Reviewer to our comment to reviewer 8M6u: https://openreview.net/forum?id=fvG6ZHrH0B&noteId=GM02jaKd5z.
Thank you for your patience. We just uploaded our proposed additions and changes to make the text more accessible.
Thank you for your positive feedback. We greatly appreciate your thoughtful consideration of the potential impact on experimental neuroscientists, a crucial audience we indeed hope to reach!

To address your concern, we propose to add a new section to the main text, simplify the language of our presentation, and add additional material in the appendix.

## We will include a new section titled: "Implications on experimental neuroscience"

Following is a draft:

Animal behavior exhibits strong resilience to changes in their neural dynamics, such as the continuous fluctuations in the synapses or slight variations in neuromodulator levels or temperature. Hence, any theoretical model of neural or cognitive function that requires fine-tuning, such as the continuous attractor model for analog working memory, raises concerns, as they are seemingly biologically irrelevant. Moreover, unbiased data-driven models of time series data and task-trained recurrent network models cannot recover such continuous attractor theories precisely. Our theory shows that this apparent fragility is not as devastating as previously thought: despite the "qualitative differences" in the phase portrait, the "effective behavior" of the system can be arbitrarily close, especially in the behaviorally relevant time scales. We show that as long as the attractive flow to the memory representation manifold is fast and the flow on the manifold is sufficiently slow, it represents an approximate continuous attractor. Our theory bounds the error in working memory incurred over time for such approximate continuous attractors. **Therefore, the concept of continuous attractors remains a crucial framework for understanding the neural computation underlying analog memory, even if the ideal continuous attractor is never observed in practice.** Experimental observations that indicate the slowly changing population representations during the "delay periods" where working memory is presumably required, do not necessarily contradict the continuous attractor hypothesis. Perturbative experiments can further measure the attractive nature of the manifold and their causal role through manipulating the memory content.

## Simplifying Language
We will revise the manuscript to reduce the use of technical jargon and complex theoretical language where possible. This will involve clearly defining key terms and concepts upfront and using more intuitive explanations throughout the text.
We will simplify Theorem 1 to only talk about invariant manifolds (locally invariant manifolds indeed unnecessarily complicate the statement). 

We will furthermore leave out concepts such as omega-limit sets and only include it in the supplementary material for the curious theoretician to show how to generalize certain concepts to other stability structures (in this case going from fixed points to considering limit cycles as well). Similarly, we will also relegate discussions of the Hausdorff distance to a less prominent section in the supplementary materials (as this part of the theory is still important to guarantee that the persistence invariant manifold is close to the original continuous attractor).

Additionally, we include an additional supplementary section to explain through less technical and in more intuitive terms the dynamical systems terms that are essential. This includes compact, normally hyperbolic, invariant, diffeomorphism and heteroclinic/connecting orbits. We will further explain what the neighborhood of a vector field entails (see also our proposed intuitive definitions list below).

Finally, in this new supplementary section, we will include a subsection with a visual illustration of our claims, aiming to tap into human geometric intuition. In this visualization, we show how models that behave almost like a perfect analog memory system correspond to a volume of models in the space of dynamical systems. 

We will furthermore provide intuitive definitions of several key concepts used in our paper:


- Manifold: A part of the state-space that locally resembles a flat, ordinary space (such as a plane or a three-dimensional space, but more generally $n$-dimensional Euclidean space) but can have a more complicated global shape (such as a donut).
- Invariant set: A property of a set of points in the state space where, if you start within the set, all future states remain within the set and all past states belong to the set as well.
- Normally Hyperbolic Invariant Manifold: A behavior of a dynamical system where flow in the direction orthogonal to the manifold converges (or diverges) to the manifold significantly faster than the direction that remains on the manifold.
- Diffeomorphism: A diffeomorphism is a stretchable map that can be used to transform one shape into another without tearing or gluing.
$C^1$ neighborhood of a $C^1$ function: A set of functions that are close to the function in terms of both their values and their first derivatives.
Dear Authors,

Due to your new experiments and the fact that most of your responses were convincing to me, I will wait until the end of the discussion period. I will raise my score to 6 if you can address my concerns regarding the bifurcation and stability analysis; otherwise, I will raise it to 5.

Kind regards,
Reviewer
I thank the authors for their detailed and impressive responses.

Broadly, my comments have been answered. That said, they were largely about clarity, so it is hard to verify if the paper does indeed make more sense.

Certainly including the details of the network is important for reproducibility (sec 4.1) and apologies for making comments that sound like they're suggesting these details should be removed. That was not my intention! Rather, it seemed strange to put such standard details in the main text rather than the appendix, when the space could be used to explain so many other things.

I continue to think the paper should get in, and will raise my score to 6 on the assumption that the resulting paper will indeed be much clearer. (I agree with reviewer 8M6u that this paper in its first version has an audience problem)
I really appreciate the authors' responses and new experiments, particularly the additional experiments with LSTMs, GRUs, and RNNs trained on a 2D task. 

- Regarding the first weakness I mentioned, my main concern was not about the role of numerical experiments in proving the generality of the theory. Rather, it was mostly about:

1) Whether and how the theory itself is applicable to higher-dimensional systems in the sense that the conditions of the theorems can be met for high-dimensional systems (especially as many systems in neuroscience are high-dimensional). Nevertheless, I appreciate the revision of Theorem 1 to show that normal hyperbolicity is both sufficient and necessary for invariant manifolds to persist. 

2) Bifurcation and stability analysis in Appendix (S2) are not only restricted to a low-dimensional system but also to a system with specific parameters (eq. (9)). One can at least consider such analysis for a more general case, e.g., W = [w11 w12; w21 w22]. Then, one can mention that in some cases, even when having a theory for low-dimensional systems, it can be extended in a similar way to higher-dimensional systems, for instance, through center manifold theory, where the higher-dimensional system has the same low-dimensional center manifold. 


- Regarding the perturbation p(x) (the second mentioned weakness), thank you for the clarification. Certainly, dynamical systems with nonhyperbolic equilibria and/or nonhyperbolic periodic orbits are not structurally stable, meaning that any ϵ-perturbation  (in the sense of Definition 1.7.1 on p. 38 of [1]) will induce a bifurcation of the system. But, please note that in Sect. 3.1, based on your response, "$l$" must be nonzero. Otherwise, one can consider $l=0$, which implies there is no continuous attractor, which might lead to confusion, as it did for me. So, I suggest changing "$l$" to  "$l \neq 0$" in Sect. 3.1.

- Finally, I agree with Reviewer CAVv that it might help to further clarify Sect. 4.1 to demonstrate the relationship between equations (6) and (7).

------------------------------------

[1]  J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, Springer-Verlag, New York, 1983.
The silence from the authors is slightly concerning. I will be waiting for another 24 hours before finalizing my score, though I must admit that it is hard to not take this as an indication of low confidence by authors regarding the applicability of their work to experimental neuroscience.
I appreciate the authors' rebuttal, the responses mostly address my concerns. I particularly appreciate the authors' additional experimental results with the 2D toroidal attractor. I understand that the DSA score is relative – I was hoping it could quantitatively characterize the similarity between various approximate attractors or help in clustering approximations with different numbers of fixed points, but the authors' response is illuminating. Also, related to another reviewer's concern, it might help to explicitly show the steps in the Euler-Maruyama integration to show the relationship between (6) and (7).

With the proposed changes and writing updates, I'm happy to increase my score to 7. I think this is a valuable contribution to theoretical neuroscience and dynamical systems theory, and hope to see it accepted to NeurIPS.
With the proposed changes, I believe this work is an important addition to the theoretical neuroscience literature. I will increase my score to match this belief. I do believe there is a way for authors to increase my score even further if the authors commit to addressing the following concern:

I believe one of the main readers of this work, though the authors probably do not intend it that way, will be experimental neuroscientists. I believe the current writing is too heavy for them to be able to understand this work and start testing some of these ideas with experiments. Could you comment on how you would go about making changes to the writing to be more accessible to this audience? In my mind, some of the theoretical contributions can be toned down in favor of accessibility to a more general audience, which in turn would increase your impact.

I believe your responses to all reviewers are satisfactory. I appreciated the new experiments in short time and I will be championing for the acceptance of this work. I do believe if you can provide a satisfactory answer to my concern above, I feel comfortable recommending this work for a spotlight.