SlowGuess/NeurIPS_2023_OCR huggingface dataset:
As discussed above, when  $L > 1$  the unmodified system does not possess true steady states at all positions: following initialization, an idealized bump state is immediately distorted, and then systematically drifts to a local minimum of the energy functional (Fig. 2). In order to define an energy landscape over a continuum of positions, it is thus necessary to first formalize the concept of states that have reached an energy minimum but also span a continuum of positions. To do so, we considered the minima of the energy functional under a constraint on the center of mass of the bump. In practice, such minima are found by idealized bump initialization, followed by gradient descent on the energy functional under the constraint. A distorted bump then dynamically evolves to minimize the Lyapunov function, while maintaining a fixed center of mass at its initial position (Supplementary Fig. 2). See SM for details on the constrained optimization procedure and its validation.
Since the goal of the constraint is to parametrize all states along the approximated continuous attractor, its exact definition is not crucial as long as the optimization is applied along a dense representation of positions. By finding the minima of the energy functional along such a dense sample of positions, it is possible to precisely measure the energy landscape of the system along each one of the attractors. As expected, the evaluated energy landscape obtained using gradient optimization achieved lower energy values compared to the energy landscape obtained using idealized bumps (Fig. 5a, orange traces).
Using this formal definition of the position dependent energy, it is possible to formulate a learning objective for the modification weights  $\mathbf{M}$ , designed to flatten the energy landscape. We denote by  $E^{k,l}$  the constrained minimum of the energy at position  $k$  in map  $l$ , evaluated using the constrained gradient minimization scheme described above. We approached our goal by attempting to equate the values of  $E^{k,l}$  for all  $k$  and  $l$ , using an iterative gradient-based scheme (SM). It is straightforward to evaluate the gradient of  $E^{k,l}$  with respect to  $\mathbf{M}$ , since up to first order in  $\Delta \mathbf{M}$ ,
$$
E ^ {k, l} (\mathbf {M} + \Delta \mathbf {M}) - E ^ {k, l} (\mathbf {M}) \simeq - \frac {1}{2} \vec {R} ^ {k, l ^ {T}} (\Delta \mathbf {M}) \vec {R} ^ {k, l} \tag {3}
$$
where  $\vec{R}^{k,l}$  is the firing rate population vector at the constrained minimum corresponding to  $E^{k,l}$ . We note that the constrained minima themselves are modified due to the change in  $\mathbf{M}$ , but these modifications contribute only terms of order  $(\Delta \mathbf{M})^2$  to the change in the energy (see SM).
In each iteration of the weight modification scheme,  $E^{k,l}$  and  $\vec{R}^{k,l}$  were first evaluated numerically for all  $k$  and  $l$  using constrained gradient optimization and the values of  $\mathbf{M}$  from the previous iteration (starting from vanishing modifications in the first iteration). Next, in iteration  $i$ , we sought adjustments  $\Delta \mathbf{M}_i$  to the weight modifications  $\mathbf{M}$ , that equate the energy across all  $k$  and  $l$ , to leading order. As in the previous section, Eq. 3 yields an underdetermined set of linear equations (assuming that  $N > 2L + 1$ ), where the least square solution was chosen in each iteration.
Fig. 5b-c shows results obtained over several gradient optimization iterations. The mean absolute value and standard deviation of weight modifications decreased with the iteration number (Fig. 5b), and the flatness of the energy landscape improved systematically in successive iterations (Fig. 5c, top, and Supplementary Fig. 3). The standard deviation of the energy across locations and maps decreased by almost four orders of magnitude after four iterations, and the maximal deviation of the energy from its mean decreased dramatically as well (Fig. 5c, bottom).
We next examined the consequences of improved flatness of the energy landscape on the network's stability. Qualitatively, individual states exhibited much less drift after five iterations (Fig. 6a),
Figure 5: Energy landscape evaluation and modifications using constrained gradient optimization. a, Energy landscape evaluated using constrained gradient optimization (orange) for maps #3 (top) and #5 (bottom), out of  $L = 10$  embedded maps as used in Figs. 2 and 3. For reference, energy landscapes evaluated using idealized bumps are plotted as well (blue traces, identical to the orange traces from Fig. 3a). As expected, the energy after gradient optimization (orange trace) is lower. Re-evaluating the energy landscape to leading order in the weight modifications yielded a precisely flat energy landscape when evaluated using the pre-modified network steady states (yellow). b, Top: mean absolute value of the weights in the original connectivity matrix (blue) and in modification adjustments for each gradient optimization iteration, as a function of  $L$ , the total number of embedded maps. Bottom: corresponding standard deviation. Error bars are ±1.96 SEM (SM). c, Top: standard deviation of the energy landscape without (blue) and with the weight modification for each gradient optimization iteration as a function of  $L$ . Bottom: same as top, but showing the peak absolute difference between all energy values and their mean. Error bars are ±1.96 SEM.
compared to the unmodified scenario (Fig. 2b), and also in comparison with the scheme based on idealized bumps (Fig. 3c). To systematically quantify the improved stability, we first validated that approximated steady states were reached three seconds after initialization (Fig. 6b), and that the bump score was not significantly affected by these modifications (Fig. 6c) as in the idealized bump approach (Fig. 4a-b). Next, we examined two measures of stability (as in Fig. 4c-d). The mean squared change in the firing rate of all the neurons, across 3s from initialization was dramatically improved, by almost three orders of magnitude, compared to the pre-modified networks for the smaller values of  $L$  (Fig. 6d). Measured drifts were attenuated significantly as well (Fig. 6e). Note that, as described above, drifts which are below or equal to the discretization precision of a single unit correspond to perfect stability. For completeness, drifts were also measured using the phase of the population activity vector and demonstrated similar results (Supplementary Fig. 4). Taken together, these results show that with few constrained gradient optimization iterations, a dramatic increase can be achieved in the inherent stability of a network forming a discrete set of continuous attractors.
Interference between distinct manifolds is highly detrimental for the function of RNNs designed to represent multiple continuous attractor manifolds. Naively, it is sufficient to embed only a second manifold in a network to destroy the continuity of the energy landscapes in each attractor. This leads to the loss of the continuity of steady states, which are replaced by a limited number of discrete attractors. In tasks that require storage of a continuous parameter in working memory, these effects are manifested by systematic drifts that degrade the memory over time. Sensory inputs can, hypothetically, pin and stabilize the representation [26, 1], but in the absence of correcting external inputs the representation is destined to systematically drift to the nearest local energy minima, thereby
Figure 6: Effects of weight modifications obtained using constrained gradient optimization on the stability of bump states. a, Bump trajectories as in Figs. 2b and 3c, but with added weight modifications obtained after five constrained gradient optimization iterations. Qualitatively, the drifts almost vanish (compare with Figs. 2b and 3c, and note cyclic boundary conditions). b, Same as Fig. 4a, but after five iterations of the constrained gradient optimization scheme. c-e, Same as Fig. 4b-d, but with superimposed results obtained after five constrained gradient optimization iterations (yellow traces). Adding weight modifications did not reduce the bump score (panel c). The network stability improved significantly (panels d-e) compared to the unmodified network (blue traces) and to the modified network using the idealized bumps scheme (orange traces). Error bars are  $\pm 1.96$  SEM (SM).
impairing network functionality. This has been thought to be a fundamental limitation of such networks.
Here we reexamined this question by adding small weight modifications, tailored to flatten the high dimensional energy landscape along the approximate attractor, to the naive form of the connectivity matrix. We found that appropriately chosen weight modifications can dramatically improve the stability of states along the embedded attractors. Furthermore, the objective of obtaining a flat energy landscape appears to be largely decoupled from the question of capacity, which is also limited by interference between multiple attractors. Indeed, introducing the weight modifications did not qualitatively affect the dependence of the bump score on  $L$  (Fig. 6c). In particular, the kink in this function occurs at approximately  $L = 20$  (for  $N = 600$ ) both in the pre- and post-modified networks. Theoretically, the position of the kink is expected to roughly match the location of a sharp transition in the large  $N$  limit, while keeping  $L / N$  fixed. A recent work [6] suggests that the capacity may depend logarithmically, and thus weakly, on the prescribed density of steady states. It will be interesting to explore the relation of this result to our framework as it was obtained for RNNs composed of binary neurons, with a linear kernel support vector machine based learning rule.
We explored our schemes in 1D to reduce the computational cost, but it is straightforward to extend our approach to 2D. One notable difference between 1D and 2D environments is that the number of neurons required to achieve a good approximation to a continuous attractor, even for a single map, scales in proportion to the area in 2D, as opposed to length in 1D. However, for a given number of neurons, there is no substantial difference between the two cases in terms of the complexity of the problem: the number of equations scales as  $NL$ , and the number of parameters (synaptic weights) scales as  $N^2$ . Since the random permutations are completely unrelated to the spatial organization of the firing fields, quenched (frozen) noise is expected to behave similarly in the two cases.
For simplicity, we assumed that each cell is active in each environment, but it is straightforward to adapt the architecture to one in which the participation ratio,  $p$ , defined as the average fraction of maps in which each cell participates, is smaller than unity. Measurements in CA3, performed
in the same cells in multiple environments, indicate that CA3 cells are active only in a subset of environments, with  $p \sim 15\%$  as a rough estimate [3]. Clearly, the quenched noise in the system increases in proportion to the number of embedded maps  $L$ , and it will be interesting in future work to assess how the roughness of the energy landscape depends on  $N$ ,  $L$ , and  $p$ . We note that even though small  $p$  implies low interference, it also implies a reduction in the number of neurons participating in each map, and in each bump state. This is expected to reduce the resilience of each attractor to the quenched noise. Hence, the overall effect of  $p$  on the roughness of the energy landscape (when keeping  $N$  fixed) is non-trivial.
Previous works have suggested that synaptic scaling [32] or short term facilitation [19, 38] mechanisms can stabilize working memory in networks with heterogeneous connectivity. These mechanisms were achieved, however, in networks which are equivalent to the single map case but with random added heterogeneity. In this respect, the source of heterogeneity in these networks is different from the one in this work, where heterogeneity arises from the embedding of multiple coexisting attractors. It would be interesting to investigate whether a synaptic facilitation mechanism [19] can be implemented in the multiple map case and further improve the network's stability alongside the energy-based approach proposed here. However, since in a discrete set of continuous attractors each neuron is participating in multiple representations, a synaptic scaling mechanism [32] is unlikely to achieve such stabilization. In addition, it may also be interesting to explore other potential methods for the design of a synaptic connectivity that supports multiple, highly continuous attractors, other than the one explored here. These might include generalization of approaches that were based on the pseudo-inverse learning rule for embedding of correlated memories in binary networks [29], or training of rate networks to possess a near-continuum of persistent states by requiring stability of a low-dimensional readout variable [12].
Our approach is based on the minimization of a loss function that quantifies the energy landscape's flatness. This raises several important questions from a biological standpoint. First, the connectivity structure obtained after training is fine-tuned, raising the question of whether neural networks in the brain can achieve a similar degree of fine tuning. A similar question applies very broadly to CAN models, yet, there is highly compelling evidence for the existence of CAN networks in the brain [45, 2, 48, 46, 39, 21, 15]. We also note that out of an infinite space of potential weight modifications only a specific set (least-squares) was chosen, leaving many unexplored solutions to the energy flattening problem which may relax the fine-tuning requirement. Second, our gradient-based learning rule for the minimization of the loss function was not biologically plausible. In recent years, however, many important insights were obtained on computation in biological neural networks by training RNN models using gradient based learning [43, 4, 33, 44, 24, 30, 20, 37, 16, 22, 42]. The (often implicit) assumption is that biological plasticity in the brain can reach similar connectivity structures as those obtained from gradient based learning, even if the learning rules are not yet fully understood. Thus, our results should be viewed as a proof of principle - that multiple embedded manifolds, previously assumed to be inevitably unstable, can be restructured through learning to achieve highly stable, nearly continuous attractors. It will be of great interest to seek biologically plausible learning rules that could shape the neural connectivity into similar structures. Such rules might be derived from the goal of stabilizing the neural representation during memory maintenance, either based solely on the neural dynamics, or on corrective signals arising from a drift of sensory inputs relative to the internally represented memory [23].
The study was supported by the European Research Council Synergy Grant no. 951319 ("KILONEU-RONS"), and by grant nos.1978/13, and 1745/18 from the Israel Science Foundation. We further acknowledge support from the Gatsby Charitable Foundation. Y.B. is the incumbent of the William N. Skirball Chair in Neurophysics. This work is dedicated to the memory of Mrs. Lily Safra, a great supporter of brain research.
[38] Alexander Seeholzer, Moritz Deger, and Wulfram Gerstner. Stability of working memory in continuous attractor networks under the control of short-term plasticity. Stability of working memory in continuous attractor networks under the control of short-term plasticity, 15(4):e1006928, 2018.
[39] Johannes D. Seelig and Vivek Jayaraman. Neural dynamics for landmark orientation and angular path integration. Nature, 521(7551):186-191, 2015.
[40] H Sebastian Seung. How the brain keeps the eyes still. Proceedings of the National Academy of Sciences, 93(23):13339-13344, 1996.
[41] W. E. Skaggs, J. J. Knierim, H. S. Kudrimoti, and B. L. McNaughton. A model of the neural basis of the rat's sense of direction. Advances in neural information processing systems, pages 173-180, 1995.
[42] Ben Sorscher, Gabriel C Mel, Samuel A Ocko, Lisa M Giocomo, and Surya Ganguli. A unified theory for the computational and mechanistic origins of grid cells. Neuron, 111(1):121-137, 2023.
[43] David Sussillo. Neural circuits as computational dynamical systems. Current opinion in neurobiology, 25:156-163, 2014.
[44] David Sussillo and Larry F Abbott. Generating coherent patterns of activity from chaotic neural networks. Neuron, 63(4):544-557, 2009.
[45] T. J. Wills, C. Lever, F. Cacucci, N. Burgess, and J. O'Keefe. Attractor dynamics in the hippocampal representation of the local environment. Science (New York, N.Y.), 308(5723):873-876, may 2005.
[46] Klaus Wimmer, Duane Q Nykamp, Christos Constantinidis, and Albert Compte. Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nature neuroscience, 17(3):431-439, 2014.
[47] Guangyu Robert Yang, Madhura R Joglekar, H Francis Song, William T Newsome, and XiaoJing Wang. Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience, 22(2):297-306, 2019.
[48] K. Yoon, M.A. Buice, C. Barry, R. Hayman, N. Burgess, and I.R. Fiete. Specific evidence of low-dimensional continuous attractor dynamics in grid cells. Nature neuroscience, 16(8):1077, 2013.
[49] K. Zhang. Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: A theory. Journal of Neuroscience, 16(6):2112-2126, 1996.