SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 3: The hippocampal model learns to path integrate on a 1D track using a ring attractor. a Position selective (place cell) inputs drive basal dendrites of the sensory layer p (HPC). b Hidden units (MEC) are connected to two sets of "conjunctive cells" which each connect back to one of the hidden neurons (g) and either the leftward (for  $\mathbf{g}_{v_L}$ ) or rightward (for  $\mathbf{g}_{v_L}$ ) velocity of the agent allowing velocity information to enter the network. Synaptic strengths of the return connections from the conjunctive cells to the MEC hidden units, as well as those for the MEC recurrent connectivity (collective denoted  $\mathbf{w}_{g_A}$ ), are randomly initialised and plastic. c After training, reordering the hidden units by the position of peak activity reveals a ring attractor in the synaptic weight matrices. Centre-surround recurrent connectivity stabilises an activity bump which is then "pushed" around the attractor manifold by asymmetric connections from the conjunctive cells, integrating velocity. Bands of zero weights show MEC neurons which have become perpetually inactive (aka "died"). The bottom panel displays the matrix row-averages, utilizing the circular symmetry of the environment to align rows before averaging. d Learning plateaus after 15 mins of simulated time. e Path integration ability is demonstrated in a lesion study: after 10 seconds in the normal oscillatory mode the network is placed into sleep mode (aka generative mode), lesioning the position-dependent sensory inputs. Despite this HPC continues to accurately encode position, evidence that the MEC ring attractor is path integrating the velocity inputs and sending predictions back to HPC. Lower panel shows the accumulated decoding error as well as the mean±SEM over 50 trials.
First, we set the dynamics of the model to "wake" mode ( $\theta = 1$ ) and measure the basal and apical activations of one of the sensory neurons for 60 seconds. Close correspondence (Fig. 2c) confirms that the network accurately "autoencodes" the high-dimensional sensory inputs through the compressed hidden layer. Since all activation functions are linear this implies that  $\mathbf{w}_{g_B}$  and  $\mathbf{w}_{p_A}$  are pseudoinverses. Next, we place the network in "sleep" mode ( $\theta = 0$ ) and allow the generative model to run freely. The autocorrelation of the generated hidden states ( $\mathbf{g}(t|\theta = 0)$ , displayed fully in the supplement) match that of the true environmental latents ( $\mathbf{z}(t)$ ), Fig. 2d, implying the generative model has statistics closely matching those of the true underlying generative process.
3.2 Learnable path integration with a hidden ring attractor
Next we turn our attention to the hippocampal formation's role in spatial navigation, and our central result. The environment consists of an agent randomly moving around a 1 m 1D circular track (motion and cell data is generated using the RatInABox package [55]). The basal compartment of each HPC neuron is spatially tuned to a single different Gaussian input however non-Gaussian
randomly spatially tuned inputs work as well (see supplement Fig. S2b):
$$
\left[ \mathbf {p} _ {B} (t) \right] _ {i} = \exp \left[ - \frac {\left(x (t) - x _ {i}\right)}{2 \sigma^ {2}} \right]. \tag {8}
$$
$x(t)$  is the position of the agent and  $\{x_i\}_{i=1}^{N_p}$  are the centres of the Gaussian inputs ( $\sigma = 6\mathrm{cm}$ ), intended to simulate hippocampal place fields, evenly spaced at  $1\mathrm{cm}$  intervals along the track. MEC (i.e. the hidden layer,  $\mathbf{g}(t)$ ) is matched in size  $N_g = N_p = 100$  with rectified tanh activation functions on both dendritic compartments ( $\sigma_{g_B}(x) = \sigma_{g_A}(x) = \max(0, \tanh(x))$ ) and HPC (the sensory layer  $\mathbf{p}(t)$ ) is linear ( $\sigma_{p_A}(x) = x$ ). Two populations of conjunctive cells (Fig. 3a & b) feed into the apical compartments of the MEC recurrent units. Random initialisation of  $\mathbf{w}_{g_B}$  means that MEC neurons start off with random non-Gaussian spatial tunings.  $\mathbf{w}_{g_A}$  and  $\mathbf{w}_{p_A}$  are also randomly initialised.
The network is trained for 30 minutes with learning plateauing after 15 (Fig. 3d). A lesion study, designed to test path integration, is then performed as follows: First, the network is run for 10 seconds normally (i.e. with theta-oscillating periods of wake and sleep). Since the simulated HPC neurons receive place-tuned inputs uniformly ordered along the track (i.e.  $x_{j} > x_{i}\forall i,j > i$ ) an activity heatmap of HPC reveals a bump of activity accurately tracking agent's position (Fig. 3e, left). The network is then placed into a sleep phase ( $\theta = 0$ ) for 20 seconds. This amounts to a full sensory lesion since top-down MEC inputs, not bottom-up place-tuned sensory inputs, drive HPC. Despite the full sensory lesion, hippocampal activity remains approximately unperturbed and the activity bump continues to accurately track position, slowly accumulating errors (Fig. 3e right). Since our HPC layer has no recurrent connectivity it cannot support this post-lesion activity on its own. Instead feed-forward drive from an MEC ring attractor, which we turn our attention to now, is responsible for maintaining the HPC code.
To find the ring attractor we must first reorder the MEC cells. We do this according to the position of the peak of their receptive fields (defined in the supplement). After reordering, the recurrent connectivity matrix can be seen to have acquired a centre-surround connectivity profile. Nearby MEC cells were, on average, strongly and positively recurrently connected to one another. Those far apart weakly inhibit one another (Fig. 3c, left; band of strong positive weights along diagonal flanked by weak negative weights). This profile matches that of a quasi-continuous ring attractor: local excitatory and long-range inhibitory connections stabilise a bump of activity on the attractor manifold in the absence of sensory input [56]. Weights from the conjunctive cells acquired asymmetric connectivity (Fig. 3c, middle & right) skewed towards the velocity direction for which they are selective. These asymmetric connections enable conjunctive cells to "push" the activity bump around the manifold, integrating velocity (see supplement for a visualisation of the MEC bump attractor). Theoretical work on ring attractors has demonstrated that for accurate path integration the asymmetric weights must be proportional to the derivative of the symmetric weights [56], approximately observed here. A noteworthy observation is that some MEC neurons become perpetually inactive; this is a consequence of the fact that both top-down and bottom-up synapses into the hidden layer are plastic and can fall to zero (Fig. 3c bands of zero-weights) satisfying a trivial  $g_{A} = g_{B} = 0$  solution for minimising the prediction error. Despite this, not all MEC neurons die and the surviving subset are sufficient for path integration. In supplementary section 5.4.2 we discuss additional results showing when the network learns robust path integrate under a variety of plasticity, initialisation and noise manipulations.
Crucially, what sets this model apart from others [19, 20, 21, 22] is that the network is not optimized using a conventional path-integration objective and backpropagation. Instead, it has been demonstrated how path integration can naturally arise in a biologically constrained network subject to a much simpler (yet more broadly applicable) local objective, in cases where idiotetic velocity signals are available to the hidden layers.
3.3 Remapping: transfer of structural knowledge between environments
Finally, we demonstrate how our trained network can transfer structural knowledge – which here means the ring attractor and thereby path integration – between environments. We start by training the network as in section 3.2; the only difference is that for simplicity we choose to fix  $\mathbf{w}_{g_B} = \delta_{ij}$  giving rise to MEC representations which, like HPC, are unimodal (this constraint can be relaxed and, in the more general case, MEC units typically have multiple receptive fields, Fig S4d, reminiscent of grid cells). We then simulate a hippocampal “remapping” event by shuffling the sensory inputs to the HPC layer (Fig. 4a & b, top panel) and retraining the network for a further 30 minutes but this time
Figure 4: Remapping and transfer of structural knowledge between environments. a After training (as in Fig. 2) place cell inputs are shuffled to simulate a "remapping" event observed when an agent moves to a new environment. The agent then retrans for an additional 30 minutes: during this period internal MEC weights, and weights from the conjunctive cells to MEC are held fixed while MEC  $\leftrightarrow$  HPC weights remain plastic. b Receptive fields of the HPC and MEC neuronal populations at different stages in the experiment: Initially after remapping HPC and MEC inputs are randomised. MEC relearns rate maps as they were before remapping but with a constant phase shift. Note: neurons are ordered by the position of their peak activity on the track before remapping and this ordering is maintained in subsequent panels. c The error  $(\pm$  SEM over 50 trials) after 1 second of path integration is shown at different stages of the experiment. Although path integration is initially disrupted after remapping it recovers despite no relearning of the MEC synapses where the ring attractor is stored.
holding weights in the hidden layer,  $\mathbf{w}_{g_A}$ . Only the HPC  $\leftrightarrow$  MEC synapses ( $w_{g_B}$  &  $w_{p_A}$ ) remain plastic during retraining. Biologically this may be accounted for by the observation that cortical plasticity is substantially slower than hippocampal plasticity [57].
During biological remapping events place cells remap independently whereas grid cells remap en masse with entire modules shifting by the same constant phase [58]. This observation is reproduced in our model: after retraining MEC units regroup with receptive fields as they were before remapping but with a constant phase shift along the track. This re-emergence of structure occurs because the ring attractor seeds a bump of activity on the attractor manifold (during the "sleep" phases of retraining) onto which the shuffled HPC inputs then bind. Since nothing constrains where on the circularly symmetric attractor manifold this regrouping can initiate, only relative correlations, modulo a phase shift, are preserved.
Decoding error one second after a sensory lesion is tested just before remapping, just after remapping and after retraining (Fig. 4c). After the remapping path integration abilities temporarily disappear because the MEC ring attractor is still tuned to the old and invalid HPC receptive fields. After relearning – and despite no adjustments to the MEC weights,  $\mathbf{w}_{g_A}$ , where the ring attractor is stored – path integration recovers to almost the level before remapping. This differs substantially from other local models of path integration learning [35, 36] which don’t consider plasticity on the ring attractor inputs. In these models, adaptation to a new environment necessarily requires complete relearning of the ring attractor. Instead our model exploits the basic fact that movement (path integration) in one environment is fundamentally the same as in another, one must simply learn a new mapping to/from the ring attractor, “translating” it to fit the new sensory stimuli.
4 Discussion
We propose that the hippocampal formation resembles a Helmholtz machine, simultaneously learning an inference and generative model of sensory stimuli. Like previous models [23] medial entorhinal
cortex (MEC) sits hierarchically above the hippocampus (HPC) to which it sends generative predictions. Our model differs in the learning rules and neural dynamics: local prediction errors are minimised between distinct dendritic compartments receiving bottom-up and top-down signals. Theta oscillations regulate internal neural dynamics, switching the network between wake and sleep phases. In a navigation task our MEC model forms a ring attractor capable of path integration. Despite simple learning rules and dynamics our model retains key cognitive capabilities of the hippocampal formation including the ability to transfer knowledge across different sensory environments.
Local learning rules are commonly recognised as essential in biologically plausible learning algorithms [43]. However, the importance of learning scheduling – how neural systems coordinate or multiplex distinct phases of forward and backward information flow – is often overlooked [59]. Neural oscillations such as theta, hypothesized to temporally coordinate communication between neuronal populations [60], likely play an underexplored role in this regard (neural “bursting” has also been pointed out as a potential solution to multiplexing [61]). One advantage of the wake-sleep algorithm, which this study suggests neural oscillations can support, compared to forward and backward sweeps is that, during convergence, the two phases become highly similar, allowing learning to proceed without affecting perception.
While our discussion has primarily focused on theta oscillations as a mechanism for learning, they have also been proposed as a mechanism for short-range future prediction via so-called "mind-travel" [62]. During the latter phase of each theta cycle (i.e. the sleep phase) gain amplified velocity signals might rapidly drive the MEC activity bump along the manifold allowing the agent to assess nearby upcoming locations. This complimentary proposition could neatly integrate into the framework proposed here and emphasizes the need for further investigation into the multifaceted functions of neural rhythms within the hippocampal/entorhinal system.
Beyond theta oscillations, both faster gamma cycles [63] and the slower physiological states of sleep and wake [64] have been associated with learning. Based on our model we suggest a tentative hypothesis that theta oscillations may be favored due to an optimality criterion; whilst faster oscillations could be a mechanism to prevent extreme drift during sleep that might disrupt learning their frequency might by upper bounded biophysically by the neural time constants associated with the biophysical processes supporting dendritic gating the soma. These ideas, their relevance to other brain regions involved in generative learning, 2D spatial dynamics, and offline memory consolidation/replay remain exciting questions for future theoretical and experimental investigation.
(12):763-775, December 2015. doi: 10.1016/j.tins.2015.10.004. URL https://doi.org/10.1016/j.tins.2015.10.004.
[63] Kwan Tung Li, Junhao Liang, and Changsong Zhou. Gamma oscillations facilitate effective learning in excitatory-inhibitory balanced neural circuits. Neural Plasticity, 2021:1-18, January 2021. doi: 10.1155/2021/6668175. URL https://doi.org/10.1155/2021/6668175
[64] William E. Skaggs and Bruce L. McNaughton. Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science, 271(5257):1870-1873, March 1996. doi: 10.1126/science.271.5257.1870. URL https://doi.org/10.1126/science.271.5257.1870.