REMI: Reconstructing Episodic Memory During Internally Driven Path Planning
Authors: Zhaoze Wang
Genela Morris
Dori Derdikman
Pratik Chaudhari
Vijay Balasubramanian
Venue: NeurIPS 2025 poster ()
Keywords: grid cells; place cells; hippocampus; medial entorhinal cortex; spatial navigation and planning
Primary area: neuroscience_and_cognitive_science
TL;DR: We propose REMI, a unified, system-level theory for cue-triggered goal retrieval, path planning, and reconstruction of sensory experience along a planned route using grid cells, place cells, and sensory cells.

Abstract: Grid cells in the medial entorhinal cortex (MEC) and place cells in the hippocampus (HC) both form spatial representations. Grid cells fire in triangular grid patterns, while place cells fire at specific locations and respond to contextual cues. How do these interacting systems support not only spatial encoding but also internally driven path planning, such as navigating to locations recalled from cues? Here, we propose a system-level theory of MEC-HC wiring that explains how grid and place cell patterns could be connected to enable cue-triggered goal retrieval, path planning, and reconstruction of sensory experience along planned routes. We suggest that place cells autoassociate sensory inputs with grid cell patterns, allowing sensory cues to trigger recall of goal-location grid patterns. We show analytically that grid-based planning permits shortcuts through unvisited locations and generalizes local transitions to long-range paths. During planning, intermediate grid states trigger place cell pattern completion, reconstructing sensory experiences along the route. Using a single-layer RNN modeling the HC-MEC loop with a planning subnetwork, we demonstrate these effects in both biologically grounded navigation simulations using RatatouGym and visually realistic navigation tasks using Habitat Sim.

REMI: Reconstructing Episodic Memory During Internally Driven Path Planning
Zhaoze Wang! zhaoze@seas, upenn. edu
Genela Morris 2.39 genelam@tlvmc. gov.
Dori Derdikman derdik@trehmionn.cc.i
Pratik Chaudhari
Vijay Balasubramanian 5.6.71 vijay@physics. upwcs. edu
pratikac@seas upenn. edu
'Dept. of Electrical and Systems Eng., Univ. of Pennsylvania
"Tel Aviv Sourasky Medical Center *Gray Faculty of Medical and Health Sciences, Tel Aviv University
'Rappaport Faculty Medicine, Technion Israel Institute of Technology Deept. of Physies, Univ. of Pennsylvania Santa Fe Institute Rudolf Peierls Centre for Theoretical Physics, University of Oxford Equal contribution
Abstract
Grid cells in the medial entorhinal cortex (MEC) and place cells the hippocampus (HC) both form spatial representations. Grid cells fire in triangular grid patterns, while place cells fire at specific locations and respond to contextual cues. How do these interacting systems support not only spatial encoding but also internally driven path planning, such as navigating to locations recalled from cues? Here, we propose a system-level theory MEC-HC wiring that explains how grid and place cell patterns could be connected to enable cue-triggered goal retrieval, path planning, and reconstruction of sensory experience along planned routes. We suggest that place cells autoassociate sensory inputs with grid cell patterns, allowing sensory cues to trigger recall of goal-location grid patterns. We show analytically that grid-based planning permits shortcuts through unvisited locations and generalizes local transitions long-range paths. During planning. intermediate grid states trigger place cell pattern completion, reconstructing sensory experiences along the route. Using single-layer RNN modeling the HC-MEC loop with a planning subnetwork, we demonstrate these effects in both biologically grounded navigation simulations using RatatouGym and visually realistic navigation tasks using Habitat Sim. Codes for experiments, simulations, and vision encoder available at 1,2,3,
Introduction
Grid and place cells the medial entorhinal cortex (MEC) and hippocampus (HC) form complementary
spatial codes for navigation [1-3]. Grid cells (GCs) in MEC fire in periodic triangular patterns as
animals move through space [4-6], spanning multiple spatial scales [7-10] potentially shaped by inhibitory gradients within attractor networks [1]]. Path integration theories propose that GCs track position by integrating movement [4, 12, 13), and recurrent neural networks (RNNs) trained for this task similarly develop grid-like activity [14-18]. Hippocampal place cells (HPCs) fire specific
Projecct Page: https://zhaozewanng..o http://aaozwwaaggggtuub...oooo iofremi
GRatatouGym (Simulation Suite): https///rattoougym githuub..i BBotleneck MAE:
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

locations [1, 2] and remap across contexts [19-23]. forming sparse, orthogonal representations that minimize contextual interference [24-26]. They auto-associate sensory sequences relayed through MEC during exploration [27-30], linking related memories via continuous attractor dynamics [31-34], and emerge in RNNs that auto-encode spatial experience [21-23].
36], it remains unclear how these representations support internally driven navigation, such as recalling goals from partial cues and planning novel routes. This ability is fundamental to flexible, goal-directed behavior, allowing animals to adapt to novel situations without extensive relearning. Evidence from rodent rest and sleep shows that hippocampal sequences replay past trajectories and generate novel ones through unvisited locations [37, 38], suggesting an underlying mechanism for offline planning.
Despite extensive study of grid and place cell function [14, 16, 18,
Two complementary frameworks have been proposed to explain planning- The successor representation (SR) posits that hippocampal place cells encode expected future occupancy [39, 40], enabling flexible replanning when goals change [41, 42] and supporting generalization across related tasks through reusable predictive representations [43]. However, SR models discretize space into learned transition states, requiring experience at many locations and limiting interpolation to unvisited areas. Grid-cell models instead propose continuous, periodic codes that support vector-based navigation [44], with stable phase relationships providing context- mentext-indeeendent spatial metrics. Yet encoding pairwise relations among locations becomes intractable as environments scale, and pure grid-based planning cannot easily capture context-speciffe constraints or hippocampal replay phenomena. Each framework offers a partial explanation of the problem, but how the brain integrates context-dependent memory with context-inees inddeedddddeennentt spatial metrics remains unresolved.
Here, we propose a unified theory of peapocampal-entoriing circuitry that explains how internally driven navigation arises from the interaction between place and grid cells. This specific hippocampal- entorhinal wiring can enable cue- cue-triggered goal retrieval and grid- based planning. Building on recent work showing that place cells autoassociatively encode sensory inputs [22], we propose that they also associate these inputs with GC activity. This coupling supports bidirectional pattern completion, whereby sensory cues can reactivate corresponding grid representations to guide planning, and grid states reconstruct the expected sensory experiences along planned routes.
We verified these effects using a single RNN trained to autoencode sensory inputs, integrate velocity signals for path integration, and form autoassociative links between sensory and grid representations. Through training, the RNN developed an internal representation of the navigable space. To enable planning, we expanded the RNN's hidden layer to include an additional planner subnetwork that drives the encoder's s dynamics with internally generated action sequences, enabling the system to traverse imagined paths between the current location and recalled goal. To test whether this framework could generalize to realistic navigation, we replaced simulated sensory patterns with visual features from panoramic images in photorealistic environment (Habitat-Sinn [4S-47]), encoded using modified Masked Autoencoder (MAE) [48]. During planning, as the network traversed imagined trajectories. intermediate sensory states were decoded by the MAE into images that closely matched the expected views at corresponding locations.
Our theory predicts that (a) during planning, HC-MEC coupling will induce "look forward" sweeps in MEC spatial representation, resembling recent experimental results [49], (b) when presented with sensory cues associated with distant locations, grid cell response patterns corresponding to those locations will be reactivated, and (c) disrupting MEC-to-HC projections should impair goal-directed navigation, while disrupting HC-to-MEC feedback should reduce planning accuracy.
2 Method
Conceptual Framework of the HC-MEC Loop. We construct a conceptual framework showing how known spatial cell types could integrate into unified system supporting both mapping and internally driven planning. Our theory builds on four established theories; (a) GCs arise from integrating velocity sequences for localization [4, 12-18]; (b) spatially modulated cells (SMCs) in MEC respond to external sensory stimuli [27] and relay sensory inputs to hippocampus; (c) HPCs autoassociate noisy sensory inputs from SMCS to reconstruct denoised sensory experience during recall [21, 22, 50-52]; and (d) while HPCs may also support planning, GCs provide a more efficient substrate for planning through their continuous, context-independen encoding [44].

---

From
Place
HC
Speed Colls
C
Grid Colle
Direction Gate
Spetitty Modudined (8MG)
Figure 1: (a) RNN model of the HC-MEC loop. The top subnetwork contains HPCs, with example emergent place fields shown on the left. The bottom subnetwork includes partially supervised GCs, as well as supervised speed, head direction, and spatially modulated cells (SMCs); example grid fields shown at left. (b) & (d) Speed cells and head direction cells are denoted as Spd and Dir, respectively. Colored regions highlight within-group recurrent connectivity indicate the partitioning of the connectivity matrix by cell groups. However, at initialization, no structural constraints are enforced. The full connectivity matrix is randomly initialized.
(b) Illustration of the path-integration network's connectivity matrix. (c) The network is trained to path-integrate 58 trials and tested on 10s trials (L 100.50 + 8.49 cm); the grid fields remain stable even in trials up 120s (L 1207.89 + 30.98 em). For each subpanel (1Os, 120s): top row shows firing fields; bottom row shows corresponding autocorrelograms. (d) Illustration of RNN connectivity matrix of full HC-MEC loop. (e-f) Example place fields (emergent) and grid fields in the full HC-MEC RNN model.
Since both GCs and SMCs are present in MEC and both project to hippocampus, GCs may be regarded as special form of sensory response derived from proprioceptive signals. This anatomical arrangement suggests core theoretical contribution, extending (c) by proposing HPCs autoassociate and pampeee-complete not only SMCs but also GCs. When sensory cues are unreliable and path integration drifts time due noise, this association provides natural solution for more robust localization than either system alone. This framework directly yields three key predictions: (1) partial sensory cues can reactivate the corresponding grid cell state via hippocampal autoassociation; (2) the recalled grid state can guide planning the grid cell manifold, enabling efficient, context- independent, and generalizable path computation; (3) during planning, intermediate grid states trigger hippocampal reactivation of the corresponding sensory representations, reconstructing the expected sensory experience along the planned route.
An RNN Model of the HC-MEC Loop. Our theoretical framework requires an RNN that can simultaneously (1) autoencode masked and noisy sensory experiences (SMCs), (2) contain grid cells capable of path-integrating noisy movement inputs, (3) form autoassociative links between SMC and GC responses, and (4) later support planning. To achieve these, we extend RNN formulations from previous models of spatial navigation cell types [14, 16, 18, 22, 53]. Specifically, consider a standard RNN that updates its dynamics as:
+1 (1)a) a) (Wim Wreef(zz)
++1
where z E dRd= is the hidden state, the input, a is the forgetting rate, and Win. Wree are the input and recurrent weight matrices. The output is given by linear readout: y/ Wout E &Rd We extend this RNN by introducing auxiliary input and output nodes, 2 and 20, and update as:
2++1
rre
W
(137
+11-a) a
u:
W
(2
W(23)
(2)
Wrout
(22)
(aa)

---

Here, 2' directly integrates the input us without a learnable projection, while 20 is probed and supervised to match simulated ground-truth cell responses. This design eliminates the need for projection matrices W and Wow Instead, Win and Wour act as surrogate mappings for the original projections. We set
learnable vector in Rd++d+tdo to allow different cells to have distinct forgetting rates, with O denoting element- wise multiplication.
Supervising Spatially Modulated Cells (SMCs). The SMCs are set as both input and output nodes, trained to reproduce the simulated ground truth. At each time step, SMC units receive simulated but partially occluded sensory inputs representing noisy environmental observations. After each recurrent update, the network is expected to reconstruct the corresponding noiseless sensory pattern and is penalized for deviations of the SMC subpopulation from the ground truth responses. These SMCs are assumed to primarily respond to sensory cues during physical traversal. Supervision constrains their dynamics to reflect tuning to these signals, while learned recurrent connections formed during training are intended to reflect Hebb-like updates in the brain that preserve this tuning structure.
Supervising Grid Cells for Path-Integration. Our theoretical model also requires grid cells that have learned to perform path integration. To achieve this, we assign two additional subpopulations within the RNN's hidden units to represent speed and allocentric head direction cells. Both subpopulations are defined as input nodes that only receive external inputs. Head direction cells are assigned preferred allocentric directions uniformly distributed over [0, 2Tr) with a fixed angular tuning width, ensuring non- non-negative responses and a population activity that lies on a ID ring (see Suppl. 3). We also simulate ground-truth grid cell activity maps. To train the network to perform path integration, the model is provided only with the initial grid cell responses at = O. For all subsequent timesteps, the network must infer the grid cell activity from the initial response together with the ongoing inputs from the speed and head direction cells.
This path integration component can be viewed as modified version of the networks described in [14-16, 18]. To verify that the network indeed learned path integration, we first trained model containing only grid, speed, and head direction cells (Figure lb). We simulated six grid modules with spatial periods scaled by the theoretical optimal factor /e [54]. The smallest grid spacing is set to 30 cm, defined as the distance between two firing centers of a grid cell [55]. The grid spacing to field size ratio is 3.26 [56]. with firing fields modeled as Gaussian blobs with radii equal to two standard deviations. We train this model on short random trajectories (5 s) but accurately path-integrate over significantly longer trajectories (10 s, 120 s) during testing (Figure Ic).
Emergence of Place Cell-Like Patterns During Autoassociation of SMCs and GCs. The central idea of our HC-MEC model is that localization using sensory inputs (SMCs) or through path integration alone can each fail under noise. Associating the two representations through hippocampal place cells makes localization more robust. To model this, we introduced a subpopulation of hidden units in the RNN that are neither input nor output neurons. They do not directly receive external signals or produce outputs. These units receive input from and project to both the SMC and GC subpopulations through recurrent connections (Figure 1d). After training the full HC-MEC network, with SMC units supervised to autoencode masked sensory inputs and GC units trained to path integrate noisy movement signals, we observed the spontancous emergence of place cell-like activity in this intermediary region. No explicit supervision or spatial constraint was imposed, and the place-like patterns emerged naturally through training, consistent with previous findings [22].
3 Recalling MEC Representations from Sensory Observations
We test our first prediction that associating sensory observations (SMC responses) with GC patterns enables recall of those patterns from partial sensory cues. Auto-association arises when input patterns incomplete degraded. To model this, we trained nine identical networks differing only in masking ratio Tmusk (0.1 to 0.9), which specifies the maximum fraction of head direction, speed, GC, and SMC inputs and initial hidden states randomly set to zero during training. Each model was trained on randomly sampled short trajectories using a fixed T'ask. with new masks generated for every trajectory and varying across time and cells. Masks were applied to both inputs and initial states of GCs, SMCs, speed cells, and head direction cells (Suppl. 5.2).
After training, we randomly selected locations in the environment and sampled the corresponding ground-truth SMC responses. Each sampled response was repeated T times to form a query. During

---

SMC
GC
HPC
Frass 0.1
fresk 0.9
B
60
69
50
50
5.50
00
30
Diebnnce
30
PC3
Pg
0
0.0
Query Time
2.0
4.0
0.8
Query Time (8)
2.0
4.8
0.0
Quary
2.0
SMC + GC
10
014
Fousa 001
PCB
2
PC
SMC
syece
yeu
TVD ira
Frac: UM
PC3
204 jM
PC
GC
C.
PC
b
Figure 2: Recalling MEC representations from sensory observations with networks trained under different masking ratios Tmask (a-d) Results from querying the trained network with fixed sensory input. (a) L2 distance between decoded and target positions using SMCs, GCs, and HPCs. (b) Top: number of identified HPCs vs. Tmuk (max 512). Bottom: number of active hippocampal units vs. m'akk (max 512). (c) Decoded positions from SMC, GC, and HPC population responses. (d) Example recall trajectories for SMC, GC, and their concatenation. Semi-transparent surfaces show PCA-reduced ratemaps (extrapolated 5x for visualization) from testing. Trajectories are colored by time; green dot marks the target.
queries, the network state was initialized to zero across all hidden-layer neurons, and the query was input only to the spatial modulated cells, while responses from all cells were recorded over 7' timesteps. At each timestep, the activity of subpopulation of cells (e.g.. SMC, GC, HPC) was decoded into a position by finding the nearest neighbor on the corresponding subpopulation ratemap aggregated during testing (S.ppl.5.4). Nearest neighbor search was performed using FAISS [57, 58] (Suppl.44.
Figure 2a shows the L.2 distance between decoded and ground-truth positions over time. The network was queried for seconds (100 timesteps), and all models successfully recalled the goal location with high accuracy. HPCs were identified as neurons with mean firing rates above 0.01 Hz and spatial information content (SIC) above 20 (Suppl. 2). The number of HPCs increased with Tmask, and location decoding using HPCs performed only for models with more than 10 HPCs. We observe a trade-off in which higher Tmaak leads to more HPCs and improved decoding accuracy but reduces the network's ability to recall sensory observations (Figure 2a.b).
To visualize the recall process (Figure 2c), we conducted Principal Components Analysis (PCA) on the recall trajectories. We first flattened the L. x Ly N ground-truth ratemap into (Lx Ly) N matrix, where L2 and Ly are the spatial dimensions of the arena and N is the number of cells in each subpopulation. PCA was then applied to reduce this matrix to (I.g I.y) x 3, retaining only the first three principal components. The trajectories (colored by time), goal responses (green dot), and ratemaps collected during testing were projected into this reduced space for visualization. In Figure 2d, the recall trajectories for all subpopulations converge indicating successful retrieval of target SMC and GC patterns. Once near the target, the trajectories remain close and continue to circulate around indicating stable dynamics during the recall process.
the target representation,
4 Planning with Recalled Representations
The recall experiment shows that aoooassooiiiinggggg spatially modulated cells (SMCs) and grid cells (GCs) through hippocampal place cells (HPCs) enables recovery of all cells' representations from
5

---

partial sensory cues. Although planning with HPCs and SMCs is feasible [40-43], their context dependence limits generalization across environments. contrast, GC patterns encode the geometric structure of space and provide context tanteettindeeeedeent substrate for planning. Planning the grid manifold allows HPC auto-association to reconstruct corresponding SMC representations from intermediate GC states, removing the need to plan directly with HPCs or SMCs. We therefore propose that planning occurs on the grid cell manifold, with sensory details later recovered through hippocampus auto-associations.
4.1 Decoding Displacement from Grid Cells
We first revisit and reframe the formalism in [44]. Grid cells are grouped into modules based on shared spatial periods and orientations. Within each module, relative phase relationships remain stable across environments [10, 55, 19,69, 60]. This stability allows the population response of grid module to be represented by single phase variable o [44], which is a scalar in ID and a 2D vector in 2D environments. This variable maps the population response onto an n-dimensional torus [61], denoted as T = R /2 = [0, )",",, where (1,2) is the dimension of navigable space.
Consider a ID case with de and d, as the phase variables of the current and target locations in some module. The phase difference is AØ oi 9.. and since o.. [0, 2.r). we have Ao (-2m, 2 However, for vector-based navigation, we instead need AΦ" such that or [ + A 2n where H2z is an element-wise modulo operation so that Φe is defined on [0, 2T)). Simply using AΦ directly is not sufficient because multiple wrapped phase differences correspond to the same phase Φt, but different physical positions on the torus. Therefore, we restrict Aő to be defined on (-H, T such that the planning mechanism always selects the shortest path the torus that points to the target phase. The decoded displacement in physical space is then d E E/2)-
For 2D space, we define Aø E I on (? by treating the two non-collincar directions as independent 1D cases. In Figure 3a, the phase variables de and Φb correspond to two points on a 2D torus. When unwrapped into physical space, these points repeat periodically, forming an infinite lattice of candidate displacements (Figure 3b). In 2D, this yields four (22) distinet relative positions differing by integer multiples of 2n in phase space. Only the point of lies within the principal domain (-T, ))² and the decoder selects Ao E E(- (-R, that minimizes | subject to Φe- = [de + AΦl
4.2 Sequential Planning
Previous network models compute displacement vectors from GCs by directly decoding from the current de and the target di [44]. However, studies show that during quiescence, GCs often fire in coordinated sequences tracing out trajectories [62, 63], rather than representing single, abrupt movements toward the target. At minimum, long-distance displacements are not directly executable and must be broken into smaller steps. What mechanism could support such sequential planning?
We first consider simplistic planning model single grid module. Phase space can be discretized into N₂ bins, grouping GC responses into N₂ discrete states. Local transition rules can be learned even during random exploration, allowing the animal to encode transition probabilities between neighboring locations. These transitions can be compactly represented by a matrix T E R.V.., where Tr gives the probability of transitioning from phase to phase j. With this transition matrix, the animal can navigate to the target by stitching together local transitions, even without knowing long-range displacements. Specifically, suppose we construct a vector yplan E IR No with nonzero entries marking the current and target phases to represent a planning task. Multiplying plan by matrix T propagates the current and target phases to their neighboring phases, effectively performing a "search" over possible next steps based on known transitions.
By repeatedly applying this update, the influence of the current and target phase spreads through phase space, eventually settling on an intermediate phase that connects start and target (Figure 3c). the animal selects the phase with the highest value after each update and renormalizes the vector, this process traces a smooth trajectory toward the target (Figure 3d). This approach can be generalized to 2D phase spaces (Figure 3e). In essence, we propose that the animal can decompose long-range planning into sequence of local steps by encoding transition probability over phases in matrix. A readout mechanism can then map these phase transitions into corresponding speeds and directions and subsequently update GC activity toward the target.

---

Max
9x
Phoaee Ircdes
Prazee
25
50
15
100
25
5o
75
100
Trajectoyy Trejeckey
L-0
L- 15
Prom
Tuibe
Figure 3: (a) The population response of single grid module forms n-dimensional torus, where multiple phase differences can connect the current and target phases. (b) Unwrapping phases into physical space yields 2 candidate displacements; only of lies within the principal domain (-#, ) (c) A Markovian process asymptotically identifies the most likely next phase that moves closer to the target. (d) Renormalizing after each update produces smooth trajectory from start to target. (e) Illustration of this process in 2D space. (f)) Learned connectivity matrix of the planning RNN using only grid cells. (g) Planned trajectories for targets reachable within 10 and 20 seconds. Blue and red crosses mark start and target locations; the reference line shows the full trajectory for visualization. The dots represented the decoded locations.
(h) planning network connected the full HC-MEC, receiving input only from GC and controlling speed and direction, drives SMC responses to update alongside GC, tracing a trajectory closely aligned with the planned GC path.
4.3 Combining Decoded Displacement from Multiple Scales
Our discussions of planning and decoding in sections 4.1 and 4.2 were limited to displacements within single grid module. However, this is insufficient when AΦ exceeds half the module's spatial period (f/2). The authors of [44] proposed combining Ań across modules before decoding displacement. instead suggest decoding Ae within each module first, then averaging the decoded displacements across modules is sufficient for planning. This procedure allows each module to update its phase using only local transition rules while still enabling the animal to plan a complete path to the target.
We start again with the ID case. Suppose there having a spatial period L, From the smallest to the largest scale, these spatial periods are fi...., The grid modules follow fixed scaling factor s, which has a theoretically optimal value of , lm- in 1D rooms and Ve in 2D [54]. Thus, the spatial periods satisfy 4; tu sf for i = 1..... m. where lo a constant parameter that does not correspond to an actual grid module.
different scales of grid cells, with each scale i
Given the ongoing debate about whether grid cells contribute to long-range planning [64, 65], we focus on mid-range planning, where distances are of the same order of magnitude as the environment size. Suppose two locations in ID space are separated by ground truth displacement d E IR,, bounded by half the largest scale (2). We can always find an index & where (1/2.... 1/2 ...,, fn/ Given undercovered. For undercovered scales, phase differences AΦ are wrapped around the torus at least one period of (-T,, æ) and may point to the wrong direction. We thus denote the phase difference due to the undercovered scales by 2g. If we predict displacement by simply averaging the decoded displacements from all grid scales, the predicted displacement
S call scales f ., Lm decodable and scales f,..... Cx
d= Eo 2mm
(Ei_ s² .2..Z,
Σm
Ao:)
In ID, the remaining distance after taking the predicted displacement dnext deurentt d. For the predicted displacement to always move the animal closer to the target, meaning daxt deurreats
7

---

suffices that mi > k + I-x-" 1-8-" < 1 requiring only m > k. In 2D, where the optimal scaling factor Ve. the condition tightens slightly to m > k + 1. Importantly, as the animal moves closer to the target, more scales become decodable, enabling increasingly accurate predictions that eventually lead to the target. In 2D, planning can be decomposed into two independent processes along non-collinear directions. Although prediction errors in 2D may lead to suboptimal path, this deviation can be reduced by increasing the number of scales or taking smaller steps along the decoded direction, allowing the animal to gradually approach the target with higher prediction accuracy.
s-1 (see Suppl. 1). This condition is trivially satisfied in 1D for as
4.4 A RNN Model of Planning
Planning Using Grid Cells Only. We test our ideas in an RNN framework. We first ask whether a planner subnetwork, modeled together with a GC subnetwork, can generate sequential trajectories toward the target using only grid cell representations. Accordingly, we connect a planning network to pre- pre-trained GC network that has already learned path integration. For each planning task, the GC subnetwork is initialized with the ground truth GC response at the start location, while the planner updates the GC state from the start to the target location's GC response by producing a sequence of feasible actions-Speciiienn specificallyy speeds and directions. This ensures the planner generates feasible actions rather than directly imposing the target state on the GCs, while a projection from the GC region to the planning region keeps the planner informed of the current GC state. The planner additionally receives the ground truth GC response of the target location through a learnable linear projection. At each step, the planner receives Wi g"" + W "2 Rdr, where g" and Ir are the goal and current GC patterns, while W and W are the input and GC-to-planner projection matrices. This combined input has the same dimension as the planner network. Conceptually, can be interpreted as the planning vector uPlun, while the planner's recurrent matrix represents the transition matrix T. The resulting connectivity matrix is shown in Figure 3f. During training, we generate random I-second trajectories to sample pairs of current and target locations, allowing the animal to learn local transition rules. These trajectories are used only to ensure that the target location is reachable within second from the current location; the trajectories themselves are not provided to the planning subnetwork. The planning subnetwork is trained to minimize the mean squared error between the current and target GC states for all timesteps.
For testing, we generate longer 10 and 20 second trajectories to define start and target locations, again without providing the full trajectories the planner. The GC states produced during planning are decoded at each step to infer the locations the animal is virtually planned to reach. As shown in Figure 3g, the dots represent these decoded locations along the planned path, while the colored line shows the full generated trajectory for visualization and comparison. We observe that the planner generalizes from local to long-range planning and can take paths that shortcut the trajectories used to generate start and target locations. Notably, even when trained on just 128 fixed start-end pairs over 100 steps, it still successfully plans paths between locations reachable over 10 seconds.
Training the Planning Network Together with HC-MEC Enables Reconstruction of Sensory Experiences Along Planned Paths. We next test whether the HC-MEC loop enables the network to reconstruct sensory experiences along planned trajectories using intermediate GC states. To this end, we train the planning subnetwork together with the entire HC-MEC model that was pre-trained to encode the room during random traversal (fmask = 1.0, see Suppl. 5.2). We fix all projections from SMCs and HPCs the planner to zero. This ensures that the planner uses only GC representations for planning and controls HC-MEC dynamics by producing inputs to speed and direction cells. SMCs and GCs are initialized to their respective ground truth responses at the start location.
Using the same testing procedures as before, we sampled the SMC and GC responses while the planner generated paths between two locations reachable within 10 seconds. We decoded GC and SMC activity at each timestep to locations using nearest neighbor search their respective ratemaps. We found that the decoded SMC trajectories closely followed those of the GCs, suggesting that SMC responses can be reconstructed from intermediate GC states via HPCs (see Figure 3h). Additionally, compared to the GC-only planning case, we reduced the number of GCs in the HC-MEC model to avoid an overly large network, which would make the HC-MEC training hard to converge. Although this resulted in a less smooth GC-decdded trajectory than in Figure 3g, the trajectory decoded from
8

---

(Pre)-Tralning BtnkMAE
Collecting Sensory Experiences From Habitat
tokon
tokon
MAE Encoder
lsarmable petich token
MAE Decoder
BinkMAE Encoder
penorama img location X.
feature map
Encoder
BIHAMAE
0
2
Figure 4: (a) (Pre-)training process the BtnkMAE that uses a modification of the Masked AutoEncoder in [48]. Image features are ignored, and only the CLS token is passed to the decoder. The decoder cross-attends to the CLS token with learnable embeddings to reconstruct the image tokens such that each image be faithfully encoded into 1024 dimensional features. See Suppl. 7. (b) Collection of SMC rate maps from Habitat Sim. (c) Decoding intermediate SMC states during path planning yields images that closely match the expected views along the planned route.
SMCs was noticeably smoother. We suggest this is due to the auto-associative role of HPCs, which use relayed GC responses to reconstruct SMCs, effectively smoothing the trajectory.
Testing REMI for Realistic Navigation Task in Habitat Sim. Finally, we tested whether the proposed framework generalizes to realistic navigation tasks and whether the reconstructed sensory embeddings not only correspond to the correct locations along the planned path but also retain sufficient information to recover the original sensory observations. To this end, we replaced the simulated SMC rate maps Rame E x...x.A L.y with visual feature embeddings extracted from the Habitat Synthetic Scenes Dataset (HSSD) [66] in the photorealistic simulator Habitat-Sim [45-47]. The new SMC rate map collected in Habitat-Sim is Riab
R
l. x x 1024
To construct Rhah. captured panoramic images 7 at each spatial location with fixed orientation and encoded them using vision encoder that produced 1024-dimensional feature vectors E(T) E IR. 1024 We additionally needed decoder D ((E x I to transform the encoded image back to the pixel space. For this purpose, we modified a pretrained Masked Autoencoder (MAE) [48] with a ViT-Large backbone, referred to BtnkMAE. The original MAE produced image features of size Rexd where is the number of patches and d the embedding dimension, which is unsuitable for our HC-MEC model that requires a single vector representation per image. To address this, we pretrained BtnkMAE on ImageNet- 1k, retaining only retrained CLS token after the encoder to represent each image before decoding. The decoder then employed DETR-style cross-attention [67] to reconstruct the image from this single embedding vector (Figure 4a; see also Suppl. for details).
We then repeated the experiment from the previous section with an additional layer normalization to stabilize training. During planning, the HC-MEC updated the SMC region intermediate states
20: 27, where 2, R|024 denotes the population firing statistics of the SMC subregion. Decoding them with D yielded images To,..., planned locations (Figure 4c).
..... Jr that closely resembled the expected views at the corresponding

---

5 Discussion
Decades of theoretical and computational studies have explored how hippocampal place cells (HPCs) and grid cells (GCs) arise, addressing the question "What do they encode?" An equally important question is "Why does the brain encode?" We suggest that place and grid representations evolved to support internally driven navigation toward vital resources such as food, water, and shelter. Thus, we ask: given their known phenomenology, how might HPCs and GCs be wired to support internally driven navigation?
Previous studies show that HPCs are linked to memory and context but exhibit localized spatial maps that lack the relational structure required for planning, while the periodic lattice of GCs supports path planning and generalization yet exhibits weak contextual tuning that limits direct recall from sensory cues. We propose that GCs and spatially modulated cells (SMCs) form parallel localization systems encoding complementary aspects of space: GCs provide metric structure and SMCs reflect sensory observations. HPCs link these systems through auto-association to enable bidirectional pattern completion. We built a single-layer RNN containing GCs, HPCs, SMCs, and a planning subnetwork to test whether this coupling enables three key capabilities: (1) recall of GC patterns from contextual cues, (2) generalizable GC-based planning to novel routes, and (3) reconstruction of sensory experiences along planned paths, eliminating the need for direct planning HPC SMC manifolds.
Our model is flexible and accommodates existing theories. In REMI, HPCs must predict both GC and SMC responses at the next timestep to pattern-complete. Since HPCs are direction-agmosticc accurately reconstructing GC and SMC activity requires encoding possible state transitions. Because adjacent locations tend to have higher transition probabilities [39], our model implicitly reflects the SR framework of HPCs, in which the transition structure is embedded in the pattern-completion dynamics. This predictive nature aligns with previous theories that predictive coding in HPCs supports planning [40], but we emphasize that using GCs may be more efficient when shortcutting unvisited locations. Lastly, while propose that HC-MEC coupling enables planning through recall and reconstruction, a parallel idea was proposed in [68], where similar coupling was argued to support greater episodic memory capacity.
Our theory makes several testable predictions. First, sensory cues should reactivate GC activity associated with a target location, even when the animal is stationary or in a different environment. Second, during planning, HC-MEC coupling will induce "look forward" sweeps in MEC spatial representation, resembling recent experimental results [49]. Finally, if hippocampal place cells reconstruct sensory experiences during planning, disrupting MEC-to-HC projections should impair goal-directed navigation, while disrupting HC-to-MEC feedback should reduce planning accuracy by preventing animals from validating planned trajectories internally.
First, existing emergence models of GCs make it difficult to precisely control GC scales and orientations [11, 17], and so avoid the complicating analysis of the simultaneous emergence of GCs and HPCs, we supervised GC responses during training. Future work on their co-emergence could further support our proposed planning mechanism. Second, our framework does not account for boundary avoidance, which would require extending the HC-MEC model to include boundary cell representations [35, 69, 70]. Finally, our discussion of planning with GCs assumes the environment of similar scale to the largest grid scale. One possibility is long-range planning may involve other brain regions [65], as suggested by observations that bats exhibit only local 3D grid lattices without a global structure [64]. Animals might use GCs for mid-range navigation, while global planning stitches together local displacement maps from GC activity.
Limitations:
Acknowledgments and Disclosure of Funding
The study was supported by NIH CRCNS grant IRO12H455444001 and in part by the NSF and DoD OUSD (R&E) under Agreement PHY-2229929 (The NSF AI Institute for Artificial and Natural Intelligence). Additional support was provided by the United States-Israel Binational Science Foundation (BSF). PC was supported in part by grants from the National Science Foundation (IIS- 2145164, CCF-2212519) and the Office of Naval Research (N00014-22-1-225, VB was supported in part by the Eastman Professorship at Balliol College, Oxford.
10

---

References
[1] Edvard I. Moser, Emilio Kropff, and May-Britt Moser. Place Cells, Grid Cells, and the Brain's Spatial Representation System. Annu. Rev. Neurosci, 31()):69-8,, July 2008.
[2] May-Britt Moser, David C. Rowland, and Edvard L. Moser. Place Cells, Grid Cells, and Memory. Cold Spring Harb Perspect Biol, 7(2):a02180) February 2015.
[3] Genela Morris and Dori Derdikman. The chicken and egg problem of grid cells and place cells. Trends in Cognitive Sciences, 27(2):225-138 February 2023.
[4] Torkel Hafting, Marianne Fyhn, Sturla Molden, May-Britt Moser, and Edvard I. Moser. Microstructure of a spatial map in the entorhinal cortex. Nature, 436(7052) August 2005.
[5] David C. Rowland, Yasser Roudi, May-Britt Moser, and Edvard I. Moser. Ten Years of Grid Cells. Annu. Rev. Neurosci. 39(1):19-40, July 2016.
[6] Edvard Moser, May-Britt Moser, and Bruce McNaughton. Spatial representation in the hippocampal formation: A history. Nat Neurosci, 20(1 (MM11):1448-144 November 2017.
[7] Caswell Barry, Robin Hayman, Neil Burgess, and Kathryn J Jeffery. Experiencee-dependnt rescaling of entorhinal grids. Nat Neurosci, 10(6):682 684, June 2007.
[8] Vegard Heimly Brun, Trygve Solstad, Kirsten Brun Kjelstrup, Marianne Fyhn, Menno P. Witter, Edvard I. Moser, and May-Britt Moser. Progressive increase in grid scale from dorsal to ventral medial entorhinal cortex. Hippocampus, 18( 12)2 December 2008.
[9] Hanne Stensola, Tor Stensola, Trygve Solstad, Kristian Frøland, May-Britt Moser, and Edvard I. Moser. The entorhinal grid map discretized. Nature, 492(7427):72-77, December 2012.
[10] Julija Krupic, Marius Bauza, Stephen Burton, Caswell Barry, and John O' symmetry is shaped by environmental geometry. Nature, 518(7538):332-235 February 2015,
Grid cell
[11] Louis Kang and Vijay Balasubramanian. geometric attractor mechanism for self- organization of entorhinal grid modules. eLife, 8, August 2019.
[12] Mark C. Fuhs and David S. Touretzky. A Spin Glass Model of Path Integration in Rat Medial Entorhinal Cortex. Neurosci, 26 26(16)::466-44777 April 2006.
[13] Yoram Burak and Ila R. Fiete. Accurate Path Integration in Continuous Attractor Network Models of Grid Cells. PLoS Comput Biol, 5(2) February 2009.
[14] Christopher J. Cueva and Xue-Xin Wei. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization, March 2018.
[15] Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin J. Chadwick, Thomas Degris, Joseph Modayil, Greg Wayne, Hubert Soyer, Fabio Viola, Brian Zhang, Ross Goroshin, Neil Rabinowitz, Razvan Pascanu, Charlie Beattie, Stig Petersen, Amir Sadik, Stephen Gaffney, Helen King, Koray Kavukcuoglu, Demis Hassabis, Raia Hadsell, and Dharshan Kumaran. Vector-based navigation using giid representations in artificial agents. Nature, 557(7705):429
May 2018.
[16] Ben Sorscher, Gabriel C Mel, Surya Ganguli, and Samuel A Ocko. A unified theory for the origin of grid cells through the lens of pattern formation. NeurIPS, 2019.
[17] Rylan Schaeffer, Mikail Khona, and Ila Rani Fiete. No Free I Lunch from Deep Learning in Neuroscience: A Case Study through Models of the Antorhinal-Hippocamne Circuit, August 2022.
[18] Rylan Schaeffer, Tzuhsuan Ma, Sanmi Koyejo, Mikail Khona, Cristóbal Eyzaguirre, and Ila Rani Fiete. Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells. 37th Conference on Neural Information Processing Systems (NeurIPS 2023), September 2023.
[19] Charlotte B. Alme, Chenglin Miao, Karel Jezek, Alessandro Treves, Edvard Moser, and May-Britt Moser. Place cells in the hippocampus: Eleven maps for eleven rooms. Proc. Natl. Acad. Sci. U.S.A., 111(52):
December 2014.
[20] John L. Kubie, Eliott R. J. Levy, and André A. Fenton. Is hippocampal remapping the physiological basis for context? Hippocampus, 30(8):851-864 August 2020.

---

[21] Marcus K. Benna and Stefano Fusi. Place cells may simply be memory cells: Memory compression leads to spatial tuning and history dependence. Proc. Natl. Acud. Sci. U.S.A., 118651))2008422211 18, December 2021.
[22] Zhaoze Wang, Ronald W. Di Tullio, Spencer Rooke, and Vijay Balasubramanian. Time Makes Space: Emergence of Place Fields in Networks Encoding Temporally Continuous Sensory Experiences. In NeurlPS 2024, August 2024.
[23] Markus Pettersen and Frederik Rogge. Learning Place Cell Representations and Context- Dependent Remapping. 38th Conference on Neural Information Processing Systems (NeurIPS
2024), September 2024.
[24] R. Monasson and S. Rosay. Transitions between Spatial Attractors in Place-Cell Models. Phys.
Rev. Lett., 115(9))09800, August 2015.
[25] Aldo Battista and Rémi Monasson. Supacity-Resoluttin Trade-Off in the Optimal Learning of Multiple Oo--imensional Manifolds by Attractor Neural Networks. Phys. Rev. Lett., 124(4):048302, January 2020.
[26] Spencer Rooke, Zhaoze Wang, Ronald W. Di Tullio, and Vijay Balasubramanian. Trading Place for Space: Increasing Location Resolution Reduces Contextual Capacity Hippocampal Codes. In NeurlPS 2024, October 2024.
[27] Amina A Kinkhabwala, Yi Gu, Dmitriy Aronov, and David W Tank. Visual cue-related activity of cells in the medial entorhinal cortex during navigation in virtual reality. eliife, 9:e43140, March 2020.
[28] Due Nguyen, Garret Wang, Talah Wafa, Tracy Fitzgerald, and Yi Gu. The medial entorhinal cortex encodes multisensory spatial information. Cell Reports, (3(10)1148813 October 2024.
[29]
Qiming Shao, Ligu Chen, Xiaowan Li, Miao Li, Hui Cui, Xiaoyue Li, Xinran Zhao, Yuying Shi, Qiang Sun, Kaiyue Yan, and Guangfu Wang. A non-canonical visual cortical-entorhinal pathway contributes to spatial navigation. Nat Commun, 15(1):4122, May 2024.
[30] Rajkumar Vasudeva Raju, Swaroop Guntupalli, Guangyao Zhou, Carter Wendelken, Miguel Lázaro-Gredine and Dileep George. Space is latent sequence: A theory of the hippocampus. Sci. Adv., 10(31 (:K3))eadm84700 August 2024.
[31] Alexei Samsonovich and Bruce L. McNaughton. Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model. J. Neurosci, 77(15):500-5922) August 1997.
[32] F. P. Battaglia and A. Treves. Attractor neural networks storing multiple space representations: A model for hippocampal place fields. Phys. Rev. E, 58))777738877753 December 1998.
[33] Misha Tsodyks. Attractor neural network models of spatial maps in hippocampus. Hippocampus, 944:4881449, 1999.
[34] E. T. Rolls. An attractor network in the hippocampus: Theory and neurophysiology. Learning & Memory, (4(11):714-731, November 2007.
[35] John O'Kefee and Neil Burgess. Geometric determinants of the place fields of hippocampal neurons. Nature, (6511111)))6 (:0881))4554444 428, May 1996.
[36] Jared Deighton, Wyatt Mackey, Ioannis Schizas, David L. Boothe Jr. and Vasileios Maroulas. Higher-Order Spatial Information for Self-Supervised Place Cell Learning, June 2024.
[37] George Dragoi and Susumu Tonegawa. Preplay of future place cell sequences by hippocampal cellular assemblies. Nature,
401, January 2011.
[38] H. Freyja Ólafsdöttir, Daniel Bush, and Caswell Barry. The Role of Hippocampal Replay in Memory and Planning. Current Biology, 28(1):R37-R50, January 2018.
[39] Kimberly L Stachenfeld, Matthew M Botvinick, and Samuel J Gershman. The hippocampus as a predictive map. Nat Neurosci, 20(11):1643-165 November 2017.
[40] Daniel Levenstein, Aleksei Efremov. Roy Henha Eyono, Adrien Peyrache, and Blake Richards. Sequential predictive learning is a unifying theory for hippocampal representation and replay, April 2024.
[41] Lila Davachi and Sarah DuBrow. How the hippocampus preserves order: The role of prediction and context. Trends in Cognitive Sciences, 19(2):22-99, February 2015,
12

---

[42] Peter Kok and Nicholas B. Brokn-B. Associative Prediction of Visual Shape in the Hippocampus. J. Neurosci, 38(31 1:688866899 August 2018.
[43] Andre Barreto, Will Dabney, Remi Munos, Jonathan J Humt, and Tom Schaul. Successor Features for Transfer in Reinforcement Learning. In NeurIPS, 2017.
[44] Daniel Bush, Caswell Barry, Daniel Manson, and Neil Burgess. Using Grid Cells for Navigation. Neuron, 87(3):507-520 August 2015.
[45] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh. and Dhruv Batra. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVE International Conference Computer Vision (ICCV), 2019.
[46] Andrew Szot, Alex Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Chaplot, Oleksandr Maksymets, Aaron Gokaslan, Vladimir Vondrus, Sameer Dharur, Franziska Meier, Wojeiech Galuba, Angel Chang, Zsolt Kira, Vladlen Koltun, Jitendra Malik, Manolis Savva, and Dhruy Batra. Habitat 2.0: Training home assistants to rearrange their habitat. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[47] Xavi Puig, Eric Undersander, Andrew Szot, Mikael Dallaire Cote, Ruslan Partsey, Jimmy Yang, Ruta Desai, Alexander William Clege, Michal Hlavac, Tiffany Min, Theo Gervet, Vladimír Vondruš, Vincent-Pierre Berges, John Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai, and Roozbeh Mottaghi. Habitat 3.0: A co-habitat for humans, avatars and robots. 2023.
[48] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked Autoencoders Are Scalable Vision Learners, December 2021.
[49] Abraham Z. Vollan, Richard J. Gardner, May-Britt Moser, and Edvard I. Moser. righ- alternating theta sweeps in entortinal-hippooampall maps of space. Nature, 63988056;995-10055 March 2025.
[50] Edmund Rolls. A computational theory of episodic memory formation in the hippocampus. Behavioural Brain Research, 215(2):180-19)) December 2010.
[51] Heekyung Lee, Cheng Wang, Sachin S. Deshmukh, and James J. Knierim. Neural Population
Evidence of Functional Heterogeneity along the CA3 Transverse Axis: Pattern Completion versus Pattern Separation. Neuron, 8755):1933-1105, September 2015.
[52] Edmund T. Rolls. Pattern separation, completion, and categorisation in the hippocampus and neocortex. Neurobiology of Learning and Memory, 129:4-28, March 2016.
[53] Dehong Xu, Ruiqi Gao, Wen-Hao Zhang, Xue-Xin Wei, and Ying Nian Wu. On Conformal Isometry of Grid Cells: Learning Distance-Pressrvving Position Embedding, February 2025,
[54] Xue-Xin Wei, Jason Prentice, and Vijay Balasubramanian. A principle of economy predicts the functional architecture of grid cells. eLife, 4:c08362, September 2015.
[55] Caswell Barry, Lin Lin Ginzberg, John Oeeefe, and Neil Burgess. Grid cell firing patterns signal environmental novelty by expansion. Proc. Nat.. Acad. Sci. U.S.A., 109(4)): October 2012.
[56] Lisa M. Giocomo, Syed Hussaini, Fan Zheng, Eric R. Kandel, May-Britt Moser, and Edvard I. Moser. Grid Cells Use HCNI Channels for Spatial Scaling. Cell, 11755)1159911777 November 2011.
[57] Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre- Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library. 2024.
[58] Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535-5477 2019.
[59] Alexandra T Keinath, Russell A Epstein, and Vijay Balasubramanian. Envirommental deforma- tions dynamically shift the grid cell spatial metric. elife, 7:e38169, October 2018.
[60] Noga Mosheiff and Yoram Burak. Velocity coupling of grid cell modules enables stable embedding of a low dimensional variable in a high dimensional neural attractor, elife, 8te48494, August 2019.
13

---

[61] Richard J. Gardner, Erik Hermansen, Marius Pachitariu, Yoram Burak, Nils A. Baas, Benjamin A. Dunn, May-Britt Moser, and Edvard Moser. Toroidal topology of population activity in grid cells. Nature, 602(7895):123-1222 February 2022.
[62] H Freyja Ólafsdóttir, Francis Carpenter, and Caswell Barry. Coordinated grid and place cell replay during rest. Nat Neurosci, 196))792-779, June 2016.
[63] J. O' D'Neill, C.N. Boccara, F. Stella, P. Schoenenberger, and Csicsvari. Superficial layers of the medial entorhinal cortex replay independently of the hippocampus. Science, 355(63211:184-188) January 2017.
[64] Gily Ginosar, Johnatan Aljadeff, Yoram Burak, Haim Sompolinsky. Liora Las, and Nachum Ulanovsky, Locally ordered representation of 3D space in the entorhinal cortex. Nature, 596(7872):40444000 409400444494000 August 2021.
[65] Gily Ginosar, Johnatan Aljadeff, Liora Las, Dori Derdikman, and Nachum Ulanovsky. Are grid cells used for navigation? On local metries, subjective spaces, and black holes. Neuron, 111(12)1188811855 June 2023.
[66] Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruy Batra, Alexander Clegg, Eric Undersander, Angel X. Chang, and Manolis Savva. Habitat Synthetic Scenes Dataset (HSSDD-00)): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation, December 2023.
[67] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-End Object Detection with Transformers, May 2020.
[68] Sarthak Chandra, Sugandha Sharma, Rishidev Chaudhuri, and Ila Fiete. Episodic and associative memory from spatial scaffolds in the hippocampus. Nature, January 2025.
[69] Trygve Solstad, Charlotte N. Boccara, Emilio Kropff, May-Britt Moser, and Edvard I. Moser. Representation of Geometric Borders in the Entorhinal Cortex. Science, 322(5909)) 1865)1866666166
December 2008
[70] Colin Lever, Stephen Burton, Ali Jeewajee, John O' Keefe, and Neil Burgess. Boundary Vector Cells in the Subiculum of the Hippocampal Formation. J. Neurosci., 29(31):9711-9777 August 2009.
[71] Mark Brandon, Andrew R. Bogaard, Christopher Libby, Michael A. Comnerney, Kishan Gupta, and Michael E. Hasselmo. Reduction of Theta Rhythm Dissociates Grid Cell Spatial Periodicity from Directional Tuning. Science, 332(6029) April 2011.
[72] Geoffrey W. Diehl, Olivia J. Hon, Stefan Leutgeb, and Jill K. Leutgeb. Grid and Nongrid Cells in Medial Entorhinal Cortex Represent Spatial Location and Environmental Features with Complementary Coding Schemes. Neuron, 94(1):83-92.c6 April 2017.
[73] James R. Hinman, Mark Brandon, Jason R. Climer, G. William Chapman, and Michael Hasselmo. Multiple Running Speed Signals in Medial Entorhinal Cortex. Neuron, 91(3):666-8 August 2016.
[74] Jing Ye, Menno Witter, May-Britt Moser, and Edvard I. Moser. Entorhinal fast-n speed cells project to the hippocampus. Proc. Natl. Acad. Sci. U.S.A., 115(7), February 2018.
[75] Francesca Sargolini, Marianne Fyhn, Torkel Hafting, Bruce L. McNaughton, Menno P. Witter, May-Britt Moser, and Edvard I. Moser. Conjunctive Representation of Position, Direction, and Velocity in Entorhinal Cortex. Science, 312(5774):758-766) May 2006.
[76] Laurenz Laurenz.Meessig,, Jonas Hauser, Thomas Joseph Wills, and Francesca Cacucci. A Developmental Switch in Place Cell Accuracy Coincides with Grid Cell Maturation. Neuron, 16(5):1
June 2015.
[77] Tale L. Bjerknes, Nenitha C. Dagslott, Edvard I. Moser, and May-Britt Moser. Path integration in place cells developing rats. Proc. Natl. Acad. Sei. U.S.A., 115(7), February 2018.
[78] Rosamund F. Langston, James A. Ainge, Jonathan J. Couey, Cathrin B. Canto, Tale L. Bjerknes, Menno P. Witter, Edvard I. Moser, and May-Britt Moser. Development of the Spatial Representation System in the Rat. Science, 328(5985):1566-5888 June 2010.
[79] Tom Wills, Francesca Cacucci, Neil Burgess, and John O' O'Keefe. Development of the Hippocampal Cognitive Map in Preweanling Rats. Science, 328(5985) June 2010.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
We have clearly stated the contribution and scope of the paper, and all contributions presented in the abstract and introduction are accurately discussed in the main text.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, we have discussed the limitations of the work the second to the last paragraph in the discussion section. For all the assumptions made in the method and derivation, we have clearly stated these assumptions as well as additional justifications using either arguments supports for previous works.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicaalle, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
15

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Yes, for all the theoretical results, we have provided the full set of assumptions and additional prooff/jusiifiationss the supplementary materials.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Yes, we have provided all the details of our network implementation, cell dynamics simulation, and training procedures. While we included an overview of this information in the main text due to space limitations, all necessary details for reproducibility are provided in the supplementary materials.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (c.g., in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
specific model and empirical evaluation, may
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
16

---

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access the model is limited in some way (e.g., to registered users), but should be possible for other researchers to have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have open-sourced all the codebases required to replicate our experiments. The code repositories are available at (1) Project Page: hutps://haazzwwang wettewawanngggbb...... io/remi; : (2) RatatouGym (Simulation Suite): https//ratatougym.iibbb..o (3) Bottleneck MAE: https///ittub..om/geee cow/graspplltl//bnitt mae.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cc
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips hattp::/mips.ccpuullc////
wwt.uubiissiioooo ssionpolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Yes, all the training and testing details are provided in the main text. We additionally include other details such as hyperparameters, optimizer, and cell simulation configuration in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
17

---

Justification:
Yes, we have added the statistical significance tests in the supplemental
materials.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Yes, all our experiments conducted single machine equipped with an RTX 5090 GPU with 36GB VRAM. The experiments can trained fairly quickly and do not impact the conclusions presented in the paper; therefore, further information is necessary.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethies https://neurips: httpp//cecctccitcceccccieieaaaee
Answer: [Yes]
Justification: Yes, this paper is fully compliant with the NeurlPS Code of Ethics. does not involve human subjects, crowdsourcing, sensitive data. All datasets and code follow privacy, licensing, and reproducibility guidelines, and we do not anticipate any harmful socictal impacts from this work.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
18

---

If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consideration due laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper focuses on understanding known neuroscience phenomenologies about place cells and grid cells and testing hypotheses with RNN model. We do not anticipate any direct applications that could lead to negative societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification:
This paper does not involve any data or models that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
19

---

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yss]
Justification: Yes, we have used the FAISS library to perform nearest neighbor search when decoding the RNN states to match the corresponding locations on the ratemap. FAISS is released under the MIT license and has been properly credited in the paper. All other code and data were developed by us. Additional packages used in our implementation are standard Python libraries from PyPI, all of which open-source.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/coooo/ooooo datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We did release any new assets in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Question:
Answer: [NA]
Justification:
This paper does involve any crowdsourcing or research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing rescarch with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
20

---

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve any crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification:
We did use any LLMs to develop the core methods in this research.
Guidelines:
The answer NA means that the method development in this research does not involve LLMS as any important, original, or non-standard components. /teeurppp.cc/oofffeerrc//22222
Please refer to our LLM policy for what should or should not described.
21

---

Condition for Positivity of Prediction Error
Given two locations, the simplest navigation task can be framed as decoding the displacement vector between them from their corresponding grid cell representations, As suggested in Section 4.3, we propose that it is sufficient for navigation even if we first decode a displacement vector within each grid module and then combine them through simple averaging. Here, examine what conditions are required to guarantee that the resulting averaged displacement always moves the animal closer to the target.
In the ID case, or along one axis of a 2D case, the decoded displacement vector through simple averaging is given by:
m
d
7
2mm
i=k+1
After taking this decoded displacement, the remaining distance to the target along the decoded direction is d - d. To ensure the animal always moves closer to the target along this axis, it suffices to show that d < d. Which is satisfied if m > k + s-1 1-s-
Proof. Assume that all decodable scales yield the correct displacement vectors, i.e., for all E {k + +1,.... ..]. we have:
to s² Adi d
Substituting into the expression
for d:
k
d d
d
Σ 3¹
Zi
1
m 2T
And thus we require
to
ded
+
(k m) d
2n
Since that & is the index that delineates the decodable and undercovered scales, (lo - d< (oo - The worst case occurs when d takes its maximum value (Eo gk+1)/2 while Z; takes its minimum value =r. Substituting these:
(A: - m) to k+1
to
k
Σ 8
(-Tr)
2
2x
s(s k
1)
$ (k - m) .kk+l <
8
1
1=1
s(s 1)
-
s k
k
s
s
- s
m
8
Notice that this bound only extend the initial assumption m > k by 1-s-t which never exceeds --1
when for ID case, and never exceeds 2 when Ve for 2D space. Therefore, simple averaging reliably decreases the distance to the goal m > in ID and m + 1 2D.
22

---

2 Spatial Information Content
We use spatial information content (SIC) [71] to measure the extent to which cell might be a place cell. The SIC score quantifies how much knowing the neuron's is firing rate reduces uncertainty about the animal's ' location. The SIC is calculated as
Pt
log2
Ti [[
E
Where E[r] the mean firing rate of the cell, ry is the firing rate at spatial bin and p; is the empirical probability of the animal being in spatial bin i. For all cells with mean firing rate above 0.01Hz, we discretize their firing ratemaps into 20 pixel x 20 pixel spatial bins and compute their SIC. We define cell to be a place cell if its SIC exceeds 20.
3 Simulating Spatial Navigation Cells and Random Traversal Behaviors
3.1 Simulating Spatial Navigation Cells
In our model, we simulated the ground truth response of MEC cell types during navigation supervise the training. We note that firing statistics for these cells vary significantly across species, environments, and experimental setups. Moreover, many experimental studies emphasize the phenomenology of these cell types rather than their precise firing rates. Thus, observed phenomenology and scale its firing rate using the most commonly reported values in rodents. This scaling does not affect network robustness or alter the conclusions presented the main paper. However, the relative magnitudes of different cell types can influence training dynamics. To mitigate this, we additionally employ a unitless loss function that ensures all supervised and partially supervised units are equally emphasized in the loss (see Suppl. 5.3.3).
simulate each type based its
We generate spatially modulated cells following the method in [22]. Notably, the simulated SMCs resemble the cells deseribed in [27]. To construct them, first. generate Gaussian white noise across all locations in the environment. 2D Gaussian filter is then applied to produce spatially smoothed firing rate maps.
Spatial Modulated Cells:
Formally, let A C R²² denote the spatial environment, discretized into a grid of size W x H. For each neuron i and location E A, the initial response sampled as i.i.d. Gaussian white noise: (x) ~ (0, 1). Each noise map is then smoothed via 2D convolution with an isotropic Gaussian kernel Goy, where o; represents the spatial tuning width of cell i. The raw cell response then given by:
RTT = C * Go.
where * denotes the 2D convolution. The spatial width o; is sampled independently for each cell using (12em, 3em). Finally, the response of each cell is normalized using min- max normalization:
Rew R max RTTh) min RMW
min (R
The SMCs used in our experiments model the sensory-related responses and non-grid cells in the MEC. Cue cells reported in [27] typically exhibit maximum firing rates ranging from 0-20Hz, but show lower firing rates at most locations distant from the cue. Non-grid cells reported in [72] generally have peak firing rates between 0-15Hz. To align with these experimental observations, we scale all simulated SMCs to have maximum firing rate of 15Hz.
Grid Cells: To simulate grid cells, we generate each module independently. For a module with a given spatial scale f, define two non-collinear basis vectors
[] and ba
8/2
b
1v3/2
These vectors generate regular triangular lattice:
C {nbb mb2 n, 2}
23

---

For each module, we randomly pick its relative orientation with respect to the spatial environment by selecting random E [0, x /3) which is used to rotate the lattice:
cos 9
- sin
R
Cs = {
sin 0 cos 0
Within each module, individual cells are assigned unique spatial phase offsets. These offsets ub, are sampled from equilateral triangle with vertices
Vi = R 0
0 -1/2
R² [V33/2 -1/2
V/2 = R
V/3
We sample phase offsets for grid cells within each module by drawing vectors uniformly from the triangular region using the triangle reflection method. Since the resulting grid patterns are wrapped around the lattice, this is functionally equivalent to sampling uniformly from the full parallelogram. The firing centers for cell in a given module are then given by:
C. = {c] + sbe | c E Co}
Finally, the raw firing rate map for cell is generated by placing isotropic Gaussian bumps centered at each location in C,:
x | c. 20
Σ exp
x E A
CLEC,
grid
where gui is the spatial tuning width of each bump and is computed 20 = f/r with 3.26 following the grid spacing to field size ratio reported in [56]. Experimental studies have reported that rodent grid cells typically exhibit maximum firing rates in the range of 0-15Hz [7, 55], though most observed values are below 10Hz. Accordingly, we scale the generated grid cells to have a maximum firing rate of 10Hz.
Many cells in the MEC respond to speed, including grid cells. head direction cells, conjunctive cells, and uncategorized cells [73, 74]. These cells may show saturating, non-monotonic, or even negatively correlated responses to movement speed. To maintain simplicity in our model, we represent speed cells as units that respond exclusively and linearly to the animal's 's movement speed. Specifically, simulate these cells with a linear firing rate tuning based on the norm of the displacement vector d at each time step, which reflects the animal's 's instantaneous speed.
Speed Cells:
Additionally, many reported speed cells exhibit high baseline firing rates [73, 74]. To avoid introducing an additional parameter, we set all speed cells' tuning start from zero their firing rate is OHz when the animal is stationary. To introduce variability across cells, each speed cell is assigned scaling factor s; sampled from A (0.2, 0.05) Hz/(cm/s), allowing cells to fire at different rates for the same speed input. Given that the simulated agent has mean speed of 10cm/s (Suppl. 3.2), the average firing rate of speed cells is approximately 2Hz. We chose lower mean firing rate than observed in rodents, as we did not include base firing rate for the speed cells. However, this scaling allows cells with the strongest speed tuning to reach firing rates up to 80Hz, matching the peak rates reported in [74] when the agent moves at its fastest speed.
All cells follow the same linear speed tuning function:
8/ ||d||
di
where dt is the simulation time resolution (in seconds), ||d]] is the displacement magnitude over that interval, and s; modulates the response sensitivity of each cell.
Direction Cells: We define direction cells based on the movement direction of the animal, rather than its body head orientation. During initialization, each cell is assigned preferred allocentric direction drawn uniformly at random from the interval [0, 2Tr). At any time during the animal's movement, we extract its absolute displacement vector d. From this vector, we compute the angular difference AO between the allocentric movement direction and the i-th cell's preferred direction. The
movement direction can be derived from the displacement vector d. Each neuron responds according
24

---

Pa
1
T= 5s
T= 30s
T 1min
T= 10min
T 30min
Figure S1: Example generated trajectories of varying lengths. From left to right: 50 trajectories of Is each, 5 trajectories of 5s, followed by single trajectories 30s, Imin, 10min, and 30min.
to wrapped Gaussian tuning curve:
|A0.12 2 dir
R₂(0) = exp
where [A0,]2 denotes the angular difference wrapped into the interval [0, (0,2 d denotes the tuning width (standard deviation) of the angular response curve. We Odir rad reduce the number of direction cells needed to span the full angular space [0, 2T), thereby decreasing the size of the RNN to improve training efficiency. Given that many head direction cells in the MEC are conjunctive with grid and speed cells [75], we set the mean firing rate of direction cells to 2Hz to match the typical firing rates speed cells.
We acknowledge that this simulation of the direction cell may only serve as a simplified model. However, in our model, direction cells serve only to provide input to grid cells for path integration, and we have verified that the precise tuning width and magnitude do not affect the planning performance or change the conclusion of the main text. Our findings could be further validated by future studies employing more biologically realistic simulation methods.
3.2 Simulating Random Traversal Behaviors
We train our HC-MEC model using simulated trajectories of random traversal, during which we sample masked observations of displacement and sensory input. The network is trained to reconstruct the unmasked values of speed, direction, SMC, and grid cell responses at all timepoints. Trajectories simulated within 100 100cm² arena, discretized 1 resolution per spatial bin. The agent's movement is designed to approximate realistic rodent behavior by incorporating random traversal with momentum.
At each timestep, the simulated agent's displacement vector d is determined by its current velocity magnitude, movement direction, and a stochastic drift component. The base velocity from a log-normal distribution with mean Mspd 10 and standard deviation asp 10 (in cm/s), such that the agent spends most its time moving below 10 cm/s but can reach up to 50 cm/s. The velocity is converted to displacement by dividing by the simulation time resolution dt, and re-sampled at each timestep with a small probability Pspd to introduce variability.
sampled
To simulate movement with momentum, add drift component that perturbs the agent's displace- ment. The drift vector computed by sampling a random direction, scaling it by the current velocity and drift coefficient Carin that determines the drifting speed. Drift direction is resampled each step with small probability Pair to simulate the animal switch their traversal direction. The drift is added the direction- based displacement, allowing agent to move directions slightly offset from its previous heading, This results in smooth trajectories that preserve recent movement while enabling gradual turns.
To prevent frequent collisions with the environment boundary, soft boundary-avoidane mechanism applied. When the agent is within daoiid pixels of a wall and its perpendicular distance to the wall decreasing, an angular adjustment applied its direction. This correction proportional to proximity and only engages when the agent is actively moving toward the wall. We set dt = 0.01 sec/timestep. Pspd 0.02, Carin 0.05, Pair 0. 0.15, and daviia 10 pixels. These values were chosen to produce trajectories that qualitatively match rodent traversal (see Fig S1).
25

---

4 Decoding Location from Population Activity
In the main text, we decode the population vector to locations in both the recall task (Section 3) and the planning task (Section 4.4). Here we present the method we used for such decoding. Given a population vector r E R at a given timestep, we decode it into a location estimate by performing nearest neighbor search over a set of rate maps corresponding to the subpopulation of cells corresponding to r. These rate maps may come from our simulated ground-truth responses or be aggregated from the network's activity during testing (see Suppl. 5.4).
Formally, let r = [r1. r2,-.. .NN] [ be the population response of a subpopulation of N cells at a given timestep, and let M E RPPN be the flattened rate maps, where each row mp corresponds to the population response at the p-th spatial bin. Here, P = W x H is the total number of discretized spatial bins in the environment A. The decoding process to find the index p" that minimizes the Euclidean distance between and mpi
p' arg min || mpll2
To efficiently implement this decoding, we use the FAISS library [8), Specifically, we employ the IndexIVFFlat structure, which first clusters the rows {mp)z=1 into & clusters using k-means. Each vector my is then assigned to its nearest centroid, creating an inverted index that maps each cluster to the set of vectors it contains.
At query time, the input vector r is first compared to all centroids to find the n-probe closest clusters. The search is then restricted to the vectors assigned to these clusters. Finally, the nearest neighbor among them is returned, and its index p" is mapped back to the corresponding spatial coordinate x". For all experiments, we set the n.clusters for the k-means to 100 and n. n.oob to 10.
5 Training and Testing of the HC-MEC Model
As described in Section 2.1, our HC-MEC model is a single-layer RNN composed of multiple sub-networks. We train two versions of this model: (1) GC-only variant, which includes only grid cells and along with speed and direction cells; and (2) the full HC-MEC loop model, which includes both MEC and hippocampal (HC) subpopulations.
In both cases, we simulate ground-truie responses for supervised and partially supervised cells using the method described in Suppl. 3. The number of simulated cells matches exactly the number of corresponding units in the RNN hidden layer. That is, if we simulate Ng grid cells, we assign precisely Ng hidden units in the RNN to represent them, and (partially--superee corresponding ground-truth activity. Additional details on this partial supervision are provided in Suppl. 5.1.
these units with the
For both models, we use six scales of grid cells, with the initial spatial scale set to 30cm. Subsequent scales follow the theoretically optimal scaling factor s = v [54]. The grid spacing to field size ratio is fixed at 3.26 [56]. Each scale comprises 48 grid cells, and the spatial phase offsets for each cell within a module are independently sampled from the corresponding equilateral triangle (Suppl. 3) with side length equals to the spatial scale of the module. In addition, both the GC-only and HC-MEC models include 32 speed cells and 32 direction cells.
The HC-MEC model additionally includes spatially modulated cells (SMCs) in the MEC subpopulation and hippocampal place cells (HPCs) in the HC subpopulation. We include 256 SMCs to approximately match the number of grid cells. These SMCs are designed to reflect responses to sensory experience and are trained with supervision approximately the total number of grid cells and SMCs (N = 256 + 288). These cells only receive input from and project to the MEC subpopulation through recurrent connections, and thus do not receive any external input. We note that, differing from [22], we did not apply a firing rate constraint on the HPCs, but still observed the emergence of place cell-like responses.
described in Suppl. 3. We also include 512 HPCs, matching
In total, unless otherwise specified, our GC-only model comprises 352 hidden units (48 x grid cells + 32 speed cells + 32 direction cells), while the HC-MEC model comprises 1120 hidden units (288 grid cells + 32 speed cells + 32 direction cells + 256 SMCs + 512 HPCs).
26

---

9
V
D
4
6
P
9
Epoch
Epoch
Epoch
Figure S2: Example place fields emerging over training, Shown are place fields from epoch 2, 20, and 50. As training progresses, the fields become increasingly specific and spatially refined.
5.1 Supervising Grid Cells
As described in Section 2.1, we partially supervise the grid cell subpopulation of the RNN using simulated ground-truth responses. Let z, E Ns denote the hidden state of the RNN units modeling grid cells at time t. During training, the HC-MEC model is trained trajectories. Along each trajectory, we sample the ground -tuth grid cell responses from the simulated ratemaps the corresponding location and denote these (r?)²- At the start of each trajectory, we initialize the grid cell units with the ground-truth response at the starting location, i.c., Zo = rz
short random traversal
From time step projections between the grid cell subpopulation and the speed and direction cells. They do not receive any external inputs. After the RNN processes the speed and direction inputs over the entire trajectory. we collect the hidden states of the grid cell subpopulation {2: (2:)²" and minimize their deviation from the corresponding ground-trute responses {x² )T The training loss function described in Suppl. 5.3.3.
T, the grid cell hidden states 2, updated solely through recurrent
We choose to partially supervise the grid cells due to their critical role in the subsequent training of the planning network. This supervision allows the model to learn stable grid cell patterns, reducing the risk of instability propagating into later stages of training. While this partial supervision does not reflect fully unsupervised emergence, it can still be interpreted as a biologically plausible scenario in which grid cells and place cells iteratively refine each other's representations. Experimentally, place cell firing fields appear before grid cells but become more spatially specific as grid cell patterns stabilize, potentially due to feedforward projections from grid cells place cells [76, 77].
We observe a similar phenomenon during training. Even with partial supervision, grid cells tend to emerge after place cells. This may be because auto-associating spatially modulated sensory representations is easier than learning the more structured path-integg inteegrttrttinn task hypothesized to be performed by grid cells. After the grid cells' pattern stabilizes, also observe that place cell firing fields become more refined (Fig. S2), consistent with experimental findings [3, 77-79].
5.2 HC-MEC Training Task
During navigation, animals may use two different types of sensory information to help localize themselves: (1) sensory input from the environment to directly observe their location; and (2) displacement information from the previous location infer the current location and potentially reconstruct the expected sensory observation. We posit that the first type of information reflected by the weakly spatially modulated cells (SMCs) in the MEC, while the second type is reflected by grid cells and emerges through path integration.
However, as previously argued in Section 2.1, both types of information are subject failure during navigation. Decades of research have revealed the strong pattern completion capabilities of hippocampal place cells. We thus hypothesize that hippocampal place cells may help reconstruct one type of representation from the other through auto-associative mechanisms.
27

---

To test this hypothesis, we simulate random traversal trajectories and train our HC-MEC model with masked sensory inputs. The network tasked with reconstructing the ground-truth responses of all MEC subpopulations from simulation, given only the masked sensory imput along the trajectory. Specifically, the supervised units-SMCs, speed cells, and direction cells-receive masked inputs to simulate noisy sensory perception. We additionally mask the ground-truth responses used initialize both supervised and partially supervised units at t = O, so that the network dynamics also begin from imperfect internal states.
To simulate partial or noisy observations. we apply masking on a per-trajectory and per-cell basis. For a given trajectory, we sample the ground-truth responses along the path to form a matrix R = (ro ......ry. rr] E RTx,, where N is the number of cells the sampled subpopulation and ry the ground-truth population response at time l. The masking ratio T'mas defines the maximum fraction of sensory and movement-related inputs, well as initial hidden states, that are randomly zeroed during training to simulate partial or degraded observations. We generate binary mask M {0, 1)7%N by thresholding matrix of random values drawn uniformly from [0, 11, such that approximately 100 x Tmusk percent of the entries are set to zero. The final masked response is then obtained by elementwise multiplication = R M.
During training, we sample multiple trajectories to form batch, with each trajectory potentially having a different masking ratio and masked positions. For both the HC-MEC model used in the recall task (Section 2) and the one pre-trained for the planning task (Section 4.4), masking ratios for SMCs and other inputs are sampled independently from the interval [0, Tmask]. Specifically, for each trial, we sample:
MsMC. Mather U((,, mmusk)
We use msse to generate the mask for SMC cells, and Mother to independently generate masks for grid cells, speed cells, and direction cells. This allows the model to encounter a wide range of noise conditions during training-foo example, scenarios where sensory inputs are unreliable but displacemeent-related cues are available, and vice versa.
5.3
RNN Implementation
5.3.1
Initialization
Our HC-MEC model includes multiple sub-regions. However, we aim to model these sub-regions with- out imposing explicit assumptions about their connectivity, as the precise connectivity- titmeccciityy-paatcuulllyy the functional connectivity between the hippocampus (HC) and medial entorhinal cortex (MEC)- unknown. Modeling these sub-regions with multiple hidden layers would implicitly enforce a uni- directional flow of information: the second layer would receive input from the first but would not project back. Specifically, a multi-layer RNN with two recurrent layers can be represented by a block-structured recurrent weight matrix:
W11
W12
W
W21
222
Where I and 222 are the recurrent weight matrices of the first and second recurrent layers, respectively, and W is the projection weights from the first to the second layer. In typical multi-layer RNN setups, 1 = 0, meaning that the second sub-region does not send information back to the first. This structure generalizes to deeper RNNs, where only the diagonal blocks W and the blocks directly below the diagonal (++1)) are non-zero.
Therefore, we model the HC-MEC system large single-layer RNN, such that all bub-blocks are initialized as non-zero, and their precise connectivity is learned during training and entirely defined by the task. To initialize this block-structured weight matrix, we first initialize each subregion independently as single-layer RNN with defined weight matrix but no active dynamics. Suppose we are modeling N, subregions, and each subregion i contains d, hidden units. We initialize the recurrent weights within each subregion using uniform distribution:
W U (-1/Vd, 1/V
28

---

For each off dfagonal block W corresponding to projections from subregion to subregion i, we similarly initialize:
W ~ (-
Note that the initialization bound is determined by the size of the source subregion j. consistent with standard practices for stabilizing the variance of the incoming signals. Once initialized, all sub-blocks are copied into their respective locations within a full recurrent weight matrix:
W
WIN
W WHC-MEC+
W
W.N.N,
with total size Σ de X Σi= d,
Additionally, as described in main text, both input and output neurons are already modeled within the HC-MEC model. As result, no additional input or output projections are required. Input neurons in this assembled RNN directly integrate external signals from the simulation, while the states of output neurons are directly probed out during training.
5.3.2 Parameters
For models used in the recall and planning tasks, use the following parameters:
Table 1: Shared parameters for GC-only and HC-MEC models
Parameter
Value
Description
N spedd
32
Number of speed cells
diirection
32
Number of direction cells
N gri (per module)
48
Number of grid cells per module
modules
6
Number of spatial scales (modules)
initial scale
30 cm
Spatial period of the smallest module Scale ratio between adjacent modules Grid spacing field size ratio Activation function of the hidden units Time for both cell simulation and RNN, Is 20 bins Initial forgetting rate of the cells
spatial.scale factor
Ve
"grid/field
3.26
activation
ReLU
dl
0.05
Oinit
0.2
learn_alpha
True
Whether the forgetting rate is learned during training Optimizer
optimizer
AdamW
learning-rate
0.001
Learning rate
batch.size
128
Batch size, each batch corresponds to single trajectory
n.epochs
50
Number training epochs
n_steps
1000
Number of steps per trajectory
Tugjectory
2s
Duration of each training trajectory
In Section 4.4 of the main text, we noted that the GC-Only model used for planning uses Ngria = 1 128, i.e. each module comprised 128 grid cells. This larger population improves planning accuracy, likely due to denser coverage of the space. However, for the full HC-MEC model used in planning, we reverted Ngrid 48, consistent with our default configuration. As discussed in the main text, the auto-assan aattoassociitive dynamics from place cells help smooth the trajectory, even when the decoded trajectory from grid cells imperfect.
5.3.3 Loss Function
We use unitless MSE loss for all supervised and partially supervised units such that all supervised cell types are equally emphasized. For each trajectory, we generate ground-truth responses by sampling the corresponding region's simulated ratemaps at the trajectory's locations, resulting After the RNN processes the full trajectory, we extract the hidden states of the relevant units to obtain (z. )- To ensure that all loss terms are optimized equally and are not influenced by the scale or variability of individual cells, perform per-cell normalization of the responses. For each region
)
29

---

Table 2: Additional parameters for HC-MEC model
Parameter
Value
Description
NsMC
256
Number of spatially modulated cells
OSMC
((1cmm, 3cm)
Width of Gaussian smoothing to generate SMCs, which controls the spatial sensitivity of SMCs (see Suppl. 3)
NHPС 512
Number of hippocampal place cells
2, we compute the mean p' Rd and standard deviation o' E Fd of the ground truth responses across the time and batch dimensions, independently for each cell. The responses are then normalized elementwise:
2 μ
The total loss across all regions is computed as the mean squared error between the normalized responses:
C C= Σ
|| +ill
where Ả controls the relative weights of each cell types. We the À 10 for both grid cells and SMCs, while À for velocity and direction cells. These relative weights are set as animals may emphasize their reconstruction of the sensory experience and relative location, rather than their precise speed and direction during their spatial traversal.
5.4 Constructing Ratemaps During Testing
After training, we test the agent using a procedure similar to training to estimate the firing statistics of hidden units at different spatial locations and construct their ratemaps. Specifically, we pause weight updates and generate random traversal trajectories. The supervised and partially supervised units are initialized with masked ground-truth responses, and the supervised units continue to receive masked ground- truth inputs at each timestep. We record the hidden unit activity of the RNN at every timestep and aggregate their average activity at each spatial location.
Let z, Rd be the hidden state subpopulation of hidden states of the RNN at time t, where d is the number of hidden units. For each unit i, the ratemap value at location x is computed as:
Σ Exx,
zi if Nx) 0
(x(x)
N(x NaN
otherwise
where [(x) is the number of times location x was visited during testing. We set the trajectory length to T 5s (250 time steps), batch_size 512, and n batches 200. We perform this extensive testing to ensure that the firing statistics of all units are well estimated. Each spatial location is visited on average 1008.13 268.2 24 times (mean standard deviation).
Recall Task
In this paper, have posited that the auto-association of place cells may trigger the reconstruction of grid cell representations given sensory observations. To test whether such reconstruction is possible, we trained nine different models, each with a fixed masking ratio Tmask (m, in the main text) ranging from 0.1 to 0.9. Following the procedure described in Suppl. 5.2, each model was trained with the same masking ratio applied to all subregions and across all trials, but the masking positions were generated independently for each trial. That for each trial, 100 x Tiask of the entries were occluded. The mask was applied both to the ground-tree -tuth responses used to initialize the network and to the subsequent inputs.
After training, we tested recall by randomly selecting position in the arena and sampling the ground-truth response of the SMC subpopulation to represent the sensory cues. This sampled response then repeated over T = 10s (200 time steps) to form constant input query. Unlike
30

---

training, the initial state of the network was set to zero across all units, such that the network dynamics evolved solely based on the queried sensory input.
Testing on Realistic Navigation
7.1 Integrating REMI into Visually Realistic Navigation Task
Having demonstrated the feasibility of our REMI framework in a simulated navigation environ- ment where trajectories generated synthetic cell responses (Gaussian random fields as SMCs), we next tested its generalization to realistic visual navigation. To this end, we let the agent explore a visually realistic environment while converting visual vasionations into simulated cell responses. Importantly, since our framework predicts that intermediate GC activity during planning can drive HPCs to reconstruct intermediate sensory experiences, we further tested whether these reconstructed sensory states could be decoded back into images that match the expected views along the planned path.
Collecting Sensory Experiences Habital
BinkMAE Encoder
1021 feature map
penorama Img location
Figure S3: Converting Lx x Ly x 512 x 1024 panorama image tensor into Reah E TR. X
We used the Habitat Synthetic Scene Dataset
(HSSD) [66] within the Habitat-Sin simulator [45-47]. convert visually realistic scenes into cell responses, we first captured panoramic images at fixed orientations across all spatial locations, forming a tensor of shape Le x Ly x 512 x 1024, where 512 1024 is the image resolution and Lg x Ly defines the environment's spatial grid. A vision encoder E(.) then maps each panoramic image / to low-dimensional feature vector (() E Rd, where d represents the number of sensory cells (SMCs) responding visual signals. During planning, decoder D(-) reconstructs images from SMC states such that D(E(T)) T. This vision encoder will thus convert the L, Ly x 512 x 1024 panorama image tensor into Riub E RLxxLyxd which can then be used to replace the original R This pair enables (1) compact representation visual observations during navigation and (2) visualization of planned trajectories by decoding intermediate sensory states back into images.
7.2 Pre-Training Bottleneck Masked AutoEncoder
To construct a generalized vision en- coder consisting of both an encoder E(.) and decoder D(-), we used the Masked Autoencoder (MAE) frame- work [48]. The standard MAE with a Vision Transformer (ViT) backbone encodes each image into
(raii-Training BtnkMAE
R20M
MAE Encoder
patch token
MAE Decoder
of patch- wise features of shape p x d, where p is the number of image patches and d is the embedding dimension. In our HC-MEC model, however, each location-specific panoramic im- age must be represented by single d-dimensional vector, requiring com- pression from d to d. Simply pooling patch features would discard spatial structure, whereas retaining all patch tokens would produce inputs too high-dimensional for our recurrent ar- chitecture.
Tolkens (ulaposedd
poe embed
BinkMAE Encoder
BinkMAE Decoder
Figure $4: Illustration of our BtnkMAF. The original image patch tokens X1:p are discarded after passing through the BtnkMAE encoder, which retains only the CLS token Xels compact visual representation. For the REMI experiment, a ReLU activation was added after the encoder to enforce non- negative firing rates consistent with biological realism. The decoder then employs DETR-style cross-attention between set of learnable query embeddings q1:p and Xais reconstruct the discarded patch tokens X1.p-
31

---

Bottleneck MAE (BtnkMAE). To address this, we developed BinkMAE, modified MAE with VTT backbone that compresses visual information into single learned representation. the original MAE, the encoder outputs features (elss:1.p) E R(++1)xd where Xels is null classification token and Xi:p are patch embeddings. In BtnkMAE, we retrain the encoder such that it discards X1:p and passes only Xels to the decoder. To reconstruct the discarded patch tokens, employ DETR-style cross-attention [67], where learned query embeddings
X1pp = Letoss (11:p: Xels) R X1:p-
Finally, X1:p are fed into the MAE decoder to reconstruct the full image.
Figure S5: First row: original ImageNet 1K images. Second row: reconstructed ImageNet Ik images.
Pretraining BtnkMAE on ImageNet 1k. We pretrained BinkMEE on ImageNet 1k for 100 epochs to learn general visual compression rule, namely distill image information into single CLS token while maintaining decodability. We initialized BtnkMAE with pretrained weights from the original MAE, loading only parameters from the shared architecture. The CLS token weights were discarded and reinitialized, and each learnable patch embedding was augmented with an additional positional embedding to encode spatial relationships among patches. The encoder learning rate was to 0.1 times that of the decoder better preserve its pretrained structure. Unlike the original MAE, which is trained with random masking, BtnkMAE was trained on full images to learn compressed representations rather than reconstructing masked inputs. During training, we computed two losses, one identical to the original MAE objective that bypassed the bottleneck and another that passed through the bottleneck, allowing the model to learn compression while retaining general visual feature representations. Example views of original and reconstructed images are shown in Figure S5.
Figure S6: First row: original panoramic images. Second row: reconstructed panoramic images.
Fine-Tuning BtnkMAE in Habitat Sim Scene. We then fine-tuned the model on panoramic images from Habitat Sim environment. These panoramic images, originally with resolution of 512 x 1024, were resized to 512 512 using bicubic interpolation to match standard ViT input dimensions. During pretraining on ImageNet-Ike 1k, the model was trained on images of size 224 224. To accommodate
32

---

Finally, after fine-tuning BtnkMAE on the Habitat Sim scenes, we froze the encoder to convert each location-specific panoramic image into a 1024-dimensional feature vector. For an environment of shape Lg X Ly. this produced ratemap tensor Raab E R Lrx byyx 1024, which replaced the SMC ratemap tensor Rame. We then repeated the HC-MEC training described in Section and the planning experiments in Section 4.4. During planning. the HC-MEC network updated the SMC region to intermediate states &p.... ................................ 2r. We collected these states, each representing 2; E 1 corresponding to the agent's internal planning or virtual navigation through the environment. These states were passed to the trained and frozen BtnkMAE decoder, which successfully reconstructed the higher resolution, we recomputed fixed sinusoidal cosine positional embeddings corresponding to the 512 x 512 input size and fine-tuned the model specifically on panoramic views from the target navigation scenes. The original and reconstructed image pairs are shown in Figure S6.
7.3 Testing REMI Visually Realistic Navigation Task
images resembling the expected visual views along the planned paths.
8 Validation of Robustness
While our experiments rely on supervised grid cell that learns path-integration, we sought confirm that our framework does not critically depend on ideal grid cells with idealized hexagonal lattices or noiseless inputs. To test this, we conducted additional experiments that systematically distorted the simulated ground truth grid cell representations. We trained 18 HC-MEC and corresponding planner models, 10 to examine the impact of noise and 8 to assess geometric distortions. Each model was evaluated on 4,096 planning trials. trial was considered successful if the agent's final position was within 10 cm of the goal, and the success rate was computed across all trials. On average, the goal was located 07.54 25.53 em from the starting position.
For the noise experiments, we added post-activation Gaussian noise (standard deviation 0.1 to 1.0) to all hidden units of the HC-MEC at each time step during training. No noise was added at test time (planning phase).
Here, we report results based on decoding spatial locations from SMC states using their ground truth ratemaps, though results consistent when decoding from GCs. In all cases, the mean decoded distance to the goal remained below 10 cm in a 100 cm x 100 cm environment, and success rates remained high across noise levels. These findings indicate that the planner performs robustly even with noisy or distorted grid cell representations.
cm
Table 3: Performance of the HC-MEC planner under varying noise levels.
Noise (std)
Distance to Goal (cm)
Success Rate
Detour Ratio
0.1
2.20 + 5.89
0.99
1.66
0.2
2.35 + 5.68
0.99
2.00
0.3
1.82 = 5.09
0.99
1.88
0.4
3.75 8.84
0.98
2.10
0.5
2.62 3.43
0.98
2.36
0.6
3.12 3.72
0.97
2.33
0.7
3.31 5.66
0.99
2.55
0.8
2.98 2.78
0.99
2.60
0.9
3.00 3.99
0.98
2.31
1.0
5.30 + 6.98
0.91
2.70
Next, we added noise to the ground-truth GC signal that used at the first time step during planning (the initial GC state).
33

---

Table 4: Planning performance of REMI under varying noise levels.
Noise (std)
Distance to Goal (em)
Success Rate
Detour Ratio
0.1
2.45 + 2.49
0.99
2.16
0.2
2.40 = 4.37
0.99
1.91
0.3
1.89 + 5.41
0.99
1.98
0.4
4.10 = 7.66
0.96
2.54
0.5
4.36 + 5.01
0.95
3.28
0.6
4.05 5.48
0.96
2.44
0.7
4.01 + 6.70
0.97
2.53
0.8
4.79 6.00
0.91
2.63
0.9
8.43
0.91
2.67
1.0
7.72 + 13.24
0.85
2.49
Additionally, we trained another 8 models without noise but with sheared grid fields. We incrementally adjust the angle between the two co-linear axes of the generated grid cells used for supervision, and repeated the test above on the 8 trained models.
Table 5: Planning performance of REMI under geometric distortion grid lattices.
Shearing Angle
Residual Goal (em)
Success Rate
Detour Ratio
40
9.53 16.39
0.86
2.45
45
7.88 = 17.66
0.91
2.10
50
7.40 18.18
0.91
2.12
55
5.48 11.38
0.92
2.38
60
7.66 + 16.29
0.87
2.26
65
2.47 -.47 7.13
0.98
2.04
70
5.13 = 10.22
0.91
2.44
75
6.38 + 14.26
0.91
2.12
80
4.46 11.31
0.94
1.91
34


Reviews:
[{"review_id": "ez0j5WuFYr", "reviewer": "Reviewer_u9fY", "summary": "This paper proposes REMI, a framework explaining how animals use both grid cells and place cells for spatial navigation and path planning. The authors suggest that place cells auto-associate sensory information with grid cell patterns, enabling animals to recall goal locations from sensory cues and then plan paths using stable grid cell representations. They demonstrate this with a model that shows how grid cells can support sequential planning through local transition rules that generalize to long-distance navigation, while place cells can reconstruct sensory experiences along planned paths.", "questions": "Any arbitrarily complicated recurrent structure can be written as a single layer RNN. Since you train the HC-MEC and planning networks in very different ways and at different times, why does it make sense to think of them as the 'same' RNN?\nThe paper suggests a connection to successor features, which at a high-level seems plausible. Is there a rigorous connection that can be specified demonstrating that the same future state transition probabilities are encoded in the trained model?\nHow accurate is your actual path integration? Do you have quantitative metrics showing this for different time scales? How does it compare to other neural path integrators in terms of accuracy?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The work provides a compelling theoretical framework that unifies the functional roles of grid cells and place cells, addressing the important question of why animals maintain two distinct spatial representation systems. The authors successfully model the HC-MEC loop in a single RNN that simultaneously exhibits grid cell path integration, place cell emergence, and planning capabilities. It was unclear if the planning network is properly thought of as part of the same single layer RNN, as it was added on afterwards, but regardless it is an interesting model. The work makes concrete, testable experimental predictions about how sensory cues should reactivate grid cell patterns and how selective brain region disruptions should affect navigation, which provide important links to empirical work. \n\nA significant limitation is the reliance on partially supervised grid cells rather than demonstrating their natural co-emergence with place cells, which somewhat undermines the biological plausibility claims and leaves questions about how this system would actually develop in real neural circuits. However, this limitation is acknowledge by the authors and leaving it to future work seems appropriate given the strong contributions of the remainder.", "was_revised": true, "final_justification": "Running the model on the new dataset is a nice addition. I don't have any other questions and appreciate the responses to my original queries.", "rebuttal": "We thank reviewer u9fY for their thoughtful review and the endorsement of our work. We are excited to see that you think this is a compelling theoretical framework that unifies the functional roles of grid cells and place cells. As you have mentioned, we indeed attempt to propose this framework to construct a system-level theory to explain the empirical works on each neural sub-region, as collectively these patterns must be preserved by evolution due to their functional importance. \n\n> **Experiments on practical and real datasets**: We would first like to highlight an additional contribution: as suggested by reviewer b2n6, we conducted experiments in a visually realistic simulation environment (Habitat-Sim) using realistic datasets.\n\nIn our model, SMCs represent sensory observations. While the main text uses simulated sensory signals, here we replace them with features derived from visual observations from the Habitat Synthetic Scenes Dataset (HSSD): the agent captures images during exploration, which are passed through a vision encoder to generate SMC signals. We follow the same training procedure for HCMEC and planner models as in Section 4.4 and find that the model successfully reconstructs visual features from this realistic dataset.\n\nImportantly, our vision encoder produces features decodable back into images, allowing visualization of scenes along the planned path. All intermediate SMC states generated during planning can be decoded into images that closely match the expected scenes at their respective planned locations. We provide key implementation details below:\n\n- We discretize the environment into an $X_w \\times X_h$ spatial grid and capture $W \\times H$ allocentric panorama images at each grid point.\n\n- We fine-tuned a generalized visual encoder (a modified Masked AutoEncoder (MAE; He et al., 2021)) on images from this specific simulation gym. This MAE learns to encode images into compact feature representations, which can later be decoded back to the original images. Compactly, it can be expressed as $D(E(I)) \\approx I$, where $I$ is the image, $E: \\mathbb{R}^{W \\times H} \\to \\mathbb{R}^{512}$ is the encoder, and $D: \\mathbb{R}^{512} \\to \\mathbb{R}^{W \\times H}$ is the decoder.\n\n- At each location, the image is encoded into a 512-dim vector used as the SMC response, enabling training of HCMEC and the planner as in the main text.\n\n- During planning, the HCMEC updates the SMC region to intermediate states ${z_0, \\dots, z_T}$, where $z_i \\in \\mathbb{R}^{512}$. Decoding them with $D$ yields images ${I_0, \\dots, I_T}$ closely resemble the expected views at the corresponding planned locations.\n\n**Here we address the questions in the review:**\n\n> **It was unclear if the planning network is thought of as part of the same single-layer RNN:**  \n\nWe train the planning and HCMEC networks in a two-stage setting primarily because they may correspond to two separate behavioral processes. HCMEC training models the phase when the animal explores the environment and builds representations of it, whereas the planner models how, once a spatial representation is formed, the animal might virtually navigate on it to plan paths. At the implementation level, however, this is equivalent to initializing a single network with both HCMEC and planner units, then alternating between training to encode the space and to plan for navigation.\n\nAdditionally, we used the term \"single layer\" to emphasize that in the HCMEC model, both HC and MEC can project to each other, unlike prior models that typically assume unidirectional information flow. As the reviewer noted, we are leveraging the fact that any arbitrarily complex RNN can be represented as a single-layer RNN. This allows us to model bidirectional interactions between multiple neural subpopulations using a single recurrent weight matrix. These interactions are entirely determined by the optimization objective, without the need to define or handcraft structures.\n\n> **On partial supervision of grid cells** \n\nOur core contributions and claims do not depend on co-emergence of grid cells, or having idealized grid cell responses.\n\nFirst, we are showing that if place cells (PCs) auto-associate grid cells and spatially modulated cells (SMCs), the agent can use sensory cues to recall GC patterns. Both recall and reconstruction of sensory experiences during planning work, as long as GC and SMC patterns are spatially stable.\n\nWe will modify Section 4 to include a short mathematical argument that shows that planning with grid cells does not require idealized GC representations. It will consist of the following claims. Within a single grid-scale, the displacement vector can be correctly decoded even if the grid field undergoes shearing and compression (in 1D these two are directly related to grid scale, while the 2D case can be decomposed into 1D cases on non-colinear axes). Larger displacements across multiple grid-scales in sheared and compressed grid-fields can be pieced together by the decoding process. Decoding is also robust with sub-optimal grid-scales. The equation on Line 276 can be written in a more general form $$ \\hat{d} = \\frac{1}{2\\pi m} \\cdot \\left( \\sum_{i=1}^{k} \\ell_{i} \\cdot Z_i + \\sum_{j=k+1}^{m} \\ell_{j} \\cdot \\Delta \\phi_i \\right) $$Under optimal scaling, $\\ell_i = \\ell_{0} s^i$ but so long as there are sufficiently many scales, the predicted displacement is guaranteed to move the animal closer to the target. \n\nUnlike place cells **the grid cell patterns may not be learned but are believed to be dynamically self-organized via a network that was pre-wired by evolution**. For instance, genetically deleting the gene encoding a critical NMDA receptor subunit specifically disrupts grid patterns without affecting other spatial cell types (Gil et al., _Nat Neurosci._, 2018), which indicates that the underlying molecular components and the capacity to form grid-like patterns are at least partly encoded in the genome. Existing theoretical literature also provides evidence suggesting the grid cells may not be learned but are made by a pre-wired dynamical attractor network (Fuhs & Touretzky, J. Neurosci. 2006; Burak & Fiete, PLoS Comp. Bio., 2009; Gardner et al., Nature. 2022). The supervised grid cell region can be directly replaced with one of the existing computational and theoretical models (Cueva & Wei, ICLR, 2018; Kang & Balasubramian, eLife, 2019; Sorscher et al. NeurIPS, 2019) of grid cells without affecting our claims of recall and reconstruction during planning. Here we only use the grid cells as pattern generators that generate grid-like patterns (idealized or not), and ask, “if we have these patterns, could the animal use them for recall and planning”.\n\n> **Connection to Successor Features**\n\nHere, we analyze why, at the implementation level, our network must encode future state. In REMI, HPCs must predict both GC and SMC responses at the next timestep in order to pattern-complete. Since HPCs are direction-agnostic, accurately reconstructing GC and SMC activity requires encoding possible state transitions. Because adjacent locations tend to have higher transition probability (Stachenfeld et al., 2017), our model implicitly reflects the successor representation framework of HPCs, where transition structure is embedded in the pattern-completion dynamics. \n\nRegarding our discussion of grid cell transition probabilities, we primarily use it as a conceptual framework to understand how the RNN might generalize from planning short trajectories to longer ones, and why such generalization is only feasible with the periodic grid structure. It is difficult to check the mechanism of the RNN, e.g., whether or not it instantiates a Markov chain that we have alluded to. We will preface the paragraph on Line 238 to emphasize this point.\n\n> **Path Integration Accuracy**\n\nWe tested the path integration accuracy of models with (Figure 1d) and without (Figure 1b) Spatially Modulated Cells (SMCs). Both models contain the same number of Grid, Direction, and Speed Cells, with the full HCMEC model additionally including SMCs and Place Cells. The HCMEC model was trained with a masking ratio for each trial sampled uniformly from $\\mathcal{U}[0, 1]$. \n\nTo test path integration, the GC subregion was initialized with the ground truth grid state. During testing, the model received unmasked speed and direction inputs. In the full HCMEC model, the SMC subregion additionally received SMC signals. Grid cell subregions were expected to update their internal grid states accordingly.\n\nWe evaluated each model over 100 seconds across 512 independent trials. GC responses were decoded into spatial coordinates using nearest neighbor search, and errors were computed as Euclidean distances (cm) from the ground truth at [1, 5, 10, 20, 50, 100] seconds in a 100 cm × 100 cm environment (1cm/spatial pixel):\n\n- Full HCMEC model: [1.96 ± 1.58, 2.63 ± 3.26, 3.14 ± 6.32, 3.76 ± 8.23, 7.67 ± 17.06, 11.43 ± 22.47] (in cm)\n- Grid cell only model: [1.89 ± 2.92, 2.76 ± 5.34, 3.37 ± 7.11, 4.86 ± 10.77, 5.17 ± 12.39, 5.28 ± 12.74] (in cm)\n\nBoth models show high path-integration accuracy over time. The error may result from the discretization of space and time in our simulation, which introduces small inaccuracies that accumulate during path integration. In contrast, biological systems may use additional signals such as boundary cells for error correction (Eli et al. Dynamic self-organized error-correction of grid cells. 2018). Future work may incorporate such signals to improve accuracy. The full HCMEC model shows slightly lower path-integration accuracy than the GC-only model, likely due to training with masked velocity, speed, and direction inputs.\n\nAll experiments were conducted with agents moving at speeds sampled from a right-skewed log-normal distribution with mean and standard deviation of 10 cm/s. Since speed affects the temporal resolution of input signals, our results are robust across different timescales."}, {"review_id": "ZbIihiGvnF", "reviewer": "Reviewer_b2n6", "summary": "This paper proposes a computational framework, REMI, to explain how animals might integrate hippocampal place cells (HPCs) and medial entorhinal grid cells (GCs) to support spatial memory and path planning. The authors hypothesize that GCs and PCs work together to enable animals to recall goal locations from sensory cues and plan paths, even through unvisited areas.\n\nTo test this, the authors develop a single-layer recurrent neural network (RNN) model that simulates the HC-MEC loop, incorporating sensory, speed, and direction inputs. GCs are partially supervised to emulate path integration, while HPCs emerge through recurrent dynamics. The network demonstrates that sensory cues at goal locations can trigger recall of corresponding GC states via HPCs. These GC representations can then be used for planning, with grid-space trajectories decoded into speed and direction commands. Furthermore, the model shows that intermediate GC states can drive the reconstruction of sensory experiences via HPCs during planning.\n\nThe paper also formalizes grid-space planning mechanisms, including decoding displacements using grid phase differences, learning local transition rules for sequential planning, and combining multiple spatial scales for more accurate long-range path planning.", "questions": "1. How does the proposed planning mechanism compare to other RL-based navigation models in terms of performance and efficiency?\n\n2. The paper claims HPCs reconstruct sensory experiences during planning. Is there empirical evidence (e.g., firing patterns) to support this claim in the model?\n\n3. How does performance (e.g., recall accuracy, planning success) vary with different levels of partial supervision or noise in the GC signals?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "** Strengths\n\n1. The use of a unified RNN architecture that simulates HC-MEC dynamics and planning within a biologically plausible framework is well-conceived and systematically developed.\n\n2. The exposition is generally clear and logically structured, progressing from biological background and motivation to model design, experiments, and implications.\n\n3. The paper addresses a central question in spatial cognition—why animals maintain dual spatial representations—and provides a mechanistic hypothesis with experimentally testable predictions. The framework may guide future research on memory-guided navigation and its neural implementation.\n\n4. The integration of auto-associative memory recall with grid-cell-based planning is a novel and biologically plausible hypothesis. It extends prior work by showing how GC-based planning can generalize to unvisited locations and reconstruct sensory experiences.\n\n** Weakness\n\n1. The term \"auto-association\" is central but not rigorously defined in the context of the HC-MEC loop.\n\n2. The proposed model lacks experiments on practical and real datasets.", "was_revised": true, "final_justification": "Thank the authors for the response, which addresses my concerns. I'll keep my positive rating unchanged.", "rebuttal": "We thank reviewer b2n6 for their thoughtful review and respond below to the comments\n\n> **Definition of auto-association**\n\nWe adopt the term from prior literature on the hippocampus's role in memory (Treves & Rolls, Hippocampus, 1994; ET. Rolls. Front. Syst. Neurosci. 2013.). Formally, we define an auto-association system as a function $f: \\mathbb{R}^n \\to \\mathbb{R}^n$ such that $f(\\mathbf{x}') = \\mathbf{x}$ if $\\mathbf{x}’ \\approx \\mathbf{x}$. In the scope of this paper, consider two vectors representing the Spatially Modulated Cells (SMCs) and Grid Cells (GCs), denoted as $\\mathbf{x_{\\text{smc}}}$ and $\\mathbf{x_{\\text{gc}}}$, respectively. These vectors, at any point during navigation, may be masked into $\\mathbf{x_{\\text{smc}}}'$ and $\\mathbf{x_{\\text{gc}}}'$ due to imperfect observation or path integration. We suggest that place cells serve as the function $f(\\cdot)$ such that $f([\\mathbf{x_{\\text{smc}}}' ; \\mathbf{x_{\\text{gc}}}' ]) = [\\mathbf{x_{\\text{smc}}} ; \\mathbf{x_{\\text{gc}}} ]$.  \n\n> **Experiments on practical and real datasets**\n\nWe conducted additional experiments using the Habitat Synthetic Scenes Dataset (HSSD) in Habitat-Sim, a visually realistic simulation environment. In our model, SMCs represent sensory observations. While the main text uses simulated sensory signals, here we replace them with features derived from visual observations from the Habitat Synthetic Scenes Dataset (HSSD): the agent captures images during exploration, which are passed through a vision encoder to generate SMC signals. We follow the same training procedure for HCMEC and planner models as in Section 4.4 and find that the model successfully reconstructs visual features from this realistic dataset.\n\nImportantly, our vision encoder produces features decodable back into images, allowing visualization of scenes along the planned path. All intermediate SMC states generated during planning can be decoded into images that closely match the expected scenes at their respective planned locations. We provide key implementation details below:\n\n- We discretize the environment into an $X_w \\times X_h$ spatial grid and capture $W \\times H$ allocentric panorama images at each grid point.\n\n- We fine-tuned a generalized visual encoder (a modified Masked AutoEncoder (MAE; He et al., 2021)) on images from this specific simulation gym. This MAE learns to encode images into compact feature representations, which can later be decoded back to the original images. Compactly, it can be expressed as $D(E(I)) \\approx I$, where $I$ is the image, $E: \\mathbb{R}^{W \\times H} \\to \\mathbb{R}\\_+^{512}$ is the encoder, and $D: \\mathbb{R}\\_+^{512} \\to \\mathbb{R}^{W \\times H}$ is the decoder.\n\n- At each location, the image is encoded into a 512-dim vector used as the SMC response, enabling training of HCMEC and the planner as in the main text.\n\n- During planning, the HCMEC updates the SMC region to intermediate states ${z_0, \\dots, z_T}$, where $z_i \\in \\mathbb{R}^{512}$. Decoding them with $D$ yields images ${I_0, \\dots, I_T}$ that closely resemble the expected views at the corresponding planned locations.\n\n>**Comparison to existing RL-based navigation methods**\n\nWe intend this work primarily to suggest answers to the question of why animals maintain two systems to encode space. We propose that grid cell representations serve as efficient, generalizable encodings for planning across diverse environments, but they lack sensory detail and therefore require hippocampal place cells (HPCs) to pattern-complete the corresponding sensory information along the planned path. The most relevant metric for this study is the successful path-planning ratio; we presented an extensive test of the success planning ratio of the HCMEC model in the last section of this rebuttal. \n\nIn the paper, to emphasize biological realism, we adopt a simplistic continuous-time RNN for modeling. However, the proposed framework is directly applicable to more advanced architectures such as GRUs or LSTMs, which may yield improved performance. We now additionally trained a variant with a simple layer normalization step before the hidden state update, which significantly increases the success rate to 0.99, making the approach potentially relevant for robotics applications.\n\n> **Empirical evidence supporting HPC reconstructs sensory experiences during planning**\n\nTo address this, we first test whether hippocampal place cells (HPCs) specifically contribute to trajectory reconstruction. This can be evaluated by checking whether intermediate HPC states during planning decode into a valid trajectory. For each planning trial, we record all HPC states and decode each time step into a location using nearest neighbor search over ratemaps aggregated during testing.\n\nWhile the main text focuses on decoding using spatially modulated cells (SMCs) and grid cells, here we compare planning performance across 4096 trials using HPC-decoded versus SMC-decoded trajectories. We find that HPC-decoded trajectories also reliably reach the goal:\n\n| Cell Type | Distance to Goal (cm) | Success Rate |\n| ------------- | ---------------- | ------------ |\n| SMCs | 4.27 ± 5.22 | 0.96 |\n| HPCs | 4.97 ± 7.80 | 0.94 |\n\nA complementary perspective is to test whether the reconstructed experience can be decoded beyond nearest neighbor search. The experiment in Experiments on practical and real datasets provides strong support: we train a compact image encoder-decoder, replace SMC signals with encoded image features, and find that the reconstructed sensory signals along the planned path can be decoded back into meaningful images.\n\n> **Performance varies with different levels of partial supervision or noise**\n\nWe conducted additional experiments to systematically distort the simulated ground truth grid cells. We trained 18 HCMEC and corresponding planner models (10 to evaluate the impact of noise and 8 to assess geometric distortions). Each model was evaluated on 4,096 planning trials. A trial is considered successful if the agent's final position is within 10 cm of the goal. Success rate is computed over all trials. On average, the goal is 57.54 ± 25.53 cm away from the start.\n\nFor the noise experiments, we added post-activation Gaussian noise (standard deviation 0.1 to 1.0) to all hidden units of the HCMEC at each time step during training. No noise was added at test time (planning phase).\n\nDue to space constraints, we report results based on decoding from SMCs, though we verified that results are consistent when decoding from GCs. In all cases, the mean distance to the goal remains below 10 cm (in a 100 cm × 100 cm environment), and success rates remain high across noise levels. This suggests that the planner can effectively operate even with noisy GC representations.\n\n| Noise (std) | Distance to Goal (cm) | Success Rate | Detour Ratio |\n| ------------- | ---------------- | ------------ | ------------ |\n| 0.1           | 2.20 ± 5.89      | 0.99         | 1.66         |\n| 0.2           | 2.35 ± 5.68      | 0.99         | 2.00         |\n| 0.3           | 1.82 ± 5.09      | 0.99         | 1.88         |\n| 0.4           | 3.75 ± 8.84      | 0.98         | 2.10         |\n| 0.5           | 2.62 ± 3.43      | 0.98         | 2.36         |\n| 0.6           | 3.12 ± 3.72      | 0.97         | 2.33         |\n| 0.7           | 3.31 ± 5.66      | 0.99         | 2.55         |\n| 0.8           | 2.98 ± 2.78      | 0.99         | 2.60         |\n| 0.9           | 3.00 ± 3.99      | 0.98         | 2.31         |\n| 1.0           | 5.30 ± 6.98      | 0.91         | 2.70         |\n\nNext, we added noise to the ground-truth GC signal that is used at the first time step during planning.\n\n| Noise (std) | Distance to Goal (cm) | Success Rate | Detour Ratio |\n| ------------- | ---------------- | ------------ | ------------ |\n| 0.1           | 2.45 ± 2.49      | 0.99         | 2.16         |\n| 0.2           | 2.40 ± 4.37      | 0.99         | 1.91         |\n| 0.3           | 1.89 ± 5.41      | 0.99         | 1.98         |\n| 0.4           | 4.10 ± 7.66      | 0.96         | 2.54         |\n| 0.5           | 4.36 ± 5.01      | 0.95         | 3.28         |\n| 0.6           | 4.05 ± 5.48      | 0.96         | 2.44         |\n| 0.7           | 4.01 ± 6.70      | 0.97         | 2.53         |\n| 0.8           | 4.79 ± 6.00      | 0.91         | 2.63         |\n| 0.9           | 5.54 ± 8.43      | 0.91         | 2.67         |\n| 1.0           | 7.72 ± 13.24    | 0.85         | 2.49         |\n\nAdditionally, we trained another 8 models without noise but with sheared grid fields.\n\n| Shearing Angle | Residual to Goal (cm) | Success Rate | Detour Ratio |\n| -------------- | --------------------- | ------------ | ------------ |\n| 40             | 9.53 ± 16.39          | 0.86         | 2.45         |\n| 45             | 7.88 ± 17.66          | 0.91         | 2.10         |\n| 50             | 7.40 ± 18.18          | 0.91         | 2.12         |\n| 55             | 5.48 ± 11.38          | 0.92         | 2.38         |\n| 60             | 7.66 ± 16.29          | 0.87         | 2.26         |\n| 65             | 2.47 ± 7.13           | 0.98         | 2.04         |\n| 70             | 5.13 ± 10.22          | 0.91         | 2.44         |\n| 75             | 6.38 ± 14.26          | 0.91         | 2.12         |\n| 80             | 4.46 ± 11.31          | 0.94         | 1.91         |"}, {"review_id": "yxHyfbwufE", "reviewer": "Reviewer_b4bC", "summary": "The authors present a single-layer RNN model of the MEC-HPC circuit, incorporating speed cells, direction cells, spatially modulated cells , and grid cells, with place cells emerging during training. The model learns path integration and can retrieve spatial locations from masked sensory cues. Additionally, the authors introduce a planning submodule—implemented both as a theoretically motivated algorithm and a trained RNN—which generates navigation trajectories between current and goal locations.", "questions": "- Could the authors clarify the use of the term “partially supervised” about grid cells? Based on the description, it appears that grid cell activity is fully supervised against ground truth responses.\n- Could the authors elaborate on the path integration mechanism? Specifically, how do speed and direction cells project to grid cells? Do any neurons with conjunctive tuning to location, speed, and direction emerge during training?\n- Why is the SMC trained separately on masked sensory inputs? My understanding is that auto-association should be learned during memory formation (i.e., path integration), and masking should be applied only during testing to evaluate recall.\n- Has the planner RNN been analyzed or dissected to confirm that it implements the proposed planning mechanism described in Sections 4.1–4.3?", "limitations": "Yes.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n- The work unifies the MEC–HPC circuit within a single-layer RNN, avoiding hand-crafted projection matrices between modules.\n- The main claims are substantiated by both simulations and theoretical derivations.\n- Ablation and comparison studies are provided, such as evaluations of path integration using GCs alone versus the full model, and recall performance with GC, SMC, and GC+SMC populations.\n\n**Weaknesses**\n\n- The novelty of the contribution is not clearly articulated. Many of the core results—such as grid cell path integration, emergence of place cells, and reconstruction from GC activity—have been previously explored in the literature.\n- In Section 3, while the model demonstrates goal retrieval via sensory cues, this property does not appear to be integrated into the planning process. Instead, the planning submodule receives a separate, direct goal input, rather than leveraging retrieval from GCs.\n- The training procedures for different model properties (path integration, memory retrieval, planning) seem to be conducted separately, rather than via a unified training protocol. Details regarding training procedures for each task are sometimes unclear.\n- Minor issues:\n  + Figures lack clarity; for example, the markers in Figure 2c are difficult to distinguish.\n  + Not all notations are clearly defined.", "was_revised": true, "final_justification": "The topic addressed in this work is interesting, and the authors have demonstrated the model’s capability for goal-directed path planning as well as the important role of GCs. However, I still have two major concerns:\nAs noted in the rebuttal, the reconstruction of sensory experience from GC patterns is described as “a core contribution we aim to emphasize in this paper.” To my understanding, similar questions have been addressed in prior work, for example, Chandra, S. et al., Nature, 2025.\nIn their further reply, the authors acknowledged that their work extends Chandra, S. et al., “emphasizing how the same underlying wiring logic might also support complex goal-directed planning.” However, in the subsequent planning experiments, it appears that the goal-recall component was not explicitly incorporated.\nThese points limit my ability to fully support acceptance of the paper.", "rebuttal": "We thank the reviewer b4bC for their insightful comments which we address below.\n\n> **The novelty of the contribution**\n\nThe main contribution of this paper is to answer the questions: \"Why do animals maintain both place cells and grid cells?\" and \"What can animals do with these two representations to support internally motivated navigation tasks?\" The answers to these questions form the core contributions of this work:\n\n- First, we propose that place cells serve to auto-associate the spatially modulated cells (SMCs) and grid cells (GCs), such that the two types of cells are tightly coupled within an environment and dynamically rewired across environments. Here, we do not intend to answer why place cells emerge, but rather to refine the functional role of place cells proposed in previous theories.\n\n- While this is an extension of previous place cell emergence theories, it makes predictions that cannot be captured by prior models, e.g., auto-associating SMCs and GCs allows retrieval of GC patterns at goal locations.\n\n- We subsequently show that using grid cells for planning is more efficient than direct planning with place cells or SMCs. Then, we explain why direct planning with place cells and SMCs, while feasible, may not be necessary. This auto-association also allows reconstruction of sensory experience from GC patterns, and we would like to highlight that this is not a previously known prediction and is a core contribution we aim to emphasize in this paper.\n\n- Finally, we extend the previous theoretical framework of grid-cell-based navigation (Bush et al., Using Grid Cells for Navigation, Neuron, 2015). Our extended framework provides a feasible implementation for both brain and ANNs without requiring an exponential number of neurons to encode pairwise displacements.\n\n> **Experiments on practical and real datasets**: \nWe would also like to highlight our additional contribution that, as suggested by reviewer b2n6, we conducted experiments on realistic datasets using the Habitat Synthetic Scenes Dataset (HSSD) in Habitat-Sim, a visually realistic simulation environment. \n\nIn our model, SMCs represent sensory observations. While the main text uses simulated sensory signals, here we replace them with features derived from visual observations from the Habitat Synthetic Scenes Dataset (HSSD): the agent captures images during exploration, which are passed through a vision encoder to generate SMC signals. We follow the same training procedure for HCMEC and planner models as in Section 4.4 and find that the model successfully reconstructs visual features from this realistic dataset.\n\nImportantly, our vision encoder produces features decodable back into images, allowing visualization of scenes along the planned path. All intermediate SMC states generated during planning can be decoded into images that closely match the expected scenes at their respective planned locations. We provide key implementation details below:\n\n- We discretize the environment into an $X_w \\times X_h$ spatial grid and capture $W \\times H$ allocentric panorama images at each grid point.\n\n- We fine-tuned a generalized visual encoder (a modified Masked AutoEncoder (MAE; He et al., 2021)) on images from this specific simulation gym. This MAE learns to encode images into compact feature representations, which can later be decoded back to the original images. Compactly, it can be expressed as $D(E(I)) \\approx I$, where $I$ is the image, $E: \\mathbb{R}^{W \\times H} \\to \\mathbb{R}^{512}$ is the encoder, and $D: \\mathbb{R}^{512} \\to \\mathbb{R}^{W \\times H}$ is the decoder.\n\n- At each location, the image is encoded into a 512-dim vector used as the SMC response, enabling training of HCMEC and the planner as in the main text.\n\n- During planning, the HCMEC updates the SMC region to intermediate states ${z_0, \\dots, z_T}$, where $z_i \\in \\mathbb{R}^{512}$. Decoding them with $D$ yields images ${I_0, \\dots, I_T}$ that closely resemble the expected views at the corresponding planned locations.\n\n> **Integrating goal retrieval into the path-planning process**\n\nIn the main text, we separated retrieval from planning to maintain a clearer separation of concerns across sections. Following the reviewer’s suggestion, we conducted an additional experiment that integrates the recall process directly into planning. We will include this additional result in the supplemental materials.\n\n> **Detailed training procedures**\n\nSupplemental Materials contain a detailed description of the simulation and training procedure. This is referred to on Line 127, 176, and 183. We will make this more prominent. We will also open-source our training code and model weights to ensure reproducibility.\n\n> **Figure clarity and defining all notations**\n\nThanks. We will revise Fig 2. We would be grateful if you can point us to notations that were not defined, and fix the issue.\n\n> **On partial supervision of grid cells**\n\nOur core contributions and claims do not depend on co-emergence of grid cells, or having idealized grid cell responses.\n\nFirst, we are showing that if place cells (PCs) auto-associate grid cells and spatially modulated cells (SMCs), the agent can use sensory cues to recall GC patterns. Both recall and reconstruction of sensory experiences during planning work, as long as GC and SMC patterns are spatially stable.\n\nWe will modify Section 4 to include a short mathematical argument that shows that planning with grid cells does not require idealized GC representations. It will consist of the following claims. Within a single grid-scale, the displacement vector can be correctly decoded even if the grid field undergoes shearing and compression (in 1D these two are directly related to grid scale, while the 2D case can be decomposed into 1D cases on non-colinear axes). Larger displacements across multiple grid-scales in sheared and compressed grid-fields can be pieced together by the decoding process. Decoding is also robust with sub-optimal grid-scales. The equation on Line 276 can be written in a more general form $$ \\hat{d} = \\frac{1}{2\\pi m} \\cdot \\left( \\sum_{i=1}^{k} \\ell_{i} \\cdot Z_i + \\sum_{j=k+1}^{m} \\ell_{j} \\cdot \\Delta \\phi_i \\right) $$Under optimal scaling, $\\ell_i = \\ell_{0} s^i$ but so long as there are sufficiently many scales, the predicted displacement is guaranteed to move the animal closer to the target. \n\nUnlike place cells **the grid cell patterns may not be learned but are believed to be dynamically self-organized via a network that was pre-wired by evolution**. For instance, genetically deleting the gene encoding a critical NMDA receptor subunit specifically disrupts grid patterns without affecting other spatial cell types (Gil et al., _Nat Neurosci._, 2018), which indicates that the underlying molecular components and the capacity to form grid-like patterns are at least partly encoded in the genome. Existing theoretical literature also provides evidence suggesting the grid cells may not be learned but are made by a pre-wired dynamical attractor network (Fuhs & Touretzky, J. Neurosci. 2006; Burak & Fiete, PLoS Comp. Bio., 2009; Gardner et al., Nature. 2022). The supervised grid cell region can be directly replaced with one of the existing computational and theoretical models (Cueva & Wei, ICLR, 2018; Kang & Balasubramian, eLife, 2019; Sorscher et al. NeurIPS, 2019) of grid cells without affecting our claims of recall and reconstruction during planning. Here we only use the grid cells as pattern generators that generate grid-like patterns (idealized or not), and ask, “if we have these patterns, could the animal use them for recall and planning”.\n\n> **Path integration mechanism and projection from speed and direction cells ...**\n\nWe describe the path integration mechanism in Section 2.1.1. Specifically, speed and direction signals are directly input into designated speed and direction cells, which in turn project to grid cells through learned recurrent connections embedded within the single recurrent weight matrix (Eq. 2; Figure 1b, d). Grid cells are trained to infer the ground-truth GC responses from their initial state and the sequence of speed and direction signals relayed via these input cell types. While we did not explicitly analyze the emergence of conjunctive cells in our model, we observed band-like tuning patterns in the direction cells. Due to NeurIPS rebuttal constraints, we are unable to include these visualizations here but will provide them in the supplementary materials.\n\n> **Why is the SMC trained separately on masked sensory inputs?**\n\nThe SMC is not trained separately on masked sensory inputs. We extend classic auto-association theories of place cells by proposing that they auto-associate not only SMCs but also grid cell patterns. This leads to two key predictions: sensory cues can recall GC patterns, and intermediate GC states during planning can be used to reconstruct sensory experiences.\n\nTo trigger auto-association, the original signals must be distorted. Prior work (Wang et al., Time Makes Space, NeurIPS 2024) used masking for this purpose. In our model, SMCs simulate sensory input and GCs simulate path integration. Since these signals are not causally linked, we apply independently sampled masking ratios. The training process remains fully integrated, not separated into stages.\n\n> **Has the planner RNN been analyzed or dissected to confirm that it implements the proposed planning mechanism described in Sections 4.1–4.3?**\n\nThe discussion in Section 4.1 and 4.3 is validated by the fact that we can decode back the location and trajectory of the animal correctly using SMCs and GCs. Section 4.2 provides a conceptual framework to think of how the RNN could generalize to planning for long trajectories after learning to plan for short trajectories. It is difficult to check the mechanism of the RNN, e.g., whether or not it instantiates a Markov chain that we have alluded to. We will preface the paragraph on Line 238 to emphasize this point."}, {"review_id": "zuIkZyG95g", "reviewer": "Reviewer_KTLJ", "summary": "This paper presents a computational model, REMI, that proposes a novel, system-level functional architecture for the hippocampus-medial entorhinal cortex (HC-MEC) circuit in the service of goal-directed navigation. The central hypothesis is that hippocampal place cells (HPCs) act as an auto-associative memory system. This system is posited to link two distinct but complementary streams of information processed in the MEC: context-rich, sensory-driven representations from spatially modulated cells (SMCs) and context-invariant, metric spatial representations from grid cells (GCs). This framework seeks to explain the functional utility of maintaining these two seemingly parallel spatial coding schemes.", "questions": "The authors must perform extensive robustness analyses. They should systematically add noise, geometric distortions (e.g., shear, compression), and phase shifts to the \"ground truth\" GC signals during both training and testing. The key question is whether the planning mechanism degrades gracefully or fails catastrophically. This would, at a minimum, characterize the fragility of their approach.\n\nThe introduction and discussion sections must be expanded to provide a detailed, nuanced, and fair comparison between the REMI and SR frameworks. This discussion should clearly articulate the architectural and functional differences and, most importantly, highlight specific experimental phenomena or predictions where the two models diverge.\n\nAll experiments must be repeated multiple times (e.g., 5-10 runs) using different random seeds for all stochastic elements of the model and data generation. All figures presenting quantitative results (e.g., Figure 2a, L2 distance over time) must include error bars (e.g., shaded regions representing standard deviation or standard error of the mean) to visualize the variance across runs.\n\nThe conceptual leap from the discrete-state Markovian planning model (visualized in Fig. 3c-e) to the continuous-state RNN implementation needs to be better articulated. The paper claims the RNN's recurrent matrix represents the transition matrix T, but this is an analogy. A more detailed explanation of how the continuous dynamics of the planner RNN approximate the iterative, search-like process would significantly improve the clarity of Section 4.", "limitations": "If the authors address the questions above, then yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The foremost strength of this paper is its ambition and success in proposing a unified, system-level theory that integrates disparate cognitive functions—sensory perception, spatial representation, memory, and planning—into a single, coherent computational framework.\n\nPerhaps the most novel and impactful contribution is the concept of \"reconstructing experience\" during planning. This provides a concrete, mechanistic model for complex cognitive phenomena like hippocampal preplay and mental simulation. The model suggests that planning is not merely an abstract calculation of a trajectory in a state space, but an embodied, imaginative process where the agent can internally \"visualize\" the sensory consequences of a potential path. \n\nThe technical implementation of the REMI model is well-conceived and appropriate for the questions being investigated. The authors develop a modified single-layer RNN that integrates input and output nodes directly into the recurrent dynamics (Equation 2). This architectural choice elegantly eliminates the need for separate input and output projection matrices, simplifying the model while enabling the joint simulation of the HC-MEC loop and the planner within a single, unified structure.\n\nDespite its conceptual strengths, the paper is built on a methodological flaw: the decision to use partial supervision to train the GC subnetwork. The authors justify this choice by stating their primary focus is on planning rather than emergence. While this is a pragmatic decision, it fundamentally undermines the strength, generality, and biological plausibility of their conclusions. The success of the entire planning mechanism may be critically, and perhaps entirely, contingent on having access to this idealized spatial representation. It is highly uncertain whether the proposed planning algorithm would function effectively, or at all, with the noisy, distorted, and imperfect grid representations that would likely arise from a more realistic unsupervised learning process or those observed in biological recordings. This makes the model's impressive performance potentially brittle and, worse, an artifact of the unrealistic assistance provided by the supervision.\n\nThe authors' treatment of SR is cursory and dismissive. They claim that SR-based models require visiting all locations in the environment and therefore cannot explain the ability to take shortcuts through unvisited areas. This is an oversimplification. The flexibility of SR depends heavily on the nature of the state representation and the policy generalization used; some formulations of SR can indeed support novel trajectories, e.g. see https://arxiv.org/abs/2107.08426, https://papers.nips.cc/paper_files/paper/2017/hash/350db081a661525235354dd3e19b8c05-Abstract.html. The paper makes no attempt to formally or empirically compare REMI and SR. For instance, could an SR model, where states are defined by SMC patterns, also learn to reconstruct intermediate states during planning? What are the unique, falsifiable predictions of REMI that would allow it to be experimentally distinguished from SR?\n\nAnother weakness that compromises the paper is the lack of statistical validation for the experimental results. The main quantitative and qualitative results, presented in Figures 2 and 3, are shown without any error bars, confidence intervals, or indication that the simulations were run multiple times with different random initializations.", "was_revised": true, "final_justification": "The paper has been substantially improved and all my concerns were addressed. I have therefore increased my score to accept.", "rebuttal": "Thank you for the insightful comments. We are glad you see this as an ambitious and unified theory for sensory perception, spatial representation, memory, and planning. We agree that a key novelty of this work is that planning involves explicit reconstruction of experiences and an embodied, imaginative process where the agent visualizes the sensory consequences of a potential path.\n\nWe appreciate your comments. Our responses are below.\n\n> **On partial supervision of grid cells**\n\nOur core contributions do not depend on co-emergence of grid cells or idealized grid responses.\n\nFirst, we are showing that if place cells (PCs) auto-associate grid cells and spatially modulated cells (SMCs), the agent can use sensory cues to recall GC patterns. Both recall and reconstruction of sensory experiences during planning work, as long as GC and SMC patterns are spatially stable.\n\nWe will revise Section 4 to include a brief mathematical argument showing that planning with grid cells does not require idealized responses. Within a single grid scale, the displacement vector can be decoded even if the field is sheared or compressed (in 1D, this changes scale $\\ell$; in 2D, it reduces to two non-colinear 1D components). Larger displacements across multiple grid-scales in distorted grid-fields can be pieced together by the decoding process. Decoding is also robust with sub-optimal grid-scales. The equation on Line 276 can be generalized as $$ \\hat{d} = \\frac{1}{2\\pi m} \\cdot \\left( \\sum_{i=1}^{k} \\ell_{i} \\cdot Z_i + \\sum_{j=k+1}^{m} \\ell_{j} \\cdot \\Delta \\phi_i \\right) $$ So long as there are sufficiently decodable scales, the predicted displacement reliably moves the agent toward the target.\n\nUnlike place cells, _**grid cell patterns might not be learned but are believed to be self-organized by a pre-wired network**_. For instance, deleting an NMDA receptor subunit gene disrupts grid patterns without affecting other spatial cells (Gil et al., Nat Neurosci., 2018), suggesting a genetic basis for grid-like activity. Theoretical works also suggest that grid patterns emerge from dynamical attractor network (Fuhs & Touretzky, J. Neurosci., 2006; Burak & Fiete, PLoS Comput Biol, 2009; Gardner et al., Nature, 2022). Our supervised GC module could be aslo replaced by existing models (Cueva & Wei, ICLR, 2018; Kang & Balasubramanian, eLife, 2019; Sorscher et al., NeurIPS, 2019) without affecting our conclusions. We use grid cells simply as spatial pattern generator and ask: if such patterns exist, could the animal use them for recall and planning?\n\n> **.... to the \"ground truth\" GC signals during both training and testing**\n\nTo clarify, at training time, the GC sub-network receives the ground-truth GC signal as input at the first time-step, path-integrates the hidden state forward, and receives a loss that measures the deviation from the ground-truth GC signal at each time-step. At test time, it receives the ground-truth only at the first step. We discuss new experiments based on your suggestion in the next section.\n\n> **Statistical validation for the experimental results**\n> **perform extensive robustness analyses by systematically adding noise, geometric distortions (e.g., shear, compression) ...** \n\nWe conducted additional experiments to systematically distort the simulated ground truth grid cells. We trained 18 HCMEC and corresponding planner models (10 to evaluate the impact of noise and 8 to assess geometric distortions). Each model was evaluated on 4,096 trials. A trial is successful if the final position is within 10 cm of the goal. On average, the goal is 57.54 ± 25.53 cm away from the start.\n\nFor the noise experiments, we added post-activation Gaussian noise (standard deviation 0.1 to 1.0) to all hidden units of the HCMEC at each time step during training. No noise was added at test time (planning phase).\n\nDue to space constraints, we report results based on decoding from SMCs, though we verified that results are consistent when decoding from GCs. In all cases, the mean distance to the goal remains below 10 cm (in a 100 cm × 100 cm environment), and success rates remain high across noise levels. This suggests that the planner can effectively operate even with noisy GC representations.\n\n| Noise (std) | Distance to Goal (cm) | Success Rate | Detour Ratio |\n| - | - | - | - |\n| 0.1 | 2.20 ± 5.89 | 0.99 | 1.66 |\n| 0.2 | 2.35 ± 5.68 | 0.99 | 2.00 |\n| 0.3 | 1.82 ± 5.09 | 0.99 | 1.88 |\n| 0.4 | 3.75 ± 8.84 | 0.98 | 2.10 |\n| 0.5 | 2.62 ± 3.43 | 0.98 | 2.36 |\n| 0.6 | 3.12 ± 3.72 | 0.97 | 2.33 |\n| 0.7 | 3.31 ± 5.66 | 0.99 | 2.55 |\n| 0.8 | 2.98 ± 2.78 | 0.99 | 2.60 |\n| 0.9 | 3.00 ± 3.99 | 0.98 | 2.31 |\n| 1.0 | 5.30 ± 6.98 | 0.91 | 2.70 |\n\nNext, we added noise to the ground-truth GC signal that is used at the first time step during planning.\n\n| Noise (std) | Distance to Goal (cm) | Success Rate | Detour Ratio |\n| - | - | - | - |\n| 0.1 | 2.45 ± 2.49 | 0.99 | 2.16 |\n| 0.2 | 2.40 ± 4.37 | 0.99 | 1.91 |\n| 0.3 | 1.89 ± 5.41 | 0.99 | 1.98 |\n| 0.4 | 4.10 ± 7.66 | 0.96 | 2.54 |\n| 0.5 | 4.36 ± 5.01 | 0.95 | 3.28 |\n| 0.6 | 4.05 ± 5.48 | 0.96 | 2.44 |\n| 0.7 | 4.01 ± 6.70 | 0.97 | 2.53 |\n| 0.8 | 4.79 ± 6.00 | 0.91 | 2.63 |\n| 0.9 | 5.54 ± 8.43 | 0.91 | 2.67 |\n| 1.0 | 7.72 ± 13.24 | 0.85 | 2.49 |\n\nAdditionally, we trained another 8 models without noise but with sheared grid fields.\n\n| Shearing Angle | Residual to Goal (cm) | Success Rate | Detour Ratio |\n| - | - | - | - |\n| 40 | 9.53 ± 16.39 | 0.86 | 2.45 |\n| 45 | 7.88 ± 17.66 | 0.91 | 2.10 |\n| 50 | 7.40 ± 18.18 | 0.91 | 2.12 |\n| 55 | 5.48 ± 11.38 | 0.92 | 2.38 |\n| 60 | 7.66 ± 16.29 | 0.87 | 2.26 |\n| 65 | 2.47 ± 7.13 | 0.98 | 2.04 |\n| 70 | 5.13 ± 10.22 | 0.91 | 2.44 |\n| 75 | 6.38 ± 14.26 | 0.91 | 2.12 |\n| 80 | 4.46 ± 11.31 | 0.94 | 1.91 |\n\nWe also trained 10 models with 60-degree grid cell lattices, without noise, using different random seeds. The mean distance to the goal was 3.14 ± 2.30 cm.\n\n> **Experiments on real datasets**: We highlight an additional contribution: as suggested by reviewer b2n6, we repeated our experiments in a visually realistic environment (Habitat-Sim).\n\nIn our model, SMCs represent sensory observations. While the main text uses simulated sensory signals, here we replace them with features derived from visual observations in the Habitat Synthetic Scenes Dataset (HSSD). The agent captures images during exploration, which are processed by a vision encoder to generate SMC signals. We follow the same training procedure as in Section 4.4 and find the model successfully reconstructs visual features along the path.\n\nThe vision encoder produces features that can be decoded back into images, allowing visualization of scenes along the planned path. Intermediate SMC states generated during planning decode into images that closely match expected views at their corresponding planned locations. Key implementation details are as follows:\n\n- We discretize the environment into an $X_w \\times X_h$ grid and capture $W \\times H$ allocentric panorama images at each location.\n\n- We fine-tuned a visual encoder (a modified Masked AutoEncoder; He et al., 2021) on images from this simulation. It encodes images into compact features that can be decoded back: $D(E(I)) \\approx I$, where $E: \\mathbb{R}^{W \\times H} \\to \\mathbb{R}^{512}$ and $D: \\mathbb{R}^{512} \\to \\mathbb{R}^{W \\times H}$.\n\n- Each image is encoded into a 512-dim vector used as the SMC response, enabling training of HCMEC and the planner as before.\n\n- During planning, the HCMEC updates the SMC region to intermediate states ${z_0, \\dots, z_T}$, where $z_i \\in \\mathbb{R}^{512}$. Decoding these yields a sequence of images that closely resemble the expected views at planned locations.\n\n> **Successor Representation**\n\nWe will remove the sentence “SR-based models require visiting all locations…” on Line 57.\n\nIntroduction\n“Successor representation (SR) frameworks (e.g., Stachenfeld et al., Nature Neurosci., 2017; Levenstein et al., bioRxiv, 2024) propose that hippocampal place cells (HPCs) encode expected future state occupancy under a given policy, enabling flexible behavior across related tasks. These models provide compelling accounts of generalization in structured environments and task transfer (Barreto et al., NeurIPS, 2017; Abdolsheh et al., ICML, 2021). However, most SR models often emphasize discrete state spaces represented by HPCs, which exhibit strong contextual tuning. In contrast, grid cells in the medial entorhinal cortex (MEC) display continuous and periodic spatial structure that supports vector-based navigation. It is less context-specific and potentially allows greater generalization across environments. If grid cells encode spatial transition likelihoods, as suggested by the stability of their phase relationships, they may therefore offer a complementary substrate for planning.”\n\nDiscussion\n“Our framework provides a complementary perspective to existing SR models. In REMI, HPCs must predict both GC and SMC responses at the next timestep in order to pattern-complete. Since HPCs are direction-agnostic, accurately reconstructing GC and SMC activity requires encoding possible state transitions. Because adjacent locations tend to have higher transition probability (Stachenfeld et al., 2017), our model implicitly reflects the SR framework of HPCs, where transition structure is embedded in the pattern-completion dynamics. This may be validated experimentally: selectively disrupting HPC-to-MEC projections may degrade planning under REMI, but not SR frameworks.”\n\n>**Conceptual leap from the discrete-state Markovian planning model**\n\nOur discussion of the Markov chain serves primarily as a conceptual framework to explain how the RNN might generalize from short to long trajectories, and why such generalization depends on the periodic grid structure. It is challenging to directly verify whether the RNN instantiates the specific Markov chain we describe. We will revise the paragraph on Line 238 to clarify this intent."}]

Author rebuttal:

Meta-review: The paper introduces REMI, a unified, system-level theory for cue-triggered goal retrieval, path planning, and reconstruction of sensory experience along a planned route. Before the rebuttal, reviewers raised concerns regarding the lack of real-world datasets, unclear definition of “partially supervised,” noise robustness, and missing baseline comparisons. After the rebuttal, three reviewers recommended acceptance, while one insisted on rejection, primarily due to the lack of comparison with Chandra et al., Nature 2025. Although the authors provided a strong rebuttal, the reviewer remained unconvinced.

After reviewing the exchange, the AC agrees with the authors that the focus and model designs of the two papers are different and complementary.

Given that the merits of the work outweigh its limitations, the AC recommends acceptance (poster). The authors are strongly encouraged to incorporate the reviewers’ comments into the final version.
Decision: Accept (poster)