Title: REMI: Reconstructing Episodic Memory During Internally Driven Path Planning
Venue: NeurIPS 2025
Accepted: yes
Keywords: grid cells; place cells; hippocampus; medial entorhinal cortex; spatial navigation and planning
Abstract: Grid cells in the medial entorhinal cortex (MEC) and place cells in the hippocampus (HC) both form spatial representations. Grid cells fire in triangular grid patterns, while place cells fire at specific locations and respond to contextual cues. How do these interacting systems support not only spatial encoding but also internally driven path planning, such as navigating to locations recalled from cues? Here, we propose a system-level theory of MEC-HC wiring that explains how grid and place cell patterns could be connected to enable cue-triggered goal retrieval, path planning, and reconstruction of sensory experience along planned routes. We suggest that place cells autoassociate sensory inputs with grid cell patterns, allowing sensory cues to trigger recall of goal-location grid patterns. We show analytically that grid-based planning permits shortcuts through unvisited locations and generalizes local transitions to long-range paths. During planning, intermediate grid states trigger place cell pattern completion, reconstructing sensory experiences along the route. Using a single-layer RNN modeling the HC-MEC loop with a planning subnetwork, we demonstrate these effects in both biologically grounded navigation simulations using RatatouGym and visually realistic navigation tasks using Habitat Sim.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a computational model, REMI, that proposes a novel, system-level functional architecture for the hippocampus-medial entorhinal cortex (HC-MEC) circuit in the service of goal-directed navigation. The central hypothesis is that hippocampal place cells (HPCs) act as an auto-associative memory system. This system is posited to link two distinct but complementary streams of information processed in the MEC: context-rich, sensory-driven representations from spatially modulated cells (SMCs) and context-invariant, metric spatial representations from grid cells (GCs). This framework seeks to explain the functional utility of maintaining these two seemingly parallel spatial coding schemes.\n\nSTRENGTHS AND WEAKNESSES:\nThe foremost strength of this paper is its ambition and success in proposing a unified, system-level theory that integrates disparate cognitive functions—sensory perception, spatial representation, memory, and planning—into a single, coherent computational framework.\n\nPerhaps the most novel and impactful contribution is the concept of \"reconstructing experience\" during planning. This provides a concrete, mechanistic model for complex cognitive phenomena like hippocampal preplay and mental simulation. The model suggests that planning is not merely an abstract calculation of a trajectory in a state space, but an embodied, imaginative process where the agent can internally \"visualize\" the sensory consequences of a potential path. \n\nThe technical implementation of the REMI model is well-conceived and appropriate for the questions being investigated. The authors develop a modified single-layer RNN that integrates input and output nodes directly into the recurrent dynamics (Equation 2). This architectural choice elegantly eliminates the need for separate input and output projection matrices, simplifying the model while enabling the joint simulation of the HC-MEC loop and the planner within a single, unified structure.\n\nDespite its conceptual strengths, the paper is built on a methodological flaw: the decision to use partial supervision to train the GC subnetwork. The authors justify this choice by stating their primary focus is on planning rather than emergence. While this is a pragmatic decision, it fundamentally undermines the strength, generality, and biological plausibility of their conclusions. The success of the entire planning mechanism may be critically, and perhaps entirely, contingent on having access to this idealized spatial representation. It is highly uncertain whether the proposed planning algorithm would function effectively, or at all, with the noisy, distorted, and imperfect grid representations that would likely arise from a more realistic unsupervised learning process or those observed in biological recordings. This makes the model's impressive performance potentially brittle and, worse, an artifact of the unrealistic assistance provided by the supervision.\n\nThe authors' treatment of SR is cursory and dismissive. They claim that SR-based models require visiting all locations in the environment and therefore cannot explain the ability to take shortcuts through unvisited areas. This is an oversimplification. The flexibility of SR depends heavily on the nature of the state representation and the policy generalization used; some formulations of SR can indeed support novel trajectories, e.g. see https://arxiv.org/abs/2107.08426, https://papers.nips.cc/paper_files/paper/2017/hash/350db081a661525235354dd3e19b8c05-Abstract.html. The paper makes no attempt to formally or empirically compare REMI and SR. For instance, could an SR model, where states are defined by SMC patterns, also learn to reconstruct intermediate states during planning? What are the unique, falsifiable predictions of REMI that would allow it to be experimentally distinguished from SR?\n\nAnother weakness that compromises the paper is the lack of statistical validation for the experimental results. The main quantitative and qualitative results, presented in Figures 2 and 3, are shown without any error bars, confidence intervals, or indication that the simulations were run multiple times with different random initializations.\n\nQUESTIONS:\nThe authors must perform extensive robustness analyses. They should systematically add noise, geometric distortions (e.g., shear, compression), and phase shifts to the \"ground truth\" GC signals during both training and testing. The key question is whether the planning mechanism degrades gracefully or fails catastrophically. This would, at a minimum, characterize the fragility of their approach.\n\nThe introduction and discussion sections must be expanded to provide a detailed, nuanced, and fair comparison between the REMI and SR frameworks. This discussion should clearly articulate the architectural and functional differences and, most importantly, highlight specific experimental phenomena or predictions where the two models diverge.\n\nAll experiments must be repeated multiple times (e.g., 5-10 runs) using different random seeds for all stochastic elements of the model and data generation. All figures presenting quantitative results (e.g., Figure 2a, L2 distance over time) must include error bars (e.g., shaded regions representing standard deviation or standard error of the mean) to visualize the variance across runs.\n\nThe conceptual leap from the discrete-state Markovian planning model (visualized in Fig. 3c-e) to the continuous-state RNN implementation needs to be better articulated. The paper claims the RNN's recurrent matrix represents the transition matrix T, but this is an analogy. A more detailed explanation of how the continuous dynamics of the planner RNN approximate the iterative, search-like process would significantly improve the clarity of Section 4.\n\nLIMITATIONS:\nIf the authors address the questions above, then yes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a single-layer RNN model of the MEC-HPC circuit, incorporating speed cells, direction cells, spatially modulated cells , and grid cells, with place cells emerging during training. The model learns path integration and can retrieve spatial locations from masked sensory cues. Additionally, the authors introduce a planning submodule—implemented both as a theoretically motivated algorithm and a trained RNN—which generates navigation trajectories between current and goal locations.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The work unifies the MEC–HPC circuit within a single-layer RNN, avoiding hand-crafted projection matrices between modules.\n- The main claims are substantiated by both simulations and theoretical derivations.\n- Ablation and comparison studies are provided, such as evaluations of path integration using GCs alone versus the full model, and recall performance with GC, SMC, and GC+SMC populations.\n\n**Weaknesses**\n\n- The novelty of the contribution is not clearly articulated. Many of the core results—such as grid cell path integration, emergence of place cells, and reconstruction from GC activity—have been previously explored in the literature.\n- In Section 3, while the model demonstrates goal retrieval via sensory cues, this property does not appear to be integrated into the planning process. Instead, the planning submodule receives a separate, direct goal input, rather than leveraging retrieval from GCs.\n- The training procedures for different model properties (path integration, memory retrieval, planning) seem to be conducted separately, rather than via a unified training protocol. Details regarding training procedures for each task are sometimes unclear.\n- Minor issues:\n  + Figures lack clarity; for example, the markers in Figure 2c are difficult to distinguish.\n  + Not all notations are clearly defined.\n\nQUESTIONS:\n- Could the authors clarify the use of the term “partially supervised” about grid cells? Based on the description, it appears that grid cell activity is fully supervised against ground truth responses.\n- Could the authors elaborate on the path integration mechanism? Specifically, how do speed and direction cells project to grid cells? Do any neurons with conjunctive tuning to location, speed, and direction emerge during training?\n- Why is the SMC trained separately on masked sensory inputs? My understanding is that auto-association should be learned during memory formation (i.e., path integration), and masking should be applied only during testing to evaluate recall.\n- Has the planner RNN been analyzed or dissected to confirm that it implements the proposed planning mechanism described in Sections 4.1–4.3?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a computational framework, REMI, to explain how animals might integrate hippocampal place cells (HPCs) and medial entorhinal grid cells (GCs) to support spatial memory and path planning. The authors hypothesize that GCs and PCs work together to enable animals to recall goal locations from sensory cues and plan paths, even through unvisited areas.\n\nTo test this, the authors develop a single-layer recurrent neural network (RNN) model that simulates the HC-MEC loop, incorporating sensory, speed, and direction inputs. GCs are partially supervised to emulate path integration, while HPCs emerge through recurrent dynamics. The network demonstrates that sensory cues at goal locations can trigger recall of corresponding GC states via HPCs. These GC representations can then be used for planning, with grid-space trajectories decoded into speed and direction commands. Furthermore, the model shows that intermediate GC states can drive the reconstruction of sensory experiences via HPCs during planning.\n\nThe paper also formalizes grid-space planning mechanisms, including decoding displacements using grid phase differences, learning local transition rules for sequential planning, and combining multiple spatial scales for more accurate long-range path planning.\n\nSTRENGTHS AND WEAKNESSES:\n** Strengths\n\n1. The use of a unified RNN architecture that simulates HC-MEC dynamics and planning within a biologically plausible framework is well-conceived and systematically developed.\n\n2. The exposition is generally clear and logically structured, progressing from biological background and motivation to model design, experiments, and implications.\n\n3. The paper addresses a central question in spatial cognition—why animals maintain dual spatial representations—and provides a mechanistic hypothesis with experimentally testable predictions. The framework may guide future research on memory-guided navigation and its neural implementation.\n\n4. The integration of auto-associative memory recall with grid-cell-based planning is a novel and biologically plausible hypothesis. It extends prior work by showing how GC-based planning can generalize to unvisited locations and reconstruct sensory experiences.\n\n** Weakness\n\n1. The term \"auto-association\" is central but not rigorously defined in the context of the HC-MEC loop.\n\n2. The proposed model lacks experiments on practical and real datasets.\n\nQUESTIONS:\n1. How does the proposed planning mechanism compare to other RL-based navigation models in terms of performance and efficiency?\n\n2. The paper claims HPCs reconstruct sensory experiences during planning. Is there empirical evidence (e.g., firing patterns) to support this claim in the model?\n\n3. How does performance (e.g., recall accuracy, planning success) vary with different levels of partial supervision or noise in the GC signals?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes REMI, a framework explaining how animals use both grid cells and place cells for spatial navigation and path planning. The authors suggest that place cells auto-associate sensory information with grid cell patterns, enabling animals to recall goal locations from sensory cues and then plan paths using stable grid cell representations. They demonstrate this with a model that shows how grid cells can support sequential planning through local transition rules that generalize to long-distance navigation, while place cells can reconstruct sensory experiences along planned paths.\n\nSTRENGTHS AND WEAKNESSES:\nThe work provides a compelling theoretical framework that unifies the functional roles of grid cells and place cells, addressing the important question of why animals maintain two distinct spatial representation systems. The authors successfully model the HC-MEC loop in a single RNN that simultaneously exhibits grid cell path integration, place cell emergence, and planning capabilities. It was unclear if the planning network is properly thought of as part of the same single layer RNN, as it was added on afterwards, but regardless it is an interesting model. The work makes concrete, testable experimental predictions about how sensory cues should reactivate grid cell patterns and how selective brain region disruptions should affect navigation, which provide important links to empirical work. \n\nA significant limitation is the reliance on partially supervised grid cells rather than demonstrating their natural co-emergence with place cells, which somewhat undermines the biological plausibility claims and leaves questions about how this system would actually develop in real neural circuits. However, this limitation is acknowledge by the authors and leaving it to future work seems appropriate given the strong contributions of the remainder.\n\nQUESTIONS:\nAny arbitrarily complicated recurrent structure can be written as a single layer RNN. Since you train the HC-MEC and planning networks in very different ways and at different times, why does it make sense to think of them as the 'same' RNN?\nThe paper suggests a connection to successor features, which at a high-level seems plausible. Is there a rigorous connection that can be specified demonstrating that the same future state transition probabilities are encoded in the trained model?\nHow accurate is your actual path integration? Do you have quantitative metrics showing this for different time scales? How does it compare to other neural path integrators in terms of accuracy?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}